Custom post types with custom directories

Is something like this possible without running MultiSite (which I believe would force the blog to live at blog.domain.com)?

Not quite. There are two basic modes multisite installation can operate in:

  • sub-domains ( site.domain.com )
  • sub-directories ( domain.com/site )

As far as I remember in subdirectory installation main blog is also forced to have /blog/ at start of permalinks for compatbility.

So for your requirements sub-directory install with main blog at domain.com (posts at domain.com/blog/) and additional blog at domain.com/open/ seems to fit.

tech