Two WordPress Installs — Failed page on One Defaults to the Other

I’m wondering if you simply copied your files from the sub directory’s install into the root directory? That wouldn’t be how you want to create a multisite. Check out the instructions here:
http://codex.wordpress.org/Create_A_Network

Secondly, if you go into your sub directory’s install, look at the wp-config.php file and compare it to the wp-config.php file in the root directory; do they have the exact same DB info? For a second install, you need to at least give the database a prefix so it can separate the two installs within the schema.

Hope that helps!

EDIT (from comments below): if you look in the .htaccess file of your root install you’ll see this:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

You’re going to need an additional RewriteCond to exclude the directory of your second install – otherwise the root wp install will handle the request.

The documentation to do that using mod_rewrite is here:
http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html