Login on Multiple Sub-Domains without Multisite

You can dynamically update the WP_HOME and WP_SITEURL constants in wp-config.php to reflect the requested host:

define( 'WP_SITEURL', 'http://' . $_SERVER[ 'HTTP_HOST' ] );
define( 'WP_HOME', 'http://' . $_SERVER[ 'HTTP_HOST' ] );

As long as visitors don’t click a link containing a different fully-qualified domain (or otherwise send a request to a different subdomain via a form action or some such), they should remain on the subdomain that was originally requested.

That said, I don’t see any reason why the admin dashboard being served on domain.com alone would prevent you from adding a pop-up jQuery login form… If anything, I think it’s kind of strange to serve the admin dashboard from those category-centric subdomains when it provides administrative functions for the entire site regardless of which (sub)domain it’s accessed from. Intuition would lead me to think the administration dashboard for category1.domain.com would administer category1 alone.