Login “Lightbox” for specific category content

Don’t know about a plugin that does that for you but you can use the WordPress native Thickbox:

First include the Thickbox script and style in your category only if the user is not logged in (simply copy/paste in your theme’s functions.php file)

function add_thickbox_script_and_style(){
    if(is_category('YOUR_CATEGORY') && !is_user_logged_in()){
        wp_enqueue_script('jquery');
        wp_enqueue_script('thickbox',null,array('jquery'));
        wp_enqueue_style('thickbox.css', "https://wordpress.stackexchange.com/".WPINC.'/js/thickbox/thickbox.css', null, '1.0');
    }
}
add_action('init','add_thickbox_script_and_style');

Then open up your category.php (or archive.php depending on your theme , or even better is a custom file for that category category-id.php) and add this code before the loop:

if (is_category('YOUR_CATEGORY') && !is_user_logged_in()){
    echo '<div id="lform" style="display:none;">';
    wp_login_form();
    echo '</div>';
    echo 'Please <a class="thickbox" href="#TB_inline?height=200&width=200&inlineId=lform">Login</a> to see the content';
}else{


//your loop goes here

}