Can you call a template file without assigning template to a page in the admin panel?

You can hook into template_redirect like this:

function custom_template_redirect() {
    global $wp;

    if ($wp->query_vars['pagename'] == 'my-page-slug') { // check the page slug
        status_header(200); // a 404 code will not be returned in the HTTP headers if the page does not exists

        include(TEMPLATEPATH . "/test.php"); // include the corresponding template
        die();
    }
}
add_action( 'template_redirect', 'custom_template_redirect' );