How to set a fall back template for a custom post type in a plugin?

To provide a default template that can be overwritten by a theme hook into template_include like the linked questions suggest. You get the template WordPress wants to use as a parameter. If that is not the file you want – replace it with your plugin’s file:

add_filter( 'template_include', 'wpse_57232_render_cpt', 100 );

/**
 * Provide fall back template file for a custom post type single view.
 *
 * @return void
 */
function wpse_57232_render_cpt( $template )
{
    // Our custom post type.
    $post_type="my_custom_cpt";

    // WordPress has already found the correct template in the theme.
    if ( FALSE !== strpos( $template, "/single-$post_type.php" ) )
    {
        // return the template in theme  
        return $template;
    }

    // Send our plugin file.
    if ( is_singular() && $post_type === get_post_type( $GLOBALS['post'] ) )
    {
        // return plugin file
        return dirname( __FILE__ ) . "/single-$post_type.php";
    }

    // Not our post type single view.
    return $template;
}