Create custom [sourcecode] shortcode, the right way?

You’re asking about X, but I’ll answer with Y. IMHO, there are too many exceptions to handle to grant another solution.

SyntaxHighlighter Evolved has the following Pro Tip (strong, uppercase, in the original):

TIP: Don’t use the Visual editor if you don’t want your code mangled. TinyMCE will “clean up” your HTML.

Not only that, it’s quite common to have plugins and themes messing around with the_content. Rough (and totally unscientific) measure: 10 occurrences within 50 plugins; 15 occurrences within 50 themes.

Another Pro Tip:

Proposed solution

1) To handle the code, I think it’s either a Custom Field/Box in the Edit Post screen or a bridge between the Post and a “private” Custom Post Type that will store the code. I prefer the latter. Steps:

• paste the embeddable code into a <textarea> rendered with edit_form_after_title
• use esc_html to display in the backend
• and '<pre class="pretty">'.htmlentities($code).</pre> in the frontend

2) To handle the syntax highlight, it’s up to the reader to choose its preferred method. Google Prettify is literally 2 lines of code and adding a class to the <pre> tag. Enqueue the script only when the shortcode is present.

Sample plugin

Custom post type

Post post type

The following is a skeleton and only outlines the logic. And here the links for some goodies:
_{Working skeleton at Gist
Full plugin at GitHub}

<?php
/**
 * Plugin Name: Skeleton for the plugin Snippets Shortcode
 * Plugin URI: https://gist.github.com/brasofilo/6804951
 */

/**
 * Based on Plugin Class Demo
 * https://gist.github.com/toscho/3804204
 */
add_action(
    'plugins_loaded',
    array ( B5F_Snippets_Shortcode::get_instance(), 'plugin_setup' )
);

/**
 * Main class
 *
 * Fires all classes
 * Handles save_post action for other classes
 */
class B5F_Snippets_Shortcode {
    protected static $instance = NULL;
    public $post_type="snippet";    
    public function plugin_setup() {
        new B5F_SS_Cpt();
        new B5F_SS_Posts_Pages_Metabox();
        new B5F_SS_Shortcode();
        add_action( 'save_post', array( $this, '_save_post' ), 10, 2 );
    }
} 

/**
 * Register Snippet post type and render Textarea Field after the Title
 *
 * CPT is hierarchical 
 * Custom Metabox for CPT is left empty, can be used as Language Selector
 * Uses save_post for CPT 
 */
class B5F_SS_Cpt {
    public function __construct() {
        add_action( 'init', array( $this, '_cpt' ) );
        add_action( 'edit_form_after_title', array( $this, 'input_text_area' ) );
    }
}

/**
 * Metabox to select the Snippet and to display the correspondent Shortcode
 *
 * Displayed in Posts and Pages post types
 * Add jQuery to listen to the Snippets Dropdown changes and update the sample Text Field
 * Uses save_post for other Post Types 
 */
class B5F_SS_Posts_Pages_Metabox {
    public function __construct() {
        add_action( 'add_meta_boxes', array( $this, '_meta_boxes' ) );
    }
}

/**
 * Snippets Shortcode
 *
 * First gets the Snippet Post ID, then its associated Meta Data, 
 * finally filters the string with http://php.net/manual/en/function.htmlentities.php 
 *
 * Uses Google Code Prettify
 * https://code.google.com/p/google-code-prettify/
 */
class B5F_SS_Shortcode {
    public function __construct() {
        add_shortcode( 'snippet', array( $this, '_shortcode' ) );
    }
}