Including file or library from other plugin

In your plugin add a custom action to let other plugins start after your basic code has done the work:

// load basic classes
do_action( 'my_library_loaded', plugin_dir_path( __FILE__ ) );

Other plugins can start their work now like this:

add_action( 'my_library_loaded', 'other_plugin_init_handler' );

They will never do anything if your base plugin is not active.

The other plugin’s start function gets the correct path now as parameter:

function other_plugin_init_handler( $base_path )
{
    require_once $base_path . 'classes/Template_Handler.php' );

    $template = new Template_Handler;
}

You could also offer a custom class load function in the base plugin. The basic idea here is: Do not let other plugins guess a path.