How can I insert a shortcode in the title tag of another?

No, you can’t do that, as the title argument is passed as a string.

Given the information you provided, I suppose that your [dt_fancy_title] shortcode does not actually execute shortcodes.

The syntax for this function should be:

function dt_fancy_title_callback( $params, $content = null ) {

    // extract the title argument from the params, setting default value
    extract( shortcode_atts( array(
        'title' => 'default title'
    ), $params ) );

    // execute your function, whatever it does

    // call for shortcodes to be executed within your shortcode
    $content = do_shortcode( $content );

    // return the content
    return $content;

}
add_shortcode( 'dt_fancy_title', 'dt_fancy_title_callback' );

If you do not call the do_shortcode() in your callback function, the shortcode texts inside will be interpreted as a string.