Custom attribute for the title tag with wp_title()

Since all _wp_render_title_tag does is check for title-tag theme support and wrap in <title> tags, there is really no reason why your existing implementation “shall not pass”, since the proper implementation is already identical via:

<title itemprop="name"><?php echo wp_get_document_title(); ?></title>

when _wp_render_title_tag does:

echo '<title>' . wp_get_document_title() . '</title>' . "\n";

(since Theme Check is a guideline check, so what if it can’t tell that a standard actually has been followed, that should in theory not stop it from passing?)

But in any case, you can get around this and improve the existing implementation at the same time by adding a customization override filter… by unhooking the existing action (as suggested by @birgire) and (my addition) hooking a wrapping function that calls _wp_render_title_tag and applies the filter to it:

if (has_action('wp_head','_wp_render_title_tag') == 1) {
    remove_action('wp_head','_wp_render_title_tag',1);
    add_action('wp_head','custom_wp_render_title_tag_filtered',1);
}

function custom_wp_render_title_tag_filtered() {
    if (function_exists('_wp_render_title_tag')) {
        ob_start(); 
        _wp_render_title_tag(); 
        $titletag = ob_get_contents();
        ob_end_clean();
    } else {$titletag = '';}
    return apply_filters('wp_render_title_tag_filter',$titletag);
}

Since it’s better practice to have a filter available anyway……. Then you can add your own use case customizations easily by using the new filter:

add_filter('wp_render_title_tag_filter','custom_wp_render_title_tag');

function custom_wp_render_title_tag($titletag) {
    $titletag = str_replace('<title>','<title itemprop="name">',$titletag);
    return $titletag;
}

Of course it would be much cleaner if the core function was simply updated to:

function _wp_render_title_tag() {
    if ( ! current_theme_supports( 'title-tag' ) ) {
        return;
    }
    echo apply_filters( 'wp_render_title_tag' , '<title>' . wp_get_document_title() . '</title>' . "\n" );
}