So this may be hacky..

I mean, compared to overriding the default Featured Image panel/component in the Gutenberg block editor, which is not exactly hard, but this one is much simpler and basically just needs a few lines of custom coding. (read till the end to see what I mean)

And I’ve also tried & tested it working (with both the old/classic editor and the Gutenberg block editor) on WordPress 5.4.2 (the latest release as of writing) as well as WordPress 5.4.1.

So for what you’re trying to do, you can override wp.media.model.Query.prototype.sync() (defined in wp-includes/js/media-models.js) which is the function used by the default Featured Image modal for querying attachments via AJAX (admin-ajax.php?action=query-attachments).

Full (JS) Code

wp.media.model.Query.prototype.sync = function( method, model, options ) {
    var args, fallback;

    // Overload the read method so Attachment.fetch() functions correctly.
    if ( 'read' === method ) {
        options = options || {};
        options.context = this;
        options.data = _.extend( options.data || {}, {
            action:  'query-attachments',
            post_id: wp.media.model.settings.post.id
        });

        // Clone the args so manipulation is non-destructive.
        args = _.clone( this.args );

        // Determine which page to query.
        if ( -1 !== args.posts_per_page ) {
            args.paged = Math.round( this.length / args.posts_per_page ) + 1;
        }

        if ( wp.media.frame && 'featured-image' === wp.media.frame.options.state ) {
            // Here I'm allowing image and PDF files only.
            args.post_mime_type="image, application/pdf";
        }

        options.data.query = args;
        return wp.media.ajax( options );

    // Otherwise, fall back to `Backbone.sync()`.
    } else {
        /**
         * Call wp.media.model.Attachments.sync or Backbone.sync
         */
        fallback = Attachments.prototype.sync ? Attachments.prototype : Backbone;
        return fallback.sync.apply( this, arguments );
    }
};

But the only thing I added there is this conditional block:

if ( wp.media.frame && 'featured-image' === wp.media.frame.options.state ) {
    // Here I'm allowing image and PDF files only.
    args.post_mime_type="image, application/pdf";
}

Which checks if the currently opened modal is the featured image modal and if so, then we modify the post_mime_type argument which defines the MIME type(s) of the attachments being queried.

And that’s how you can use this approach to force the Featured Image modal to include files other than images.

Sample PHP Code

Update Jun 13^th (to other readers): Just to clarify why should we check for the media-models script, which is because we’re
overriding something in that script, so if the script is not being
loaded on the current page, then there’s no need to echo/enqueue our
custom script. 🙂

The following is the PHP code I used to echo the script in the footer on the post editing screen for the default post and page post types:

add_action( 'admin_print_footer_scripts', function () {
    $screen_ids = [ 'post', 'page' ];
    if ( in_array( get_current_screen()->id, $screen_ids ) &&
        wp_script_is( 'media-models' ) // check if media-models.js was enqueued
    ) :
        ?>
            <script>
                // See/copy the FULL code above and place it here..
            </script>
        <?php
    endif;
} );

And if you put the script in an external JS file, you should also ensure that the media-models script has been enqueued:

add_action( 'admin_enqueue_scripts', function () {
    $screen_ids = [ 'post', 'page' ];
    if ( in_array( get_current_screen()->id, $screen_ids ) &&
        wp_script_is( 'media-models' ) // check if media-models.js was enqueued
    ) {
        wp_enqueue_script( 'my-script', '/path/to/my-script.js', [], '', true );
    }
} );