Is it faster to query records using $wpdb instead of Wp_Query?

Yes, custom SQL queries are in essence faster than a custom instance of WP_Query. The disadvantage of a custom SQL query is that you are loosing a lot, like object cache, filters and actions, etc, etc, and it is also discouraged to use custom SQL if WordPress already offers that specific functionality.

WP_Query does have the option however to only query post ID’s (which is really fast and very lean) by means of adding and using the fields parameter in your query arguments. You can also just use get_posts which uses WP_Query but does not return the post object but just the array of posts

So, you can basically do the following

$args = [
    'nopaging' => true,
    'fields' => 'ids' // Just get post ID's
];
$q = get_posts( $args );
var_dump( $q );