yet another: custom post type with pagination not working in WordPress

I’m guessing you are confusing the query with your pagination logic. You are saying if get_query_var('paged') is false get get_query_var('page') but you are NOT on a static front page and then else kicks in and sets $args['paged'] = 1 so essentially always giving you the first set of posts in the loop.

I would try writing the query this way instead.

$paged = ( get_query_var('paged') ) ? get_query_var('paged') : 1; // If we have a 'paged' parameter, set $paged to that value, if not set it to 1

$args = array(
   'post_type' => 'post',
   'posts_per_page' => 6,
   'paged' => $paged, // Our current paged parameter set above.
   'order' => 'desc',
   'orderby' => 'date',
   'tax_query' => array(
       array(
           'taxonomy' => 'category',
           'field'    => 'slug',
           'terms'    => get_field('category_to_display'), // category from the Advanced Custom Field variable
       ),
   ),
);


$the_query = new WP_Query( $args );

also, I noticed you are setting next post to older post

next_posts_link( '« Older Posts' );

and previous post to newer post

previous_posts_link( 'Newer Posts »' );

wouldn’t you want that the other way around? But then it might just be me who’s not thinking straight on that one!