WP Query with offset breaks wp_pagenavi or any pagination

I had the same issue.

Check out if this works for you:

<?php
  $paged = (get_query_var('paged')) ? get_query_var('paged') : 1;
  $per_page = 9;
  $default_offset = 4;

  if ($paged == 1) {
    $offset = $default_offset;
  } else {
    $offset = (($paged - 1) * $per_page) + $default_offset;
  }

  $args = array(
    'post_type' => 'post',
    'posts_per_page' => $per_page,
    'order'   => 'DESC',
    'offset' => $offset,
    'paged' => $paged
  );

  $loop = new WP_Query($args);

  while ($loop->have_posts()) : 
    $loop->the_post();
?>
  ... HTML
<?php 
  endwhile; 
  wp_pagenavi(array('query' => $loop)); wp_reset_postdata();
?>