How do I turn off 301 redirecting posts (not canonical)?

This appears to have to do with redirect_guess_404_permalink() called on line 96 of wp-includes/canonical.php. Just to test, I added a return false; to the first line of the redirect_guess_404_permalink() function, and that seemed to stop this odd behavior. I’m poking around a bit, but so far I don’t see a good way to fix this without editing that core WordPress file (which I am personally opposed to doing in a production environment, since it makes core updates more difficult and accident prone). I wish there was a good filter/action hook to use in redirect_guess_404_permalink to shortcut this behavior. I’ll keep poking a bit and update this answer if I find a good solution.

EDIT

I may have found a fix, which I tested briefly and worked.

Edit (again) Added some logic (which replicates the checks done in canonical.php to perform the redirect) to check for certain query parameters. Not as well tested as the last edit, so let me know how it works. If not 100% working, it should at least get you in the right direction (and check canonical.php).

add_action('template_redirect', 'remove_404_redirect', 1);
function remove_404_redirect(){
  if (is_404()){
    $id = max(get_query_var('p'), get_query_var('page_id'), get_query_var('attachment_id'));
    $redirect_url = false;
    if ($id && $redirect_post = get_post($id)) {
      $post_type_obj = get_post_type_object($redirect_post->post_type);
      if ($post_type_obj->public)
        $redirect_url = get_permalink($redirect_post);
    }
    if (!$redirect_url)
      remove_filter('template_redirect', 'redirect_canonical');
  }
}

This will work because the undesired redirects are only occurring if the page is initially a 404, so we just check for 404 and, if it is, remove the redirecting filter. YAY!