Disable Attachment Pages Completely

You can filter default rewrite rules and remove those for attachments:

function cleanup_default_rewrite_rules( $rules ) {
    foreach ( $rules as $regex => $query ) {
        if ( strpos( $regex, 'attachment' ) || strpos( $query, 'attachment' ) ) {
            unset( $rules[ $regex ] );
        }
    }

    return $rules;
}
add_filter( 'rewrite_rules_array', 'cleanup_default_rewrite_rules' );

Don’t forget to re-save your permalinks once. WordPress will generate new rules without anything related to attachments.

Now, the attachment page URL gives 404. You can also add that redirect to prevent the 404 page, it’s useless in this case. But I’m not sure how to catch the redirect – is_attachment() will not work if the rewrite rules are removed.

Update:

WordPress will still offer the attachment page pretty URLs in media library and media insertion dialog. You can filter this as well:

function cleanup_attachment_link( $link ) {
    return;
}
add_filter( 'attachment_link', 'cleanup_attachment_link' );

In this case, even when you insert your attachment into post ans select “Link to attachment page”, the image will be inserted without the link.