Redirect feed to 404 page

There are three simple steps to get what you want:

1. Hook into 'template_redirect' to act before the feed is displayed.
2. If it is a feed send a 404 status header and
3. load the 404 template.

Here is a simple plugin that does it:

<?php # -*- coding: utf-8 -*-
/**
 * Plugin Name: T5 404 Feed
 * Description: Sends a 404 status code for all feeds and loads the <code>404.php</code> template.
 */

add_action( 'template_redirect', 't5_404_feed', 1 );
function t5_404_feed()
{
    if ( is_feed() )
    {
        status_header( '404' );
        locate_template( array ( '404.php', 'index.php ' ), TRUE, TRUE );
        exit;
    }
}

I am not so sure that 404 is the proper status code. 410 (Gone) may fit better.