How do I remove featured images and captions from a wordpress RSS feed?

Have a look at http://codex.wordpress.org/Customizing_Feeds and try using a template for your feed.

There are a variety of ways to customize your feeds, through the use
of third-party software you install and add-on to your WordPress site,
or by manually changing the feed templates to meet your needs.

Find a tutorial here http://gregrickaby.com/custom-rss-template-wordpress/. There registering and creating a RSS template is explained in two steps. To begin you’ld maybe simply copy and paste the offered code and customize it to your needs afterwards.

In that example are the following lines you simply have to delete to no longer display the image in your feed:

<image>
    <url><?php echo esc_url( $postimage ); ?>"/></url>
</image>

---

Maybe Feed Template Customize might help you as well as this plugin modifies RSS feeds and ATOM feeds as you want.