I want to add a custom field in the rss title of each post, but don’t want it to be shown on the site

I’ve never actually tried this, but I think the following should work:

function wpse_96096_append_author( $title, $post_id ) {
    if( is_feed() ) {
        $author = get_post_meta( $post_id, '{YOUR CUSTOM FIELD KEY}', true );
        $title = $title . ' by ' . $author;
    }
    return $title;
}
add_filter( 'the_title', 'wpse_96096_append_author', 10, 2 );

What this does is add a filter to the title field that is only applied when is_feed() returns true. When it does, it queries the custom field and tacks it on to the title.