FYI, you should never hack the WordPress core files. Instead, WordPress provides the handy rss2_item
action hook. All we need to do is check if we’re on a category feed or not:
function wpse_99336_category_feed_item() {
if ( is_category() && get_the_post_thumbnail() ) {
printf ( '<featured-item>%s</featured-item>',
wp_get_attachment_url( get_post_thumbnail_id() )
);
}
}
add_action( 'rss2_item', 'wpse_99336_category_feed_item' );