Retrieving category’s color with get_term_meta

From your question, it seems that you want to fetch the meta for the post. In which case you would want get_post_meta which would look something like this:

echo get_post_meta( get_the_ID(), 'color', true );

Change the second value (‘color’) to whatever you have used for the meta key – I took a wild guess.