Display site language setting in source code

Is it possible to display <!-- plugin version 123 (Fr) --> if the
language is French?

Yes, it is possible and you can use get_bloginfo( 'language' ) which returns a language tag like en-US for English (US).

So if you just want to retrieve the first 2-or-3 character code (e.g. en for en-US and en-UK) of the language tag, you can do:

list ( $lang ) = explode( '-', get_bloginfo( 'language' ) );
echo '<!-- plugin version 123 (' . ucfirst( $lang ) . ') -->';