Add a link back to site on ‘edit profile’ page for users?

Put this code in the functions.php file.

add_action( 'show_user_profile', 'my_show_extra_profile_fields' );
add_action( 'edit_user_profile', 'my_show_extra_profile_fields' );

function my_show_extra_profile_fields( $user ) { ?>
    <h3>Go back to home</h3>
    <table class="form-table">
        <tr>
            <td>
                <a href="https://wordpress.stackexchange.com/questions/56172/<?php echo home_url(); ?>" title="Go back">Home</a>
            </td>
        </tr>
    </table>
<?php }

Check this tutorial for more detail.

You can put any mark up you want, I just modified the example and have put a link to go to the homepage.
Hope this helps!