How to tranform Headlines into linkable permalinks?

Analysed situation

When you take a look at your (original) example “GitHub” and it’s source code, you see the following:

<a name="updating-submodules" class="anchor" href="#updating-submodules">
    <span class="octicon octicon-link"></span>
</a>

Prerequisites

Basically this is possible (even pretty easy), but you’ll have to have theme templates that properly use WordPress Templates Tags. The functions that are needed are the_permalink() or the underlying get_the_permalink() and an action/callback on that filter.

Styles/Icons

First off, WordPress registers dashicons per default, so they are ready for enqueuing. So we simply need:

add_action( 'wp_enqueue_scripts', 'wpseDashiconsFrontEnd' );
function wpseDashiconsFrontEnd()
{
    wp_enqueue_style( 'dashicons' );
}

To use them, we pass its CSS class dashicons-[name] to any element where we are going to use it on. Take a look at the Dashicons site, click icon and look at the name for it on top.

Add the anchor link:

Now when WordPress calls the permalink to a post, it normally looks close to the following:

the_title(
    sprintf(
        '<h2><a href="https://wordpress.stackexchange.com/questions/136624/%s" title="https://wordpress.stackexchange.com/questions/136624/%s">',
        get_the_permalink(),
        get_the_title()
    ),
    '</a></h2>',
);

Then we want to transform it to something like the following:

the_title(
    sprintf(
        '<h3><a href="#%1$s" name="%1$s" class="anchor"><span class="anochor-icon dashicons-admin-links"></span>',
        get_the_title()
    ),
    '</a></h3>',
);

Obviously you want to have .anchor span with display: none; per default in your main stylesheet and only show it on h3:hover .anchor .anchor-icon with display: inline-block;.

Additional info

If this is not about a list of headlines for posts, pages or any other custom post type, then you’ll need an action on the content filter that makes a DOMDocument::XPath search to add the needed MarkUp.

You could as well just write a 10 line custom plugin that hooks into get_the_title() and adds the needed output.