add_filter() doesn’t work in loop

It doesn’t work in your template because the query has already happened before the template is loaded. If you want to add your filter only under certain conditions, you need to hook an earlier action, like pre_get_posts, and check if the query is for the home page:

function add_my_orderby_filter( $query ) {
    if ( $query->is_home ) {
        add_filter('posts_orderby', 'orderby_last_modified');
    }
}
add_action( 'pre_get_posts', 'add_my_orderby_filter' );

This would go in your functions.php along with your orderby_last_modified function.

See the WordPress Action Reference for the order actions are executed in a request.