How to modify the output of wp_terms_checklist when used within the built-in category metabox on edit posts?

Yes, as the documentation for wp_terms_checklist() stated, you can use the walker argument (which is part of the $args parameter for the wp_terms_checklist_args hook) to use a custom walker which modifies the output of the Category checklist generated by wp_terms_checklist(), e.g. the one in the “Categories” meta box on wp-admin/post.php.

Example where the custom walker is in a folder named includes:

add_filter( 'wp_terms_checklist_args', 'my_wp_terms_checklist_args', 10, 2 );
function my_wp_terms_checklist_args( $args, $post_id ) {
    require_once __DIR__ . '/includes/class-my-walker-category-checklist.php';

    $args['walker'] = new My_Walker_Category_Checklist;

    return $args;
}