doing an ajax request always outputs 0

Your custom functions should always go in your theme’s functions.php file (if they are not within plugin territory). You should never try to modify any core files outside the wp-content folder (except wp-config.php) as they will be overridden by future updates.

Now, there are 2 ways to do what you want.

1 – Admin-ajax

This simple function will serve you nice to get a hello! as an output:

add_action('wp_ajax_my_ajax_handler', 'my_ajax_handler');
add_action('wp_ajax_nopriv_my_ajax_handler', 'my_ajax_handler');
function my_ajax_handler (){
    _e('Hello!','text-domain');
    exit();
}

Now, try and access the admin-ajax.php directly to see if you have an answer, by visiting wp-admin/admin-ajax.php?action=my_ajax_handler. This should work for both logged-in and non-logged-in users.

2 – REST-API

Even simpler. Take a look into this:

add_action( 'rest_api_init', function () {
    //Path to ajax function
    register_rest_route( 'Roman/v1', '/say_hello/', array(
            'methods' => 'GET', 
            'callback' => 'my_hello_function' 
    ) );
});

And a function to say hello:

function my_hello_function(){
    return __('Hello!','text-domain');
}

Now, if you send a request to http://example.com/wp-json/Roman/v1/say_hello/, you will get a nice welcome as JSON, like:

{"Hello!"}

Which you can use in your jQuery script.