Add Password Generator on password protected page

It is not possible to modify the post submit meta boxes with a filter.

But you can do that with JavaScript. This script will generate 10-12 random alphanumeric string and put it in the password field (if it is empty) when you click the Password protected radio button.

$('#visibility-radio-password').click(function () {
    // If there is no password
    var $password = $('#post_password')
    if ( !$password.val() ) {
        $password.val(Math.random().toString(36).slice(2));
    }
});

Here you can learn how to correctly add JS scripts to admin pages – https://codex.wordpress.org/Plugin_API/Action_Reference/admin_enqueue_scripts