do_action('register_form'); doesn’t echo the form. It runs inside the form. You can use it to add information or maybe alter some values… maybe. So the answer is that you can’t insert the registration form with that mechanism. And while there are function in wp-login.php, the form isn’t one of them.
What you probably want is wp_loginout.
function loginout_sc_wpse_107419() {
wp_loginout();
}
add_shortcode('loginout','loginout_sc_wpse_107419');
The form itself is not part of any function that you can reuse, so far as I know. The code for it is here, though. You will have to create that form yourself, using that code as a model.