How does register_post_type know how and which function to use from the add_action function?

You are confusing between function and hook. register_post_type() is a function, not a hook.

When you are using register_post_type( 'book', $args ), you are calling the register_post_type() function to create a Custom Post Type named book.

register_post_type() needs to be used on the init hook which is fired at the beginning of wordpress initialisation, that’s why you are using add_action( 'init', 'codex_book_init' ) and using register_post_type() there.