although I can play around with the CSS
If you are not going to use CSS (or jQuery) to modify the position of these elements, you have to create your own login form: Customizing_the_Login_Form#Make_a_Custom_Login_Page.
There is no hook in wp-login.php
that allows to move these elements inside the form field.