Just to answer the direct question of where you might hook in to add to your nav menu, this technique is all over the web, mainly in the context of appending a search box to the nav.
If the theme location for your menu is main-nav then this will add an extra list item to the menu list items.
add_filter('wp_nav_menu_items', 'add_signup_form', 10, 2);
function add_signup_form($items, $args) {
if( $args->theme_location == 'main-nav' )
$items .= '<li><form>Signup form in here</form></li>';
return $items;
}
As I said in my comments above, I don’t think this is very semantic HTML and I guess the popularity around the web of this technique is due to that hook being almost the only one when nav menus were introduced.
And if you’re in a position to edit theme files to put in this hooked function, then you may as well edit the header file(s) and put in more meaningful HTML to achieve what you want.
You can only really append rather than insert using this hook, but again I’d feel that inserting a form into the navigation was unsemantic myself.