Add span in top menu item with nested itens

You can use the walker_nav_menu_start_el filter:

add_filter( 'walker_nav_menu_start_el', 'wpse_add_arrow',10,4);
function wpse_add_arrow( $item_output, $item, $depth, $args ){
    //Only add class to 'top level' items on the 'primary' menu.
    if('primary' == $args->theme_location && $depth ==0){
        $item_output .='<span class="arrow"></span>';
    }
    return $item_output;
}

This assumes that you’re using wp_nav_menu( array( 'theme_location' => 'primary') ); to display the menu.

Updated answer in response to comments. To add the span classes to only top-level elements which have children you need to use a custom walker:

class SH_Arrow_Walker_Nav_Menu extends Walker_Nav_Menu {
    function start_lvl(&$output, $depth, $args) {
        $indent = str_repeat("\t", $depth);
        if('primary' == $args->theme_location && $depth ==0){
            $output .='<span class="arrow"></span>';
        }
        $output .= "\n$indent<ul class=\"sub-menu\">\n";
    }
}

To use this you’ll need to set the walker argument in wp_nav_menu:

wp_nav_menu( array( 
     'theme_location' => 'primary', 
     'walker'=> new SH_Arrow_Walker_Nav_Menu()
 ) );