Remove a class from a using Walker

You don’t need to use the Walker to remove classes from nav menu items. There is a filter, nav_menu_css_class which can be used to modify the classes on menu items.

Filters the CSS class(es) applied to a menu item’s list item element.

This, for example, would remove the classes from all menu items:

add_filter( 'nav_menu_css_class', '__return_empty_array' );

But you don’t want that, you only want to remove classes for menus that are using a specific walker. One of the arguments passed to callbacks for this filter is $args, which contains the arguments used when the menu is output. This will include which Walker was used. So in the callback to the function you could check the Walker and remove all classes if it’s yours:

function wpse_309490_nav_menu_css_class( $classes, $item, $args, $depth ) {
    if ( is_a( $args->walker, 'My_Walker_Nav_Menu' ) ) {
        $classes = [];
    }

    return $classes;
}
add_filter( 'nav_menu_css_class', 'wpse_309490_nav_menu_css_class', 10, 4 );

In your example markup though, it looks like your top-level items do have a class, and a custom one at that. In that case you can use the $depth argument to add a class for top-level items and no classes for other items:

function wpse_309490_nav_menu_css_class( $classes, $item, $args, $depth ) {
    if ( is_a( $args->walker, 'My_Walker_Nav_Menu' ) ) {
        if ( $depth === 0 ) {
            $classes = ['d-inline-block'];
        } else {
            $classes = [];
        }
    }

    return $classes;
}
add_filter( 'nav_menu_css_class', 'wpse_309490_nav_menu_css_class', 10, 4 );