How to create this custom menu walker?

You need to set-up your own menu walker (link to WordPress Codex), and in there add custom start_el and end_el overrides. So for example your start_el and end_el may look something like this:

function start_el(&$output, $item, $depth=0, $args=array()) {
    $output .= "<div>" . esc_attr($item->label);
}

function end_el(&$output, $item, $depth=0, $args=array()) {
    $output .= "</div>\n";
}

This tutorial might be useful too:
https://code.tutsplus.com/tutorials/understanding-the-walker-class–wp-25401