it doesn’t return false because the menu exists in every case, you’re just unsetting all of the menu items. you’re not setting an items_wrap arg, so that defaults to the ul wrapper. maybe try removing it by setting it explicitly to just the child items:
'items_wrap' => '%3$s'
note, you’ll then have to manually wrap your menu in ul tags in that case.