<br> tags are not the solution to the problem that you were hoping they would be. Menu labels were never intended to contain arbitrary HTML. HTML does slip through in places, but this appears to be a bug and is inconsistent.
For Post Types
No. You can’t.
$menu[ $ptype_menu_position ] = array( esc_attr( $ptype_obj->labels->menu_name ), $ptype_obj->cap->edit_posts, $ptype_file, '', $menu_class, $ptype_menu_id, $menu_icon );
Changing this would require changes to WordPress itself, and you’re likely to face heavy resistance.
You might be able to use CSS to adjust how overflow occurs, but this will depend on the exact label you used, its length, and the types of characters inside it.
Note that adding a <br> to the label would also add it everywhere else that label occurs, causing problems. For example the labels get used in dropdown UI’s and as headings.
Oddly enough, the sub-menu labels don’t get escaped, so it is possible to add <br> tags to sub-menus, although it will lead to unintended problems:
For General Admin Menu Pages
You can add <br> tags by using them when adding the menu, there is no escaping that removes them. This is very likely a bug.
add_menu_page(
'Custom Menu Title',
'custom menu<br>🤔🤔<br>test<br><img width="120" src="https://one.wordpress.test/wp-content/uploads/2021/04/Screenshot-2021-04-08-at-20.35.02-1024x564.png" />',
'manage_options',
...
So What Can I Do?
Ignore <br> tags and recognise them as the X Y Problem they were all along. You should have asked how to solve your problem, not how to implement your solution.
By sharing code, an obvious solution appears, use the non-breaking hyphen instead of a normal hyphen:
‑
This may not solve the problem for all cases though, after all super-long-hyphenated-text will still overflow if given no other opportunities. Fundamentally the label you’ve chosen is too long. Shortening it is the only reliable and futureproof solution.