Adding a button to the format toolbar for custom styles

If you added a console.log in the file, it should have appeared.
For a better debugging experience, you can wrap your code inside an IIFE which needs wp to be sure it is loaded:

((wp) => {
  ... // your code
})(window.wp);

Also i’m not sure you can use JSX syntax here, your usage of RichTextToolbarButton should be instead:

const { createElement } = wp.element;

...

return createElement(
  RichTextToolbarButton, 
  {
    icon: "editor-underline",
    title: "Finno underline",
    onClick: onToggle,
    isActive
  }
);