A component in the Customizer is just a way for core to allow for plugins to determine whether or not the functionality is loaded. Functionality in the Customizer involves much more than just adding a panel. It also involves logic for previewing, enqueuing scripts, and so on. So there are components for widgets and nav menus in order to provide a way for a plugin to turn off all of a component’s functionality, without having to know all of the hooks that a component integrates with. Read more on #33552 for why the filter was introduced. Both widgets and nav menus have panels, but they also have classes (WP_Customize_Widgets and WP_Customize_Nav_Menus respectively) which manage the registration of the panels and all of the other things the functionalities need. You can certainly create your own custom classes to organize your Customizer code in the same way. The Customize Posts plugin does this by hooking into the same filter to give plugins the opportunity to turn the functionality of if needed: https://github.com/xwp/wp-customize-posts/blob/9550566/php/class-customize-posts-plugin.php#L70-L71
