Change sidebar headers from H1?

The tags used for the headings would be set wherever the sidebars are registered. Look for register_sidebar() in the theme’s functions.php file (or a file that has been included into the functions file).

It will look like this (example from here):

register_sidebar( array(
    'name'          => __( 'Main Sidebar', 'textdomain' ),
    'id'            => 'sidebar-1',
    'description'   => __( 'Widgets in this area will be shown on all posts and pages.', 'textdomain' ),
    'before_widget' => '<li id="%1$s" class="widget %2$s">',
    'after_widget'  => '</li>',
    'before_title'  => '<h2 class="widgettitle">',
    'after_title'   => '</h2>',
) );

Just change the tag in before_title and after_title.

For what it’s worth, and just in my opinion, the ‘only one h1 tag’ thing is just a superstition of so-called SEO experts and there’s probably nothing wrong with having h1 tags in the sidebar. What’s more important is the hierarchy of the headings. If the headings in the sidebar don’t make sense as subheadings of the main h1, then they shouldn’t be marked up as such. If you use the W3C Markup Validator with “Show Outline” enabled you can see a heading-level outline of the page. That outline should make logical sense, and sometimes more than one h1 is the way to achieve that.