WordPress taxonomy and archive custom class

Add another layer of condition to your is_archive() test, so that your archive class is only output when the other two are NOT true:

function fotogruf_layout_class(){  
    if ( is_search() ) {
        echo $search_layout_class;
    }

    if ( (is_post_type_archive('gallery') ) || ( taxonomy_exists('gallery_category') ) ){
        echo $gallery_layout_class;
    }

    if ( is_archive() ) {
        if ( ( !is_post_type_archive('gallery') ) || ( !taxonomy_exists('gallery_category') ) ) {
            echo $archive_layout_class;
        }
    }       
}

WP’s built in body classes might do this for you automatically if implemented. Maybe not to the level of detail you are looking for, however. https://developer.wordpress.org/reference/functions/body_class/