Adding an HTML wrapper to the gallery can be done using the post_gallery filter. Here’s a fully commented example.

add_filter( 'post_gallery', 'gallery_custom', 10, 3 );
/**
 * Filters the default gallery shortcode output.
 *
 * If the filtered output isn't empty, it will be used instead of generating
 * the default gallery template.
 *
 * @see gallery_shortcode()
 *
 * @param string $output   The gallery output. Default empty.
 * @param array  $attr     Attributes of the gallery shortcode.
 * @param int    $instance Unique numeric ID of this gallery shortcode instance.
 */
function gallery_custom( $output, $attr, $instance ) {
    // Remove the filter to prevent infinite loop.
    remove_filter( 'post_gallery', 'gallery_custom', 10, 3 );

    // Add opening wrapper.
    $return = '<div class="mydiv">';

    // Generate the standard gallery output.
    $return .= gallery_shortcode( $attr );

    // Add closing wrapper.
    $return .= '</div>';

    // Add the filter for subsequent calls to gallery shortcode.
    add_filter( 'post_gallery', 'gallery_custom', 10, 3 );

    // Finally, return the output.
    return $return;
}

• In gallery_custom(), it’s important to remove the post_gallery filter before calling gallery_shortcode() because otherwise we’d run into an infinite loop.

• Note that output needs to be concatenated to $return. In your original code, $return is continuously being overwritten because = is used instead of .= after the string has been initialized.

• The main output is generated using the standard gallery output function, gallery_shortcode( $attr ); Our filter will not be applied in this call because we’ve removed it at this point.

• After the gallery output has been concatenated to $return, we add the closing HTML tag and add our filter back so that it will be run the next time the gallery shortcode function is called.

• Finally we return the output.

Alternate solution: Replace shortcode function:

Here’s another approach to solving the problem. This time the default gallery output function, gallery_shortcode() is removed from the gallery shortcode string. Then, a replacement function, wpse_custom_gallery_shortcode() is wired up to the original gallery shortcode string.

// Replace the default  shortcode function with a custom function.
add_action( 'init', 'wpse_replace_gallery_shortcode' ); 
function wpse_replace_gallery_shortcode() {
    remove_shortcode( 'gallery', 'gallery_shortcode' );
    add_shortcode( 'gallery', 'wpse_custom_gallery_shortcode' );
}

// Customized gallery shortcode function.
// See gallery_shortcode() for documentation.
function wpse_custom_gallery_shortcode( $attr ) {
    $gallery = gallery_shortcode( $attr );

    if ( $gallery ) {
        return '<div class="mydiv">' . $gallery . '</div>';
    }
}