Gallery image caption inside link

You should get the attachment differently.

Use wp_get_attachment_url for the file url
https://codex.wordpress.org/Function_Reference/wp_get_attachment_url

In your case it’s an image so I would suggest wp_get_attachment_image_src so you can define an image size.
https://developer.wordpress.org/reference/functions/wp_get_attachment_image_src/

This way you can greate your own “template” to echo the image link.

To get the image metadata you can simply use get_post( $id ) since an image is a post type aswell (attachment).

(Edit)
An example:

foreach ( $attachments as $id => $attachment ) {
    $large_image = wp_get_attachment_image_src( $id, 'full' );
    $small_image = wp_get_attachment_image_src( $id, 'medium' ); // Cange the size to anything

    $output .= "<{$itemtag} class="gallery-item">";
    $output .= "<{$icontag} class="gallery-icon">";
    $output .= "<a href="https://wordpress.stackexchange.com/questions/241916/{$large_image[0]}">";
    $output .= "<img src="{$small_image[0]}" alt="{$attachment->post_title}" />";
    if ( $captiontag && trim($attachment->post_excerpt) ) {
        $output .= "
            <{$captiontag} class="wp-caption-text gallery-caption">
            " . wptexturize($attachment->post_excerpt) . "
            </{$captiontag}>";
    }
    $output .= "</a>";
    $output .= "</{$icontag}>";
    $output .= "</{$itemtag}>";

    if ( $columns > 0 && ++$i % $columns == 0 )
        $output .= '<br style="clear: both" />';
}

error code: 523