Is there a way to define a default caption to all uploaded images

There’s really no documentation for it yet, but you’ll probably be able to do it hooking to the attachment_fields_to_save filter and inserting the default caption there.

From the Codex:

attachment_fields_to_save
applied to fields associated with an
attachment prior to saving them in the database. Called in the
media_upload_form_handler function. Filter function arguments: an
array of post attributes, an array of attachment fields including the
changes submitted from the form

It is defined on wp-admin/includes/media.php:

// TESTED :)
function wpse300512_image_attachment_fields_to_save($post, $attachment) {
if ( substr($post['post_mime_type'], 0, 5) == 'image' ) {
    if ( '' === trim( $post['post_title'] ) ) {
        $post['post_title'] = preg_replace('/\.\w+$/', '', basename($post['guid']));
        $post['errors']['post_title']['errors'][] = __('Empty Title filled from filename.');
    }
    // captions are saved as the post_excerpt, so we check for it before overwriting
    if ( '' === trim( $post['post_excerpt'] ) ) {
        $post['post_excerpt'] = 'default caption';
    }
}

return $post;
}

add_filter('attachment_fields_to_save', 'wpse300512_image_attachment_fields_to_save', 10, 2);

UPDATE:
I managed to test it, and it works as is. Just drop it on your functions.php 🙂