Customize all image captions

Depending on whether the captions are using the same CSS class or HTML tag, you can add a :before pseudo element to the CSS. This will add an image before every <caption> tag on your website.

caption:before {
    content: '';
    display: inline-block;
    background: url('your-image-url-here');
    background-repeat: no-repeat;
    background-size: contain;
    background-position: center center;
    height: 15px;
    width: 15px;
}

Alternatively, if you wanted to use the WordPress glyph icons, try the following:

caption:before {
    content: '\f129';
    font-family: dashicons;
}

And don’t forget to enqueue the dashicons in your functions.php file:

function enable_frontend_dashicons() {
  wp_enqueue_style( 'dashicons' );
}
add_action( 'wp_enqueue_scripts', 'enable_frontend_dashicons' );