WordPress adding scaled images that don’t exist (1536×1536 and 2048×2048)

I found the culprit!

WordPress 5.3 introduced additional image sizes which can be found via /wp-includes/media.php.

Updating my function, like so, removed the extra sizes:

function remove_default_image_sizes( $sizes) {
    unset( $sizes['large']); // Added to remove 1024
    unset( $sizes['thumbnail']);
    unset( $sizes['medium']);
    unset( $sizes['medium_large']);
    unset( $sizes['1536x1536']);
    unset( $sizes['2048x2048']);
    return $sizes;
}
add_filter('intermediate_image_sizes_advanced', 'remove_default_image_sizes');