Reject upload of wrong-sized images using the Media Uploader

In your handler, if you set ‘error’, the error message will be displayed and will cancel the upload

add_filter( 'wp_handle_upload_prefilter', 'custom_upload_filter' );

function custom_upload_filter( $file ) {
    $image_info   = getimagesize( $file['tmp_name'] );
    $image_width  = $image_info[0];
    $image_height = $image_info[1];

    if ( $image_with !== 800 || $image_height !== 600 ) {
        $file['error'] = __( 'Images must be sized exactly 800 * 600', 'your_textdomain' );
    }
    return $file;
}

If your user attempts to upload a different size, the message will be:

“thefile.png” has failed to upload due to an error  
Size must be exactly 800 * 600

Note that wp_handle_upload_prefilter comes very early in upload processing, so you may want to test if the file has been properly uploaded (from HTTP standpoint) and is an image before testing the size.

Ref: funtion wp_handle_upload() in the core file wp-admin/includes/file.php