If the custom field will be always empty when there is no pdf file, you can check if it is not empty before displaying the link:
<?php
$pdf_name = get_post_meta($post->ID, 'pdf_name', true);
if($pdf_name) {
?>
<a class="pdfDownload" href="<?php bloginfo('url'); ?>/wp-content/uploads/<?php echo $pdf_name; ?>">PDF Download</a>
<?php
}
?>
But, if there is a possibility that the custom field can hold a value even if there is no actual pdf file attached, then you can check if the pdf file actually exists by its path before displaying the download URL:
<?php
$pdf_name = get_post_meta($post->ID, 'pdf_name', true);
$uploads_dir = wp_upload_dir();
$pdf_path = $uploads_dir['basedir'] . "https://wordpress.stackexchange.com/" . $pdf_name;
if(is_file($pdf_path)) { ?>
<a class="pdfDownload" href="<?php bloginfo('url'); ?>/wp-content/uploads/<?php echo $pdf_name; ?>">PDF Download</a>
<?php
}
?>