Here’s the code for what you’re trying to do (accessing post custom value):
<?php
global $post;
$prev_post = get_adjacent_post();
$next_post = get_adjacent_post( false, '', false );
$prev_post_id = $prev_post->ID;
$next_post_id = $next_post->ID;
// this should, according to your code above, be the http://example.com/your/img.jpg string
$prev_img_path = get_post_custom_values( 'your_key_name', $prev_post_id );
$next_img_path = get_post_custom_values( 'your_key_name', $next_post_id );
$prev_def_path = get_bloginfo('stylesheet_directory').'/default_next_img_inside_theme_folder.jpg';
$next_def_path = get_bloginfo('stylesheet_directory').'/default_prev_img_inside_theme_folder.jpg';
echo '<img src="'.!empty($prev_img_path) ? $prev_img_path : $prev_def_path.'" alt="Permalink to previous post" />';
echo '<img src="'.!empty($next_def_path) ? $next_def_path : $next_img_path.'" alt="Permalink to next post" />';
?>