Display time difference (6 hours ago) in a Soliloquy caption

In your filter function you should pass post ID to get_the_time() function, otherwise you will get the date from current post.

function wpse_relative_date( $date, $format, $post ) {
   return human_time_diff( get_the_time('U', $post), current_time( 'timestamp' ) ) . ' ago';
}
add_filter( 'get_the_date', 'wpse_relative_date', 15, 3 );

References:

• get_the_time() function
• get_the_date filter