What is “sermon date”, is it the custom field? If so, your meta_key is the name of your custom field used for that date. If not, how are you getting the sermon date?
Might be something like this:
$args = array(
'post_type' => 'sermons',
'meta_key' => 'sermon_date',
'orderby' => 'meta_value',
'meta_type' => 'DATE'
);
$query = new WP_Query( $args );