How to display custom post type ordered by a custom field date

The correct array key to use is meta_key, not key. You can only use key as part of a meta_query subarray, where the ‘meta’ part is implied.

This is all described in the codex.

Try

<?php $activities = new WP_Query( array(
    'post_type' => 'timeline',
    'posts_per_page' => -1,
    'meta_key'      => 'activity-date',
    'orderby'       => 'meta_value_num',
    'order'         => 'ASC'                            
)); ?>