How can I fetch all the dates from custom fields from various different custom post types and show / list them at one place in ascending order?

get_post() and get_post_meta() should work to collect the data and array_multisort() should be able to sort it into ascending order.

    // Query the database to get the posts from your post types

    $project_args = array( 'post_type' => 'projects', "numberposts" => -1 );
    $task_args = array( 'post_type' => 'tasks', "numberposts" => -1 );
    $material_args = array( 'post_type' => 'materials', "numberposts" => -1 );

    $projects = get_posts( $project_args );
    $tasks = get_posts( $task_args );
    $materials = get_posts( $material_args );

    $list_items = array();
    $timestamps = array();

    // Loop through each set of posts adding the data you want to an array and their timestamp to another array
    foreach($projects as $project){ 
        array_push( $list_items, array( 
            'name' => $project->post_title, 
            'date' => get_post_meta( $project->ID, 'custom_date'),
            )
        ); 
        array_push( $timestamps, strtotime( get_post_meta( $project->ID, 'custom_date') ) );    
        }

    foreach($tasks as $task){ 
        array_push( $list_items, array( 
            'name' => $task-> post_title, 
            'date' => get_post_meta( $task->ID, 'custom_date'),
            )
        ); 
        array_push( $timestamps, strtotime( get_post_meta( $task->ID, 'custom_date') ) );       
        }

    foreach($materials as $material){ 
        array_push( $list_items, array( 
            'name' => $material-> post_title, 
            'date' => get_post_meta( $material->ID, 'custom_date'), 
            )   
        );
        array_push( $timestamps, strtotime( get_post_meta( $material->ID, 'custom_date') ) );       
        }

    //Sort both arrays based on the timestamp   
    $array_multisort( $timestamps, $list_items );

    //Output as you like
    print_r( $list_items );

For multiple dates, just add additional meta fields to the foreach loops and be sure to assign the date you want to sort by to the $timestamps variable.

    foreach($materials as $material){ 
        array_push( $list_items, array( 
            'name' => $material-> post_title, 
            'sort_date' => get_post_meta( $material->ID, 'sort_date'),
            'date_1' => get_post_meta( $material->ID, 'date_1'),
            'date_2' => get_post_meta( $material->ID, 'date_2'),
            'date_3' => get_post_meta( $material->ID, 'date_3'), 
            )   
        );
        array_push( $timestamps, strtotime( get_post_meta( $material->ID, 'sort_date') ) );     
        }