Get all comments associated with a specific page ID (comment_post_ID)

Based on developer documents of get_comments functions, it is using args like WP_Comment_Query::__construct Method which is accepts post_id in the argument’s array. So, the code will be like:

foreach($assignments as $assignment) {

    echo $assignment->post_title;
                                
    $args = array(
        'number' => 0,
        'status' => 'approve',
        // shows all comments, but it shouldn't
        'post_id' => $assignment->ID
    );
                                
    $comments = get_comments( $args );
                                
    if ( $comments ) {
        foreach ( $comments as $comment )   {
            echo '<li>';
            echo $comment->comment_content;
            echo '</li>';
        }
    }

}

Links:

https://developer.wordpress.org/reference/functions/get_comments/
https://developer.wordpress.org/reference/classes/wp_comment_query/__construct/