If newest post of category is newest post in general, skip first post of category

So the easiest way to do this would be to store the ID of the first post (1) then in each of your category loops you can use the post__not_in property like so:

// inside the first loop at the top.
$latest_post_id = get_the_ID();

// WP_Query for fetching each category
$category_query = new WP_Query( [
  // other parameters
  'post__not_in' => [ $latest_post_id ],
] );

Now to exclude a category in WP_Query you can use category__not_in which takes an array of category ID’s. It’s definitely worth checking out the wordpress codex for WP_Query