Ok I found a solution by following the recommandation of s_ha_dum to use pre_gets_posts.

All the following code goes to functions.php

1 Create a new rewrite rule

function rewrite_clean()
{
  // my CPTs
  $types = array('projects', 'works');
  foreach($types as $type)
  {
    add_rewrite_rule('^'.$type.'/([^/]*)/?$', 'index.php?post_type=".$type."&term=$matches[1]','top');
  }
}

add_action('init', 'rewrite_clean');

transform the following format

?post_type=projects&term=architecture

to the following format

projects/architecture

2 Overwrite the post query

// check the vars in url and if there is term then only display from this term
function add_cat_cpt_filter($query)
{
  if($query->is_archive() && get_query_var( 'term' )  && get_query_var( 'post_type' ))
  {
    // get the id of the category by using the slug
    $id = get_category_by_slug(get_query_var( 'term' ))->term_id;
    if($id){
      // add a filter to the query
      $query->set('cat', $id);
    }
    else{
      // redirect if the term doesn't exist
      wp_redirect( "https://wordpress.stackexchange.com/".get_query_var( 'post_type' ));
      exit;
    }
    return;
  }
}
add_action( 'pre_get_posts', 'add_cat_cpt_filter' );

The code above will filter the current archive page with the term used in the URL.

Gotcha

Then I got an issue due to Custom Post Type Permalinks plugin, it kept displaying a category template while I wanted to keep my CPT archive template and just add a filter on it.

The solution is simply to disable and reenable the plugin, it did the trick for me.