Remove Taxonomy Slug when No Taxonomy is Assigned to Custom Post Type?

With helpful tips from Milo, I was able to remove the taxonomy slug when no taxonomy is assigned to the custom post type by adding the following code to the updated functions above:

add_filter( 'request', 'project_request_filter' );
function project_request_filter( $request ){
if( array_key_exists( 'project' , $request )
    && ! get_term_by( 'slug', $request['project'], 'project' ) ){
        $request['set'] = $request['project'];
        $request['name'] = $request['project'];
        $request['post_type'] = 'set';
        unset( $request['project'] );
}
return $request;
}