How do I modify the tag on my Category Archive page?

The default WordPress themes such as Twenty Sixteen and Twenty Twenty-One are using the automatic <title> tag feature (or the title-tag theme support) introduced in WordPress 4.1, so with that feature, if you want to modify the title, you would want to use the document_title_parts hook:

add_filter( 'document_title_parts', 'my_document_title_parts' );
function my_document_title_parts( $title ) { // $title is an *array*
    if ( is_category() ) {
        $title['title'] = 'Posts in ' . single_cat_title( '', false );
    }

    return $title;
}

UPDATE

Actually, looking at the code (or ugly hack) in the edited question, I think my sample code above did not work because the is_category() returned a false and it’s probably because the “Weekly Updates” is a Page (post of the page type) and not actually a default category archive at /category/<category slug>, e.g. example.com/category/uncategorized.

So let me put it this way: For themes like Twenty Sixteen which support the title-tag, you should use the document_title_parts hook to filter the title tag value, i.e. <title>THIS PART</title>. But as for the actual code which modifies the title, it will be up to you on how should you write your code.

Nonetheless, try this (and be sure to remove the ugly hack in your code):

add_filter( 'document_title_parts', 'my_document_title_parts' );
function my_document_title_parts( $title ) {
    if ( false !== strpos( $title['title'], 'Weekly Updates' ) ) {
        $title['title'] = 'Weekly Updates';
    }

    return $title;
}
/* NOTE: Try using a greater number as the callback priority, if the default (10)
 * doesn't work - example with 9999 as the priority:
add_filter( 'document_title_parts', 'my_document_title_parts', 9999 );
*/

And note that the hook has one parameter which is $title and it’s an array that consists of (but not all items present on all pages):

• ‘title’
  (string) Title of the viewed page.
• ‘page’
  (string) Optional. Page number if paginated.
• ‘tagline’
  (string) Optional. Site description when on home page.
• ‘site’
  (string) Optional. Site title when not on home page.

Also, if you want to change the title separator (like -), then you can use the document_title_separator filter.