Customizing the output of the archive and category widget without altering the original behavior of the widget

No, there are no filters for doing this. You would need to create your own version of the widgets in order to achieve this.

You can find the default WordPress widgets in the following folder: /wp-includes/widgets in your WordPress installation.

Simply copy the code from within the respective widget .php files in to your theme or plugin:

class-wp-widget-archives.php

and

class-wp-widget-categories.php

Then just edit the widget code to your liking.

Finally, you’ll need to register the widgets using the register_widget() function as per the WordPress Codex page – https://codex.wordpress.org/Function_Reference/register_widget. Which should look something like this:

function myplugin_register_widgets() {
    register_widget( 'CustomArchiveWidget' );
    register_widget( 'CustomCategoriesWidget' );
}

add_action( 'widgets_init', 'myplugin_register_widgets' );