{$taxonomy}_row_actions for all custom taxonomies

You could use the tag_row_actions hook (which was deprecated in WP v3.0.0, but then restored in WP v5.4.2) to target any taxonomies (Tags/post_tag, Categories/category, a_custom_taxonomy, etc.), so just replace the category_row_actions with tag_row_actions:

add_filter( 'tag_row_actions', 'category_row_actions', 10, 2 );

Or you could use get_taxonomies() with an early hook like admin_init (that runs after init), and then add your filter for the target taxonomies like so:

add_action( 'admin_init', function () {
    // This adds the filter for *all* taxonomies.
    foreach ( get_taxonomies() as $taxonomy ) {
        add_filter( "{$taxonomy}_row_actions", 'category_row_actions', 10, 2 );
    }

/*
    // This one adds the filter for *public* taxonomies only.
    foreach ( get_taxonomies( array( 'public' => true ) ) as $taxonomy ) {
        add_filter( "{$taxonomy}_row_actions", 'category_row_actions', 10, 2 );
    }
*/
} );