How to remove tag base from permalinks

It’s probably there to help visually and programmatically distinguish tag URLs from pages; I’m not sure, I’m not a core developer of WordPress.

But http://wordpress.org/extend/plugins/wp-no-tag-base/ and http://wordpress.org/extend/plugins/wp-no-category-base/ work well and do not make permanent changes to files or GUIDs, so they can be deactivated with no ill effects.

Use the plugins as “regular” plugins, or as mu-plugins http://codex.wordpress.org/Must_Use_Plugins