Update

As the Answer wasn’t best practice, I replace it with a link to my simple plugin that even does more than the OP needed. You can take it as reference on how to (re)move different things in the admin UI.

It’s freely available Open Source Software. I encourage you to fork it.

Functionality

The Plugin removes:

• All Dashboard Widgets except for “Incoming Links”
• All built in taxonomies and leaves only the “nav-menu” taxonomy
• The built in post type “Post”
• Changes the option “show on front” to “pages”
• Changes the initial posts query to search for the page with the lowest ID
• All admin bar items and leaves only the site name that can be used to access the public view
• Admin Menu items and submenu items: Posts, Links, Comments
• The admin menu items for Plugin & Theme Editor
• All Settings admin menu items and replaces them with the “All Settings” Page and makes it only accessible for ‘manage_options’ capability
• Removes the “capitalPdangit” filter from content, title and comment text, in case you add those filter in CPT related stuff

Detailed Description & Download

WP Strip Naked