This all depends on your theme. Most themes includes a 404.php
template to display a “well this is embarrassing” page or an index of previous posts.
For example:
- My consulting site shows an index: http://jumping-duck.com/does-not-exist
- One of my client sites shows a basic “404 not found” error: http://votebode.com/does-not-exist
They’re both running the latest version of WordPress (3.3.2 as of this answer). Both sites define their own 404 behavior through a themed 404.php
template.