Should the actual /category/ directory be 404? Is that normal WP behaviour

Yes, it’s normal behavior. WordPress uses a set of rewrite rules to process requests. Here are the rules that will match requests related to categories:

[category/(.+?)/feed/(feed|rdf|rss|rss2|atom)/?$] => index.php?category_name=$matches[1]&feed=$matches[2]
[category/(.+?)/(feed|rdf|rss|rss2|atom)/?$] => index.php?category_name=$matches[1]&feed=$matches[2]
[category/(.+?)/page/?([0-9]{1,})/?$] => index.php?category_name=$matches[1]&paged=$matches[2]
[category/(.+?)/?$] => index.php?category_name=$matches[1]

As you can see, all of them require, that the request is longer than /category/.

And it makes sense.

  • category/(.+?)/feed/(feed|rdf|rss|rss2|atom)/ – will match the feed for category
  • [category/(.+?)/(feed|rdf|rss|rss2|atom)/ – is a shorter URL for feed
  • category/(.+?)/page/?([0-9]{1,})/ – is support for pagination
  • category/(.+?)/ – is just category listing (first page)

So all of the URLs above are related to given term in this taxonomy and you know which term, because it is defined in URL.

On the other hand, what should be displayed when you go to /category/? No term is defined, so you can’t select any one of them. So should it show you all posts on your site? (Blog index already does it).

Sometimes it makes sense to show the list of categories on such URL, but it isn’t a common practice.

You can always add your custom rewrite rule to process such requests.