Adding this code to the top of the index.php file in your theme will return a 403 if the theme directory is accessed directly:
if ( ! defined( 'ABSPATH' ) ) {
header( 'HTTP/1.0 403 Forbidden' );
die();
}
However, you never see that code in any theme’s index.php file, because it’s unnecessary. I only include it so that this answer has an actual solution in it before I try to convince you you don’t actually need to solve anything:
it makes no sense to return a 500 error when this is not a server problem
There is a server problem if you try to access a PHP file, index.php, which is not supposed to be accessed directly.
There are dozens and dozens of files in WP that will throw an error if you try to do this, like these:
https://www.dailywritingtips.com/wp-content/themes/dailywritingtips/functions.php
https://www.dailywritingtips.com/wp-settings.php
The only reason you’re not getting crawl errors on those is because there’s no links to them anywhere and Google’s not trying to crawl them (don’t worry, links on this site are nofollow).
So your actual problem is Google attempting to index this URL, which would be happening because there’s a link to it somewhere that Google has crawled.
The solution to that is to find out why Google is crawling this URL to begin with and removing the link that’s responsible, but frankly, if you can’t find it, I wouldn’t spend any more time on it.
Google’s seeing an error response code on a URL you don’t even want indexed, and users shouldn’t be accessing anyway, so this isn’t actually causing any problems. Like you said, this happens on some pretty popular sites, even ones from WordPress VIP like TechCrunch, and they’re doing just fine.