I’ve been looking in the DB to see where WP stores the modified dates for taxonomies but I’ve been unable to find the right table.
It doesn’t. There is no last modified date for terms, that data simply doesn’t exist in the tables in vanilla WP. If you want it, you’re going to have to track and store it yourself in code via hooks, preferably term meta.
when I call get_the_modified_date($term_id)
You can’t call that function on term IDs, it only works for post IDs, which gives us a clue to why you’re getting strange results. The documentation and that functions signature clearly state they want a post ID.
some of the taxonomy terms return the date but others return false.
Because any ID you pass in will be a post ID, passing in the term with ID 4, will give you the modified date for the post with ID 4. If you check again, saving terms in WP Admin has no bearing at all on those dates, due to the aforementioned fact that terms don’t have modified and publish dates.
Naturally, there are term IDs that don’t match posts, so you’ll get false. e.g. request the modified date and pass in 200000000, there won’t be a post with that ID, even if there is a term with that ID.