I’m currently working on a theme framework and one of it’s classes is a custom category template class which does just what you are asking for: it adds a select field to the category edit screen that lets you select a template, just like with pages, and i just published it as a plugin
and if you are not up with using a plugin then simply take a look at the plugins code which is a single class that you can even use from your functions.php file.
Keep up the excellent work, I read few content on this web site and I believe that your blog is rattling interesting and has got circles of good info .