WordPress can only use one Theme at a time. You cannot (and, really, should not) change Themes on-the-fly. (There are edge cases; search the Plugin repository for Theme Switcher Plugins for implementation examples.)
That said, that doesn’t mean that a single Theme can’t output different styles based on the current context; and, really, that’s what you’re after here.
The first step is understanding the Template Hierarchy; in your case, you’re primarily interested in Category templates. The primary takeaway here is that you can use specific template files for specific category contexts; i.e. you can define a category-foobar.php template file to output the Foobar category archive index.
The second step is understanding the body_class() template tag, and the context-specific CSS classes that this tag adds to the HTML <body> tag. The primary takeaway here is that WordPress will add category-specific CSS classes to the HTML <body> tag for specific category contexts; i.e. WordPress will add category and category-foobar as classes on the Foobar category archive index, which you can target via CSS with e.g. body.category-foobar.
The rest becomes a matter of HTML and CSS, which is best left as an exercise for the reader.