Setting theme dynamically is certainly possible, but not too convenient since it’s hardly “normal” mode of operation.
In my experience this works best if you stay away from higher level template/stylesheet hooks and instead filter underlying options. In that way you can trick WP into thinking that configured theme is different from what it is and use it.
For a code example there are quite a few theme switchers around. You can take a look at Toolbar Theme Switcher plugin’s code for one take on it (disclosure — I am its developer).