Indeed, if your site has a static page on the front, the condition is_page will be true and WP will never get to the elseif.
If you want a different template for your front page, you should exclude it from the first condition so it will return false in that case. Like this:
if ((is_single() || is_page()) && !is_front_page())
Or you can switch the statements around: first test for home/front and if that fails for single/page.