Is it possible to place custom post type files inside a folder in theme directory?

You may want to look at the TwentySixteen theme, which does it this way:

In single.php:

get_template_part( 'template-parts/content', 'single' );

Several templates are grouped in the ‘template-parts’ folder. You could use a similar logic for your custom post type.

It seems that the technique comes from _underscores.