Is it possible to slow down site with a bulky child theme?

Yes. There is a definitely a point where you just want to create a new theme.

Unless you are writing more than a thousand lines of code, I would not be worried about performance problems with your child theme, if the parent theme is well-constructed.

Adding conditional statements to test if scripts and styles are needed for certain pages will improve site response, not reduce it. The amount of processing time for the server to test three PHP if statements is negligible.