The main difference is semantical. A page is something that stands by itself, while a post type in a categorization of content. This semantical difference results in different admin flow for both.
The only other difference is that pages can have flexible URLs and you can basically put a page anywhere.
As far as core API goes, you can do everything in any of them.
So to answer your question directly, when to use one over the other depends on how you think the admin will use it. For example for most sites there is only one contact page, using a CPT for that will just clutter the admin UI. OTOH plugins that enable you to create many forms will use a CPT as it is results in a more organized admin for them