WP does not do this by default since there is no guarantee that attachment isn’t still being used by some other post.
Basic logic would be to hook into delete_post, query for child attachments and run wp_delete_attachment() on each.
I did quick search in plugin repository and came up with tiny plugin that does just that (seems unmaintained so test before use) – Post Data Delete Advanced.