Don’t worry about the duplicated content issue. First of all, the canonical link fixes most browsers and search engine crawlers that will be reading your site (and it’s actually been there for a while).
Second, the alleged “duplicate content penalty” that people have told you about (the things you’ve read that say it’s bad for SEO) is a myth. Google developers have been quoted repeatedly as saying so. Here’s a great article that references them and explains it in more detail than I care to go into.
So yes, you’re fine. Don’t worry about paginated comments on your site. They actually make the experience better for the user and don’t harm your SEO score.