How to insert widget areas specific to certain pages (or posts, etc.)?

Use Conditional Tags to show content only if a certain condition is met.

In your case, you’re probably looking to use is_front_page().

<aside>
    <ul>

    <?php
        if ( function_exists( 'dynamic_sidebar' ) ) {
             if ( is_front_page() ) {
                 if ( ! dynamic_sidebar( 'frontpage-widget-area' ) ) {
                     echo '<li>No sidebars for the frontpage.</li>'; // some default output
                 }
             } else {
                 if ( ! dynamic_sidebar( 'primary-widget-area' ) ) {
                     echo '<li>No sidebars for posts/pages.</li>'; // some default output
                 }
             }
        } else {
            echo '<li>Sidebars disabled.</li>'; // some default output
        }
    ?>

    </ul>
</aside>

That’s assuming, the two widget areas have been properly registered via register_sidebar() beforehand.