Register sidebar ‘before’ and ‘after’ html as div rather than li tags

dynamic_sidebar() does not output a <ul>, so what your code should look like is as follows:

<div id="sidebar">
    <?php dynamic_sidebar(); ?>
</div>

and that, along with <div> tags in before_widget and after_widget, will give you nested divs.