How to override the wordpress default widget markup

http://codex.wordpress.org/Function_Reference/unregister_widget

add something like this to functions.php:

if (!function_exists('my_unregister_default_wp_widgets')) {
    function my_unregister_default_wp_widgets() { 
        unregister_widget('WP_Widget_Calendar');
        unregister_widget('WP_Widget_Recent_Posts');
    }
    add_action('widgets_init', 'my_unregister_default_wp_widgets', 1);
}

Also… you can look at the http://codex.wordpress.org/Widgets_API examples to see how you can extend the WP_Widget class to create custom widgets that you want.