Get Default Post Category From Settings

You can access to default category ID via default_category option:

$default_category = get_option('default_category');

Now you can use selected() function to generate the selected attribute:

<?php 
$args = array(
    'type' => 'post',
    'hide_empty' => 0   
    );
$categories = get_categories( $args );
$default_category = get_option('default_category');
?> 
<select name="category_name">
<?php
foreach($categories as $category){
    if ($category->name != 'Uncategorized') {
        if($action=="edit"){
            $selected_category = get_the_category($post_id); 
            $selected_category = $selected_category[0]->cat_name;
            if($category->name==$selected_category){
                echo '<option value="'.$category->term_id.'" selected> '.$category->name.'</option>';
            }else{
                echo '<option value="'.$category->term_id.'"> '.$category->name.'</option>';
            }
        }else{
            echo '<option value="'.$category->term_id.'" '. selected( $category->term_id, $default_category, false ) .'> '.$category->name.'</option>';
        }
    }
}
?>

Maybe you want to use wp_dropdown_categories, which can simplify your code:

<?php

if( $action=="edit" ){

    $selected_category = get_the_category( $post_id );
    $selected_category = $selected_category[0]->term_id;

} else {

    $selected_category = get_option('default_category');

}

$args = array(
    'hide_empty' => 0,
    'name'       => 'category_name',
    'selected'   => $selected_category
);

wp_dropdown_categories( $args );

?>