Change the second line of your code to this:
$catlist = get_categories( 'hide_empty=0&parent=" . $cat_id );
Instead of child_of parameter, use the parent parameter of the get_categories() function; this will display the direct descendants (i.e. children only) of the category and not grandchildren of the category.
For more information visit the Codex page for get_categories().