How to display only subcategory

The easiest way to do this would be to use wp_list_categories instead. It has depth parameter which will do exactly what you want (you can use custom walker class, if you need different output than default one).

Another way would be to write your own code (but it won’t be a beautiful one). You can use something like this:

$args = ...
$categories = get_categories( $args );  // $category_ID as child_of
foreach ($categories as $k=>$category) {
    if ( $category->parent != $category_ID) {  // it's not direct child
        $parent_category = get_term($category->parent, 'product_cat');
        if ( $parent_category->parent != $category_ID ) {  // it's not grandchild either
            unset($categories[$k]);
        }
    }
}