get_terms not working when child_of is used

I got it solved on my own so, wanted to answer my own question.

in options table the results are cached, so I queried it and delete that.

SELECT * FROM `wp_options` WHERE `option_name` like '%department%'

department is my taxonomy name here.

You can check the function _get_term_hierarchy in taxonomy.php in wp-includes folder.