How to remove set_post_thumbnail_size() in child themes?

What exactly do you mean by nothing happens ?

Technically you don’t need to remove set_post_thumbnail_size(), as I understand from code – simply making your own call later will overwrite size.

There are two ways you can handle this:

  1. Adjust your setup function to later priority. Child theme is processed before parent theme. So when you hook with default priority – setup functions are also executed in that order. So add_action( 'after_setup_theme', 'twentyten_child_theme_setup', 11 );

  2. Plug your own twentyten_setup() function. It is wraped in if ( ! function_exists( 'twentyten_setup' ) ) condition in Twenty Ten. So simply copy it to your theme and make changes you need – your version will run instead of original one.