Making an add_filter() call from within an add_filter() call

You can add or remove hooks from inside other hooks if you get the timing right but I don’t understand why you are making this so complex.

function set_title($title) {
  global $wp_query,$post,$address;
  $address = $wp_query->query_vars['address'];
  if ($address) {
    return $address;
  }
}
add_filter('wp_title', 'set_title');

If you need the share the value, use a static variable:

function my_address() {
  static $address="";
  global $wp_query;
  if (empty($address)) {
    $address = $wp_query->query_vars['address'];
  }
  return $address;
}

function set_title($title) {
  return my_address();
}
add_filter('wp_title', 'set_title');

Or wrap the whole thing in a class.