How to sort a custom wordpress query by combination of meta values?

What you are wanting to do is possible– MySQL can do math– but WP_Query can’t handle this natively. You’d need to apply a filter or two. You would end up sorting on generated values which is not especially efficient. That is, MySQL, would read all of the rows in the table for those two keys, do some math, and then sort.

My advice would be to store a third key– say, profitability— when you save the post keys, and then sort on that instead of trying to generate this is SQL. Something, very crudely, like this:

function generate_profitability_wpse_138683($meta_id,$object_id,$meta_key,$_meta_value) {

  if ('paidusd' === $meta_key) {
    $tips = get_post_meta($object_id,'totaltips');
    if (!empty($tips)) {
      $profit = (int)$_meta_value/(int)$tips;
    }
  } elseif ('totaltips' === $meta_key) {
    $paid = get_post_meta($object_id,'paidusd');
    if (!empty($paid)) {
      $profit = (int)$paid/(int)$_meta_value;
    }
  }

  if (!empty($profit)) {
    update_post_meta($object_id,'profitability',$profit);
  }
}
add_action('updated_post_meta','generate_profitability_wpse_138683',10,4);