PHP: How to access db the right way in plugin?

I don’t understand if it works for you or not.

For me the code is not wrong, maybe can be optimized.

For the ajax request, if it is made in the backend, yo do not need to pass the ajax url using wp_localize_script. From WP 2.8 a global variable ajaxurl is defined to be used in ajax calls in the admin area.
So you can delete the wp_localize_script and use directly ajaxurl in the javascript.

Second optimization in the db request. Now it works, (or at least it should for me) but if you need only the variable $row->url you can use $wpdb->get_var, like so:

$url = $wpdb->get_var("SELECT url FROM " . $wpdb->prefix . "custom_table");
if ( filter_var($url, FILTER_VALIDATE_URL) ) {
    return $items . "<li class="menu-item"><a href="" . esc_url($url) . "" target="_blank">Custom Link</li>";
} else {
   return $items;
}

What is strange here is the missing of a ‘WHERE’ clause in your SQL. Seems you save only one value in a custom table. If so, is much better using the standard options table and using get_option / update_option to get and save the values.