How do I link to a php file in my plugin directory?

You can use admin.php?action=my_action and WordPress will fire the equivalent action hook:

// How to get the URL
$url = admin_url( "admin.php?action=wpse_21460_export&any_other_arguments" );

// How to handle the URL
function wpse_21460_export() {
   // Do your export and exit
}

add_action( 'admin_action_wpse_21460_export', 'wpse_21460_export' );

See how WordPress formats the hook name to admin_action_{value_of_action_param}