Are your database tables using the InnoDB engine? If so, and the plugin is just for your website, start a transaction, run the inserts, commit the transaction (or rollback on error).
global $wpdb;
$wpdb->query('start transaction');
// do the inserts
$wpdb->query('commit');