Insert images into wordpress post with a query

Yes, it is possible, WordPress does not have an API, but some useful functions.

I would load all your rows into array like

$your_table = ['post_id' => 'image', 'post_id' => 'image', …]

Then execute a foreach loop

foreach($your_table as $post_id => $image)
{
    //full path to the image
    $image_path = _YOUR_FULL_ABSOLUTE_PATH.$image;

    /**
      * Get an ID of the inserted attachment
      * This takes care of resizing image and stuff…
      * The fourth param I would sugest you to keep true, since in case of duplicate entry in your table, it will not have been inserted more than once
      */
    $featured_image = wp_insert_attachment(array('attachment_meta'), $image_path, $post_id, true);

    //assign featured image
    add_post_meta($post_id, '_thumbnail_id', $featured_image);
}

‘attachment_meta’ study here