Where should I save an API key of my user when installing my plugin?

Using the Settings API to display the field and the Options API to store the value would be a good way to go.

Your hunch is correct — creating a custom table to store an API key is not the way to go.

Edit

Here’s a complete example (copied from this oldie but goodie https://wpengineer.com/2139/adding-settings-to-an-existing-page-using-the-settings-api/) that adds a custom setting to the Options – General page:

// Register and define the settings
add_action('admin_init', 'ozhwpe_admin_init');
function ozhwpe_admin_init(){
    register_setting(
        'general',                 // settings page
        'ozhwpe_options',          // option name
        'ozhwpe_validate_options'  // validation callback
    );

    add_settings_field(
        'ozhwpe_notify_boss',      // id
        'Boss Email',              // setting title
        'ozhwpe_setting_input',    // display callback
        'general',                 // settings page
        'default'                  // settings section
    );

}

// Display and fill the form field
function ozhwpe_setting_input() {
    // get option 'boss_email' value from the database
    $options = get_option( 'ozhwpe_options' );
    $value = $options['boss_email'];

    // echo the field
    ?>
<input id='boss_email' name="ozhwpe_options[boss_email]"
 type="text" value="<?php echo esc_attr( $value ); ?>" /> Boss wants to get a mail when a post is published
    <?php
}

// Validate user input
function ozhwpe_validate_options( $input ) {
    $valid = array();
    $valid['boss_email'] = sanitize_email( $input['boss_email'] );

    // Something dirty entered? Warn user.
    if( $valid['boss_email'] != $input['boss_email'] ) {
        add_settings_error(
            'ozhwpe_boss_email',           // setting title
            'ozhwpe_texterror',            // error ID
            'Invalid email, please fix',   // error message
            'error'                        // type of message
        );
    }

    return $valid;
}

I changed the code so that the option appears on the General Options page. It won’t be too difficult to modify this code to use your API key field.