The Settings API – What is the purpose of default value in register_setting?

Excerpt from the register_setting() documentation:

‘default’
(mixed) Default value when calling get_option().

So the purpose of the default argument is to provide a default value which get_option() would return if the option does not exist (or has not yet been saved) in the database.

And register_setting() sets the option’s default value via filter_default_option() which is hooked on the default_option_<option name> filter: (see source on GitHub)

if ( array_key_exists( 'default', $args ) ) {
    add_filter( "default_option_{$option_name}", 'filter_default_option', 10, 3 );
}

Therefore, because update_option() uses $old_value = get_option( $option );, i.e. it uses get_option() which runs the above hook, then if the option hasn’t yet been saved in the database, the old value would then be the default value and thus the option wouldn’t be saved if the new and the default/old values are the same just like you can see in the example below:

$option  = 'my_test_option';
$default="My default value";

register_setting( 'my_test_page', $option, [
    'default' => $default,
] );

delete_option( $option ); // JUST FOR TESTING

// So at this point, the option doesn't yet exist.
var_dump( get_option( $option ) );                   // string(16) "My default value"
// this displays a custom value because the 2nd parameter ($default) is specified:
var_dump( get_option( $option, 'a custom value' ) ); // string(14) "a custom value"

// Now let's save the option.
$old_value = get_option( $option );  // this returns "My default value"
var_dump( $old_value === $default ); // bool(true)
// bool(false), because $default is the same as $old_value
var_dump( update_option( $option, $default ) );
// bool(true), because 'foo' is not the same as $old_value
var_dump( update_option( $option, 'foo' ) );

// Retrieve the saved option.
$old_value = get_option( $option );
var_dump( $old_value ); // string(3) "foo"

// Update the option. (set the value to the default value)
var_dump( $old_value === $default ); // bool(false)
// bool(true), because the old value is now 'foo'
var_dump( update_option( $option, $default ) );

As for this: (italic formatting added by me)

When accessing the option on the front end using the shortcode
[my-test-option] it is blank. This is expected when I haven’t
visited the settings page, as nothing has been saved to the
database
.

Yes, that’s expected, and it’s because you used the admin_init hook to register the setting on the admin side only (wp-admin).

If you used a hook which runs both on the admin and non-admin sides of the site, e.g. init, then you would’ve seen the default option value instead of the blank output.