How to do store, retrieve and write attributes in the registerFormatType method in Guttenberg?

So inline formats don’t have a state as such, they aren’t discrete objects like blocks. Instead you apply a format to a slice of text, and the UI is generated on the fly.

That UI is then given attributes that are currently active, an activeAttributes array is passed that contains those attributes. Note only active attributes are passed. If you simply apply a format to text, you have not set any attributes yet, so this array will be empty. So you cannot rely on the attributes always being present, or having default values.

When you want to change those attributes or add them, you can’t just update or set them, instead you need to call applyFormat with the value ( available in props ), specifying which format, and the full list of attributes to apply ( it doesn’t update, it overwrites ). Then you need to call onChange with the new value that applyFormat returned afterwards.

Here is an example that slims yours down to just a name attribute:

wp.richText.registerFormatType(
    'test/set-link',
    {
        attributes: {
            name: 'data-set',
        },
        title: 'Sets Link',
        tagName: 'a',
        className: 'sets-link',
        edit: (props) => {
            return [
                el(wp.blockEditor.RichTextToolbarButton, {
                    icon: 'block-default',
                    title: 'Sets Link',
                    isActive: props.isActive,
                    onClick: () => {
                        props.onChange(
                            wp.richText.toggleFormat( props.value, {
                                type: 'test/set-link'
                            })
                        )
                    }
                }),
                props.isActive && ( el( components.Popover,
                    {
                        position: 'bottom center',
                        headerTitle: 'Sets Attributes',
                    },
                    [
                        el( components.TextControl, {
                            placeholder: 'Set Name',
                            value: props.activeAttributes.name ? props.activeAttributes.name : '',
                            onChange:  (newName) => {
                                newval = wp.richText.applyFormat(
                                    props.value,
                                    {
                                        type: 'test/set-link',
                                        attributes: {
                                            name: newName
                                        }
                                    }
                                );
                                props.onChange( newval );
                            },
                        }),
                    ]
                ))
            ]
        },
    }
)

Note that it would be much smaller if you used the WP Scripts tool, which would allow you to use JSX/React directly without bundling the libraries.

I also fixed a typo, and a bug where isActive was always set to true on the toolbar button