How to add color icons to front of custom block color palettes

If you mean the indicator for the currently selected colour here:

enter image description here

Then No.

The problem is a misunderstanding of what the circles colour represents. It’s not telling you the current colour of the buttons text, it’s telling you the chosen/selected colour of the text, and you have not chosen a colour yet, it has not been overriden. Hence the blank circle with a cross.

To change it to the current colour even when none is provided would require you to build several core WordPress JS packages with custom versions that modify the JS components to show it.

Even then, figuring out what that colour should be when the user has not chosen a colour from the palette will be very difficult, especially when the colour may not be in the palette of options available, and arbitrary CSS from blocks, block styles, block variations, parent blocks, the theme, child themes, custom CSS, and plugins, as well as core itself could have changed the block.

Is this something that can be incorporated into our theme.json file?

No, because that’s not how the control works. theme.json configures things, but it can’t be used to replace React components or make more extensive changes. If it could do this it would probably already have been done, most likely via global styles.