Add custom classes for blocks in editor based on custom attributes

You can do this with the editor.BlockListBlock filter. More info here. This allows you to do something like the following:

const withCustomAttributeClass = createHigherOrderComponent( ( BlockListBlock ) => {
    return ( props ) => {
        const { attributes } = props;
        const { yourCustomAttribute } = attributes;
        const class = yourCustomAttribute ? 'my_custom_class' : '';

        return <BlockListBlock { ...props } className={ class } />;
    };
}, 'withCustomAttributeClass' );

addFilter(
    'editor.BlockListBlock',
    'your-plugin/custom-attribute-class',
    withCustomAttributeClass
);

In the above, I am just checking if the attribute exists and then am applying a class, but you could do all sorts of more complicated things here. Note that this filter will apply the class to every block with that custom attribute. However, props contains all of the block information, so you could exclude certain blocks as needed if you wanted.