Just as @MarkKaplun mentioned in a comment, it’s possible with CSS. Also with Javascript.
CSS Approach
Let’s look at the Calendar widget, as an example.
The id for the widget, in the available widgets list, is of the form:
widget-{integer}_calendar-__i__
and in the sidebar it’s:
widget-{integer}_calendar-{integer}
If we want to display the calendar dashicon before the widget title, then we can e.g. enqueue a custom stylesheet for the widgets.php file, with something like:
div[id*="_calendar-"].widget .widget-title h3:before{
content: "\f145"; font-family: dashicons; margin-right: 0.5rem;
}
Here we use *= to find a substring match within the id selector.
It will display like this:
If needed we could restrict this further to the right or left parts:
#widgets-right
#widgets-left
or the available widgets:
#available-widgets
The widgets.php page will also have body.widgets-php if needed for general stylesheets.