How to edit original wordpress HTML source code and add icon?

That HTML is generated in the get_comment_reply_link function:

https://github.com/WordPress/WordPress/blob/0418dad234c13a88e06f5e50c83bcebaaf5ab211/wp-includes/comment-template.php#L1654

Which gives us what’s needed. Here are 3 options:

  1. The function takes an arguments array, and reply_text is one of the options, so just set this value to what you wanted e.g. 'reply_text' => '<i class="material-icons"> Reply </i>'
  2. There’s a filter at the end of the function that gives you the opportunity to modify the entire links HTML named comment_reply_link, look up the docs for that filter for details
  3. You don’t need to modify the HTML markup at all, this can be done with CSS

Changing the call to get_comment_reply_link will require changes to your theme, possibly the creation of a comments template. You will need to adjust wp_list_comments to take a callback argument with a function to display the comment, allowing you to change its HTML

Using the filter can be done in a themes functions.php or in a plugin.

Using CSS will require you to enquire with the material design library you’ve chosen.