WordPress themes can be set up in many different ways, which is both a blessing (you can do things however you like) and a curse (there’s not always a single file you need to look at to make a particular change).
You’re on the right track, looking into the theme. What you’ll need to do first is to create a child theme. That way, when you update your theme, your edits won’t be overwritten.
Once you have a child theme, you’ll want to look in the parent theme to find out how they are calling the particular nav menu you want to modify. Many themes use more than one menu, so you need to isolate that one. Then, look and see how it is called. You will see something like this:
<?php wp_nav_menu(
// everything inside this array will vary, depending on your theme
array(
'menu' => 'topnav'
'walker' => new custom_walker()
)
); ?>
What you want to find out is whether there is a “walker” in the array. (There is above.) If there is, you’re in for more complexity, as your theme (now the “parent” theme) is already using a custom walker that builds different HTML than WP Core does by default. You’ll then need to search through your parent theme’s files. Most often the walker class is inside functions.php but not always. Find that class, then copy it into your child theme’s functions.php and then change the name of the class.
Or, if the nav menu you want to affect doesn’t use a walker at all, go into your child theme’s functions.php and create a walker.
In either case, you now need to look into what it takes to create a walker. Look for examples online and start very simple, and then as you run into questions you can check back here with a new question.
One last note – if you’re doing something like top navigation and you have a static set of pages to include there, along with your custom HTML, it might be a lot simpler to just not use a WP nav menu. You can use other functions like wp_list_pages() to list all the Pages, or even hard-code, depending on your needs and how willing you are to maintain the code.