If the values of the constants defined in constants.php are hardcoded strings, then there is no need to escape them when they are used in PHP code. Escaping functions like esc_html__ are typically used for dynamic user input that could contain potentially malicious content like HTML tags, JavaScript code, etc.
However, if your constants are being used in an output context where user input is being displayed, such as in an HTML tag or in a JavaScript file, then it is important to make sure that the constant values are properly sanitized and escaped to prevent XSS attacks. This is especially true if an attacker is able to modify the value of the constant, either through a vulnerability in your application or by gaining access to the server.
To prevent this, you should ensure that your constants.php file is not accessible to the public, and that any sensitive information, such as database credentials or API keys, are not included as constants. You can also consider storing such information outside of the web root, and using server-side code to retrieve it as needed.