I believe your problem stems from the upload process itself. If so, you do not need to customize an error message. Instead, you need to update your upload settings, and/or publish your file size limits to your community.
This article describes what causes that error, and how to address it.
Good luck!