Definitely wear ears while hunting and at the range. I shot without ears for about 20 years then one day my ears started ringing and never stopped. That was 20 years ago. I have tinnitus. 40% loss in my left ear and 20% loss in my right ear. If you think ringing constantly is not that bad think again. I was miserable for a few years until I got used to it. Now I have a heard time hearing, forget about holding a conversation with a lot of background noise, and I hear crickets 24/7.
You may not notice a shot while hunting, but the nerves in your ears are being damaged with each shot. There is only one thing in my life that I would like a do over on and it would be shooting with ears every time I have ever pulled a trigger in my life.