Personally I think it's an inflamed nerve/nerves in the head, odd as that sounds. It bugs me that when I tell my doctors that there is an actual physical pain inside my ears they dismiss it because they don't see any fluid behind the ear drum or redness inside the ear canal. I even told my ENT that it hurts to put a Q-tip in my ear and got the standard "don't put anything in your ear smaller than you elbow blah blah blah" response as a way to deflect the fact that she has no clue what the issue could be.
I wasn't too keen on the infection theory myself as I take really good care of myself, lots of vitamin D, etc. and almost never get colds. I can't remember the last time I got the flu to be honest. But things did get really bad after I had major foot surgery back in November so I just assumed I picked something up from the hospital. I admit it really freaked me out because, like I said, I haven't gotten sick in years and thought I was doing everything right when it comes to staying healthy. Having gotten conflicting opinions from my primary doc verses my ENT I'm thinking that maybe I didn't have an infection and my problems are all related to my inner ear and its attending nerves being inflamed for some unknown reason.
At any rate, I'm pulling for you to get some answers to what's going on with you. Try no to feel too bad if the next ENT doesn't give you much relief though. I think sometimes we just need to give ourselves a break from all these doctors and just allow our bodies to go thru whatever it is they're going thru. All the appointments and jumping thru hoops to get answers can really add to the stress we are dealing with.
Of course best wishes always.