So I finally got in to see an ENT yesterday.
He ordered an MRI to rule out anything serious, did a few checks, then said I'm fine and to learn to live with it. Also said I can't have hyperacusis because I'm too young (I'm 33), that only older people get it due to age-related cochlear damage. He said it's just anxiety and not actually hyperacusis.