Anxiety can definitely play a part. My personal experience have been mixed though. Some sounds induce multi-day spikes, and in those cases, I'm not always aware that the spike is reducing, since it happens so slowly. Over time, I believed that spikes were stacking on top of each other, but in reality I basically still had my baseline intact. My baseline has increased permanently though, due to very safe sound levels, but not to the extent that I've initially thought.
My loudness hyperacusis has not worsened in recent months, and I have a few reference points which I test to see where my tolerances are. It's a very rough estimate, but works reasonably well.
If you feel that exposure to everyday sounds has irritated your auditory system and worsened the tinnitus and hyperacusis, I'd advise you to lower your exposure, just to manage the conditions. Exposing yourself to the current levels is obviously not effective since you seem to worsen from it. Have you tried custom molded high fidelity ear plugs? A -25 dB filter will still let through most frequencies as it tries to attenuate similarly to a "flat eq". I think e.g. a water tap running wouldn't cause nearly as much spiking or discomfort when using such hearing protection. I personally think it's best to find a "level" in which the auditory system isn't being irritated, and from there very very gently test the boundaries, and I think high fidelity ear plugs are a great way to do so through our everyday sounds.
It isn't completely black and white though, since tinnitus/hyperacusis severity isn't necessarily a 1:1 correlation to hearing loss. It's pretty clear that we can worsen the tinnitus to unfathomable levels without actually having severe hearing loss.