It's worse because you fear it. Which is totally normal, everyone goes through that at different stages of tinnitus. I am a pretty stable person but tinnitus brought me to my knees for over 6 months. Benzos helped me through that, no doubt, but there's a time you have to give it up. You're tapering just fine, maybe even on the cautious side, so there's nothing you can do any better. If you're seeing improvement, great. And if one day it's a bit worse again, no worries, it'll get better again.