There can be a balance. I used to live the life where everything revolved around sport spectating. I got too mad when my teams lost. I said no to too many things because it interfered with a sporting event I wanted to see. I'd drink too much - before, during and after - for said sporting event which wasted even more time than the event itself.
A few years ago, I gave up making it so important. I still love sports and love watching them. I just made myself a promise not to prioritize a sporting event in my life unless it was a tournament, playoffs, championship, etc. The more I allowed myself to miss a game, the more I found myself not getting so mad when they lost.
I still love my sports teams as much as the next, and I can still enjoy watching games and not give up on my teams. It's just not a life priority anymore. Sports enrich my life now instead of owning my life.
Find that balance if you can.
edit:
I wanted to add this too. When I was wrapped up in sports, it made me that crazy fan too. Hate the enemy, boo them, cuss at them, and all that insanity. Worry about the refs, worry about the commissioner's views, the players' off the field activities.
The change I made changed all of that too. Most people like some sort of art. Some like ballet, some like the orchestra, some like static works of art. Sports are my form of art. I don't hate the enemy anymore. I don't care about the refs - well, not to the point of ruining my day. When I watch sports now, I enjoy the art of athletes doing insane things I wish I could. I want my team to win, but damned if I don't love watching anyone on the field or court doing something spectacular.