How did it go? Did you end up going?
I should put a disclaimer here that probably the best thing is to not drink but I do enjoy it, so it is always a fight between my anxiety and my party instinct.
I know exactly how you feel. For me, it isn't so much about the amount of people, but the amount of time I would be expected to hang around and drink. If I came for one or two hours and left for no good reason (and hiding the anxiety), it wouldn't go over well and I never wanted the drama. I went to my best mate's birthday back in March and dreaded it. Even going there was an achievement. Then, I went and sat there with the darkness, the loud music, the constant worrying about my physical sensations. I did lose it a bit and had to excuse myself for about 30 minutes. But I managed to just about hold it together. Sort of fortunately, he got absolutely wasted well before me and the night ended a bit earlier than thought.
What usually happens is I feel super anxious during the first 1 or 2 drinks, and looser after that. But if I drink too much, the anxiety the next day pretty much wipes out the day for me.
There was another instance where I was leaving to go somewhere and they really wanted to organise a big farewell for me. The parties where I am the centre are the worst. I had to keep putting it off due to organising moving and everything and needing time to relax myself (which to be fair, was true). Eventually I caught up with only 2 mates a couple of nights before I was leaving, and just had to say clearly I couldn't drink too much.
But I also dread long social occasions even when I don't drink too much or at all, so it sucks. Anxiety tries to rob us of enjoyment of some of life's great moments.
I hope you did alright if you went mate. I've been there, panicking, going to the bathroom, going outside for some air. It truly sucks. I'm still here. If you didn't go also, you have our support here and hopefully your mate's understanding.