Dealing with guest count and fiance's anxiety?

My (31f) fiance (26m) and I are planning on getting married next fall at our local renaissance faire. We do not want a traditional wedding and don't believe in spending a lot of money on the event ($10k or so is our limit). He doesn't really believe in weddings in the first place, thinks they are a waste entirely and would rather elope, but he is very selflessly willing to have a wedding because celebrating with my family is important to me.


The problem is I have a large family and grew up close with neighbors who are more like family than neighbors. His family is spread out (east coast, west coast, and New Zealand) and probably no one but his immediate family would come (parents and brother). We have a few friends we want to invite, as well. Between us, the total guest count would be 105 (kids included). He was... not thrilled about this - he has a lot of anxiety around being the center of attention or having to socialize with a bunch of people he doesn't really know. My solution was we invite only a limited number to the ceremony/reception and, because the guests also get all day admission to the ren faire, we invite the rest of the people JUST to the faire, saying we were limited to the number we could have at the actual wedding but wanted them to come celebrate too. Probably, we would also include a gift card for food ($15 or $20 per person, kids included) to make up for not having them at the reception where food will be served.
This list for the ceremony/reception is still 68 people (including 12 kids, and this number includes us and our immediate families which total 11 people, so it's really more like 57 extended family and friends). We also expect that some of the older aunts/uncles and his family who live far away will not come, which is fine with us. That would probably bring the guest count from 68 to 55 or so.
I feel like this is a perfect compromise in my mind, but he is FREAKING out about it. He keeps saying how it's such a higher number than he thought and I can tell he doesn't want to do it at all. It's not the cost, it's just the people themselves. The thing is, he doesn't have anxiety problems any other time, and I'm trying to be understanding and compromise, but this is incredibly important to me. I've been willing (and happy) to change pretty much every aspect of our wedding EXCEPT inviting my family - the whole point is that I want to celebrate with my family who I am close with. Otherwise, why have a wedding at all? I have been to (and often in) all of their weddings and we love celebrating together. I know this is somewhat foreign to him since he grew up without seeing his extended family often, but I grew up seeing my cousins every weekend. I can't imagine this day without them.
He also was originally okay with the secondary invite solution, but thinks it's too many people now. The thing is, we probably would barely see the additional people we invite, if at all. It's a big enough faire and there wouldn't be an expectation of seeking them out. It's really more about treating them to a day at the faire to make up for not inviting them to the wedding.
I feel like a hypocrite because I've had anxiety and depression issues my while life - I know how immobilizing it can be to face a severe anxiety issue. I guess I feel disappointed that he isn't trying harder to fight through it, which I know is unfair. It has been a problem in the past that he has avoided family gatherings when too much of my extended family has been invited, and its made me very upset. My ex did that and it was a huge point of contention for us. It signifies to my family that he doesn't want to get to know them and by extension doesn't value me (which is NOT true, but I know thats how it reads). I spent 3 weeks in New Zealand with his extended family, which I had severe anxiety about, both because travel/being outside of my comfort zone is hard for me, and staying with his family in a foreign country where a lot is unfamiliar and frankly their accents could be really thick! Lol
So my questions are this: How do we solve this in a way that works for both of us?Am I being too stubborn or unreasonable?Is there a better solution, or should I continue to try to talk to him about how important this is to me and how do I do that best?Is my solution about inviting people to the faire separately good enough? And if you were a family member or friend who wasn't super close to me (but expected an invite), would you see this as a good solution to not being invited to the ceremony but being able to still celebrate?
Sorry this is so long but I appreciate any advice you have!!!