Guest list add or cut

I have these 3 "friends" from high school, who I am having the honest hardest time trying to figure out whether to add them or cut them from guest list. A little back story... when my fiancé and I really first started dating they threw a fit because we didn't coke to a party they hosted because he didn't know anyone. They even asked me if he liked me as much as I did him... my fiancé has never liked two of them because he sees them as bad influences on me. Now that we are older and things they only talk to me when they want to get to gether the 4 of us, however two of them hangout all the time and now 2 of them are moving into together so I'm sure it will become the 3 of them and then me only when they choose.



However got together with them the other night and they straight up asked if they were invited I panicked and said yeah but only if you want too. They said well duh of course they did.
Now I feel pressured but I truly feel like they really are not that good of friends.
Please anyone help me! Many family members say invite them anyway who cares. Another family member said no dont because they sound like terrible friends (especially because they straight up asked me).
Fiancé does not care anymore he just wants me to stop stressing about it