Turns out you cannot use Google Calendar Zapier integration to invite someone to recurring event.
When you test your Zap, it will correctly invite the person to the first instance of the meeting but not to future events.
This is a tricky issue to catch, and should be clear on the integration description. I spent a lot of time figuring it out, and made some people miss out on very important meeting.
It seems to be a known issue -
Hi there
Welcome to the Community! 
Zapier’s “New Event” trigger doesn’t capture recurring/repeated events by design, since recurring events can occur infinitely into the future. However, you can use the “Event Start” trigger to have a Zap that fires on recurring events.
If you want to add to every single occurrence of a recurring event, instead of just a single instance of the event. You may setup your Zap like this:
Hopefully, this helps! 
Using the "Recurring Event ID" was indeed the solution to the issue. Thank you!
Yay! Thanks for letting us know, 
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.