Question

Are Zaps "Threadsafe"???

  • 31 July 2021
  • 7 replies
  • 383 views

Userlevel 1

Zapier was working really well for us when we had just a few Zaps doing a few simple things. Because of that early success we started using Zapier for more and more automated tasks that are increasingly complex. Now I think we are running into issues where Zaps are stepping on each other but I can’t find anything online about the execution model that Zapier uses for controlling Zaps.

 

So we have two Zaps: Zap A & Zap B

 

Each Zap, when run “by itself”, works perfectly. The definition of “by itself” is when an instance of Zap A or B runs at a time when no other instance of Zap A or B will be executed for so much time later that it is essentially guaranteed that the first Zap A or B instance finishes before any other Zap A or Zap B instance could be run.

 

Both Zap A and B are straight forward, 1) Do Something, 2) Put a new entry in the same google sheet, 3) do some more stuff and exit. The google sheet is acting as a look up table so when a client finishes their part of the task, which might be between 1 minute or 1 month later, a Zap C & D can do a look up in the google sheet to bring context to the finished task, remove the google sheet entry, and move on.

 

The Issue is that when Zap A and B are triggered within a second of each other it appears they are stepping on each other’s toes when the google sheet operations of the Zap happens. So when the client part of both zaps finish we find out that one of the entries isn’t in the google sheet so one of the clean up Zaps C & D will fail and one will succeed.

 

So this brings up some questions that I can’t find anywhere. I believe Zapier should write a “Best Practices” or “How to design your Zaps” guide so that people can avoid these types of issues.:

  1. Can multiple instances of Zap A run at a time? So if Zap A’s trigger happens twice at essentially the same time, will one instance of Zap A from the first trigger run to completion and then the second trigger of Zap A will run? Essentially, when I write a Zap, can I make the assumption that it is impossible that another instance of Zap A will run at the same time? Or do I need to defensively design my Zaps to account for the potential of two instances of Zap A running at the same time? Basically, can I assume Zap A has a mutex around it?
  2. Can Zap A and Zap B run at the same time or does my Zapier instance require one zap to run at a time. If Zap A and Zap B are triggered at the same time will they run in separate threads of execution at the same time (and therefore, can easily step on each other if they are editing the same spreadsheet)? Based on behaviors I’ve seen I’m suspecting the answer is yes, they run at the same time, but I want to make sure.
  3. When you are operating on a single google sheet, if Zap A and Zap B operating on the same sheet at the same time, could their actions happen at the same time or are they queued up? So if Zap A and B both write an entry at the same time is there something in the Zapier engine that says “this is for the same sheet so queue up these writes so they happen one at a time” or could both those writes literally happen at the same time?
  4. Assuming Zaps are running at the same time, if Zap A is doing a “Find Row” at the exact same time as Zap B is doing a “Create Spreadsheet Row” are those instructions at least autonomous?
  5. How do I protect the reading and writing of the google sheet to guarantee that there is no way multiple Zaps could be operating on it at the same time. Delay on Queue seems promising but it’s description had a warning saying that delays in the services used in tasks after the queue make it so they can’t guarantee autonomous operations. Is there a way to put a “mutex” around a google sheet so I can do a look up followed by a write as a guaranteed autonomous action?

This post has been closed for comments. Please create a new post if you need help or have a question about this topic.

7 replies

Userlevel 7
Badge +14

Hi @Ryan R. 

Check out this article about Zap trigger types: https://zapier.com/help/create/basics/set-up-your-zap-trigger

FYI: Zaps either trigger instantly, or will trigger between 1-15 minutes depending on your Zap plan.

NOTE: GSheets has a slightly different trigger type.

 

Some of this will depend on which apps are being used as the Zap trigger.

For example, check out the help articles for GSheets related to Zaps: https://zapier.com/apps/google-sheets/help

 

Can multiple instances of Zap A run at a time? 
So if Zap A’s trigger happens twice at essentially the same time, will one instance of Zap A from the first trigger run to completion and then the second trigger of Zap A will run?
Essentially, when I write a Zap, can I make the assumption that it is impossible that another instance of Zap A will run at the same time?
Or do I need to defensively design my Zaps to account for the potential of 2 instances of Zap A running at the same time?
Basically, can I assume Zap A has a mutex around it?

Zaps by default can be triggered and run in parallel, so you’d need to design your Zaps to account for this potential use case you’ve described.

 

Can Zap A and Zap B run at the same time or does my Zapier instance require one zap to run at a time.
If Zap A and Zap B are triggered at the same time will they run in separate threads of execution at the same time (and therefore, can easily step on each other if they are editing the same spreadsheet)?
Based on behaviors I’ve seen I’m suspecting the answer is yes, they run at the same time, but I want to make sure.

Depends on how the Zaps are configured.

Zap can run in parallel if triggered at the same time.

Zap steps can take different lengths of time to process.

Delay steps can be added to help space out or have Zaps run in order they are added to a queue.

Check out Delay After Queue: https://zapier.com/help/create/customize/add-delays-to-zaps#delay-after-queue

 

When you are operating on a single GSheet, if Zap A and Zap B operating on the same sheet at the same time, could their actions happen at the same time or are they queued up?

So if Zap A and B both write an entry at the same time is there something in the Zapier engine that says “this is for the same sheet so queue up these writes so they happen one at a time” or could both those writes literally happen at the same time?

Both can happen at the same time.

 

The Google Sheets trigger is marked "instant" but it still takes a few minutes to trigger

The triggers for Google Sheets are unique among Zapier triggers.
When there is a trigger event in the spreadsheet, Zapier gets a notification webhook from Google about this. After that, Zapier sends Google Sheets a request for new data, so it uses both the polling and instant trigger methods.
This process takes about 3 minutes overall.

While not being "instant", these triggers are faster than regular polling ones, as they don't depend on the polling interval of the plan your account uses.

 

How do I protect the reading and writing of the google sheet to guarantee that there is no way multiple Zaps could be operating on it at the same time.

Delay After Queue seems promising but it’s description had a warning saying that delays in the services used in tasks after the queue make it so they can’t guarantee autonomous operations. 

Is there a way to put a “mutex” around a GSheet so I can do a look up followed by a write as a guaranteed autonomous action?

Advanced:

  1. You can daisy chain Zaps together using Webhooks: https://zapier.com/apps/webhook/help
  2. Also, check out Sub-Zaps: https://zapier.com/apps/sub-zap-by-zapier/integrations

 

Userlevel 1

The triggering “Setup your Zap trigger” and “Data deduplication in Zaps” articles are irrelevant to this question. For this question, you can assume two triggers of Zap A (for one example) or the triggers to Zap A and B individually for the other example, that happen essentially at the same time are each valid. Doesn’t matter if they are instant or polled. The important part is that the Zaps are called upon to start executing at essentially the same time.

 

Also, this issue isn’t specifically about Google sheets so reading up on it further than we already have won’t help. We’ve been writing Zaps for over a year now.

Moving to webhooks or subzaps won’t help with this issue either.

 

Maybe this will help… Here is a very simplified version of the issue that I think we are running into. Both Zap A and B essentially do this when operating on a google sheet:

  • Look for an empty row or a row with info we want to change
  • Do stuff
  • Write the row

What we want to happen is:

  • Zap A: Triggered
  • Zap B: Triggered (at roughly the same time)
  • Zap A: Find row in GS
  • Zap A: Do Stuff
  • Zap A: Write row in GS
  • Zap A: Completes
  • ...now Zap B runs, mutually exclusive from Zap A
  • Zap B: Find row in GS
  • Zap B: Do Stuff
  • Zap B: Write row in GS
  • Zap B: Completes

Instead what I think is happening is:

  • Zap A: Triggered
  • Zap B: Triggered (at roughly the same time)
  • Zap A: Find row in GS
  • Zap B: Find row in GS (finds same row)
  • Zap A: Do Stuff
  • Zap A: Write row in GS
  • Zap A: Completes
  • Zap B: Do Stuff
  • Zap B: Write row in GS (which overwrites the Zap A row)
  • Zap B: Completes

You could also take this same scenario and replace “Zap A” with “Zap A1” and “Zap B” with “Zap A2” to get across the idea of not only different Zaps stepping on each other but multiple instances of the same Zap stepping on each other.

We did just learn about Delay on Queue and I can see that potentially helping solve this issue most of the time BUT even it’s documentation says it can’t **guarantee** mutual exclusion of these operations if there are delays. We will be adding it into our zaps next week and seeing if it helps.

So the goal of my questions (1-5) in the original post I was hoping someone would help us peak under the hood a little to better understand the potential concurrency issues we might be having (or confirm my example above could never happen) and give us some ideas, design patterns, tools, etc to prevent this from happening.

Userlevel 7
Badge +14

@Ryan R.

Webhooks and Sub-Zaps would be the way to ensure Zaps run in the desired order.

  1. You can daisy chain Zaps together using Webhooks: https://zapier.com/apps/webhook/help
  2. Also, check out Sub-Zaps: https://zapier.com/apps/sub-zap-by-zapier/integrations

 

Example using Webhooks:

Zap 1

  1. Trigger: GSheet - New/Updated Row
  2. Action: [app] - [event]
  3. Action: POST - Webhook (this would trigger Zap 2)

Zap 2

  1. Trigger: Webhook - Catch Hook
  2. Action: [app] - [event]

 

Example using Sub-Zaps

  1. Trigger: GSheet - New/Updated Row
  2. Action: Sub-Zap - Call Sub-Zap (this calls a Sub- Zap and waits until the Sub-Zap finishes to continue to the next Zap step)
  3. Action: [app] - [event]
Userlevel 1

But what if Zap 1 is called by a webhook two times within a millisecond of each other? Does the second instance of Zap 1 wait for the first instance of Zap 1 to finish before it starts?

 

How do I make sure that only one instance of Zap 1 is running at a time in Zapier?

Userlevel 7
Badge +14

@Ryan R.

But what if Zap 1 is called by a webhook two times within a millisecond of each other? Does the second instance of Zap 1 wait for the first instance of Zap 1 to finish before it starts?

No, but using a Delay After Queue step can help Zaps run in sequential order: https://zapier.com/help/create/customize/add-delays-to-zaps#delay-after-queue

 

TIP: When in doubt, test it out.

Userlevel 1

Troy, as I stated in the original post, I just learned about Delay after Queue and we are going to start updating our Zaps with it this week. I believe it has the potential the make this issue **better** but not solve the problem 100% because of this statement in the Delay after Queue documentation you linked to:

 

The Delay After Queue action does not guarantee that the steps following it will never run simultaneously. Slowdowns in Zapier’s infrastructure and auto or manual Zap run replays after errors may cause some steps to still run at the same time.

 

Are there any other tools Zapier has to prevent race conditions?

It would be great is Zapier had a flag they could set for SubZaps where we can say “only one instance of this subzap can run at a time”

Userlevel 7
Badge +14

@Ryan R.

The Delay After Queue app is the best option available to have triggered Zaps run in sequential order.

The key is to test how long it takes the Zap to complete, then make sure your Delay After Queue time encompasses this time to allow for some variance.