Feedback - Webhook deduplication

  • 29 September 2022
  • 4 replies
  • 116 views

Userlevel 1

Hi,

Zapier Status (https://status.zapier.com/) says instant triggers were 100% reliable yesterday, but I have a webhook from Zendesk to Zapier which received some 500 Server Error as well as some 504 Gateway Timeout errors between 1pm - 1:20pm eastern time yesterday.

 

 

That’s how it looks on the Zendesk side. In Zapier, the Zap with the webhook trigger actually triggered 3 times.

The incoming webhooks from Zendesk included a Request Header titled X-Zendesk-Webhook-Invocation-Id which ideally should permit deduplication of requests in a situation like this. If Zendesk encounters a 5xx error and retries, but Zapier successfully processed the webhook trigger despite the error, then the next request bearing the same X-Zendesk-Webhook-Invocation-Id header should be responded with a 200 OK response to tell Zendesk it’s not necessary to keep retrying, but should not trigger the Zap.

I understand of course that the X-Zendesk-Webhook-Invocation-Id header is not a standard header, is there any way to indicate to Zapier in the trigger settings what header to look for to deduplicate?

If not, is this forum the best place to make this a feature request?


This post has been closed for comments. Please create a new post if you need help or have a question about this topic.

4 replies

Userlevel 7
Badge +8

Hey @David Smedberg , 

 

i might not be the best person to comment on this as Im a beginner when it comes to webhooks, but wouldnt a filter after the trigger help in this case to avoid losing tasks in Zapier? Not sure of that helps :) 

Userlevel 1

Hey @David Smedberg , 

 

i might not be the best person to comment on this as Im a beginner when it comes to webhooks, but wouldnt a filter after the trigger help in this case to avoid losing tasks in Zapier? Not sure of that helps :) 

Hi @MohSwellam … it might, but there’s a couple of challenges:

  1. I’d have to use some external app, such as Storage for Zapier or Airtable, to store the invocation ID, and if that system doesn’t reply faster than the retry logic of Zendesk, then the deduplication function wouldn’t work properly. I can’t control the speed of the Zendesk retries, it looks like they are 15-30 second increments, and so in that tight time period the retry might pass the deduplication filter before the initial delayed request adds the invocation ID to the storage system.
  2. That increases my Tasks, which I am always careful about since that is what we are billed for. If the application layer can handle it instead of me doing it in the automation, that is preferable.
Userlevel 7
Badge +12

Hi @David Smedberg 

To me it looks like Zapier may not have responded quick enough for Zendesk, thus Zendesk is sending the request multiple times and Zapier is processing it each time.. just not always fast enough for Zendesk. 

To my knowledge, Zapier does not perform deduplication against webhooks with the standard Catch Webhook trigger. 
 

Userlevel 1

@GetUWired yeah, I mostly agree. Although the 504 response code would have been sent by Zapier’s reverse proxy…. meaning that it is not so much “not fast enough for Zendesk”, as “not fast enough for Zapier’s proxy” so it is in Zapier’s control.

 

I’m not an expert, but I think Zapier’s proxy should close the connection to the application at the same time as it returns a timeout to the client, and the application should treat a closed connection to the proxy as a cancellation of the request.