Question

Microsoft Dynamics 365 CRM error: The app did not respond in time. It may or may not have completed successfully.

  • 6 August 2023
  • 5 replies
  • 613 views

Userlevel 2
Badge

I am using the Microsoft Dynamics 365 CRM app and are finding inconsistent “The app did not respond in-time. It may or may not have completed successfully” errors for the Create Lead step.

Firstly, we are using the 1.17 version of this public application, not the latest 1.18, the reason being is the payload data used in the 1.18 is broken for us, it does not send certain field names with the right format (logical name) and hence we can’t use the latest version in it’s current state. This is only for creating/writing data, we use the latest version for find/lookups.

What we are finding with various Zaps that use the Create Lead step in most cases the task is in fact completing because the data has been created in CRM, but Zapier sees it as a failure with timeout related errors. With auto-replay on, this task is retried until a success response occurs, however this in nearly all cases is now duplicating data (sometimes multiple times if repeated tasks continue to be marked as stopped/error.

Turning off auto-replay is a solution but then because of the inconsistent errors occurring, the Zap itself will also be at risk of being automatically turned off due to repeated errors without handling.

From my own API testing in Postman I’ve observed the Dynamics 365 CRM web API responses in our environment take more than 30 seconds in some cases. The fact we cannot increase the timeout of the HTTP request is the main issue. Sometimes the response is within the threshold, sometimes it’s not, however in nearly all cases the task being marked as Stopped/Error has completed. The amount of fields being mapped doesn’t seem to have much bearing on the issue, I’ve had the same issue with smaller payloads vs larger ones too.

With the release of the App Extensions feature I am considering creating my own Create Lead action, so I can use the latest version of the public app while making sure the payload data is what it should be, I still can’t do anything about the timeout value however.

We really could do being able to have a way to define a custom timeout value for the HTTP request though, some APIs will always be slower and there is nothing you can do about it. I can’t see why we can’t have the option for various apps. We can create an artifical delay action step with a significantly long “delay for” value in Zaps right now, allowing certain API requests/actions to run longer is surely possible within the platform?

 


5 replies

Userlevel 7
Badge +11

Hey there, @jamesmacwhite! 👋

I couldn’t see any existing bug reports regarding these timeout errors you’re seeing but it may be worth contacting our Support team to investigate this further in case this is a wide spread issue that needs investigating. I don’t think it’s going to be possible to increase the timeout period for that particular action unfortunately.


If Microsoft Dynamics 365 CRM taking longer than 30 seconds to respond, it could be that Microsoft’s servers are receiving an unexpectedly high amount of traffic or encountering issues or temporary downtime at the time that the request is being made. Usually autoreplay would be the solution here but, as you mentioned, it’s resulting in duplicate leads being created. That will be down to the Zap not checking again (after it errors) to see if the lead already exists before attempting to create the lead.

So I think the way to get around this might be to switch from the two separate actions to use a single Find Lead action on v1.17 of the app, but enable the option to create a new lead if an existing one isn’t found (turning it into a Find or Create Lead action). That way, if the action errors it should, in theory, then check again to see if the lead was already created when it’s replayed automatically and prevent it from creating a duplicate lead. Do you think that approach could work? 

Looking forward to hearing from you!

Userlevel 2
Badge

Hi @SamB,

Thanks for your input. I have previously discussed this issue with support, but because of the use of 1.17 app version, there’s not much official support provided as it needs to be the latest version, unfortunately due to issues with the 1.18 version, I can’t use any functions that attempts to write data i.e. POST with the 1.18 version given the payload data is wrong, although find/lookup requests work fine, so it’s sort of a split use between the older app version for writing data and latest for read actions.

Your suggestion around creating the lead from a find is interesting and something I’ll look into, thank you for the tip. I still have concerns around the timeout error being thrown because even with auto replay, eventually this triggers the threshold for automatically disabling a Zap (given these errors are transient but not really following a pattern). I have implemented another Zap using Zapier Manager to alert me to this event, but all this stems from the timeout being fixed at 30 seconds (I think). I also don’t want the find process to neccersarily block a lead from being created. The purpose of leads is multiple under a contact i.e. one contact might have multiple leads. If you are just matching on properties like First Name, Last Name and Email, this could stop a legitimate lead being created if it’s about two different lines of enquiry from two different processes.

I get that Zapier’s HTTP requests need to complete within a reasonable time, but we have another Microsoft Dynamics 365 CRM intergration that offloads these actions to separate queue worker for this exact reason, so it can run for much longer without blocking front end requests. While I get that Zapier tasks can’t run forever either. The “delay for” action exists and can artificially make a task run longer, I don’t really see why we can’t have the ability to set the HTTP timeout value on public apps, given it’s literally just a single value to pass to the HTTP client as an integer. I’m not talking about a massive increase either, 60 seconds might do it, or possibly 90/120 to be absolutely safe. I think 30 seconds is too low as a default and clearly isn’t working for us with the constant reports of an error occurring randomly, when in fact it’s a timeout and the data was created, but now Zapier sees it as a failure and retries it.

The issue we have with this scenario with the timeout not controllable is no matter what we do, we are going to face an issue. We can do nothing and let auto replay do it’s job, but the problem with that is auto replay is being coecred into replaying tasks which potentially didn’t actually fail in the first place. We can turn off auto replay but then the errors happening will automatically turn the Zap off itself eventually causing further problems and even with auto replay on this will also eventually happen as well. You can see the catch 22 in the scenario.

I believe the Dynamics 365 API actually allows for up to 10 minutes, so the API requests can run for longer periods without issue based on official documentation for the API itself.

Zapier really needs to considering allowing the timeout to be increased on some apps. No two Microsoft Dynamics 365 environments are the same and some are going to be slower to respond than others e.g. if you have bigger schema, fields etc. Please let us raise the timeout or increase it in the public app, begging you on behalf of other Dynamics 365 users who user Zapier who I’ve read have had similar reports.

Userlevel 2
Badge

Looking at the find lead action with the optional create step, I can see while the find lead action is limited to certain fields to search across, there is the topic field on a lead which I could implement a unique enough value to ensure I don’t trip up on other leads not related to a process that could be picked up by the search.

I’ll try implementing a Find/Create action instead and see if that improves things regarding errors. I think this will certainly prevent duplicate data if replayed and the extra task usage I can live with (although not ideal) but I still don’t think this will address the other fact of repeated errors that are actually timeouts, will also eventually cause the Zap to be turned off automatically, because of repeated errors, given even with replays the task can still error out with another timeout.

I still strongly highlight the need for looking at the 30 seconds timeout limit, I feel this is the main problem here. I cannot guartenee the API response time or influence it. The payloads are reasonable and don’t involve multiple entities, so realistically, it’s about as small as it can be, If our Microsoft Dynamics 365 environment responds slower than the 30 seconds what can I reasonably do within the parameters available, not a lot.

Userlevel 2
Badge

I implemented the Find action instead of a single create and that should resolve the duplicate data problem with the find step introduced so thanks @SamB for the idea and suggestion. I have further been looking into timeout issue some more.

There’s a possible explanation for timeouts although I can be sure, but I suspect it might be due to workflows/processes running on the CRM environment side during the POST request, triggered by certain values provided in the lead data.

Payload test 1, matching how Zapier has been creating leads with the previous Crete Lead and now Find/Create Lead step amendment.

Time 20.29s. That response time isn’t great, but still under 30 seconds, so should in theory be OK and under the threshold. I did further tests and the times were lower average around 10 seconds instead, but a worst value seemed to be in the 20 seconds areas, but that’s only leaving a buffer of about 10 seconds before Zapier deems the request as timed out.

Payload test 2: This links an existing contact record to the lead, instead of using the ccl1000_createnewcontact field with a true value (this is bespoke functionality within our CRM environment) to essentially create the contact record from the lead without having to create a contact field.

Time: 6.17s. We have a significantly shorter response time, which potentially hints at some form of process or workflow slowing down the first test.


This could suggest that leveraging the workflow that creates a contact upon a lead being created within our CRM system could in fact by slowing down the response overall.

If the timeout total is 30 seconds, the better solution might be to not trigger this process when creating a lead and instead break out the process into two steps.

Step 1: Find/Create Contact (Maximum of 30 seconds):

  • Check if Contact exists based on data such as First Name, Last Name, Email
  • If a contact is found return the contactid for linking to the lead
  • If no contact is found, create a contact with the data provided and return the contactid created

Step 2: Find/Create Lead (Maximum of 30 seconds):

  • Check if the Lead exists based on data such as First Name, Last Name, Email and Subject/Topic (prevents tagging other leads unrelated but with the same personal details)
  • If a lead is found don’t create the lead as it should have already been created
  • If no lead is found, create the lead but link to the contactid provided in the Find/Create contact step.

You should have 2 x 30 seconds to create the contact and lead in two steps instead, currently the contact and lead are being created in Payload test 1 in a single request, which could be where things can be unpredictable.

From API testing with Postman find requests generally seem to be completed under a second, you can further improve the response time by using $select to not return all fields on the record if you get a match, but even without doing this the response was still quick under 100ms in most cases.

It would seem the Create action/creating a contact through a workflow could be slowing down things, so abstracting that out into the Zapier action may be better and more reliable, it adds an extra action and hence task usage, but in an effort to get things in a reliable window with timeouts, it might be the better option.

Userlevel 7
Badge +11

I’m so sorry for missing your replies previously, @jamesmacwhite

Thanks so much for sharing your findings here that’s super helpful context to have. Seems like you’ve got a solution in place for now but I can totally see how it would be beneficial to be able to increase the timeout limit for actions, or even just for specific actions. So I’ve passed your request here over to the Product team. I can’t make any promises around when or if that functionality would become available but we’ll be sure to share news of it here in the Community once it is. 🙂

Reply