Question

Breaking up line ltems into multiple batches


Hi,

I have built two custom integrations (apps) using the Zapier CLI to integrate our Payroll system with our HRIS. For context we have 150 employees and this could grow to 200 in the next 6 months.

The Payroll integration has a trigger that returns a line item with for each employee with the employee’s leave balance. The line items are passed to an action provided by the HRIS integration which updates the leave balance for each employee.

The problem is that the API for our HRIS is rate limited to 60 requests per minute. This rate limit cannot be increased (I’ve asked already) and there is no bulk update method available to update leave balances in a single request. Given the execution time limit for an action is 30 seconds and we are limited to 1 request per second the theoretical maximum number employee leave balances we can update is 30.

As we have 150 employees we cannot even hope to update everyone’s leave balance in a single action and it is not cost effective to loop through each employee and update them one at a time.

What I would like is a way to batch up the line items and process ~25 employees at a time. Any recommendations?


This post has been closed for comments. Please create a new post if you need help or have a question about this topic.

3 replies

Userlevel 7
Badge +8

Hi @warner 

 

I assume that by “there is no bulk update method available to update leave balances in a single request. “ you mean for the 150 employees, but you can update 25 employees at a time? 

 

I think the best way to achieve what you are trying to do is to use a Code by Zapier step where you can divide your data into batches of 25 employees, send them to your HRIS system from within the Code step. Before the code step, add a Delay by Zapier step > Delay after Queue and set it to 1 min so that after the zap runs it would wait for 1 min before running the next zap. 

 

Hope this helps 

Hi @MohSwellam,

The API only supports updating employee leave balances one at a time. Therefore, a single action trying to process 150 line items would have to make 150 requests within 30 seconds, which is not possible.

The grouping of employee leave balances into batches of 25 was an arbitrary starting point. In reality I’d have to tune the batch size based on real-world testing.

Thanks for your help , but at this stage the best solution I can see is to write a custom action that will slice the line items into a JSON object containing one line per batch and then loop over the batches.

This feels clumsy and I’m surprised that the Zapier CLI doesn’t have a more elegant way of handling this scenario.

 

Userlevel 7
Badge +8

Hi ​​​@warner 

 

In this case you will need to use 1 action per employee since the API doesn't support line items