Skip to main content
Question

Paginating results from a trigger

  • 28 May 2024
  • 1 reply
  • 23 views

I’m trying to determine best practices when a user sets up a trigger. Instead of polling ALL records from our platform, shouldn’t we limit the query in some way? Perhaps limiting to new data since the last poll or at least the last 24 hours. I know Zapier de-duplicates based on ID. Here’s my code:

const options = {
  url: `https://XXX/${bundle.authData.event_id}/sales-report?checkout_timetgt]=2000-01-01&per_page=2000`,
  method: 'GET',
  headers: {
    'Accept': 'application/json',
    'Authorization': `XXXX`
  },
  params: {

  }
}

return z.request(options)
  .then((response) => {
    response.throwForStatus();
    const results = response.json;

    // You can do any parsing you need for results here before returning them

    //return results.data;
    const lists = resultse"data"].map((item) => {
      return {
        ...item,
        id: item:"record_id"],
      };
    });

    return lists;
  });

Shouldn’t I limit my 2000-01-01 to perhaps yesterday? Then, I can limit my per_page to 100 or so. Lastly, I could then determine if I have more than 100 in the call and loop within the “return z.request...” area. Is that how I should be looking at this?

1 reply

M
Userlevel 4
Badge +9

Hi there 👋

 

This has been addressed via our support channel at https://developer.zapier.com/contact but I’ll post an answer here as well 👍

Polling results should definitely be limited using a sort or limit parameter, depending on what your API endpoint has available. 
 
Limiting your results is different to pagination however, which is described here: https://platform.zapier.com/build/pagination-trigger and relates to using a trigger to populate a dynamic dropdown, when the user can click Load More to load additional pages of data.
 
For regular polling triggers, the deduplication process here: https://platform.zapier.com/build/deduplication will need the results sorted in reverse chronological order and the params to limit those results for every poll request.
 
Polling requests and any scripting are limited to 30 seconds per poll, or the trigger will timeout. That and other operating constraints are noted here: https://platform.zapier.com/build/operating-constraints 


It is advisable to limit the number of records a trigger’s poll returns for that reason. The Zap does not need to see ALL existing records on every poll, only new records are meant to trigger the Zap. Often the polling request is limited to the 50-100 most recent records for this reason.
 
This does require a method of sorting on the endpoint though, typically by date_created, though based on this example, using checkout_time might make more sense, if those can be sorted in desc so the most recent checkouts are listed first. 
 

Reply


Terms & Conditions