Zapier looping skipping first row storing in SQL database
Hello Friends,
I need help related to looping multiple records trying inserting multiple rows into SQL database, but the first record sometimes gets skipped, while the rest insert fine.
I am getting data from emailme form which i wanted to store in SQL database but data i am storing in rows. I mean to say if my form build with 10 fields then those 10 fields data (label as question and value as answer) is storing in database in different rows with some repeated information.
Like form fields are event_id, event_location, event_name, what is your name, what is your age, where do you live.
and to achieve this i have multiple task in one zap with loop to store all the fields data in different rows.
I am using multiple zap steps to achieve this, like i can say 16 steps in one zap, steps are as mentioned below
Webhook to catch the submitted data
Formatter by zapier to get event id only
Formatter by zapier to get event name only
Formatter by zapier to fix multiple images been uploaded
Formatter by zapier to get all questions
Formatter by zapier to get all answers
Delay by zapier (added new step to add delay before loop started)
Loop by zapier to loop through all the fields (questions and answers)
SQL server - Find Row via Custom Query to identify if record already submitted ( if not submitted it will go towards Add New Row and if submitted then it will go towards Update Row)
Path - to split into paths
Path - if record not exists
Code by zapier if field value is empty then put NA as answer
SQL Server New Row
Path - if record exists
Formatter by zapier if field value is empty then put NA as answer
SQL Server Update Row
Please help if its possible to store all the records in database everytime.
FYI: Sevent Step Delay i have added recentlly. Because after my research i found Zapier sometimes starts the loop before the array is fully built or before index 0 is ready, that is why sometimes first loop is skipped. To fix this have added delay before the loop
Page 1 / 1
Hello @kitchen.jam
You can probably cut this down to 5 steps and avoid the missing row issue. I believe it's how the loops are fed that causes the issues. I suggest using a code block to fix the shape of the data before looping. There are a couple of approaches use a single SQL step that inserts only if not there and fetch what’s there and filter it with Python code.
I think the first approach is best and here’s and idea of how to do it. (I have validated the code for errors, it’s just to show what’s possible)
Step 1. Code by Zapier (Python) “Build Rows”
Inputs you provide to the code step event_id, event_location, event_name, question and answer
# Returns rows like: # [{event_id, event_location, event_name, question, answer}, ...] from typing import Any NA = "NA"
def to_text(v: Any) -> str: if v is None: return "" if isinstance(v, (list, tuple)): return ", ".join(str(x) for x in v if str(x).strip()) return str(v)
def labelize(k: str) -> str: import re k = k.replace("_", " ").strip() k = re.sub(r"(?<=[a-z])(?=[A-Z])", " ", k) return k[:1].upper() + k[1:]
event_id = (input_data.get("event_id") or "").strip() event_location = (input_data.get("event_location") or "").strip() event_name = (input_data.get("event_name") or "").strip() if not event_id: raise Exception("Missing event_id")
rows = [] exclude = {"event_id","event_location","event_name"} exclude |= {k for k in input_data if k.startswith("zap_meta_")}
i = 0 for k, v in input_data.items(): if k in exclude: continue val = to_text(v).strip() or NA rows.append({ "event_id": event_id, "event_location": event_location, "event_name": event_name, "question": labelize(k), "answer": val, "order": i, }) i += 1
MERGE dbo.FormResponses AS target USING (VALUES (?, ?, ?, ?, ?)) AS src(event_id, event_location, event_name, question, answer) ON target.event_id = src.event_id AND LOWER(LTRIM(RTRIM(target.question))) = LOWER(LTRIM(RTRIM(src.question))) WHEN NOT MATCHED THEN INSERT (event_id, event_location, event_name, question, answer) VALUES (src.event_id, src.event_location, src.event_name, src.question, src.answer);
Map from the loop value event_id event_location event_time question answer
Optional DB-side guard
CREATE UNIQUE INDEX UX_FormResponses_event_question ON dbo.FormResponses (event_id, question);
I hope this makes sense.
Best of luck, Badger
Hello Badger, thank you for your response.
Actually i don’t wanted to use code by zapier because in my previous topic i asked to optimize the tasks, basically i wanted to reduce the number of tasks.
So i was suggested to replace code by zapier with Formatter.
this issue where first row is skipping sometimes it was happening before code by zapier replace with formatter and its still there.
so i was looking around that what is the issue and i found
Zapier sometimes starts the loop before the array is fully built or before index 0 is ready.
Result: First loop (index 0) is skipped or half-populated.
and to fix this i have added the delay before the loop, to ensure that all data loaded before looping starts. But not working for me.
Hi @kitchen.jam
For us to have true context, post screenshots showing:
how your Zap steps are outlined
how your Zap steps are configured in the CONFIGURE tab while in EDIT mode with the field mappings visible so we can see the field types
the DATA IN/OUT you are trying to use up until the issue with the looping
Okay let me explain in details (and let me know if you need more detail)
Webhook to catch the submitted data from emailMe Forms
Formatter by zapier to get event id only
Formatter by zapier to get event name only
Formatter by zapier to fix multiple images been uploaded
There are few more steps like this to fix images
Formatter by zapier to get all questions
Formatter by zapier to get all answers
Delay by zapier (added new step to add delay before loop started)
Loop by zapier to loop through all the fields (questions and answers)
SQL server - Find Row via Custom Query to identify if record already submitted ( if not submitted it will go towards Add New Row and if submitted then it will go towards Update Row)
Path - to split into paths
Path - if record not exists
Formatter by zapier if field value is empty then put NA as answer
SQL Server New Row
Path - if record exists
Formatter by zapier if field value is empty then put NA as answer
SQL Server Update Row
This post has been edited by a moderator to remove potential personally identifiable information (PII). Please remember that this is a public forum and avoid sharing personal or potentially sensitive details.
@kitchen.jam
Can you post screenshots showing the DATA OUT from Zap step 1, so we can see how the data is originally structured?
TIP: Click field labels to see tooltips with more info.
NOTE: Max # of Loop Iterations is 500, but you have it set to 5,000
Hello Troy, thank you for your response, sure here is the screenshots from first step
You can open a Zap Run history to see the details including the DATA IN/OUT for each step to help you trace the data flow and troubleshoot to confirm each Loop iteration is being processed.