Skip to main content
Question

Incorrect Results When Using Zapier ChatGPT

When I use my ChatGPT ZAP to upload a TXT file for analysis and response, the document uploads properly and I am able to get responses. The problem is that the responses are incorrect. I have tried multiple different variations of how to phrase the questions, different version of ChatGPT, etc. All result in incorrect answers. 

However, when I login to ChatGPT directly and upload the same document and ask the same questions, I get the proper answers. Not sure why using the Zapier process yields incorrect results. I have another Zap I use to analyze PDF files and that ZAP works perfectly. 

Is the Zapier ChatGPT limited or work differently than chatting directly with ChatGPT? If so, is there a way to get responses as if you are chatting directly with ChatGPT using Zapier? Can I create a Python script or some other solution to get accurate results? 

I also noticed that when I am chatting directly, I can upload an Excel file for analysis but cannot do so using the Zapier integration. I am just trying to understand why there is such a difference in responses between Zapier and chatting directly with ChatGPT? 

13 replies

Troy Tessalone
Userlevel 7
Badge +14

Hi @smccarthy956 

For us to have more context, post screenshots with how your Zap steps are outlined and configured.

 

 

https://help.openai.com/en/articles/8983675-what-types-of-files-are-supported

⚡ Troy Tessalone - AutomationAce.com | Premier Certified Zapier Expert | #1 Zapier Community Contributor
smccarthy956
Userlevel 1
Badge +1

Here are the steps:

 

Troy Tessalone
Userlevel 7
Badge +14

@smccarthy956 

Is the mapped variable with the file url a publicly accessible file?

Help for using files in Zaps: https://help.zapier.com/hc/en-us/articles/8496288813453

 

⚡ Troy Tessalone - AutomationAce.com | Premier Certified Zapier Expert | #1 Zapier Community Contributor
smccarthy956
Userlevel 1
Badge +1

Yes, the file uploads no problem and chatgpt can access it. I have tested it. 

Troy Tessalone
Userlevel 7
Badge +14

@smccarthy956 

Things to try…

  • Copy/paste the returned file URL from Knack in Step 1 into a new/different private browser to see if it starts a direct download of the file. (If not, then it means the file url is not publicly accessible.)
  • Try using a different Model.
  • You can try adding a Delay step to make sure the file finishes uploading before asking questions about it.

 

 

⚡ Troy Tessalone - AutomationAce.com | Premier Certified Zapier Expert | #1 Zapier Community Contributor
smccarthy956
Userlevel 1
Badge +1

Yes, the URL works if I copy and paste it into a different browser that is not logged in. 

Troy Tessalone
Userlevel 7
Badge +14

@smccarthy956 

Things to try...

  • Try using a different Model.
  • You can try adding a Delay step to make sure the file finishes uploading before asking questions about it.
  • Try uploading the file directly using the New Files field instead of the previous ChatGPT Upload File Zap action
  • Make sure the question being asked in the Message field is relevant to the file being uploaded.

 

⚡ Troy Tessalone - AutomationAce.com | Premier Certified Zapier Expert | #1 Zapier Community Contributor
smccarthy956
Userlevel 1
Badge +1

Have tried all of the above with the same results. Some answers are correct and some are just plain wrong. 

Troy Tessalone
Userlevel 7
Badge +14

@smccarthy956 

Have you tried including additional guidance for the AI?

  • Include context before/after the mapped variable in the Message field
  • Include Assistant Instructions
  • Response Instructions

Make sure to check the field descriptions for guidance.

 

 

⚡ Troy Tessalone - AutomationAce.com | Premier Certified Zapier Expert | #1 Zapier Community Contributor
smccarthy956
Userlevel 1
Badge +1

Going back to my original point, if I upload the same file to chatgpt’s interface, it answers everything perfectly even if I provide no context. I don’t understand how Zapier is getting different answers when this works perfectly using the standard Chatgpt interface. The data is not that complicated. 

Troy Tessalone
Userlevel 7
Badge +14

@smccarthy956 

When you are using the ChatGPT web app it can use context across chats.

The way you have the ChatGPT Zap action step configured is running in a silo.

So it’s not a true 1:1 comparison and that may account for some of the variance you are seeing with the results.

⚡ Troy Tessalone - AutomationAce.com | Premier Certified Zapier Expert | #1 Zapier Community Contributor
smccarthy956
Userlevel 1
Badge +1

So how do you get it to work like chatgpt if I was using the interface?

Troy Tessalone
Userlevel 7
Badge +14

@smccarthy956 

Try using these fields and check the field descriptions.

 

⚡ Troy Tessalone - AutomationAce.com | Premier Certified Zapier Expert | #1 Zapier Community Contributor

Reply


Terms & Conditions