Frankly, I am having a host of problems with the S3 zap on the Upload file. 1) There is a omission in the s3 connect dialog that essentially prevents you from creating a S3 connection to the US-East-2 region. The drop-down does not have this option. If you had buckets in that region, it basically means you cannot upload anything using this Zap. Zapier apparently hasn’t prioritized this fix very high - at least there is no ETA on a fix. In any case, I got around this by moving my bucket to us-west-1, a workaround that took me some 3-4 hours of work and is probably infeasible for anybody with a complicated AWS deployment on us-east-2. 2) A much more serious and showstopper issue is that the upload zap seems to be randomly chopping up and trunctating the file being uploaded. This is the file loaded with the zap - Note the size: 89.2 kb
This is the same file that has been uploaded through S3 console. Note the size is 181 kb.
Essentially, thanks to the truncation, my .docx and .pdf files are essentially corrupted and unusable in S3.
I am a little baffled by how such a basic error is happening on a vanilla upload to S3. I am also assuming this will be a widespread problem unless I am missing something obvious. The original file is in a Google Drive if that is relevant (this also correctly shows 181Kb size)
Can folks comment on their usage and whether anybody else is having this problem and what you have done to fix it…
My zap is about as basic as it gets. 1. It is triggered by a new line being created in a Google sheet. The Google sheet has the info for the upload step, in terms of s3 bucket, s3 key and the source file URL (Google Drive) 2. Second step is the S3 zap. which should be doing the upload. The zap upload success message also shows the wrong size (about 91,0xx bytes) which is about 89.2
Page 1 / 1
Hi @anatesanStream
For us to have full context, post screenshots with how the steps are outlined and configured.
Is your bucket in a different region than the one you selected when connecting your S3 account to Zapier? If so, you will need to reconnect and select the matching region.
Note that often (depending on the security policy of the S3 account you have connected to Zapier), we are able to see buckets from regions other than the selected region, but we are not able to interact with them. For example, if you have a bucket in US East (Ohio) (which is us-east-2), you may be able to select that bucket in a drop-down, but if you selected us-east-1 when you connected your account, we won't be able to upload files to that bucket.
If you are still having trouble after this, please drop us an email so we can troubleshoot further with you!
Zapier’s S3 connection dialog shows a list of regions that you specify on the connect. In this drop-down list, us-east-2 is NOT present. So, you are out of luck if your buckets are currently located in us-east-2.
If your buckets are in us-east-2, There is not much you can do to get the S3 upload zap working other than moving your buckets to a different region. In my case, I moved them to us-west-1. But other parts of my deployment and code on the AWS side were affected and all this drama was solely due to zapier simply not allowing a S3 connection to specify 1 of 4 US regions! This seems like a bug/ommission on their part. But it has real ramifications to 25% of the AWS user community who have buckets in us-east-2.
picture below is of the drop-down on connection… us-east-2 does not show up in this drop-down at all. You can try this yourself. Create a new connection in the S3 upload zap. Try to set the region to us-east-2… it's missing.
This is however, NOT the main problem I am reporting. I worked around this by moving my buckets to a supported region. I still need a fix to the problem that my uploaded files seem to be getting truncated by the time they get to my S3 buckets and the file is getting corrupted (unless I am missing something)
OK - I created a simple Test zap to illustrate the problem. Here are all the details and screenshots… Not sure how I simplify this test case any more, but happy to hear your thoughts.
This is the zap… (looks like file does not always get truncated… It can even expand!)
This is the screenshot of the source file I tested with - this screenshot is details from step 1 of the zap above
Source file details on step 1 of the zap
Source file details on Step 2 of the zap (before I hit Test)
Source file details as shown in the Step 2 before I hit Test button
This is the output AFTER I hit the test button and the S3 upload had finished
S3 File Upload results - successfully uploaded but with the wrong file size
After I ran the zap, I could see the file on my S3 bucket. I downloaded the file from S3 and compared it with the original source file. You can see the difference in file sizes
The S3 file is corrupted… I cannot open it in my app. I can however correctly open the source file as a png file.
If I am doing something obvious wrong… please LMK. I am completely stuck with this problem on the zapier front and its an important part of my overall automation flow.
If Zapier support is looking at this, you can try this out yourself and presumably you can reproduce the same problem.
Finally, are there other AWS S3 users who are using this upload Zap? These problems seem so basic (both the connection dialog missing a major US AWS region + files getting mangled on an upload) that I am wondering how more users are not encountering the same behaviour.
Would also love to hear if I am missing something obvious which I can quickly fix at my end.
If you are actively using this zap, please comment on your usage and if your file uploads to S3 are all working ok. Thanks.
@anatesanStream
Can you post screenshots with how your Zap steps are configured in EDIT mode?
@anatesanStream
Also, best to submit feedback and feature requests via a ticket to Zapier Support for it to be properly logged: https://zapier.com/app/get-help
@anatesanStream
Looks like you are using a GDrive file VIEW link, rather than a GDrive File direct download link.
I verified this by modifying my test-zap as shown below… I added a new Formatter step to extract the object ID and recomposed the URL in format (1) above in the s3upload zap for the file location.
Revised test zap with a new Formatter step to extract the file_object_id from standard Gdrive URLs
Would be good if zapier included all this in the standard doc for this zap.
To send a file in a Zap, you need to use a file object or certain types of URL links.
File objects are the bundles of data that represent the actual file. This is different from the name of the file or other file attributes. For example, the actual image of an apple is the file object, while the file name might be “apple.jpg”.
If you’re using a URL link to a file, it has to be:
Publicly-accessible: it can’t require a login to be accessed.
Downloadable: the app you’re sending it to must be able to upload it to their server.
Some apps will create a text (.txt) file if you enter text values into a field that accepts files.
Limitations
Some file fields may not accept text.
Files larger than 100 MB may result in a time out error.
You can only add 1 file per field. If you need to add additional files, you’ll need to add separate action steps.
Tip
You can test your URL link by opening it in a private or incognito browser window. If it prompts you to download a file, the link can be used. If it prompts you to log in to an app, loads directly in the web browser instead of downloading, or results in an error code, the link can’t be used.
1. Check if your previous step provides a file
If a previous step has a File field with an (Exists but not shown) value, that is a file object.
If a previous step has an Attachments field, it may have a file object or a URL link to a file that you can use.
2. Add a file to your action step
If your field accepts files, you will see a file icon.
Insert your file into the field by selecting it from an earlier step.
The problem is that if you deal with Google Drive, you will get many URLs - attachment links, download links, parent links and on and on. If you deal with Google Mail to get attachments, its even worse - there are a huge number of URLs to puzzle over.
So, practically, its pretty much a guessing game to figure out what will work for your use case and zap, even if you read generic articles like these.
This is not a Zapier-induced problem, but it was a non-trivial problem to figure out what to use. For my use case, I had to actually reconstruct the right URL myself in Google Sheets/zap by breaking apart one of the standard Google Drive file URLs to get the object ID.
In any case, I am glad that the pointer you gave me helped me get to a solution that works…. Thanks again.
Tip
icon.