Best answer

Read data from RSS feed every hour?


I need to get/parse data from a public RSS feed, which is done via trigger in Zapier. However I need it to run on a limited schedule (only between 8am-4pm on weekdays). AFAIK the latter is accomplished via “Schedule by Zapier” + “Filter by Zapier” and the schedule is also a trigger. I’m pretty sure I can’t have 2 triggers in 1 zap.

 

So either (1) I use the “Schedule by Zapier” as my trigger and then get data from the RSS feed downstream, or (2) get data from the RSS feed as the trigger and limit the schedule downstream. Either way I’ve spent over 6 hours trying to figure it out with no luck. Has anyone created a similar zap so far?

icon

Best answer by laurie 29 July 2021, 19:39

Hey @a1angeles!

Can you share more about the data you’re trying to pull from the RSS feed and where you’re going to send it? For example, do you want to collect it all between 8-4 and the release it all at once? Do you want to pull every item in the feed and release it one at a time? This part of the workflow would help me point you into how to get this completely working the way you’re after.

 

But, here’s a good start for the time and RSS feed part: 

Start with your Schedule trigger and trigger every hour 

Add a Filter step that checks if the current time is between 8 am and 4pm:

Next, you can add a Webhooks by Zapier step using a GET. In the GET, the URL you’ll go to is your RSS feed.

That will pull *everything* in the current feed, so depending on what you want to do with that information, there will be a few ways to work with that...

View original

10 replies

Userlevel 6
Badge +6

@a1angeles you can use the filter app and integrated time function as second step in your zap

Here you find in general more infos to times: https://zapier.com/help/create/customize/insert-the-time-your-zap-runs-into-a-field

1. create a filter step

2. use an operator to compare pub date: 

 

@Wemakefuture Your reply is a good starting point but there’s still a lot of work to be done. In your example Pub Date has to be reformatted in an earlier step/task so that the variable “HH:mm” would make sense to Zapier.

I’m getting closer to making my original goal work but I’m trying to find more guidance on what can be entered into the variable fields. But I do appreciate the help.

In my original question I mentioned 2 options:
 (1) Using “Schedule by Zapier” as the trigger
 (2) Using “RSS by Zapier” as the trigger

 

Thus far I’ve tried using option 2. The reality is that the RSS feed/source updates 2x an hour, so this task alone consumes 1440 tasks in a 30-day month and I have the starter (750 tasks/mo) plan.

If I could restrict the option 2 trigger to run between 8am and 4pm that number goes down to 480 tasks. Better yet, the clients are fine with just 4 updates a day, which translates to 120 tasks in a month.

 

Since option 2 consumes way too many tasks I have no choice but to go with option 1. So far it seems like I can trigger the zap to run weekdays at 4 specific times, so if what I entered in the “Time of Day” field below is accepted, then I should be in good shape:

 

Now the only gotcha is that I can’t find a way to have Zapier get/parse the data from the public RSS feed I’m using, since I can’t seem to find a version of “RSS by Zapier” that is not a trigger. Does Zapier have a way to get/parse data from an RSS feed as a 2nd or 3rd step in a zap?

Userlevel 3
Badge

Hey @a1angeles!

Can you share more about the data you’re trying to pull from the RSS feed and where you’re going to send it? For example, do you want to collect it all between 8-4 and the release it all at once? Do you want to pull every item in the feed and release it one at a time? This part of the workflow would help me point you into how to get this completely working the way you’re after.

 

But, here’s a good start for the time and RSS feed part: 

Start with your Schedule trigger and trigger every hour 

Add a Filter step that checks if the current time is between 8 am and 4pm:

Next, you can add a Webhooks by Zapier step using a GET. In the GET, the URL you’ll go to is your RSS feed.

That will pull *everything* in the current feed, so depending on what you want to do with that information, there will be a few ways to work with that...

Thanks @laurie for the quick reply!

In a nutshell, we’re trying to get local ocean/water temperature data that will be displayed on digital signage screens around the Scripps Institution of Oceanography (SIO). Stuff like local temperature, humidity, wind, etc. is easy to get via our digital signage platform, but water temperature data is not readily available.

SIO operates buoys offshore that record a lot of data including water temperature, and the easiest way for me to access the data is using a public RSS feed which is updated 2x an hour.

The folks at SIO will be happy if the water temperature data on their digital signage screens updates 4x a day, and that limit will save a lot of tasks in my account. And there’s no need to run the zap at, say, 3AM when no one is at SIO and the displays are off anyway.

I’ll experiment with the Webhooks by Zapier GET. That might be the missing piece of this puzzle!

Userlevel 3
Badge

Oh, that’s way easier than what I was preparing for. When you do your GET in the webhooks, each item comes back numbered. So, you can just map the details for item1 and that will send the most recent update each time.

Hi @laurie . I’m stuck at the Webhook/GET action. I entered the URL for the RSS feed and “Retest and Review” works, but when I move on to “Retest and Continue” the Loading... notification stays on the screen along with the spinning horseshoe.

This is the URL for the RSS feed: https://www.ndbc.noaa.gov/data/latest_obs/46254.rss
Maybe this particular RSS feed is not GETtable?

Userlevel 3
Badge

@a1angeles As long as you can load the URL in a browser window, we should be able to get it. If it’s working on one retest option but not the other, it’s likely just a browser hangup (or possibly internet gremlins :) )

Did items from the feed pull in on the Retest and Review? If so, you can just move on to your next step and map those fields.

Hi @laurie . I think it’s working. For now I set it to run at 8am and 12pm so I’ll check in tomorrow.

 

The last “surprise” I had to deal with was formatting. When using “RSS by Zapier” as the trigger, I would get clean, pretty text for parsing. The Webhook/GET is functional, but the water temp data only appeared in this format:

<strong>Water Temperature:</strong> 70.7&#176;F (21.5&#176;C)<br />

I use Formatter → Text → Split Text to locate the data I’m after (70.7 in the example above) and then do it all again to remove the extra data after the temperature, which unfortunately uses an extra step. Not ideal, but at least it’s working.

My last message is a different issue so I’ll end this thread. Thanks @laurie for your tip on the webhook. That solved a large chunk of the challenge.

Reply