Question

Creating a loop for rss feed and check for new ones

  • 28 December 2022
  • 4 replies
  • 166 views

Hi!

 

I want to achieve the following. I have a RSS-feed. Every time I open the URL it changes content so I can't use the check for new content at the beginning.

 

I want a random item from the RSS-feed, which hasn't been used earlier, to create a WordPress post from it. How can I get just one value from the feed, checks that, if it hasn't been used continue to create a post and if it is used go back and pick another value from the feed and do the same check over and over as long as there are no more items to check. I want a new WordPress post every hour.

 

Can someone help me explaining this? I also have Airtable.


This post has been closed for comments. Please create a new post if you need help or have a question about this topic.

4 replies

Userlevel 7
Badge +14

Hi @rvd20 

Good question.

For some guidance about the concept of looping, check out this help article:

 

Userlevel 7
Badge +12

Hi @rvd20 

You will need to get pretty custom and very advanced to accomplish something like this… but you could do this with some code, a google sheet, and webhooks. Keep in mind a while loop (a loop that runs while a logic isn’t met) could be a bit dangerous in Zapier and consume a lot of tasks if it takes a while to find an article that hasn’t been used. Be sure to set this up carefully and perhaps add some breaks to the zap (not shown in the below)


The Google Sheet should have 2 worksheets. 1 for the import feed and 1 for selected articles. Here is a sample set up https://docs.google.com/spreadsheets/d/1dmbu8V6jKOmUs9GoGXWt4NqG5W6FcvTQH1BctZG_640/edit?usp=sharing 

You can import an RSS feed to Google Sheets using the import feed formula

=IMPORTFEED(URL,,1,100).. where URL is the url to the RSS Feed, 1 tells it to return the headers, and 100 is the number of items to load)

In Zapier you need the following 2 zaps. 
Zap 1:

Trigger: Scheduler - Set up to however often you want to create a post
Action: Get Many Spreadsheet Rows (Google Sheets Action). set it up to start on row 2 (omitting the header and to pull back 100 rows). This will pull back all of the rows on your spreadsheet

Javascript Code Action to Randomly select one of the articles. In the below there are two inputs. rows and headers which is just the headers from your feed separated by 2 pipes (||). 
 

Here is the code to randomly grab one and remove the one that was grabbed which will help later if the same article is randomly selected

let rows = JSON.parse(inputData.rows)
let headers = inputData.headers.split("||")
let formatted_rows = rows.map(row => {
let temp = {};
for (var i=0;i<row.length;i++) {
//process each col
temp[headers[i]] = row[i];
}
return temp;
})

let random_selection_index = Math.floor(Math.random()*formatted_rows.length)
let selected = formatted_rows[random_selection_index]
formatted_rows.splice(random_selection_index,1)
return {selected_article: selected, unselected_articles_raw: JSON.stringify(formatted_rows)};


Action: Lookup a spreadsheet Row in Google Sheets. check off the box to create a row if none is found. Set it up to look up something unique about the article like the title, if nothing is found then it should create a new row. 
 



Paths by Zapier: 1 path for if Zap Data was found is false. In this path, you will create the post using the selected article info from the code block.
1 path for if zap data was found is true. In this path, you will trigger your second zap via a lookup table to run until it finds a new article to post. 

You need to trigger it via a webhook POST request. You will have a different url than what is shown below. You can get it by setting up Zap 2 below. 



ZAP 2: Zap that will loop itself until a new article is found. Most of this zap is similar to the first one. 
Trigger: Catch RAW Hook in Webhooks By Zapier. This url goes into the POST webhook from the previous zap. 

Action: Code block. Similar to the above code block in that it selects a random article from the list provided. 

let json = JSON.parse(inputData.json)
let articles = JSON.parse(json.unselected_articles);

let random_selection_index = Math.floor(Math.random()*articles.length)
let selected = articles[random_selection_index]
articles.splice(random_selection_index,1)
return {selected_article: selected, unselected_articles_raw: JSON.stringify(articles)};

then repeat the google sheets find/create row and the path actions, just like what was in the first zap. 

Thanks for explaining @GetUWired !

 

There are only 2 issues I have with Google Sheets and importing RSS-feed:

  • It does not import more then 250 records. My product feed is over 1600 records
  • It does not automatically check if there are new products being added

Also sometimes there are products being added who have the same productname and description (because of a different price). Can those automatically be removed from Google Sheets?

Userlevel 7
Badge +11

Hey @GetUWired! Just wondering if you have any thoughts or workarounds for the import limitations that @rvd20 mentioned in their previous reply?

CC’ing @Troy Tessalone here as well in case he’s got any creative ideas?