Skip to main content
Question

How many tasks do I need for scraping forum posts?

  • February 21, 2025
  • 2 replies
  • 14 views

Hi,

 

I want to use Zapier to invoke a Octoparse task to scrape and get the lastest posts on a forum every day. Then analyze the contents of each post via AI model(such as OpenAI GPT 4o) and generate a report.

 

In such a case, how many task do I need in my Zapier account? Or more accurately, which plan should I use? Can a single task process unlimited posts? Or do I need 100 tasks to process 100 posts?

 

Since the posts are on a forum, I cannot know the number of posts need to be processed accurately.

This post has been closed for comments. Please create a new post if you need help or have a question about this topic.

2 replies

michaeltoth
Forum|alt.badge.img+1
  • New
  • February 21, 2025

Hey ​@ccw 

You can read more about how Zapier measures tasks in this article: https://help.zapier.com/hc/en-us/articles/8496196837261-How-is-task-usage-measured-in-Zapier

For your use case, you will use tasks each time the Zap runs (i.e. for each post you process). The number of tasks will depend on the number of steps that your Zap has. 

For an accurate estimate, I suggest you build the Zap, and then run it for a single post. Then, you can run it for an hour or a day to get a sense of volume. 


JammerS
Forum|alt.badge.img+6
  • Zapier Staff
  • February 25, 2025

Hi ​@ccw,
 
We just wanted to see how everything is going with your Zap. Feel free to reach out if you need further assistance with your Zap. We're glad to address any concerns and assist you.
 
We're looking forward to your response.