i'm working on a project for an individual who works with golf course data. all things related to the course, yardages, tee times, superintendents, slope ratings, etc. older gentlemen has been manually "scraping" data for years. i'm new to the scraping tools, but it sounds like Apify would be a great solution to run and gather all this data automatically. any suggestions on actors to use or approach methods here? i'm considering to build this man an API to pull data from a foundations database, then run the automation for websites that have data not listed in the foundations db.

So do you want a custom solution or want to create your own

I'll be creating my own custom solution for this guy.

Can you provide more information so I can help you?

currently, he pulls data from a golf foundation DB. then he manually goes to course websites and pulls similar data that is not included in the DB. from there, he compiles into a spreadsheet and distributes to clietns in csv format. we want to build an API that essentially pulls all this information automaticall from the DB, and webscrapers to handle the other course info that's not covered. ultimately, he plans to market this data to other businesses, so we want to have a custom built RESTful API that can compile everything into a SQL data base so it can be easily queried

Website link can I get?

i guess my main challenges won't be so much the NGF db, but the scrapers for the various types of websites. so i'm faced with building a resilient scraper that's scalable and maintainable

And in what aspect you want help?

https://usgolfdata.com/ i'm in the learning phase here. i'm offering my proposal to him, and may need some assistance at certain points throughout the project. need to build my network of experts and learn as i go.

Sure, I would love to help

Hey there! I’ve successfully built a scraper in Python using beautifulsoup and selenium. Now I’m realizing it might be better to be in firecrawl. Any advice?

@christopher_pe just advanced to level 1! Thanks for your contributions! 🎉

but why?

Well it sounds like firecrawl has a lot already built into it to handle a majority of the data gathering I need. My tool I built feels very manual. Finding every css block that needs to be parsed.