Resurect an Actor run
# apify-platform
g
I develop an actor that runs on Apify platform. The actor will basically take a list of URL in the INPUT and open each one of them. Simple. I run the Actor and it timed out. Fine. Then I resurrected it and strangely enough, it started loading the INPUT list of URL from the begining. Isn't it strange ?
a
message has been deleted
a
You need to persist the progress in some way for example, by using a RequestQueue, a Key-Value Store (KVS), your own external database on a VPS, or even by extracting logs (e.g., if you log something like Processed page 10/500, although I wouldn’t recommend relying on logs since that approach is rather fragile), etc.
g
The Apify Dataset is not enough ?
And if I understand you correctly, when you say "persist the progress in some way" it means that by default, resurrecting will start from scratch ? I need to handle the specific case of resurection in my actors ?
a
Exactly you need to persist the progress somehow, because Apify doesn’t do that automatically. You have to handle it yourself, whether the user aborts, a timeout occurs, or Apify internally migrates the run to another server for some reason. That’s why, before starting a run, you should check the progress using whichever option you’ve chosen.
g
Tks for your answer. very clear ! Are there any Actor implementation example somewhere ?
a
@guillim_ just advanced to level 4! Thanks for your contributions! 🎉
g
(that you would be aware of)
o
yeah, for handling URLs you should use requestQueue And for saving some important data You can use useState(): https://crawlee.dev/js/api/next/core/function/useState Also see this article for inspiration: https://docs.apify.com/academy/expert-scraping-with-apify/solutions/handling-migrations