How do I set general settings data that will be accessible to the request handler without passing it to the user data of every request. I thought there will be away to add it to the context when creating the crawler but I can't find any parameter that does this.

Hi, @toludare You're right; there isn't a configuration option that allows you to do that. You can use KVS to store data and access it in the handler. Or, as a workaround, you can use the private method `_use_state`; we'll probably make it public in the future.

python
async def main():
    crawler = BasicCrawler()
    data = await crawler._use_state()
    data['foo'] = 'bar'

    @crawler.router.default_handler
    async def handler(ctx):
        data = await ctx.use_state()

        print('State:', data['foo'])

await crawler.run(["https://www.example.com/"])


if __name__ == "__main__":
    asyncio.run(main())

Thank you. I will use

use_state

.

use_state

will be public after next realease https://github.com/apify/crawlee-python/pull/1799