How can I index all pages of a Shopify products.json endpoint? I'm building a chatbot for a Shopify store. The product data is available at example.com/products.json, but Shopify paginates the results (30 products per page) using query parameters like ?page=1, ?page=2, etc. When I add these URLs as Knowledge Base sources, Botpress strips the query parameters and treats them all as the same URL — so only the first page gets indexed. Is there a way to: Force Botpress to respect query parameters when crawling URLs? Automatically re-crawl a file-based KB source on a schedule? Use the Botpress API to programmatically update a KB source (so I can automate it via n8n or a cron job)? The store has ~180 products across 6 pages. I need all products in the KB with automatic updates when inventory changes.

yo u want prices and eanfrom idealo fnac cdiscount .. i can slap together a simple script or automation u can run urself no coding needed python or n8n style whatever u want u want it to run every day and hour? plz drop more deets like how many sites, how often u need fresh data

unfortunately i think the only way to do that is through some sort of automation like make or n8n

why dont you use tables for the products? including the price, name and anything you want to show I dont know if this would solve your issue but I think it might be a good idea

Well. Prices changing and I also want to show img. I don’t know if the tables can include a img tab

You can include an image in the table and connect the table via API to update the prices

Oh ok might try that with a n8n crawler then