Hi community! 👋 I'm Harsh, a 17-year-old develop...
# 🗣general-chat
y
Hi community! 👋 I'm Harsh, a 17-year-old developer building Nexus AI , an autonomous execution layer for AI agents. Rather than exposing a single scraping API, Nexus routes requests through different retrieval strategies depending on the target: Tier 1 → Firecrawl (default): Fast, markdown-first retrieval for public documentation, blogs, and standard websites to reduce token usage before LLM processing. Tier 2 → Dynamic rendering: If the page appears JavaScript-dependent (empty DOM, loading states, etc.), Nexus automatically escalates to a rendering-capable provider. Tier 3 → Fallback retrieval: If earlier attempts fail (timeouts, WAF responses, rate limits, etc.), Nexus can retry through another provider to improve reliability. The goal isn't to compete with Firecrawl—it's to reduce the orchestration code AI developers write by handling provider selection, retries, workflow execution, and structured extraction behind a single API. I had one question for builders who use Firecrawl in production: If an LLM autonomously discovers a URL during a workflow, what are the recommended  legal/compliance guardrails I should implement? For example: * Should I maintain a domain allowlist or blocklist? * Should my Terms of Service explicitly place responsibility on users to comply with website terms and applicable laws? * Are there best practices for handling websites that reject automated access? * What policies or safeguards would you expect from a product built on top of Firecrawl? I'm trying to build Nexus responsibly from day one, so I'd really appreciate guidance from people who've shipped products using Firecrawl. Product: https://nexus-ai-production-4eb6.up.railway.app GitHub: https://github.com/Harshpatil77/Nexus-AI Thanks in advance! 🙌