Security
Website crawler safety
Website-to-MCP is powerful because it starts from real public pages. It also needs strict limits so browser discovery does not become unrestricted automation.
Implementation
Path to ship.
Guide
Safe discovery
Safe website discovery reads public content, extracts useful links, and produces bounded tools for search or inspection. It should not silently submit forms, bypass login, or crawl arbitrary third-party domains.
Astrail labels website-generated behavior so teams can tell when a tool came from browser inspection rather than a deterministic API endpoint map.
Guide
Production path
Use browser discovery to map the workflow. Once the workflow is important, move stable actions into explicit APIs, reviewed MCP tools, or SDK methods with auth and logs.
This keeps the speed of website-to-MCP while avoiding a production surface that depends on fragile page behavior.
FAQ
Common questions.
Should website-to-MCP submit forms automatically?
Not by default. Public read workflows are the safer starting point. State-changing browser actions should require review and policy.
Why block private network targets?
Blocking private targets prevents website discovery from reaching internal services, loopback apps, and cloud metadata endpoints.