User-agent: * Disallow: /admin Disallow: /fetch # internal SSRF testing tool requires special auth header to be set to 'true'