The robots.txt example exposes sensitive-looking paths and may be mistaken for access control
Source references: 2The example labels `/admin/`, `/api/`, and `/private/` as areas to “block.” robots.txt is a public crawler hint; it does not stop browsers, attackers, or noncompliant bots, and listing paths can make them easier to discover.
If copied and relied upon for protection, administrative, API, or private pages may remain directly reachable while compliant search crawlers stop visiting them.
This is a configuration example under “Crawlability,” not implemented access control, but it explicitly lists admin, API, and private paths in a public robots.txt. Only compliant crawlers honor these directives; browsers and malicious visitors are not blocked. Copying it could disclose path names and create a false sense of protection. Users can ask the author to state that these areas still require authentication and server-side authorization, and to avoid listing sensitive paths unnecessarily.
```text# /robots.txtUser-agent: *Allow: /# Block admin/private areasDisallow: /admin/Disallow: /api/Disallow: /private/Show 1 other places
**robots.txt:**```text# /robots.txtUser-agent: *Allow: /# Block admin/private areasDisallow: /admin/Disallow: /api/Disallow: /private/