# forbid access for crawlers

User-agent: *
Disallow: /private/
Disallow: /files/