# Make changes for all web spiders
User-agent: *
Disallow: /admin 
Disallow: /common 
Disallow: /retail 
Disallow: /webalizer