User-agent: *
Allow: /
Disallow: /admin/
Disallow: /dev/
Disallow: /monitor/
Disallow: /shortener/