User-agent: * Disallow: pages/ Disallow: index.php Disallow: database/ Disallow: 1/