firq-dev-website/public/robots.txt

User-agent: *
Disallow: /assets/data/

User-agent: GPTBot
User-agent: Bytespider
Disallow: /

Sitemap: https://firq.dev/sitemap-index.xml
Link markdown, robots and contact vars 2024-10-26 11:51:37 +00:00			`User-agent: *`
robots.txt indexing change 2023-03-16 10:50:47 +00:00			`Disallow: /assets/data/`
Moving static files, adding robots, adding sitemap 2023-03-14 20:49:13 +00:00
fuck chatgpt 2023-08-07 13:45:41 +00:00			`User-agent: GPTBot`
Link markdown, robots and contact vars 2024-10-26 11:51:37 +00:00			`User-agent: Bytespider`
fuck chatgpt 2023-08-07 13:45:41 +00:00			`Disallow: /`

Moving static files, adding robots, adding sitemap 2023-03-14 20:49:13 +00:00			`Sitemap: https://firq.dev/sitemap-index.xml`