RobotsTxt — Manage robots.txt from the admin UI

[maximus]

Hi everyone,

A small but practical module — manage your robots.txt directly from the ProcessWire admin, no FTP or SSH needed.

GitHub: https://github.com/mxmsmnv/RobotsTxt

What it does:

• Visual editor under Setup → Robots.txt with dark theme and monospace font
• Two-column layout — preset sidebar on the left, editor on the right
• 10 built-in rule presets — click any card to append rules to the editor:
  • Allow all / Block all crawlers
  • Hide admin panel (/processwire/)
  • Block uploaded file assets
  • Block AI training bots (GPTBot, CCBot, anthropic-ai, Google-Extended, FacebookBot, Omgilibot)
  • Sitemap reference, Crawl-delay, Block query strings, Google only, Bing only
• Rules overview — parses the file after saving and renders a human-readable table with color-coded Allow/Disallow/Crawl-delay badges
• Status bar with file path, size, and last-modified date
• File locking (LOCK_EX) on write, CSRF protection, role-based access via robots-manager permission
• Automatic path detection via $config->paths->root

Requirements: ProcessWire 3.0.200+, PHP 8.2+

Screenshot:

MIT License.

[bernhard]

I think there is only one thing missing: a screenshot 😉