Sitemap — a new XML sitemap module with sitemap index, IndexNow, and an admin dashboard

[maximus]

I've been working on sites where the standard sitemap setup started showing cracks: XML generated on every request eating memory, no way to regenerate automatically without writing custom hooks, and no visibility into what actually ended up in the file.

So I built Sitemap — a module that generates static XML files to disk, splits output by template name, and handles the full lifecycle from generation to search engine notification.

What it does differently:

• Writes files to disk instead of rendering in memory — no RAM spike on large sites
• Pages are fetched in chunks of 500 with uncacheAll(), so memory stays flat regardless of page count
• Each template gets its own named file: sitemap-product.xml, sitemap-blog.xml, etc. — the index at sitemap.xml references them all
• Auto-regeneration via LazyCron with a configurable interval; the LazyCron hook slot is chosen dynamically to match what you configured (every hour, every 6 hours, daily, etc.) rather than always using everyHour
• A needs_regen flag is set whenever a page is saved, trashed, or deleted — visible in the admin dashboard
• IndexNow support: after generation, all URLs are submitted to api.indexnow.org in batches of 10,000
• Sitemap: directive written directly to the physical robots.txt on save and on generate
• Lock file prevents concurrent generation
• Admin dashboard at Setup > Sitemap showing file count, URL count, total size, and last generated time

Settings stored in a dedicated DB table (sitemap_settings, name/value, MEDIUMTEXT) rather than the module's data field — avoids the serialized config size limit when template settings grow large.

Image sitemap extension and hreflang alternate links for multilanguage sites are both supported.

GitHub:

• https://github.com/mxmsmnv/Sitemap

Screenshots:

Still v1.0.0, so feedback is very welcome — especially from anyone running it on a site with 10k+ pages.

[dynweb]

I tested on a site with ~12000 URLs, it seems to work fine 🙂

Right now, every user with page-edit permission can use the module, but there might be good reasons to restrict it to certain roles. Maybe you could add a "sitemap" permission to make this more granular?

[PWaddict]

Amazing, this looks like the ultimate sitemap module (haven't tested yet) but as @dynweb said a role restriction should be definitely added. I guess replacing the 24th line of ProcessSitemap.module.php with the below will do the trick:

'permission' => 'sitemap-edit',
'permissions' => array(
 'sitemap-edit' => 'Edit Sitemap settings'
),

 

[maximus]

Thank you both for the feedback!

Added a dedicated sitemap-edit permission in v1.0.1. It is created automatically on module install — just assign it to the roles you need in Access > Roles.

https://github.com/mxmsmnv/Sitemap

[psy]

@maximus another great module. 🙂

How are urlSegments handled?

[psy]

On 7/30/2026 at 12:00 PM, psy said:

How are urlSegments handled?

bump

[maximus]

@psy

Thanks for the question and the bump.

URL segments cannot be discovered automatically because ProcessWire only stores whether URL segments are enabled for a template. The actual segment values are defined dynamically by the site or module code.

I’ve added support for them in Sitemap v1.2.0 through a new hookable provider:

$wire->addHookAfter('Sitemap::collectUrlSegments', function(HookEvent $event) {
    /** @var Page $page */
    $page = $event->arguments(0);

    if ($page->template->name !== 'article') return;

    $event->return = array_merge((array)$event->return, [
        'print/',
        [
            'segment' => 'comments/',
            'changefreq' => 'daily',
            'priority' => '0.4',
        ],
    ]);
});

The hook is called for every included page whose template has URL segments enabled.

Each entry can be a relative segment string or an array containing segment (or an absolute loc) with optional lastmod, changefreq, priority, and template overrides.

Segment URLs inherit the parent page’s sitemap metadata and also pass through URL validation, exclusion rules, and deduplication.

The update is available on GitHub:
https://github.com/mxmsmnv/Sitemap

[psy]

12 hours ago, maximus said:

URL segments cannot be discovered automatically because ProcessWire only stores whether URL segments are enabled for a template.

Thanks @maximus

That suits a scenario where the urlsegment is a hardcoded string such as 'comments', along with fixed changefreq and priority.

Perhaps I should have explained more. The urlSegment may not be known. For example, in the old Blog module, the blog author name was derived from the User name. User template cannot be rendered so was added as a urlSegment, eg:

https://mysite.com/blog/authors/joe-bloggs

This hook when using SeoMaestro covers a few urlSegment scenarios:

// Add pages with urlSegments to sitemap
$wire->addHookAfter('SeoMaestro::sitemapItems', function(HookEvent $event) {
    $items = $event->return;
    $pages = wire('pages');
    $users = wire('users');
    $roles = wire('roles');
    $config = wire('config');

    $seen = [];
    foreach($items as $item) {
        if(!empty($item->loc)) {
            $seen[rtrim($item->loc, '/')] = true;
        }
    }

    $getSeoSettings = function(Page $page) {
        foreach($page->template->fieldgroup as $field) {
            if(!$field->type instanceof FieldtypeSeoMaestro) {
                continue;
            }

            $seo = $page->get($field->name);
            if(!$seo || !$seo->sitemap->include) {
                return null;
            }

            return [
                'priority' => $seo->sitemap->priority,
                'changefreq' => $seo->sitemap->changeFrequency,
            ];
        }

        return null;
    };

    $appendItem = function($url, $lastmod, array $settings) use (&$items, &$seen) {
        $key = rtrim($url, '/');
        if(isset($seen[$key])) {
            return;
        }

        $item = new \SeoMaestro\SitemapItem();
        $item->set('loc', $url);
        $item->set('lastmod', date('c', (int) $lastmod));
        $item->set('priority', $settings['priority']);
        $item->set('changefreq', $settings['changefreq']);

        $items[] = $item;
        $seen[$key] = true;
    };

    $appendPaginatedItems = function(Page $page, $totalItems, $perPage, $lastmod) use ($config, $getSeoSettings, $appendItem) {
        $settings = $getSeoSettings($page);
        if(!$settings) {
            return;
        }

        $totalPages = (int) ceil($totalItems / $perPage);
        if($totalPages < 2) {
            return;
        }

        for($pageNum = 2; $pageNum <= $totalPages; $pageNum++) {
            $appendItem(
                $page->httpUrl . $config->pageNumUrlPrefix . $pageNum . '/',
                $lastmod ?: $page->modified,
                $settings
            );
        }
    };

    $blogPostsPage = $pages->get('template=blog-posts');
    if($blogPostsPage->id) {
        $latestPost = $pages->get("template=blog-post, sort=-modified");
        $appendPaginatedItems(
            $blogPostsPage,
            $pages->count("template=blog-post"),
            8,
            $latestPost->id ? $latestPost->modified : $blogPostsPage->modified
        );
    }

    foreach($pages->find('template=blog-category, include=hidden') as $categoryPage) {
        $latestPost = $pages->get("template=blog-post, blog_categories=$categoryPage, sort=-modified");
        $appendPaginatedItems(
            $categoryPage,
            $pages->count("template=blog-post, blog_categories=$categoryPage"),
            10,
            $latestPost->id ? $latestPost->modified : $categoryPage->modified
        );
    }

    $authorsPage = $pages->get('template=blog-authors');
    $authorSettings = $authorsPage->id ? $getSeoSettings($authorsPage) : null;
    if($authorsPage->id && $authorSettings) {
        $authorRole = $roles->get('blog-author');
        foreach($users->find("roles=$authorRole, sort=title") as $author) {
            $authorSlug = wire('sanitizer')->pageName($author->title);
            if(!$authorSlug) {
                continue;
            }

            $postCount = $pages->count("template=blog-post, created_users_id=$author");
            if(!$postCount) {
                continue;
            }

            $authorUrl = $authorsPage->httpUrl . $authorSlug . '/';
            $latestPost = $pages->get("template=blog-post, created_users_id=$author, sort=-modified");
            $lastmod = $latestPost->id ? $latestPost->modified : ($author->modified ?: $authorsPage->modified);

            $appendItem($authorUrl, $lastmod, $authorSettings);

            $totalPages = (int) ceil($postCount / 10);
            if($totalPages < 2) {
                continue;
            }

            for($pageNum = 2; $pageNum <= $totalPages; $pageNum++) {
                $appendItem(
                    $authorUrl . $config->pageNumUrlPrefix . $pageNum . '/',
                    $lastmod,
                    $authorSettings
                );
            }
        }
    }

    $event->return = $items;
});

How would that hook need to be adapted to work with your module?

[maximus]

Thanks for the clarification — you’re right, my previous example made the provider look limited to hardcoded segments.

The returned routes can be calculated dynamically during sitemap generation. I’ve also updated the module to call the provider for templates with either urlSegments or allowPageNum enabled, so it now covers virtual author routes and ordinary ProcessWire pagination.

Your example can be adapted like this:

Spoiler

$wire->addHookAfter('Sitemap::collectUrlSegments', function(HookEvent $event) {
    /** @var Page $page */
    $page = $event->arguments(0);

    $routes = (array)$event->return;
    $pages  = wire('pages');
    $users  = wire('users');
    $roles  = wire('roles');
    $config = wire('config');

    $appendPagination = function(
        int $totalItems,
        int $perPage,
        $lastmod,
        string $prefix = ''
    ) use (&$routes, $config) {
        $totalPages = (int)ceil($totalItems / $perPage);

        for ($pageNum = 2; $pageNum <= $totalPages; $pageNum++) {
            $route = [
                'segment' => $prefix
                    . $config->pageNumUrlPrefix
                    . $pageNum
                    . '/',
            ];

            if ($lastmod) {
                $route['lastmod'] = date('c', (int)$lastmod);
            }

            $routes[] = $route;
        }
    };

    if ($page->template->name === 'blog-posts') {
        $latestPost = $pages->get(
            'template=blog-post, sort=-modified'
        );

        $appendPagination(
            $pages->count('template=blog-post'),
            8,
            $latestPost->id
                ? $latestPost->modified
                : $page->modified
        );
    }

    if ($page->template->name === 'blog-category') {
        $latestPost = $pages->get(
            "template=blog-post, blog_categories=$page, sort=-modified"
        );

        $appendPagination(
            $pages->count(
                "template=blog-post, blog_categories=$page"
            ),
            10,
            $latestPost->id
                ? $latestPost->modified
                : $page->modified
        );
    }

    if ($page->template->name === 'blog-authors') {
        $authorRole = $roles->get('blog-author');

        foreach ($users->find("roles=$authorRole, sort=title") as $author) {
            $authorSlug = wire('sanitizer')->pageName(
                $author->title
            );

            if (!$authorSlug) continue;

            $postCount = $pages->count(
                "template=blog-post, created_users_id=$author"
            );

            if (!$postCount) continue;

            $latestPost = $pages->get(
                "template=blog-post, created_users_id=$author, sort=-modified"
            );

            $lastmod = $latestPost->id
                ? $latestPost->modified
                : ($author->modified ?: $page->modified);

            $routes[] = [
                'segment' => $authorSlug . '/',
                'lastmod' => date('c', (int)$lastmod),
            ];

            $appendPagination(
                $postCount,
                10,
                $lastmod,
                $authorSlug . '/'
            );
        }
    }

    $event->return = $routes;
});

Because the hook is called after the base Page has already passed the sitemap inclusion checks, there is no need to repeat the SEO settings lookup. The generated routes inherit the base Page’s priority and change frequency.

URL validation, exclusion patterns, and deduplication are also handled by the module, so the separate $seen logic is no longer required.

This is available in Sitemap v1.2.1:
https://github.com/mxmsmnv/Sitemap