Jump to content

MarkupSitemap


Mike Rockett

Recommended Posts

Docs & Download: rockettpw/seo/markup-sitemap
Modules Directory: MarkupSitemap
Composer: rockett/sitemap

⚠️ NEW MAINTAINER NEEDED: Sitemap is in need of developer to take over the project. There are a few minor issues with it, but for the most part, most scenarios, it works, and it works well. However, I'm unable to commit to further development, and would appreciate it if someone could take it over. If you're interested, please send me a private message and we can take it from there.

MarkupSitemap is essentially an upgrade to MarkupSitemapXML by Pete. It adds multi-language support using the built-in LanguageSupportPageNames. Where multi-language pages are available, they are added to the sitemap by means of an alternate link in that page's <url>. Support for listing images in the sitemap on a page-by-page basis and using a sitemap stylesheet are also added.

Example when using the built-in multi-language profile:

<url>
    <loc>http://domain.local/about/</loc>
    <lastmod>2017-08-27T16:16:32+02:00</lastmod>
    <xhtml:link rel="alternate" hreflang="en" href="http://domain.local/en/about/"/>
    <xhtml:link rel="alternate" hreflang="de" href="http://domain.local/de/uber/"/>
    <xhtml:link rel="alternate" hreflang="fi" href="http://domain.local/fi/tietoja/"/>
</url>

It also uses a locally maintained fork of a sitemap package by Matthew Davies that assists in automating the process.

The doesn't use the same sitemap_ignore field available in MarkupSitemapXML. Rather, it renders sitemap options fields in a Page's Settings tab. One of the fields is for excluding a Page from the sitemap, and another is for excluding its children. You can assign which templates get these config fields in the module's configuration (much like you would with MarkupSEO).

Note that the two exclusion options are mutually exclusive at this point as there may be cases where you don't want to show a parent page, but only its children. Whilst unorthodox, I'm leaving the flexibility there. (The home page cannot be excluded from the sitemap, so the applicable exclusion fields won't be available there.)

As of December 2017, you can also exclude templates from sitemap access altogether, whilst retaining their settings if previously configured.

Sitemap also allows you to include images for each page at the template level, and you can disable image output at the page level.

The module allows you to set the priority on a per-page basis (it's optional and will not be included if not set).

Lastly, a stylesheet option has also been added. You can use the default one (enabled by default), or set your own.

Note that if the module is uninstalled, any saved data on a per-page basis is removed. The same thing happens for a specific page when it is deleted after having been trashed.

pwtalk-01.thumb.jpeg.a239282a9414e4c1882e7f8d787f4a4b.jpeg  pwtalk-02.jpeg.e0cdee14590b52c7d404f05bb9e48f21.jpeg  pwtalk-03.thumb.jpeg.9956b756b9e2aea59b6239e55bee0f4f.jpeg

Edited by Mike Rockett
New Maintainer Needed
  • Like 17
  • Thanks 1
Link to comment
Share on other sites

Bumped to 0.1.1-beta

  • Added image support:
    • You can now specify fields in module config that should be included when rendering the sitemap. Simply choose one or more image fields from the list, and any Page that uses those fields will have their images listed in the sitemap. If a description is present, it will be used as the image caption.
    • Also supports ImageExtra. To add specific details to your sitemap's image entries, use ImageExtra to add the meta fields accordingly:
      • Add a license field to set a license for the image (it must be a URL, or it won't be included)
      • Add a title, if needed
      • Add geo, location, or geolocation if you'd like to specify the geo-location of your images
  • The template field in ModuleConfig was changed from includeTemplates to sitemap_include_templates.
  • The ModuleConfig fields now show template and field labels, and not their names.
  • Small changes to the per-Page config field descriptions and labels
  • Like 1
Link to comment
Share on other sites

Hi

Thanks for this module. I was googling tonight to find an how-to for building sitemap and see this. But I have 2 stupid questions:

  1. "Home" template is not available, is it because it's included by default?
  2. Where is the sitemap? domain.com/sitemap.xml brings me to my home page. Did I did something wrong? 

Thanks

Mel

Link to comment
Share on other sites

Hi Mel, You're most welcome.

1 hour ago, mel47 said:

"Home" template is not available, is it because it's included by default?

The list on the configuration page defines templates which get the sitemap_ignore_page and sitemap_ignore_children fields. As the home page should not be ignored, the home template does not get the fields (the module would be pretty much useless otherwise). The presentation of the sitemap works out of the box without any configuration.

Edit: I think the module config should be a little clearer with the explanation. I will sort that out now.

1 hour ago, mel47 said:

Where is the sitemap? domain.com/sitemap.xml brings me to my home page. Did I did something wrong?

The sitemap should be working at that URI. If you're looking for a simple sitemap, maybe use MarkupSitemapXML and see if it renders. If it does, then it means I've done something wrong with my module, which is based on that one.

:)

Link to comment
Share on other sites

Bumped to 0.1.2-beta

  • Fixed descriptions to better explain the purpose of the fields added to templates.
    If additional non-related fields are ever added, these descriptions will be amended accordingly.
     
  • Removes sitemap cache on module-save.
    This is already done when pages are saved, unless the option in MarkupCache's config dictates otherwise.
  • Like 2
Link to comment
Share on other sites

  • 2 weeks later...

Hi Mike, wonderful module for languages and image integration!
But what can i do to fix this case?

<url>
<loc>http://www.xxx.de/</loc>
<lastmod>2017-09-10T00:03:12+02:00</lastmod>
<xhtml:link rel="alternate" hreflang="home" href="http://www.xxx.de/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://www.xxx.de/en/"/>
</url>


Unbenannt.thumb.PNG.05e4315eaab06214e47780e351eef0e9.PNG

Link to comment
Share on other sites

7 hours ago, maxf5 said:

Hi Mike, wonderful module for languages and image integration!
But what can i do to fix this case?

<url>
<loc>http://www.xxx.de/</loc>
<lastmod>2017-09-10T00:03:12+02:00</lastmod>
<xhtml:link rel="alternate" hreflang="home" href="http://www.xxx.de/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://www.xxx.de/en/"/>
</url>

@maxf5 - Glad you like it, and thanks for the post. I still need to work better multi-language support. Even at this point on the multi-lang site profile that PW ships with, the first alternate is identifcal to the main URL in terms of what they contain. Yours is a little different, and so I'd need to envisage the possibilities before working on it properly.

Link to comment
Share on other sites

@Mike Rockett There are 3 problems with the module:

1. The iso for the default language as @maxf5 mentioned.
2. Some pages are completely ignored
3. Wrong sitemap markup. Each link must have it's alternate links not just the default one.

I'm currently using the below method for a proper multi-language sitemap.

1st step
Create a php file "sitemap-xml-multilanguage.php" with the below code and add it on /site/templates/ folder.

<?php namespace ProcessWire;

if (!wire('modules')->isInstalled('LanguageSupportPageNames')) throw new WireException('Missing module: LanguageSupportPageNames');

// Exclude pages here
function selectors() {
	$selectors = array(
	'', // E.g. template!=variations
	); 
	return implode(',',$selectors);
}

function renderSitemapPage(Page $page) {
	$out = '';
	foreach (wire('languages') as $lg) {
		if (!$lg->isDefault() && !$page->{"status$lg->id"}) continue;
		$out .= "\n<url>" . 
			"\n\t<loc>" . $page->localHttpUrl($lg) . "</loc>" . 
			renderSitemapAlternateLangs($page) .
			"\n\t<lastmod>" . date("Y-m-d", $page->modified) . "</lastmod>" . 
			"\n</url>";
	}
	return $out;
}

function renderSitemapChildren(Page $page) { 

	$out = '';
	$newParents = new PageArray(); 
	$children = $page->children(selectors()); 
	
	foreach($children as $child) {
		$out .= renderSitemapPage($child);
		if($child->numChildren) $newParents->add($child); 
			else wire('pages')->uncache($child); 
	}

	foreach($newParents as $newParent) {
		$out .= renderSitemapChildren($newParent); 
		wire('pages')->uncache($newParent); 
	}

	return $out; 
}

function renderSitemapAlternateLangs(Page $page) {
	$out = '';
	foreach(wire('languages') as $lg) {
		if (!$lg->isDefault() && !$page->{"status$lg->id"}) continue;
		$iso = $lg->language_iso_code; // Custom text field for ISO under system's language template
		$out .=	"\n\t<xhtml:link rel=\"alternate\" hreflang=\"$iso\" href=\"".$page->localHttpUrl($lg)."\"/>";
	}
	return $out;
}

function renderSitemapXML(array $paths = array()) {

	$out = 	'<?xml version="1.0" encoding="UTF-8"?><urlset xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns="https://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="https://www.w3.org/1999/xhtml" xsi:schemaLocation="https://www.sitemaps.org/schemas/sitemap/0.9 https://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd">';

	array_unshift($paths, '/');

	foreach($paths as $path) {
		$page = wire('pages')->get($path); 
		if(!$page->id) continue; 
		$out .= renderSitemapPage($page);
		if($page->numChildren) $out .= renderSitemapChildren($page);
	}

	$out .= "\n</urlset>";

	return $out; 
}
header("Content-Type: text/xml");
echo renderSitemapXML();

Note: The above code if I remember correctly is based on @ryan and @kixe code.

2nd step
Add the new template "sitemap-xml-multilanguage" in PW and under URLs tab on "Should page URLs end with a slash?" select "No".

3rd step
Create a new page under "Home" with the title "sitemap.xml" by using the "sitemap-xml-multilanguage" template.

Final step
Create a text field with the name "language_iso_code" and add it under system's language template under "Setup > Templates > Show system templates? > Yes. Then on each language under "Setup > Languages" you will have to add the iso name. This is useful for the default language.

Here are the results I'm getting on http://localhost/mysite/sitemap.xml with my method:

<urlset xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns="https://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="https://www.w3.org/1999/xhtml" xsi:schemaLocation="https://www.sitemaps.org/schemas/sitemap/0.9 https://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd">
<url>
<loc>http://localhost/mysite/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/"/>
<lastmod>2017-09-02</lastmod>
</url>
<url>
<loc>http://localhost/mysite/en/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/"/>
<lastmod>2017-09-02</lastmod>
</url>
<url>
<loc>http://localhost/mysite/nea/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/nea/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/news/"/>
<lastmod>2017-08-27</lastmod>
</url>
<url>
<loc>http://localhost/mysite/en/news/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/nea/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/news/"/>
<lastmod>2017-08-27</lastmod>
</url>
<url>
<loc>http://localhost/mysite/parastaseis/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/parastaseis/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/shows/"/>
<lastmod>2017-08-07</lastmod>
</url>
<url>
<loc>http://localhost/mysite/en/shows/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/parastaseis/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/shows/"/>
<lastmod>2017-08-07</lastmod>
</url>
<url>
<loc>http://localhost/mysite/omada/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/omada/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/meet-the-team/"/>
<lastmod>2017-08-25</lastmod>
</url>
<url>
<loc>http://localhost/mysite/en/meet-the-team/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/omada/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/meet-the-team/"/>
<lastmod>2017-08-25</lastmod>
</url>
<url>
<loc>http://localhost/mysite/epikoinwnia/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/epikoinwnia/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/contact/"/>
<lastmod>2017-09-02</lastmod>
</url>
<url>
<loc>http://localhost/mysite/en/contact/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/epikoinwnia/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/contact/"/>
<lastmod>2017-09-02</lastmod>
</url>
<url>
<loc>http://localhost/mysite/nea/test-4/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/nea/test-4/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/news/test-4/"/>
<lastmod>2017-08-14</lastmod>
</url>
<url>
<loc>http://localhost/mysite/en/news/test-4/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/nea/test-4/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/news/test-4/"/>
<lastmod>2017-08-14</lastmod>
</url>
<url>
<loc>http://localhost/mysite/nea/test-3/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/nea/test-3/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/news/test-3/"/>
<lastmod>2017-08-11</lastmod>
</url>
<url>
<loc>http://localhost/mysite/en/news/test-3/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/nea/test-3/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/news/test-3/"/>
<lastmod>2017-08-11</lastmod>
</url>
<url>
<loc>http://localhost/mysite/nea/test-2/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/nea/test-2/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/news/test-2/"/>
<lastmod>2017-08-11</lastmod>
</url>
<url>
<loc>http://localhost/mysite/en/news/test-2/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/nea/test-2/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/news/test-2/"/>
<lastmod>2017-08-11</lastmod>
</url>
<url>
<loc>http://localhost/mysite/nea/test-1/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/nea/test-1/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/news/test-1/"/>
<lastmod>2017-08-11</lastmod>
</url>
<url>
<loc>http://localhost/mysite/en/news/test-1/</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/nea/test-1/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/news/test-1/"/>
<lastmod>2017-08-11</lastmod>
</url>
<url>
<loc>
http://localhost/mysite/parastaseis/test/
</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/parastaseis/test/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/shows/test/"/>
<lastmod>2017-09-04</lastmod>
</url>
<url>
<loc>
http://localhost/mysite/en/shows/test/
</loc>
<xhtml:link rel="alternate" hreflang="el" href="http://localhost/mysite/parastaseis/test/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/shows/test/"/>
<lastmod>2017-09-04</lastmod>
</url>
</urlset>

 

and here are the results from your module:

<urlset xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns="https://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="https://www.w3.org/1999/xhtml" xsi:schemaLocation="https://www.sitemaps.org/schemas/sitemap/0.9 https://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd">
<url>
<loc>http://localhost/mysite/</loc>
<lastmod>2017-09-02T05:37:50+03:00</lastmod>
<xhtml:link rel="alternate" hreflang="home" href="http://localhost/mysite/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/"/>
</url>
<url>
<loc>http://localhost/mysite/parastaseis/</loc>
<lastmod>2017-08-07T09:51:08+03:00</lastmod>
<xhtml:link rel="alternate" hreflang="home" href="http://localhost/mysite/parastaseis/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/shows/"/>
</url>
<url>
<loc>
http://localhost/mysite/parastaseis/test/
</loc>
<lastmod>2017-09-04T11:06:36+03:00</lastmod>
<xhtml:link rel="alternate" hreflang="home" href="http://localhost/mysite/parastaseis/test/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/shows/test/"/>
</url>
<url>
<loc>http://localhost/mysite/omada/</loc>
<lastmod>2017-08-25T20:27:09+03:00</lastmod>
<xhtml:link rel="alternate" hreflang="home" href="http://localhost/mysite/omada/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/meet-the-team/"/>
</url>
<url>
<loc>http://localhost/mysite/epikoinwnia/</loc>
<lastmod>2017-09-02T04:10:51+03:00</lastmod>
<xhtml:link rel="alternate" hreflang="home" href="http://localhost/mysite/epikoinwnia/"/>
<xhtml:link rel="alternate" hreflang="en" href="http://localhost/mysite/en/contact/"/>
</url>
</urlset>

 

I hope it can help you fix your module.

  • Like 1
Link to comment
Share on other sites

@PWaddict - Thanks! I totally missed out on the actual requirements there. Will work on fixes based on your code soon. Note sure why some of those pages are being ignored as I didn't experience that myself... Will look into it.

@maxf5 - After fixes, the module will create a language ISO field and automatically add it to the language template.

  • Like 2
Link to comment
Share on other sites

Wishlist/request... would love to see on a per page/template basis a sitemap priority field.

When no priority is specified, Google goes 'top page - 100%', 'sub page - 80%', 'sub-sub-page - 60%'. 

In reality however, the 'sub-page' is often the parent page in the menu. The 'sub-sub-page' has the guts & glory that you want indexed with a high priority.

Google doesn't guarantee but says it will give a sub-sub-page with a priority eg of 80% a higher priority than it's root parent with a priority of 60%.

https://www.sitemaps.org/protocol.html

  • Like 2
Link to comment
Share on other sites

I'm making progress with all the necessary changes. On my dev server (not git-pushed yet), the following is complete:

  1. Languages: Multi-language is sorted. It now renders the sitemap according to Google standards, where a page's <url> is generated for each languages with alternate langs, including current. Method is not the same as above example (page-loop), but expands on the root-first-then-children process - this allows easier flexibility in terms of what we can ignore on a page by page basis.
  2. ISOs: Haven't created the separate language ISO field yet, as my my main concern is regarding names vs ISOs. If they are normally identical, then we would only need to declare a proper ISO name for the default language only when its URL matches the default root URL (ie: default URL root = site root). In this case, perhaps it would be better to simply have a module config entry for the default language only? The rationale behind this is that the language name (other than default) should be the same as the iso (for better SEO practice, I'd guess). I haven't yet bumped into a ProcessWire site that doesn't use the ISO for each language name accordingly. If I'm wrong, please point it out, and then I'll use the iso field, per the above example. Or perhaps I should simple allow the flexibility and have the ISO name fall back to the language name if not provided via a field (the default would still need one if it is root)?
  3. Image-support: Images are now picked up in all languages, given the process-change. If an image field is not specified in another language, it will fall back to the default language. Support for ImageExtra remains.

Priority: not done yet. I'll follow a basic principal with this: if the priority is set on a specific page, then it will be added. Otherwise, the tag will be left out. If this is not the correct methodology, please let me know. (cc @psy)

Note: I still need to do some testing on this...

  • Like 4
Link to comment
Share on other sites

Quick questions: Should the module -

  1. ping sitemap engines (after cache-revocation)?
  2. add /sitemap.xml to /robots.text if it exists and is not already added? (This would be done at install-time, and removed at uninstall-time.)

For anyone who may be curious about it, I don't plan on adding XSLT support. End of the day: sitemap.xml is intended for the viewsing of search engines, and not humans. If you want a visual sitemap, best to create your own. :-)

Link to comment
Share on other sites

17 hours ago, Mike Rockett said:

ISOs: Haven't created the separate language ISO field yet, as my my main concern is regarding names vs ISOs.

[...]

Just to add to this point, @maxf5's issue can be solved by adding a default-language name to the home page. In LanguageSupportPageNames, the default language URI is the same as it would be without a language, and so it won't show even if the name is set.

Here's the setup I'm using on the built-in language profile:

LanguageSupportPageNames

sitemap01.jpeg.88db4a516cdfe4e81e91d070450186f6.jpeg

Home Page default language name

sitemap02.jpeg.de30965b3f9cc18fa473ef1b86799ba0.jpeg

Sitemap url output

    <url>
        <loc>http://pw-modules.local/</loc>
        <lastmod>2017-09-15T15:35:15+02:00</lastmod>
        <xhtml:link rel="alternate" hreflang="en" href="http://pw-modules.local/"/>
        <xhtml:link rel="alternate" hreflang="de" href="http://pw-modules.local/de/"/>
        <xhtml:link rel="alternate" hreflang="fi" href="http://pw-modules.local/fi/"/>
        <...>
    </url>

 

  • Like 1
Link to comment
Share on other sites

1 hour ago, PWaddict said:

@Mike Rockett nice "trick"! I don't remember seeing that option on LanguageSupportPageNames. I will definitely gonna use that. I'm currently using the language_iso_code field not only on my sitemap but also on site's navigation (hreflang) and the flag icons but I will delete it :)

Indeed it is - and I bumped into how it works by accident. Will be sticking to this method. v0.2 coming out shortly.

Link to comment
Share on other sites

12 minutes ago, Mike Rockett said:

Indeed it is - and I bumped into how it works by accident. Will be sticking to this method. v0.2 coming out shortly.

There is only a small "downside" if the website is already online. All links from the default language (except homepage) will change so the Page Path History module MUST be installed to avoid 404 errors from previously shared pages, google search etc. Make sure to update your README.md with all these important info.

  • Like 1
Link to comment
Share on other sites

5 minutes ago, PWaddict said:

There is only a small "downside" if the website is already online. All links from the default language (except homepage) will change so the Page Path History module MUST be installed to avoid 404 errors from previously shared pages, google search etc. Make sure to update your README.md with all these important info.

Now that I think about it, we do actually need a way to set the iso for sites that do not use a home page name for the default language.

Link to comment
Share on other sites

  • bedak changed the title to MarkupSitemap: Fatal Error after updating to 0.8.1 (PHP 8+)

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...