Jump to content
Pete

Module: XML Sitemap

Recommended Posts

I'm on the verge of converting your code back to a template, this is getting silly...

Just want to clarify that the problem is not with Pete's module. There was an obscure problem with the modules installer in the core which has been fixed. I'm not certain this is the problem in your case as you are still getting a different behavior than us, but I suggest grabbing the latest copy of ProcessWire before trying anything else.

Share this post


Link to post
Share on other sites

Just want to clarify that the problem is not with Pete's module.

My apologies for implying any such thing, just getting frustrated. When I saw this module it looked like a great timesaver, and that hasn't worked out.

I'm uploading a fresh copy of PW now, we'll see how it goes.

EDIT: Still no go!

Share this post


Link to post
Share on other sites

Can you confirm that you still get this same error message and that your ProcessWire reports it's version number as 2.2?

Exception: Unable to create path: /MarkupSitemapXML/ (in /home/theseeke/public_html/wire/core/CacheFile.php line 62) This error message was shown because you are logged in as a Superuser. Error has been logged.

thanks,

Ryan

Share this post


Link to post
Share on other sites

I just downloaded a fresh/blank copy of PW 2.2 and installed this module just to make sure it didn't have anything to do with the dev site I had tested on before. But it's working as expected, creating the dir in the right place, etc. So there must be something else that I'm missing. The behavior you are describing definitely indicates a core bug. I can't think of any other possibility. I'm going to do more testing here and hope to find and push a solution shortly.

Share this post


Link to post
Share on other sites

I'm pleased to say that this issue is resolved, and when it came right down to it, Problem Existed Between Keyboard and Chair.

Stupid me forgot to read the instructions, and installed the module to /wire/modules instead of /site/modules.

Now that it's in the right place it's working fine, cheers!

Share this post


Link to post
Share on other sites

Thanks for reporting back, glad that it's working. I was stuck trying to determine how it was doing that, so it's a relief to hear it's resolved. This has still been valuable though as we did solve a bug as a result (top of this page).

Share this post


Link to post
Share on other sites

Hi there,

I've been using this module on a couple of sites in place of the template I previously used and it certainly is a boon in keeping my template folder a lot tidier — great work!

I changed line 53 to check for access when iterating the children so that user-restricted pages do not show up in the sitemap. I think this is a saner default but could be a settting too.

foreach($page->children("check_access=1") as $child) $entry .= $this->sitemapListPage($child);

Thanks,

Stephen

Share this post


Link to post
Share on other sites

Stephen: check_access=1 is default, so you don't have to include that. If you don't want to check access, then you need to use check_access=0.

  • Like 1

Share this post


Link to post
Share on other sites

Very true. I obviously visited the sitemap straight after installing, while still logged in as admin. My mistake!

Thanks, apeisa

Share this post


Link to post
Share on other sites

Want to share a few things I've added for a magazine site I'm building:

First of all, I think "priority" and "changefreq" are fairly important if you're going to have a sitemap at all. This post has some info on Google's guidelines: http://www.eduki.com...-are-important/

What I decided to do was quickly add two global fields to PW so I can set these values manually in each page:

-sitemap_priority

-changefreq

And in the code:

public function sitemapListPage($page) {
$entry = "";
$default_priority = "0.5";
$default_changefreq = "monthly";

include $this->fuel('config')->paths->templates . "sitemap_module_defaults.inc";

if ($page->sitemap_ignore == 0 || $page->path == '/') { // $page->path part added so that it ignores hiding the homepage, else you wouldn't have ANY pages returned
$modified = date ('Y-m-d', $page->modified);
$entry = "\n <url>\n";
$entry .= " <loc>{$page->httpUrl}</loc>\n";
$entry .= " <lastmod>{$modified}</lastmod>\n";

if(!empty($page->sitemap_priority)) {
$entry .= " <priority>{$page->sitemap_priority}</priority>\n";
} else {
$entry .= " <priority>{$default_priority}</priority>\n";
}

if(!empty($page->changefreq)) {
$entry .= " <changefreq>{$page->changefreq}</changefreq>\n";
} else {
$entry .= " <changefreq>{$default_changefreq}</changefreq>\n";
}

 $entry .= " </url>";
if($page->numChildren) {
foreach($page->children as $child) $entry .= $this->sitemapListPage($child);
}
}
return $entry;
}

The sitemap_module_defaults.inc file in the templates dir is so I can set some values on the fly without doing it manually:

<?php
switch($page->template->name) {
case "blog_post":
case "blog_topic":
case "blog_topic_type":
$default_priority = "0.7";
$default_changefreq = "daily";
break;
}
?>

That's done and works fine for me.

Something I found frustrating with the sitemap module was that I couldn't add virtual pages I made. For example, if there's a page with urlSegments with some kind of page manipulation they won't show up on the sitemap, for obvious reasons. So what I did was add this method to the module:

public function sitemapListVirtualPage($httpUrl, $modified = NULL, $sitemap_priority = "0.5", $changefreq = "monthly") {
$entry = "";
$modified = date ('Y-m-d', $modified);
$entry = "\n <url>\n";
$entry .= " <loc>{$httpUrl}</loc>\n";

if($modified) {
$entry .= " <lastmod>{$modified}</lastmod>\n";
}

if($sitemap_priority) {
$entry .= " <priority>" . (float)$sitemap_priority . "</priority>\n";
}

if($changefreq) {
$entry .= " <changefreq>{$changefreq}</changefreq>\n";
}

 $entry .= " </url>";
return $entry;
}

And added this include to the init method before the output is saved to cache:


public function init() {
// Intercept a request for a root URL ending in sitemap.xml and output
if (strpos($_SERVER['REQUEST_URI'], wire('config')->urls->root . 'sitemap.xml') !== FALSE) {
// Check for the cached sitemap, else generate and cache a fresh sitemap
$cache = wire('modules')->get("MarkupCache");
if(!$output = $cache->get("MarkupSitemapXML", 3600)) {
$output = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n";
$output .= '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">';
$output .= $this->sitemapListPage(wire('pages')->get("/"));

include $this->fuel('config')->paths->templates . "sitemap_module_virtual.inc";

$output .= "\n</urlset>";
$cache->save($output);
}

header("Content-Type: text/xml");
echo $output;
exit;
}
}

And in my sitemap_module_virtual.inc file I did this:

<?php
foreach(wire('pages')->find("template=blog_topic|blog_topic_type") as $real_page) {
$output .= $this->sitemapListVirtualPage("http://" . $this->fuel('config')->httpHost . $real_page->url . "archives/", wire('pages')->get("template=blog_post,sort=-created")->modified, "0.4", "daily");
}
foreach(wire('pages')->find("template=blog_topic|blog_topic_type") as $real_page) {
$output .= $this->sitemapListVirtualPage("http://" . $this->fuel('config')->httpHost . $real_page->url . "rss/", wire('pages')->get("template=blog_post,sort=-created")->modified, "0.4", "daily");
}
?>

In this part, I have manipulated /archive/ and /rss/ sub-pages for each blog_topic and sub-topic(or blog_topic_type) using urlSegments. I wanted these to show up on the sitemap, even though they're probably not super important.

Usually I wouldn't bother doing this if it was just these kinds of pages. But what if you have countless articles with manipulated urls that don't show in the sitemap? This is perfect example of why I did this for the future as well.

I've been trying to understand how to do these things for a while, so I hope it helps someone out.

EDIT: I have a download with all of my changes if anyone's interested in taking a look: http://clintonskakun.com/processwire-docs/posts/the-xml-sitemap-module-with-priority-and-changefreq-and-more/

Edited by ClintonSkakun
  • Like 1

Share this post


Link to post
Share on other sites

Thanks for posting this, these seem like some useful additions and some good insights on sitemap.xml too.

Share this post


Link to post
Share on other sites

jukooz asked me a while back how to stick the sitemaps into a template as he's using the multisite module and currently I guess that this module will pull the sitemap for EVERY site on an install...?

This should work on a per-site basis having it in a separate template file per site as a workaround for now, but pay attention to the comments please as there are things to change per-site - please also note that this is un-tested and largely just pulled from the module and tweaked for pasting into a template for use instead of the module:

EDIT: See attachment as the forum software tries to parse a URL in the code : sitemap.txt

I've also updated the module (see first post) to v1.0.3 to check if the page is viewable before including it in the sitemap - I noticed that it was incorrectly listing pages that had no template file... oops!

  • Like 1

Share this post


Link to post
Share on other sites

Hey,

is it possible to use this module with the LanguageLocalizedURL ?

I wanne make the site multilanguage like this:

www.url.de/de/testindeutsch/

www.url.de/en/testinenglish/

Would like to hear from you... Greets Jens alias DV-JF

Share this post


Link to post
Share on other sites

While I've not tried it, I would guess that this module does not collaborate with or accommodate the LanguageLocalizedURL module in any special way. Though Pete could say for sure. 

Share this post


Link to post
Share on other sites

It doesn't accommodate it at the moment, no, but I may need this myself soon so if you can wait a week or two I may have an update (it's not at the top of my list at the moment but I might surprise myself and do it sooner ;)).

Thinking out loud, it needs:

  • Check if LanguageLocalizedURL is installed
  • Find the root page for each language
  • Generate a sitemap.xml page under each root page (it doesn't really generate a page in the database, it just outputs the sitemap if you request that URL)

I think that's it, but I need to get to grips with the LanguageLocalizedURL first.

Share this post


Link to post
Share on other sites

Pete the LLU module doesn't have separate trees, it uses a gateways page for each language in the root like "/de/" "/en/" with url segments to then get the page and switch user language.  The site structure is still all the same as without, you just use text language fields.

The parsing of the url happend automaticly through these gateway pages and it hooks into Page::path to change the url of the pages system wide. So if you do a echo $page->url you'll get the language url in the language you're currently in. Like /en/about-us/, /de/ueber-uns/. 

Have anyone tried yet if it doesn't already work?

Share this post


Link to post
Share on other sites

Soma just submitted a pull request and I merged it.

The 1.0.4 version should allow you to use this with the LanguageLocalizedURL module - let us know how you get on and thanks to Soma ;)

Share this post


Link to post
Share on other sites

Pete, thanks, but I already pulled another request, forgot to up the cache time.

Also to fully work it would also have to add the "language_published" check to see if language version of the page is really published. Will try to add that later.

Also wanted to add that I never use sitemaps for google and never will again (used to try long time ago, but it doesn't really help at all if you build the site carefully. It just eating time doing it and making sure everthing works still). It's not as easy as it first seems and can even be contra productive if not done carefully. Problem with this module as it is now, it will not find and list pages that may are added through urlSegments and I don't see a way to do it easy. Also it doesn't have weighting etc.

  • Like 1

Share this post


Link to post
Share on other sites

Thanks - merged it :)

The main reason I created it was because it gets a new site listed faster - plain and simple. That was my experience from sitemaps a few years ago at least so I hope it still happens, but it does seem to work from my experience.

I've never bothered with weighting or anything like that. Not bothered about URL segments either as for a start the crawler will find a link to that from a normal page on the site presumably, but my main goal was to get Google to crawl the page sooner than it normally would - and I think it will do this every time it looks at the sitemap and sees a new page (assuming it doesn't find it in the crawl already on its own by following links in the content).

Think of it this way - if you release a brand new site without telling Google, it will take until someone links to your site before Google is aware of its existence. On a small, personal site with no blog or comments that could be a loooong time, but the first thing I usually do is launch a site and submit it so I know it's done.

The argument against this is there are other ways of doing that as search engine companies often have pages where you can just type your domain in and they will search it sometime later, but I like to keep Google informed when I add new pages etc just in case it has a lapse in concentration and misses something :)

Share this post


Link to post
Share on other sites

just uped another pull request... :)

I think google doesn't parse the website if it has a sitemap.xml. Or has this changed? I found myself fixing others website by actually removing the sitemap form google.

Google is so fast nowadays it doesn't matter that much, as it won't be in index before index update. I could be wrong as things change all the time, but thats my personal experience and from readup. Also if you add a sitemap page to your site will help if you really worry about it. As with most seo things you have to take everything with a grain of salt at the end.

It's maybe ok if google doesn't come to your new site if it's new, but tests have shown even only 1 link to your site and google will parse it in 1 day if you have a nice structure.

Share this post


Link to post
Share on other sites

I used to post a link on my website to give my new client sites a poke. Work well. Or even in this forum.

  • Like 1

Share this post


Link to post
Share on other sites

Forgot to mention, I've pulled in the last of Soma's pull requests so this is now v1.0.5 to implement the added functionality he wrote as well as the fixes for the bugs he created :P;)

My fault for not testing :D

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Mike Rockett
      Jumplinks for ProcessWire
      Release: 1.5.56
      Composer: rockett/jumplinks
      Jumplinks is an enhanced version of the original ProcessRedirects by Antti Peisa.
      The Process module manages your permanent and temporary redirects (we'll call these "jumplinks" from now on, unless in reference to redirects from another module), useful for when you're migrating over to ProcessWire from another system/platform. Each jumplink supports wildcards, shortening the time needed to create them.
      Unlike similar modules for other platforms, wildcards in Jumplinks are much easier to work with, as Regular Expressions are not fully exposed. Instead, parameters wrapped in curly braces are used - these are described in the documentation.
      Under Development: 2.0, to be powered by FastRoute
      As of version 1.5.0, Jumplinks requires at least ProcessWire 2.6.1 to run.
      View on GitLab
      Download via the Modules Directory
      Read the docs
      Features
      The most prominent features include:
      Basic jumplinks (from one fixed route to another) Parameter-based wildcards with "Smart" equivalents Mapping Collections (for converting ID-based routes to their named-equivalents without the need to create multiple jumplinks) Destination Selectors (for finding and redirecting to pages containing legacy location information) Timed Activation (activate and/or deactivate jumplinks at specific times) 404-Monitor (for creating jumplinks based on 404 hits) Additionally, the following features may come in handy:
      Stale jumplink management Legacy domain support for slow migrations An importer (from CSV or ProcessRedirects) Feedback & Feature Requests
      I’d love to know what you think of this module. Please provide some feedback on the module as a whole, or even regarding smaller things that make it whole. Also, please feel free to submit feature requests and their use-cases.
      Note: Features requested so far have been added to the to-do list, and will be added to 2.0, and not the current dev/master branches.
      Open Source

      Jumplinks is an open-source project, and is free to use. In fact, Jumplinks will always be open-source, and will always remain free to use. Forever. If you would like to support the development of Jumplinks, please consider making a small donation via PayPal.
      Enjoy! :)
    • By BitPoet
      As threatened in Ryan's announcement for 3.0.139, I built a little module for sliding toggles as a replacement for checkboxes. Styling of the input is CSS3 only (with all the usual caveats about older browsers), no JS necessary, and may still be a bit "rough around the edges", so to speak, since I didn't have much time for testing on different devices or brushing things up enough so I'd feel comfortable pushing it to the module directory. But here's the link to the GitHub repo for now:
      InputfieldSlideToggle
      Fieldtype and Inputfield that implements smartphone-style toggles as replacement for checkbox inputs. The visualization is CSS-only, no additional JS necessary.
      Status
      Still very alpha, use with caution!
      Features / Field Settings
      Size
      You can render the toggles in four different sizes: small, medium, large and extra large.
      Off Color
      Currently, "unchecked" toggles can be displayed either in grey (default) or red.
      On Color
      "Checked" toggles can be rendered in one of these colors: blue (default), black, green, grey, orange or red.
      Screenshots

      Some examples with checkbox label


      View all Size and Color Combinations
      Small toggles Medium toggles Big toggles Extra big toggles  









    • By Orkun
      Hi Guys
      I needed to add extended functionalities for the InputfieldDatetime Module (module is from processwire version 2.7.3) because of a Request of Customer.
      So I duplicated the module and placed it under /site/modules/.
      I have added 3 new Settings to the InputfieldDatetime Module.
      1. Day Restriction - Restrict different days based on weekdays selection (e.g. saturday, sunday) - WORKING

       
      2. Time Slots - Define Time slots based on custom Integer Value (max is 60 for 1 hour) - WORKING

       
      3. Time Range Rules per Weekday - Define a minTime and MaxTime per Weekday (e.g. Opening Hours of a Restaurant) - NOT WORKING PROPERLY

       
      The Problem
      Time Slots and Day Restriction working fine so far. But the Time Range Rules per Weekday doesn't work right.
      What should happen is, that when you click on a date, it should update the minTime and maxTime of the Time Select.
      But the change on the select only happens if you select a date 2 times or when you select a date 1 time and then close the datepicker and reopen it again.
      The time select doesn't get change when you select a date 1 time and don't close the picker.
      Here is the whole extended InputfieldDatetime Module.
      The Files that I have changed:
      InputfieldDatetime.module InputfieldDatetime.js jquery-ui-timepicker-addon.js (https://trentrichardson.com/examples/timepicker/) - updated it to the newest version, because minTime and maxTime Option was only available in the new version  
      Thats the Part of the JS that is not working correctly:
      if(datetimerules && datetimerules.length){ options.onSelect = function(date, inst) { var day = $(this).datetimepicker("getDate").getDay(); day = day.toString(); var mintime = $(this).attr('data-weekday'+day+'-mintime'); var maxtime = $(this).attr('data-weekday'+day+'-maxtime'); console.log("weekday: "+day); console.log("minTime: "+mintime); console.log("maxTime: "+maxtime); var optionsAll = $(this).datetimepicker( "option", "all" ); optionsAll.minTime = mintime; optionsAll.maxTime = maxtime; $(this).datetimepicker('destroy'); $(this).datetimepicker(optionsAll); $(this).datetimepicker('refresh'); //$.datepicker._selectDate($(this).attr("id"),date); //$.datepicker._base_getDateDatepicker(); // var inst = $.datepicker._getInst($(this)); // $.datepicker._updateDatepicker(inst); /*$(this).datetimepicker('destroy'); InputfieldDatetimeDatepicker($(this), mintime, maxtime); $(this).datetimepicker('refresh'); */ // $(this).datetimepicker('option', {minTime: mintime, maxTime: maxtime}); } } Can you have a look and find out what the Problem is?
      InputfieldDatetime.zip
       
      Kind Regards
      Orkun
    • By teppo
      This module tracks changes, additions, removals etc. of public (as in "not under admin") pages of your site. Like it's name says, it doesn't attempt to be a version control system or anything like that - just a log of what's happened.
      At the moment it's still a work in progress and will most likely be a victim of many ruthless this-won't-work-let's-try-that-instead cycles, but I believe I've nailed basic functionality well enough to post it here.. so, once again, I'll be happy to hear any comments you folks can provide
      https://modules.processwire.com/modules/process-changelog/
      https://github.com/teppokoivula/ProcessChangelog
      How does it work?
      Exactly like it's (sort of) predecessor, Process Changelog actually consists of two modules: Process Changelog and Process Changelog Hooks. Hooks module exists only to serve main module by hooking into various functions within Pages class, collecting data of performed operations, refining it and keeping up a log of events in it's own custom database table (process_changelog.) Visible part is managed by Process Changelog, which provides users a (relatively) pretty view of the contents of said log table.
      How do you use it?
      When installed this module adds new page called Changelog under Admin > Setup which provides you with a table view of collected data and basic filtering tools See attached screenshots to get a general idea about what that page should look like after a while.
      For detailed installation instructions etc. see README.md.
       


    • By Gadgetto
      Status update links (inside this thread) for SnipWire development will be always posted here:
      2019-08-08
      2019-06-15
      2019-06-02
      2019-05-25
      If you are interested, you can test the current state of development:
      https://github.com/gadgetto/SnipWire
      Please note that the software is not yet intended for use in a production system (alpha version).
      If you like, you can also submit feature requests and suggestions for improvement. I also accept pull requests.
      ---- INITIAL POST FROM 2019-05-25 ----
      I wanted to let you know that I am currently working on a new ProcessWire module that fully integrates the Snipcart Shopping Cart System into ProcessWire. (this is a customer project, so I had to postpone the development of my other module GroupMailer).
      The new module SnipWire offers full integration of the Snipcart Shopping Cart System into ProcessWire.
      Here are some highlights:
      simple setup with (optional) pre-installed templates, product fields, sample products (quasi a complete shop system to get started immediately) store dashboard with all data from the snipcart system (no change to the snipcart dashboard itself required) Integrated REST API for controlling and querying snipcart data webhooks to trigger events from Snipcart (new order, new customer, etc.) multi currency support self-defined/configurable tax rates etc. Development is already well advanced and I plan to release the module in the next 2-3 months.
      I'm not sure yet if this will be a "Pro" module or if it will be made available for free.
      I would be grateful for suggestions and hints!
      (please have a look at the screenshots to get an idea what I'm talking about)
       




×
×
  • Create New...