asbjorn

Here is my solution for large sitemaps containing thousands upon thousands of pages, without the trouble of timeouts and such. I'm currently running this on website with 170.000+ pages. Relying only upon the ProcessWire API. Just a sidenote: I have no actual need to add all my pages to a sitemap …

The keywords here are sitemap index, sitemap and ProcessWire's page numbers.

Two templates:

• sitemap-index.php
• sitemap-xml.php (with page numbers activated)

The structure of mine is this:

• sitemap-index.php is domain.com/sitemap/
• sitemap-xml.php is domain.com/sitemap/sitemap/ 

And the code for each of them:

sitemap-index.php

<?php
	namespace ProcessWire;

	$out =  
	    '<?xml version="1.0" encoding="UTF-8"?>' . "\n" .
	    '<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">';
	
	$templates = "basic-page|blog-post|tag";
	$key = $pages->count("template=$templates");
	$limit = 200;

	$pageNum = ceil($key/ $limit);
	$post = $pages->get("template=sitemap-xml");

	$i = 1;

	while($pageNum >= $i){
	  $out .= "\n<sitemap>" .
	    "\n\t<loc>" . $post->httpUrl . "page$i/</loc>" .
	    "\n\t<lastmod>" . date("Y-m-d", $post->modified) . "</lastmod>" .
	    "\n</sitemap>";

	  $i = $i + 1;
	}

	$out .= "\n</sitemapindex>";

	header("Content-Type: text/xml");

	echo $out; 
?>

sitemap-xml.php

<?php
	namespace ProcessWire;

	$out =  
	    '<?xml version="1.0" encoding="UTF-8"?>' . "\n" .
	    '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">';

	$pageArray = $pages->find("template=basic-page|blog-post|tag, limit=200");

	foreach ($pageArray as $post) {
	  $out .= "\n<url>" .
	    "\n\t<loc>" . $post->httpUrl . "</loc>" .
	    "\n\t<lastmod>" . date("Y-m-d", $post->modified) . "</lastmod>" .
	    "\n</url>";
	}

	$out .= "\n</urlset>";

	header("Content-Type: text/xml");

	echo $out;
?>

Now that Google is closing down the paid service Google Site Search, and only the free ad version of their search will be available within a year, I have started looking around.

It seems that DuckDuckGo has an API that delivers data as JSON. Has anyone used this with ProcessWire and have some code and guidance to offer? Such as listing the results, open for pagination and any other tricks?

Is there a way to append $rss->itemLinkField = ''; ? For use with campaign url or other tracking methods.

E.g. such as this:

$rss->itemLinkField = 'httpUrl' . '?utm_source=RSS_Feed&utm_medium=RSS&utm_campaign=RSS_Syndication';

Except that this wont work … I have also tried:

$rss->itemLinkField = 'httpUrl';
$rss->itemLinkField .= '?utm_source=RSS_Feed&utm_medium=RSS&utm_campaign=RSS_Syndication';

Just to show what is not working, and what I am trying to achieve.

I have been trying to tweak the example from the blog post about repeater depth to work with a two level repeater. Where each "first level" should indicate the start of a Bootstrap column (e.g. col-md-4). And the last item of each "second level" should indicate the end of the previously "opened" column.

Examples:

Repeater input:
Repeater level 0
- Repeater level 1
- Repeater level 1
Repeater level 0
- Repeater level 1
- Repeater level 1

Output example:

<div class=col-md-4>
  <h2>Repeater level 0</h2>
  <p>Repeater level 1</p>
  <p>Repeater level 1</p>
</div>

<div class=col-md-4>
  <h2>Repeater level 0</h2>
  <p>Repeater level 1</p>
  <p>Repeater level 1</p>
</div>

Even though I could imagine other solutions to this exact problem, the code would be helpful because of the whole detecting and closing opened <div> tag based on level.

How would one approach such a problem?

Thank you for your suggestions. I see now that there are methods that could work with fewer pages, so I have added some additional information in my initial post.  I should have mentioned this in my question:

I have over 170 000 pages to check.

It seems either method suggested is too much to handle.

I have a web page where all templates have Norwegian (default language) and English (secondary language, ID 1036). Many of my pages have been imported using the API, and so I used a method from How to set language = active via API? to set the secondary language active.

Now I want to output a list of pages where the secondary language is inactive. I have tried the following without luck:

$pages->find("template=basic-page, status1036=0")

But this gives an error: Field does not exist: status1036

Is there another way that makes this possible?

Additional information: basic-page was just an example, I should have been more clear from the beginning. I have multiple templates and a totalt of 170 000 pages to check.

This is great. I was just looking for a way to nest repeaters a few weeks ago for a three level menu. My result was messy. But now I was able to do it better, with more options, easier editing and less code. Thank you for the time you are putting in to create this!

I can see some future possibilities: Such as where the nesting depth level defines the width of repeater matrix items on the front end. Making it possible to create a dynamic template where each item can have a different column widths (Uikit or Bootstrap). It would eliminate the need to set the width as a value (or page field value). Making the back end a little more visual (good for the overview). Perhaps with flexbox in css on the front end …

On 31.10.2016 at 7:40 PM, Michael Murphy said:

To update those on this thread, Ryan just pushed some fixes to the dev branch which fixed this issue for me, maybe also for you.

Nice. I'll check it out tomorrow!

Edit: I can confirm that the fix is working. I've been able to set up multi instance with multiple language without problems.

On 31.10.2016 at 7:40 PM, Michael Murphy said:

Would be interesting to hear what people are planning to use the multi-instance support for. My initial needs are for some content migration between two sites, but I can imagine using it for much more in the future.

I'm a web editor at a university college's web page. And we'll be using this to integrate parts of a study guide (academic presentation of study programs and its subjects) into our main web page where our primary focus is marketing of these programs.

No luck with your language suggestion, BitPoet.

At least not like this on my main site:

switch($user->language->name) {
    case 'default':
        setlocale (LC_ALL, $homepage->locale);
        break;
}

$mi_studiehandbok = new ProcessWire('/Users/asbjornness/documents/vid/studiehandbok', '/studiehandbok/');
$mi_studiehandbok->user->language = $mi_studiehandbok->languages->get('default');

And also repeating the switch-part on my secondary (multi-instanced) site. Both nb_NO to be precise in my case.

Still getting a list of the text Array with this code:

$subjects = $mi_studiehandbok->pages->find("template=subject, limit=10");

foreach ($subjects as $s) {
	echo "$s->title<br>";
}

I'm trying the recently added multi-instance support in a local environment, and has encountered some problems. There's not much code, so I'll paste all I have and some info on the installations (both are ProcessWire 3.0.33), and pinpoint the issues as I go:
 

In a template on www.testdomain.pw (local through MAMP)

<?php
// Server path to the PW installation
$path = "/Users/name/documents/testdomain/studiehandbok/";

// The root URL for the PW installation
$url = "/studiehandbok/";

// Create a new ProcessWire instance
$mi_studiehandbok = new ProcessWire($path, $url);

// get study pages
$subjects = $mi_studiehandbok->pages->find("template=study, limit=2");

// output list
foreach ($subjects as $sub) {
echo "<a href='$sub->httpUrl'>$sub->title</a><br>";
}
?>

Issues: The echo will show the correct Url. But as for title, only a list of the text Array will show. But echoing name instead of title works.
Comment: as for $url, when I had http://www.testdomain.pw/studiehandbok/ as the root URL for the PW installation, the first slash "/" after www.testdomain.pw disappeared.

In config-dev.php on the /studiehandbok/ multi-instance installation

$config->httpHosts = array('www.testdomain.pw');

Comment: This multi-installation is located in a subfolder of my main PW installation. The subfolder is as shown in $path above, called /studiehandbok/

PS:
I've also tried with another local installation as the multi-instance, also giving an error on the line of the ->title Notice: Array to string conversion in my template.

PS2:
I tried to find template=home and others. With home the title actually showed, but only then (tested a few).

I'll provide more details if necessary, but I didn't know exactly what to include.

6 hours ago, Robin S said:

 

$subjects = $page->custom_menu;
$subjects = $pages->find("id=$subjects, status!=hidden");

 

This one worked. Thanks Robin S and LostKobrakai.

40 minutes ago, horst said:

How do you or the customer can add pages to

$page->custom_menu

?
What are the settings for that. Are there hidden pages included, selectable?

I am able to select hidden pages, and save, and they are still there in the page field in the page edit mode. The field settings for custom_menu are "nothing special". A standard page field, with a custom name, title as label, PageListSelectMultiple*+ as input fieldtype. No custom values in other input fields.

13 minutes ago, Robin S said:

Hidden means "excluded from lists and searches" so doesn't affect if the page can be selected in a Page field.

Try:

$subjects = $page->custom_menu->find("status!=hidden");

 

I understand, so I should really unselect it from the page field if I want to do i correctly? I also tried your option, but the pages are still shown.

I have a page that are either ) showing an automatic list of all pages with a certain template, if no pages are selected in a page field. 2) Or showing the selected pages from the page field.

if ($page->custom_menu->count == '0') {
	$subjects = $pages->find("template=subject-page, sort=title");
}
else {
	$subjects = $page->custom_menu;
}

When I set these "subject-page"-pages to hidden. The first option excludes these from the list as they should. But the second manually selected list will still show hidden pages. I have checked that these are actually hidden, with the following code:

if ($subject->status & Page::statusHidden) {
	echo "I'm hidden!";
} else {
	echo "And I'm visible!";
}

If I am not wrong, hidden pages have been removed from page fields by themself before, because of their status. But I'm not sure. I haven't used include=hidden. I am on ProcessWire 3.0.32, the page field is using PageListSelectMultiple*+.

How would I go about if I wanted to be able to custom sort pages referenced in a Hanna Code?

Example, I want to sort the output that is the title for pages these ids, in the order specified below:

[[houses id=10|20|40|30]]

I have tried different sort options, such as sort=sort or without sort, in the PHP. But these will look for the order in the page tree and such.

I am thinking of getting started with https, and are also wondering if I should add www to my domain (because of this). I have a domain such as http://example.com/ as of today.

What would be the correct way of implemeting both these things, without messing up to much in relation to Google?

1. Redirect both http with www and http non-ww to https with www
2. I should setup preferred domain with Google Search Console to https with www
3. In PW's config.php, set www.example.com as the whitelisted domain
4. In PW template settings, change Scheme/Protocol to https

These are some of those things I think are important. The main thing here is that I have not been able to find a "easy tutorial" on how to setup https. Although I have managed to install it before, I encountered issues concerning www vs non-www when going from http to https before.

Some pointers are what I ask for. How to get started. How to handle http/https www/non-www issues.

Not sure whether this is actually API, template, dev talk or module talk … I am looking for a starting point on a wanted functionality, which I have never setup before, and am hoping for some references and ProcessWire insight that might help me on the way.

I am looking for to make a search box / live search. Which I am thinking might work like this:

1) You start typing in the search box, i.e. when you're on the front page (home.php - http://example.com/)

2) Once you have typed more than 3 letters, you are redirected to a "live and updated as you type search result" on search.php http://example.com/search/?q=queryhere. Where also the url is reloading as you type. Maybe with some delay on refreshing (when you have stopped typing for 2-3 seconds).

The refreshing and redirecting should be "ajax" smooth. And the search result is on a full page view, not just a dropdown or overlay. It might be advanced, counterintuitive, not API-related. I'm just looking for a nod in the right direction. This is for a hobby project anyway.

I am relatively new to namespaces. But although ProcessWire 2.x doesn't support it. Can I add

<?php namespace ProcessWire;

to the top of php files while running ProcessWire 2.x? Without anything breaking.

The reason I am asking is that I am thinking of modifying a small scale site with no development environment live, and preparing the template files (if possible) would be my first step.

I have unfortunately no experience with debugging / BlackFire / XDebug. I have MAMP where I do my local development, and have activated Xdebug there, and downloaded MacGDBp. But that's all I have managed to do.

With some guidance there would of course be no problem for me to contribute! Is what I am saying. When you're ready.

I am experiencing this module to be a bit slow after a while. Note that I have not read the entire thread from 1 to 13 for solutions or limitation, just quick read a few pages. Or found anything on the webpage.

A few details on my installation:

• About 330 jumplinks
• It takes about 17 second to load the Jumplinks page/list
• Registering new jumplinks also takes a good while (I am not sure if is because of saving or loading the Jumplinks page/list).
• My webpage has about 120 000 pages.
• Running 3.0.17

I can live with it being slow. Just thought I should mention it. And also hoping a little that there's a solution.

1. Yes.

2. Thanks, this is where I messed up! It works

I have now installed the plugin, moved ProcessPageEditTruncate.js to AdminCustomFiles's root folder in "modules" and tested the custom settings you have provided (thank you!) multiple places, but I cannot seem to get it right. I'll paste my code below.

ProcessPageEdit is also activated in the module settings in ProcessWire (but no other options have been setup there).

Some guidance on what I am doing wrong would be much appreciated. (I am running ProcessWire 2.7.2). 

(function ($) {
	$.fn.truncate = function(options) {

		var $fields = this,
			name = $fields.attr('name'),
			settings = $.extend({
				characters: 128,
				prefix: '',
				suffix: '',
				class: 'notes'
			}, options );

		if ($fields.parent('.LanguageSupport').length) {
			var $fields = $("#langTabs_Inputfield_" + name ).find("input, textarea");
		}

		$fields.after("<span class='" + settings.class + "'></span>");

		$fields.each(function (index, el) {
			var truncate = function () {
				var value = $(el).val(),
					typed = typeof value != 'undefined' ? value.length : 0,
					left = settings.characters - typed;
				if (left < 0) {
					$(el).val(value.substr(0, settings.characters));
					truncate();
				} else {
					$(el).next("span").text(settings.prefix + left + settings.suffix);
				}
			}

			$(el).keyup(function() { truncate(); });
			truncate();
		});

		return this;
	};

}(jQuery));

// DOM is ready
$(function () {
    // field with the name attribute title
    $("[name='meta_title']").truncate({
        characters: 55,
        prefix: 'You have',
        suffix: ' character(s) left'
    });

});

Great tip, Martijn Geerts!

Do anyone here now how to setup fields in ProcessPageEditTruncate.js? 

Readme says 4. Configure your fields in ProcessPageEditTruncate.js and save the file.

But I cannot see how to do it in the JS-file.

Great, getting better and better every week!

Just a note: The update does not show in ProcessWireUpgrade.

A major update to the translations today, in terms of adding my "site" related translations (for pro and free modules). Minor changes to the existing "wire" translations.

I have also installed 3.0.13 devns, so that is the official supported version of this language pack.

Changes made to the first post to reflect the current status.

I've been trying to wrap my head around multilanguage, which was discussed in this thread previously. But I have no luck importing (the English one in this example) into my template(s). NOTE: The domain is just an example, the real domain is different.

I have two languages:

Norwegian: http://domain.no/service-pages/?template=subject

English: http://domain.no/en/service-pages/?template=subject

When I paste these links into the browser, I get the JSON in the correct language as I hope for from each of them. But when I try to import it somewhere (even on domain.no itself), I can only get the Norwegian one to work.

The code (with /en/) that wont work is this, simply remove /en/ and it works.

<?php 

    // Get JSON string
    $json_output = file_get_contents("http://domain.no/en/service-pages/?template=subject");
    
    // Convert JSON string to Array
    $someArray = json_decode($json_output, true);

    echo "<pre>";
    print_r($someArray);
    echo "</pre>";

?>

Any suggestions?