Feature Requests

[Robin S]

7 hours ago, adrian said:

I don't have time to spend fiddling with this today. Maybe I'll revisit later, or if someone else can see what I am doing wrong, any tips would be greatly appreciated.

Please don't spend a lot of time on this. I'll see if I can work it out in the weekend.

[Robin S]

@adrian, I've tried but unfortunately I haven't had any success with getting the titles of admin pages. ?

I turned off a couple of $config settings while I was testing:

$config->sessionFingerprint = 0;
$config->sessionChallenge = false;

And I used curl directly rather than WireHttp to keep it simple. The problem is that as soon as you set the "wire" cookie value...

curl_setopt($ch, CURLOPT_COOKIE, "wire={$input->cookie->wire}");

...PW just doesn't give any response and curl times out. It doesn't matter if you try and get a PW admin page or a frontend page.

I thought it might have been an endless redirect situation but when I get the transfer information with curl_getinfo it looks like no redirects are occurring so it seems that's not the problem.

I'd love to know why PW doesn't respond but I don't know how to debug this any further.

Unless anyone can make a breakthrough with this maybe it will be a matter of manually setting titles for any links to pages that require a logged-in user. Not a great hardship. ?

[horst]

What I have found out is that sending WireHttp requests as https lacks of embedding the ssl certificate things and therefor results in false responses. When I explicitly use http and NOT https, then this code seems to work fine, as it sends exactly the same headers like the browser of the logged in session does:

$wire->session->set('wirehttprequestheaders', getallheaders());

$url = 'http://' . $wire->config->httpHost . '/cli.php';

$http = new WireHttp();
foreach($wire->session->get('wirehttprequestheaders') as $header => $value) {
    if('Host' == $header) continue;
    $http->setHeader($header, $value);
}
$response = $http->post($url);  // or ->get() or send()
if($response !== false) {
    echo '<pre>';
    var_dump($response);
} else {
    echo '<pre>';
    var_dump($http->getError());
}

My cli.php for testing looks like this:

<?php namespace ProcessWire;
include('./index.php');
if($wire->user->isSuperuser()) {
    echo '<pre>';
	var_dump(getallheaders());
} else {
    die('NO ACCESS');
}

 

I never got this timeout behave. Only false or true responses, but always within a second or less. Don't know if this is of any help.

[adrian]

Thanks @Robin S and @horst - unfortunately I am still getting this timeout behaviour, even with your approach horst, so I have no idea ?

[Robin S]

I made some progress! ?

The problem is session blocking: a session is already started and not yet closed when Tracy (or any code in a template file, like I was using for testing) attempts to get the response for the link using the same session cookie. So we need to close the existing session before using WireHttp.

It wouldn't be good to close the session during the Tracy module config save because it's probably still needed at that point, so I'm using a hook after ProcessWire::finished when it should be safe to close the existing session. I also made some other minor tweaks for efficiency: only processing the links if they have changed from the existing config data, and moving the instantiation of WireHttp outside the foreach loop.

I replaced this code with the following:

$existingConfig = $this->wire('modules')->getModuleConfigData($this);
$existingLinks = isset($existingConfig['linksCode']) ? $existingConfig['linksCode'] : '';
$savedLinks = isset($data['linksCode']) ? $data['linksCode'] : '';
if($savedLinks !== $existingLinks) {
	$this->addHookAfter('ProcessWire::finished', null, function($event) {
		// Make URLs in links panel root relative and get titles if not supplied
		$tracyConfig = $this->wire('modules')->getModuleConfigData($this);
		// Close existing session to avoid session blocking
		session_write_close();
		$allLinks = array();
		$http = new WireHttp();
		$http->setHeader('Cookie', "wire={$this->wire('input')->cookie->wire}; wire_challenge={$this->wire('input')->cookie->wire_challenge}");
		foreach(explode("\n", $tracyConfig['linksCode']) as $link) {
			$link_parts = explode('|', $link);
			$url = trim($link_parts[0]);
			$title = isset($link_parts[1]) ? trim($link_parts[1]) : '';
			$url = str_replace($this->wire('config')->urls->httpRoot, '/', $url);
			if($title == '') {
				$fullUrl = strpos($url, 'http') === false ? $this->wire('config')->urls->httpRoot . $url : $url;
				$html = $http->get($fullUrl);
				libxml_use_internal_errors(true);
				$dom = new \DOMDocument();
				$dom->loadHTML($html);
				$list = $dom->getElementsByTagName('title');
				libxml_use_internal_errors(false);
				$title = $list->length ? str_replace('|', ':', $list->item(0)->textContent) : $url;
			}
			$finalLink = $url . ' | ' . $title;
			$allLinks[] = $finalLink;
		}
		$tracyConfig['linksCode'] = implode("\n", $allLinks);
		// Calling saveModuleConfigData with underscores because we don't need hooks to run again
		$this->wire('modules')->___saveModuleConfigData($this, $tracyConfig);
	});
}

 

[adrian]

Thank very much for figuring out the issue @Robin S - I have tried implementing your code but I keep getting logged out due to the session_write_close() and nothing happens after that. I tried testing with and without SessionHandlerDB and that made no difference.

I tried to use $session->forceLogin() to log back in again after the curl request, but that didn't help either. 

Are you testing this on Windows? I wonder if that might be the difference? 

 

[Robin S]

43 minutes ago, adrian said:

I have tried implementing your code but I keep getting logged out due to the session_write_close() and nothing happens after that

Hmmm, I was testing locally on Windows but I just tested on my Linux remote hosting and it worked there too. PW itself will call session_write_close() at shutdown so I don't think that in itself should cause a logout. Perhaps you have other hooks after ProcessWire::finished that have the effect of starting a new session? Maybe try setting a high number for the hook priority - the idea being that this hook to get the links should be the last thing that runs before shutdown.

I just realised that I haven't yet dealt with the HTTPS situation - I've only been testing on HTTP so far. Will report back if/when I can make some progress with HTTPS.

[Robin S]

57 minutes ago, adrian said:

I keep getting logged out

Arrgh, I think it's due to session fingerprinting - the WireHttp request is failing Session::isValidSession(). Will try some more experimentation a bit later.

[horst]

I'm not totally following what you are doing, but sessioningerprinting should not be an issue if you look at my code the post above with using php function 

getallheaders()

This allows a complete 1:1 copy of the sended headers from the browser also through wirehttp. It includes the same UA header, same cookies, same everything.

 array(12) 
  {
    ["Host"]                      string(32) "pw-change-default-language.local"
    ["Connection"]                string(10) "keep-alive"
    ["Upgrade-Insecure-Requests"] string(1) "1"
    ["User-Agent"]                string(132) "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36 OPR/75.0.3969.149"
    ["Accept"]                    string(135) "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
    ["Sec-Fetch-Site"]            string(4) "none"
    ["Sec-Fetch-Mode"]            string(8) "navigate"
    ["Sec-Fetch-User"]            string(2) "?1"
    ["Sec-Fetch-Dest"]            string(8) "document"
    ["Accept-Encoding"]           string(17) "gzip, deflate, br"
    ["Accept-Language"]           string(35) "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"
    ["Cookie"]                    string(166) "wire=8jfqut56c2ag66iaquvb27jdpu; wire_challenge=72ie9BYKDjbw4h0%2FhBYu1wmngAyUoA3F; wires=5u0b7v49ecudnj39bmdpat4dje; wires_challenge=TmFvvRVgTxBQYsyta4tST6yQJjkzymPd"
  }

 

[adrian]

Good point @horst about using getallheaders() to ensure the fingerprint matches. I have combined that with @Robin S's code and it looks like we have a winner. Your combined brilliance seems to have got us through.

I'll test a bit more tomorrow, including on a production server with https, but hopefully we'll be good. Here is the version I am using:

            // make URLs in links panel root relative and get titles if not supplied
            $existingConfig = $this->wire('modules')->getModuleConfigData($this);
            $existingLinks = isset($existingConfig['linksCode']) ? $existingConfig['linksCode'] : '';
            $savedLinks = isset($data['linksCode']) ? $data['linksCode'] : '';
            if($savedLinks !== $existingLinks) {
                $this->addHookAfter('ProcessWire::finished', null, function($event) {
                    // Make URLs in links panel root relative and get titles if not supplied
                    $tracyConfig = $this->wire('modules')->getModuleConfigData($this);
                    // Close existing session to avoid session blocking
                    $allHeaders = getallheaders();
                    session_write_close();
                    $allLinks = array();
                    $http = new WireHttp();
                    foreach($allHeaders as $header => $value) {
                        if('Host' == $header) continue;
                        $http->setHeader($header, $value);
                    }
                    foreach(explode("\n", $tracyConfig['linksCode']) as $link) {
                        $link_parts = explode('|', $link);
                        $url = trim($link_parts[0]);
                        $title = isset($link_parts[1]) ? trim($link_parts[1]) : '';
                        $url = str_replace($this->wire('config')->urls->httpRoot, '/', $url);
                        if($title == '') {
                            $fullUrl = strpos($url, 'http') === false ? $this->wire('config')->urls->httpRoot . $url : $url;
                            $html = $http->get($fullUrl);
                            libxml_use_internal_errors(true);
                            $dom = new \DOMDocument();
                            $dom->loadHTML($html);
                            $list = $dom->getElementsByTagName('title');
                            libxml_use_internal_errors(false);
                            $title = $list->length ? str_replace('|', ':', $list->item(0)->textContent) : $url;
                        }
                        $finalLink = $url . ' | ' . $title;
                        $allLinks[] = $finalLink;
                    }
                    $tracyConfig['linksCode'] = implode("\n", $allLinks);
                    // Calling saveModuleConfigData with underscores because we don't need hooks to run again
                    $this->wire('modules')->___saveModuleConfigData($this, $tracyConfig);
                });
            }

 

[Robin S]

7 hours ago, horst said:

but sessioningerprinting should not be an issue if you look at my code the post above with using php function

I did try your suggestion but the default for $config->sessionFingerprint is "10", which means the session fingerprint is going to include the IP from $_SERVER['REMOTE_ADDR']. When WireHttp requests the page this is going to be the server IP and so it's not going to match the IP in the user fingerprint and a logout will occur. Maybe it would be possible to spoof $_SERVER['HTTP_CLIENT_IP'] or $_SERVER['HTTP_X_FORWARDED_FOR'] but this would mean users have to change from the default session fingerprinting setting.

@adrian, when you say you got it working did you test yet on a remote server?

[adrian]

6 hours ago, Robin S said:

when you say you got it working did you test yet on a remote server?

Just tested now and unfortunately it's logging me out again. I tried moving things to __destruct and removing the session_write_close but that just resulted in the timeout issue ?

 

[horst]

Oh @Robin S, you are right. Sadly the value of $_SERVER['REMOTE_ADDR'] changes between the (remote) browser and the (local) webserver that is used by wirehttp. And if $_SERVER['REMOTE_ADDR'] is part of the session fingerprinting (what it is by default) it only matches if the browser and the webserver are on the same machine. ?

 

[Robin S]

@adrian, I have an idea for a different solution to the problem. I'll report back once I've done some work on it.

[bernhard]

@adrian

would it be possible to get a refresh&clear link here?

I'm developing processmodules quite extensively those days and when adding a new nav item I always need to click "refresh" for triggering the module to catch the changes and then "clear session and cookies" to make the change visible in the menu. It would be great to get all that via one click.

We know how annoying plenty of reloads get when we have to use them a lot... ? 

Thx for considering!

[Mikie]

Hey! I have tried to search for this so apologies if I have missed it. Is it possible to set Tracy module config in config.php? Additionally, it would be great to be able to export current config for this purpose.

[adrian]

Just now, Mikie said:

Hey! I have tried to search for this so apologies if I have missed it. Is it possible to set Tracy module config in config.php? Additionally, it would be great to be able to export current config for this purpose.

Yes, you can set all of Tracy's settings as $config options.

For exporting settings, use: https://processwire.com/modules/module-settings-import-export/

This post explains it all: https://processwire.com/talk/topic/5693-new-module-type-wiremail/?do=findComment&comment=172428

[Mikie]

1 hour ago, adrian said:

Yes, you can set all of Tracy's settings as $config options.

For exporting settings, use: https://processwire.com/modules/module-settings-import-export/

This post explains it all: https://processwire.com/talk/topic/5693-new-module-type-wiremail/?do=findComment&comment=172428

Thanks Adrian that is perfect!

[bernhard]

Hey @adrian any comment on my request from here? https://processwire.com/talk/topic/24932-feature-requests/?do=findComment&comment=213302

I'm developing a process module again and need far too many clicks all the time for simply refreshing the menu ? 

 

[adrian]

@bernhard - what do you think about me including a modules refresh in the Clear Sessions & Cookies option? Just trying to avoid yet another option and a very long name, eg "Modules Refresh, Clear Session & Cookies".

Can you think of any instance where it would be a problem to also refresh modules when you really just want to clear the session and cookies?

I am actually kind of surprised that you need to also do a modules refresh, because the session/cookies option actually logs you out and back in again, so I would have thought that also refreshed modules, but I've probably just forgotten. Can you confirm that both are needed?

[adrian]

@bernhard - is that "Thanks" reaction an indication that you think that would be an OK solution?

[bernhard]

Hi @adrian that thx meant "It's late here in Austria and I'm on mobile in a train. Thx for your reply I'll look into that tomorrow" ? 

18 hours ago, adrian said:

I am actually kind of surprised that you need to also do a modules refresh, because the session/cookies option actually logs you out and back in again, so I would have thought that also refreshed modules, but I've probably just forgotten. Can you confirm that both are needed?

I've just tried that on a fresh and clean installation and the menu is only updating when I refresh modules + clear cookies/session

18 hours ago, adrian said:

what do you think about me including a modules refresh in the Clear Sessions & Cookies option? Just trying to avoid yet another option and a very long name, eg "Modules Refresh, Clear Session & Cookies".

Can you think of any instance where it would be a problem to also refresh modules when you really just want to clear the session and cookies?

I can't reliably say that. I've never ever needed to clear session+cookies other than making the manu catch up changes that I've made to process modules... I do think that a modules refresh should not be a problem though.

I'm fine with the wording of the two options we already have. There's probably no need for explicitly stating that a "clear session&cookies" does already do a modules refresh behind the scenes.

Thx for working on that request, will be much appreciated and save me from a lot of unnecessary clicks! ? 

[adrian]

Sorry about that :)

New version committed - I built it into that option and decided to rename it to "Clear Session, Cookies, & Modules Refresh"

Let me know if you have any problems with it.

[bernhard]

It works, thank you very much!! ?

What about this?

I think it looks a bit cleaner then the original...

You have a typo here anyhow: Cookies >,< & Modules Refresh

[adrian]

I don't mind your wording in the way it suggests refreshing the session, but I am not really sure "refreshing cookies" really describes what is going on.

Regarding the typo - I am a proponent of the Oxford Comma - https://www.colesandlopez.com/blog/what-is-the-oxford-comma - I wasn't taught it in school, but I do like the way it reduces ambiguity. Obviously not necessary here, but I just use it all the time these days :)