froot

actually I'm not done. I noticed that on some page where I use a simple contact form, all sanitizer APIs like $input->post->textarea('message'); $input->post->text('name'); do exactly what I expect. Proper HTML don't break the input and special HTML characters make it through no problem. so some input like <h1>two > one</h1> becomes two > one So the questions are: how I replicate that behaviour with a formdata object send over AJAX? how can I store the values not directly to the formdata but in an array or object that is then stored in some formdata key or property of the formdata object? what are the headers that I need? and what else is to consider?

I will use htmlspecialchars($input->post->message, ENT_SUBSTITUTE) on the server before storing anything and chuck the sanitizer entirely. It's up to the user to not do typos, it's just up to me to make the logic not break. Thanks for your input!

it actually works fine after all, must have been a permissions issue indeed OR I just had to refresh the modules OR the fact that some CSV-files with the same name where in the folder already (why they were there and not deleted remains a mistery). When I deleted these files, it worked fine and now no more issues so far. I just wonder if this is right fclose($fp); $this->wire('files')->unlink($file->filename); or should it be $this->wire('files')->unlink($file->filename); fclose($fp); or if it even makes a difference.

so I tried with a couple of other sanitisers, all with the same issues. The way it looks to me now is, I better sanitise the HTML in the frontend with some sort of REGEX before sending it to the server and then NOT sanitise anything on the server so that special characters like < and >, mostly submitted by mistake are still processed. There's no danger of SQL injection in PW anyway in my understanding…

so I should sanitise the input in JS before sending it to the server? with regex I suppose? I guess these are now two unrelated issues, one is the formData and the other is the sanitising, be it on client or server side. Can I avoid the url encoding by building my own much simpler object that I would send to the server? I'm not sending files anyway.

I gave the folder site/assets/files/4444/ where the CSV-files are stored 777 permissions, if that's what you meant. Doesn't help though.

I don't even remember this issue 😄 but thanks @Andy

I cannot confirm that. Reading the doc I understand that when I use formData I don't have to worry about url encoding, which explains why it actually works without the header approveOrderXHR.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded'); (and doesn't work with it). Anyways, after more investigation I have to circle back to my first suspicion, namely a problem with sanitizing. The formdata makes it to the server with or without broken HTML. I mean stuff Like John< Do><<e Main< street <>12>3 which is exaggerated and unlikely to happen but even one < breaks my entire logic. It's the further processing of the input that runs into issues and I can now confirm that the problem must have always been the santize API. $input->post->textarea('message'); strips anything past a > or <. Highly undesired behaviour. This however: $input->post->message; works just fine. But proper html like <h1>John</h1><h1>Doe</h1> is stored as is and not what I want to see in my AJAX response which is rendered to markup. How can I have the best of both worlds?

Not sure how that works, haven't used the console yet. I put $this->wire('files')->unlink($file->filename) in the tracy console when looking at that specific admin action dialogue. Then I hit "run" and it runs. But the file is not deleted and I get no errors or anything.

sorry to reawaken this thread but it seems like I'm having the exact same issue again, with the same code in the same project. This is my code after processing the CSV-file $this->successMessage = "$count CSV rows were processed."; fclose($fp); $this->wire('files')->unlink($file->filename); return true; yet the file remains in the site/assets/files folder and the error is the same "file already in the system" Any more suggestions?

XHR.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded') this header doesn't work either, need to omit it for it to work at all. Any other suggestions for headers? Can't find a clear documentation on headers anywhere, so to me it remains a mystery. I'm basically just passing simple strings and an email via AJAX, can't be that hard, can it?

it seems like when I do formData.set('content', content); the formData doesn't make it to the server, proper HTML or not. And when I do formData.set('content', JSON.stringify(content)); proper HTML is passed but broken HTML is not. The other scenario where it works, the form fields are directly stored in the formdata separately and not stored in an object (content). I guess that's why it works there and not here.

there's no URL involved so I think this header is unnecessary. It works without it in one scenario. But in another one I create a formData object which just doesn't make it through if it contains < or > var formData = new FormData(); fillFormData(formData); sendFormData(formData) function fillFormData(formData) { formfields = document.getElementsByClassName('formfield'); content = new Object(); for (i = 0; i < formfields.length; i++) { value = formfields[i].value; content[formfields[i].title] = value }; formData.set("content", JSON.stringify(content)); } function sendFormData(formData) { var XHR = new XMLHttpRequest(); XHR.onreadystatechange = function () { if (XHR.readyState !== 4) return; if (XHR.status >= 200 && XHR.status < 300) { let response = XHR.responseText; response = JSON.parse(response); console.log(JSON.parse); } }; XHR.open('POST', '', true); XHR.setRequestHeader('X-Requested-With', 'XMLHttpRequest'); XHR.send(formData); }

OK turns out this is a different issue. The sanitizer API works fine, even with broken HTML. The issue is rather, that the JS formData object sent via AJAX doesn't reach the server properly when it contains some < and/or > (special characters?). So I guess it's another header-issue. I use: XHR.setRequestHeader('X-Requested-With', 'XMLHttpRequest'); XHR.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded'); But they don't solve the issue. I never know which one is right because I never understood headers. Any ideas?

the following doesn't work… protected function bb($i) { $o = str_replace("<", "", str_replace(">", "", $i)); return $o; } $_SESSION['message'] = $this->bb($input->post->textarea('message')); nor does the following work… $_SESSION['message'] = wire('sanitizer')->purify($input->post->textarea('message')); The < or > and anything after that is not stored. How can that be?

I have a form where some user input is sent to the server via ajax and then returned to the frontend and displayed. That input is sanitised on the server like so $entry['message'] = $input->post->textarea('message'); // the server receives a formdata object that stores the user input as stringified JSON which works fine when it's proper HTML with a < and a corresponding > or < and a corresponding /> so <h1>John Doe</h1> is stripped to John Doe So far so good, but what about broken HTML tags? If the user send some BS like <h1John Doe </h1 the whole script breaks, the input is not processed properly, data is lost and the ajax reponse is empty too. How can I sanitise this and avoid this behaviour? Should that be done in the frontend before sending to the server anyway? Frontend uses Vanilla JS. The input is used to send an automatic email later on and though the email is sent, it's completely broken. I mainly need to avoid that of course, so I guess I can just check for empty values before that happens. However, the ajax response needs to have proper markup too and then I wonder if there are any other dangers? Cause I'm also storing the input in some PW fields of a page… Should I use ->purify() ? Thanks for help!

I'm building some sort of two dimensional repeater matrix for a template. So I create a field of type repeater matrix named grid_level1 and add it to a template, let's call it repeater_template. That field has a couple of repeater matrix types, but each one only has a field named grid_level2, which is also of type repeater matrix. That grid_level2 has different repeater matrix types, each of which has a different field (body, images, …). Now here's the catch: I want the grid_level2 to also include grid_level1 among its repeater matrix types. But if I do that, save it, and edit a page that is using the repeater_template, the site is caught in an infinite loop and I get a Oye… Fatal Error: Maximum execution time of 30 seconds exceeded Restricting the "item depth" to say 3 doesn't help either. What to do? What's my logic error?

Does anyone sell or offer PW website templates for end users? Is there a market for that like there is for WordPress and other frameworks/CMS ? Envato market comes to mind…

yes, you are so right. My client is reluctant to use a service like that though. Thanks anyway for the suggestion.

thank you! it works just fine

hello everyone, I hope you're enjoying your Sunday, I have a limit of 5000 mails per day on the mail server and 5000 mails per day on the web server. I'm usually sending emails over the mail server, contact form, newsletter and more, seldomly on the web server. So the mail server limit is reached rather quickly, the web server limit as good as never. Is there a way to (1) catch a failed attempt to send an email (if the limit is reached) and (2) to continue sending emails over the web server when the mail server limit is reached? And then what's the exact API to send over php as opposed to WireMailSMTP ? thanks for help!

apologies, didn't mean to be rude, I'm not complaining at all, contrary to your impressions. Thanks a lot for your help and the module works perfectly! Caught a lot of emails that went nowhere. I need to figure out a way to clean up the logs, maybe with a lazycron? or maybe split it up in more files so I don't have to handle huge files at some point. Also, maybe there's a way to access the very log folder of a web server via PW? like /var/log/

ho ly cow… rock-mail-logger-bak.txt is 10.17 GB 🤯 no wonder. what's a better practice? now with a fresh and empty rock-mail-logger.txt no issues.

chunk for chunk? Not sure how to do that tho but thanks so far

I have a couple of log modules installed. rockmaillogger is the only one that triggers a 504 Gateway Timeout Is that server specific? Something quota issue? Too much data? Or is it a bug? (As a comparison, I also have a Promailer-email log module that estimates 1629756 entries and shows the first page (1-100 entries) alright. However, that one doesn't let me move on to page 2 and further.)