Database content gets truncated

[stipps]

Hi,

I've started to work with PW about a week ago, and am trying to port one of our websites. The website features a blog, and a long list of testimonials. 

Setting it all up was more or less straigth forward, and I got it to work like I wanted to in few days tinkering with PW. 

One big problem which I have encountered while checking the website today after working on the styles a bit, is that all blog articles, all testimonials etc. seen to get trunkated when saved in the database. The last few sentences, or in case of the blog articles the last few hundred words are simply not saved. 

Has anybody ever encountered this behavior? I'm getting desperate here, in the worst case I will have to again start over with yet another cms

 

Thanks in advance,

stipps

[LostKobrakai]

Did you check your maxlength setting for the textarea field?

[stipps]

Could you please advise where to find the maxlength setting for the textarea field on PW 2.7? I seem to be unable to find it in the field editor or in the module configuration .. 

[Webrocker]

10 minutes ago, stipps said:

Could you please advise where to find the maxlength setting for the textarea field on PW 2.7? I seem to be unable to find it in the field editor or in the module configuration .. 

I could have sworn that this was an option in the fields settings -- but I just looked up and down in my install (3.0.30) and the only reference to maxlength I found is in the $sanitizer options for text/textarea, which is not what I had in mind.

Jut to make sure; you don't have a charset mix-up between what your site delivers and what your database expects? (just asking because I had such truncation issues back when WordPress moved from ISO-Latin to UTF-8, and weird things happenend with a utf-8 frontend saving into iso-latin DB tables and the other way round…)

cheers
Tom

[stipps]

Hi Tom,

Just checked the DB, it is set to UTF8 default collation, same as the meta attribute on the frontend/backend. Or is there some other place to set the charset that I missed?

If I understand correctly, the $sanitizer should not concern me unless I explicity use it in my templates, correct?

Thanks,

Fredrik

[Webrocker]

1 hour ago, stipps said:

If I understand correctly, the $sanitizer should not concern me unless I explicity use it in my templates, correct?

Yes, this is what I'd say.

Can you find any similarities when or where the truncation happens? Is it after n characters or always after a certain char?

Can you check the field table with phpmyadmin or something similar if a "tinytext" or a kind of charlength restriction is stated there? attached is a screenshot of my "body" textarea field. "mediumtext" indeed caps the amount of data the field will save, but shuld be enough for average sites (http://stackoverflow.com/questions/13932750/tinytext-text-mediumtext-and-longtext-maximum-storage-sizes#13932834

cheers
Tom

 

Edited: Hm, maybe the textarea() sanitizer is applied anyways, which would limit the amount to 16384 chars? http://cheatsheet.processwire.com/sanitizer/properties-and-methods/sanitizer-textarea-value-options/

Edited August 17, 2016 by Webrocker
Added link to sanitizer textarea doc

[LostKobrakai]

Ah, thought there would be a setting: The textarea inputfield is limited to 8 * 1024 characters and 3 * 8 * 1024 bytes. There are already lot's of topics around in the forums about that limit, but I cannot give you an answer on how to circumvent this limit or why it's this low (a text row in mysql could hold 60k bytes).

 

[stipps]

11 minutes ago, LostKobrakai said:

Ah, thought there would be a setting: The textarea inputfield is limited to 8 * 1024 characters and 3 * 8 * 1024 bytes. There are already lot's of topics around in the forums about that limit, but I cannot give you an answer on how to circumvent this limit or why it's this low (a text row in mysql could hold 60k bytes).

 

Okay, with 12.000 characters in one blog entry that question is answered, what a bummer .. maybe I will just split the content in 2 textarea fields. 

I'll have a look if I can find out if there is any way to circumvent this limitation. 

 

Thanks a lot

[Webrocker]

52 minutes ago, LostKobrakai said:

The textarea inputfield is limited to 8 * 1024 characters  (…)

 

Hi, where is this limitation set? Is that a ckeditor/handling consideration?
The data column of a textarea field is "mediumtext" -- which can store millions of chars?

Edited: For reference, here's a discussion about saitizer length and db limit:

cheers,
Tom

 

Edited August 17, 2016 by Webrocker
added link to related discussion

[LostKobrakai]

It's in InputfieldTextarea::init()

[stipps]

11 minutes ago, LostKobrakai said:

It's in InputfieldTextarea::init()

That is definetly the culprit, just changed it and can save the complete article content in just one textarea.

[Webrocker]

hi, just looked it up in my 3.0.30 install. seems the limit was raised with the 3.n version…

public function init() {
        parent::init();
        $this->setAttribute('rows', self::defaultRows);
        $this->setAttribute('maxlength', 1024*32);
        $this->set('contentType', FieldtypeTextarea::contentTypeUnknown);
    }

 

protected function setAttributeValue($value) {
		if($this->maxlength > 0 && $this->hasFieldtype === false) { 
			$value = $this->wire('sanitizer')->textarea($value, array(
				'maxLength' => $this->maxlength, 
				'maxBytes' => $this->maxlength*3, 
				'stripTags' => false, 
				)); 
		} else {
			if(strpos($value, "\r\n") !== false) {
				$value = str_replace("\r\n", "\n", $value); 
			}
		}
		if($this->stripTags) $value = strip_tags($value);
		return trim($value); 
	}

 

 

[stipps]

1 hour ago, Webrocker said:

hi, just looked it up in my 3.0.30 install. seems the limit was raised with the 3.n version…

Good to know. I originally wanted to start working with v3 directly, but had to find out that quite many modules are not compatible just yet.

[Webrocker]

Hi, I'd give it a go -- I'm working with the 3.n on several production sites w/o any issues. But I am using only few non-core modules, mind.
The only thing to look out for is the namespace addition; this presented me with some initial blank screens, due to not having it in the config.php <?php tag.

[stipps]

Hm, I have been blessed with a very .. optimistic timeframe for porting this website. But maybe I'll just do a quick backup and give it a go, thanks!