Trying to save emoji

[Zeka]

Hi.

I want to show emojis in page's title and description. But while saving a page text that goes after an emoji gets stripped. 

I'm using utf8_general_ci as charset, but for title and descriptions tables I've changed it to utf8mb4_general_ci. Even after that 4-byte emojis saves as ???? in DB. 

Is there something that I missed? 

Thanks. 

[dragan]

With the latest PW 3 dev version, I can insert emojis without any problems. Is that a richtext field, or plain-text?

I used https://emojipedia.org/black-rightwards-arrow/ with simple copy and paste. It shows up fine in the back- and frontend.

 

[Zeka]

@dragan

Thanks for your attention to the thread.

Yes, you are right, simple arrows work without problems, but try to insert more complex emoji as ?????

P.S. It's multilanguage text field. 

[dragan]

OK, I just tried it with one of these "exotic" emojis and get the same ???? as you. I guess there's a textformatter function deep inside PW's core that is responsible. I don't think changing the database charset alone will help ?

[Zeka]

But it should))))

There is only $config->dbStripMB4 option that I've found that allows to strip these complex emojis instead of saving them as ????.

Any other ideas? 

 

[Robin S]

Related issue: https://github.com/processwire/processwire-issues/issues/561

Ryan said:

Quote

In any case, your best bet is to switch to utf8mb4 so that you can support it properly. You can do this by exporting your DB to an SQL file, and updating a few things (charset=utf8 to charset=utf8mb4, and adjusting max key lengths for some fields), then re-importing.

Probably not useful in your case seeing as you have an existing installation but there is the option to choose utf8mb4 from the installer: https://processwire.com/blog/posts/pw-3.0.15/#support-for-utf8mb4-and-innodb

Also related:

 

[LostKobrakai]

On 7/1/2018 at 5:51 PM, Zeka said:

but for title and descriptions tables I've changed it to utf8mb4_general_ci

That's just the collation, but not the charset. You need to change at least the latter one.

Quote

A character set is a set of symbols and encodings. A collation is a set of rules for comparing characters in a character set.

https://dev.mysql.com/doc/refman/8.0/en/charset-general.html

Edit: I'm also kinda wondering about the "but for title and descriptions" part. Why not use utf8mb4 for everything, which is the way to actually use utf8 text in mysql. The "utf8" charset is just plain not utf8 at all, but rather a mysql abomination.