? appearing when creating pages in PW from non-PW database (charset issue?)

[DrQuincy]

Okay, so I'm pretty sure this is a charset issue, I'm just not sure how to solve it.

I have a MySQL database (non-PW) with news items in that uses utf8mb3_general_ci. I have a new PW site that uses utf8mb4_general_ci.

I am using \ProcessWire\WireDatabasePDO to connect to the non-PW database and am interating through and converting them to PW pages. It seems characters such as em dashes and the Euro symbole are coming through as ?. It's happening in the titles only but they may be becuase in the HTML field, CKEDitor is using entites.

How do I reconcile the charsets? I recall doing something like this years ago on a non-PW job and seem to remember SET NAMES might be the soluiton but if anyone could help me out that'd be appreicated. ? 

Thanks.

[DrQuincy]

I've got it working. It's really easy; before you read anything from the utf8mb3_general_ci database run this query:

$database->query('SET NAMES `utf8mb4`');

This will mean any queries in the current connection will use the same charset as PW and everything will work okay.

[poljpocket]

True, as long as you really set the charset to MB4 in the ProcessWire installer. This isn't the default I think.

[adrian]

@DrQuincy - I think you should really change your entire DB because that utf8mb3_general_ci encoding has caused you other problems as well. IIRC in your other related post I linked to a resource that explains how to make the change.

[DrQuincy]

My PW install is MB4 but my legacy one (soon to be switched off so no point converting) is MB3. Running SET NAMES `utf8mb4` before reading the Mb3 database does the trick nicely. ?