Add a Language field to FieldtypeComments

[Edison]

Normally in multi-language websites comments are associated to the language of the page where they have been submitted. A couple of weeks ago I realized that PW Comments field is not language sensitive. This results in a discussion in several languages below an article, as blog-post language is not used for comments filtering. In some cases this may be a plus as you can show a broader discussion in multiple language, but if other cases where a tight language consistency is required may become a limitation. ?

Enhancing the Comments field to become language sensitive is not really a big deal. ? It would be enough to add a language column in the database table field_comments to store the user language ID.

Thanks to this additional column, each comment would be associated with the language ID of the page where it has been submitted. To achieve that I went through a modification of FieldtypeComments. As it took some time to understand the core module, I thought to write a tutorial hoping this may be of help for others in the future. ? Apart the specific purpose of the language field, in principle this could be useful anytime is necessary to add a custom database column (field) to Comments.

FieldtypeComments is a core module located in wire/modules/Fieldtype/FieldtypeComments. As usual we will not work directly on core modules as our modification may be overwritten during PW updates. ?  We will make a copy of /FieldtypeComments directory with all its content to site/modules/FieldtypeComments. As we want to overwrite the original FieldtypeComments and keep using the module ProcessCommentsManager, we are not going to rename FieldtypeComments, its directory and files.

Now we can Login PW Admin panel, select Modules, then Site. Here we press the button Refresh. PW should inform you that FieldtypeComments module is duplicated and you have to choose which one to use. In the alert click on FieldtypeComments, it will open the module settings:

 

Please select to use /site/modules/FieldtypeComments/FieldtypeComments.module, and make sure to press the submit button on the top right. Remain on the same page and click on InputfieldCommentsAdmin. It will open another module settings.

Here select to use site/modules/FieldtypeComments/InputfieldCommentsAdmin.module and make sure to press the submit button on the top right. 

The two modules are now no longer visible under Core Modules but in Site Modules. Now you can test your comments, adding a new one and approving it with ProcessCommentsManager in the Admin panel just to make sure nothing went wrong. If you will need to revert back to the original core module, in Site Modules click on their links.  The module settings will open and there you can select the /wire/modules version.

OK now it's time to make our code modifications. Inside the directory /site/modules/FieldtypeComments you will find 14 files (3 modules, 7 php, 2 css, 2 js). ? Do not worry! we will modify only 3 files: FieldtypeComments.module, Comment.php and CommentForm.php. ?

We start with FieldtypeComments.module, where we will update the database schema. After that the new language ID column can be read/written from/to the database. We will modify two functions: getDatabaseSchema() and ___sleepValue():

public function getDatabaseSchema(Field $field) {
...
  $schema['ip'] = "varchar(15) NOT NULL default ''";
  $schema['user_agent'] = "varchar($maxIndexLength) NOT NULL default ''";
  $schema['language'] = "int unsigned NOT NULL"; 	//>>>>> ADD THIS LINE
  $schemaVersion = $field->schemaVersion;
...

public function ___sleepValue(Page $page, Field $field, $value) {
...
  $a = array(
    'id' => $comment->id,
    'status' => $comment->status,
    'data' => $comment->text,
    'cite' => $comment->cite,
    'email' => $comment->email,
    'created' => $comment->created,
    'created_users_id' => $comment->created_users_id,
    'ip' => $comment->ip,
    'user_agent' => $comment->user_agent,
    'language' => $comment->language,		// >>>>> ADD THIS LINE
  );

Now we will modify Comment.php where we will set the language property modifying __construct() and set() functions:

public function __construct() {
  ...
  $this->set('downvotes', 0);
  $this->set('stars', 0);
  $this->set('language', 0); // >>>>> ADD THIS LINE
}

public function set($key, $value) {

  if(in_array($key, array('id', 'parent_id', 'status', 'flags', 'pages_id', 'created', 'created_users_id', 'language'))) $value = (int) $value; // >>>> MODIFY THIS LINE
  else if($key == 'text') $value = $this->cleanCommentString($value);

Now that we should be capable to read/write the language ID from/to the database, we need to make each comment aware of the user language. It is CommentForm.php turn.. where we will modify processInput() function:

public function processInput() {
  ...
  $comment->parent_id = (int) $data->parent_id;
  $comment->language = wire()->user->language->id;  // >>>> ADD THIS LINE
  $errors = array();\
  ...

As in my case I had some test comments, I decided to use PHPMyAdmin to add the language column into table field_comments using SQL. ? If you plan to do the same, please make sure to backup your database before, to avoid surprises.. ? After creating the new column, I manually updated the language ID of the test comments.

ALTER TABLE `field_comments'
ADD COLUMN `language` int unsigned NOT NULL AFTER `user_agent`;

Now select a blog-post you have available in multiple languages and write new comments for each language. If no issue occurred, in the database table field_comments you should see (with PhpMyAdmin ) the new comments with the user language ID in the language column.

This sounds great ... ? but still we have an issue. ? If we look at the blog-post comments and we switch the language, we still have in the comments a discussion in multiple languages. ? We will have to filter the page comments to display only the ones corresponding to the user language. This will depend on your theme profile. As in my case I am using Uikit 3 Site/Blog Profile I had to modify _uikit.php function ukComments():

function ukComments(CommentArray $comments, $options = array()) {
    ...
    $out = "<ul id='$options[id]' class='uk-comment-list'>";
    $language = user()->language->id; 		// >>>>> ADD THIS LINE
    foreach($comments as $comment) {
      //$out .= "<li class='uk-margin'>" . ukComment($comment) . "</li>"; // >>>>> COMMENT OUT ORIGINAL LINE
      if($comment->language === $language) $out .= "<li class='uk-margin'>" . ukComment($comment) . "</li>"; // >>>>> REPLACE WITH THIS ONE
    }

After this modification only the comments corresponding to the user language will be displayed ! ?

Picky ones.. like me.. ? may notice that comments count in the blog-post header are showing the total count for all the language. For my project this is ok, as I am fine with all comments being counted, doesn't matter the language. But in case you want to fix it, that is pretty simple. Still to work on _uikit.php, now on function ukBlogPost():

function ukBlogPost(Page $page, $options = array()) {
  ...
  $moreIcon = ukIcon($options['moreIcon']);
  $categoryIcon = ukIcon($options['categoryIcon']);
  //$n = $page->get('comments')->count(); 		// >>>>> COMMENT OUT ORIGINAL LINE
  $n = $page->get('comments')->find("language=$language")->count(); // >>>>> REPLACE WITH THIS LINE
  $numComments = $n ? "<a href='$page->url#comments'>" . ukIcon('comments') . " $n</a>" : "";
  ...

I hope you may find something helpful in this tutorial ! ?

[dragan]

Nice one! I never realized the comments field type is language-agnostic (because... I have never really used it, d'oh). Maybe @ryan would be interested in making it language-aware in the core?

[MilenKo]

Oh my God! I am working on a new profile of a multilanguage corporate website and was asked to add an ability to post comments under the website blog section. So I was checking up with Google to see how Wordpress and other CMS/CMF are completting the task and to my surprise, someone ( @Edison? )just posted a solution that should work in a similar way to my understanding of comments logic. Since FieldType Comments is not language oriented (yet ? ), initially I could have added an option to translate comments etc. however on a huge website this could be an overkil of work. So I thought that logically, if someone is switching the language from default to some other lang, the comments listed should be showing only in the language selected but not in the original/default profile language. Not sure how well your solution would work, but would definitely give it a try as I hope it is a much better solution than having multiple PW installations for every language in order to achieve the comments to be posted in the default profile laguage.

 

[Edison]

Thank you @dragan, I think you are right ? , it would make sense for @ryan to make comments language-aware in the core. Should be a small adaptation, and considering comments filtering takes place at profile level, PW users could freely choose to have language-agnostic or language-aware comments.

[Edison]

Thank you @MilenKo. Currently my blog is based on Wordpress and multi-language support comes from Polylang plug-in. It has around 20K posts, now in two languages, and will be switched to Processwire in August. ? Comments are language-aware because there are unique post_id for each language (n languages -> n post records ?), so each comment is indexed to the corresponding post_id in the given language.

I think Processwire approach to multi-language is dramatically more efficient, as you have only one post_id (n languages -> 1 post record ?). I just had the issue that comments were language-agnostic, but adding this simple language_id in the comments table I have replicated the same behaviour. ??

[MilenKo]

I will test it on the startup language profile to see how it will work and will get back to you. How would PW get the language ID for the user and pick the comments? Here is a scenario I thought of - the default profile language is set to English but there are some other - Russian, French, Italian etc. So I am reading the website in Russian as it is more convenient for me, however I speak several languages and my comment is posted on French or Italian for example? Would the language get recognized, or the language ID would be picked up from the language I am reading the website on?

[Edison]

It is very simple. ? The language ID is picked up from the language the user is reading the website. This takes place in CommentForm.php, processInput() modified line: 

 $comment->language = wire()->user->language->id;

The comments are picked (filtered) when read by the profile. This takes place in _uikit.php, ukComments() modified lines:

$language = user()->language->id;
...
if($comment->language === $language) $out .= "<li class='uk-margin'>" . ukComment($comment) . "</li>"; 

 

[MilenKo]

@Edison That is what I thought would be the idea behind it. The only issue that may arise is if an user is reading the website in one language but is posting the comment in another one just because he can. Great example would be bilingual countries like Canada and other parts of the world where English & French are official so if I am reading the website in French but post a comment in English, that would still show the comment to the French part. Of course not many cases of users doing that would be out there, just noting a possibility. If a user is logged on, maybe the language can be picked up from their user profile settings, however in cases where the users are allowed to post comments without a need/ability to be logged on, that would be impossible to pick. Not sure if with the power of PHP the language can be obtained from the OS settings by default but that is getting away from the purpose and might be a subject to GDPR and other regulations of privacy.

In any way, will test your idea on the default profile and see how the functionality works. It is definitely a huge step forward towards multilanguage comments...

[Edison]

Understand @MilenKo, basically you would like users from multi-language countries (Canada or Switzerland) to be able see all the comments written in their national languages (ie: English and French for Canada, or French, German, and Italian for Switzerland). That's sounds a bit more complex, but feasible. ?

I would still keep language ID field in the comments table, but you may wish to re-think filtering and detection.

Filtering will need to identify user's country. Using third party services it is possible to get user's country from IP address. In PW environment you can find a great example of country detection in the module CookieManagementBanner by @adrian. I have not tested it, but it looks promising. You will then need a simple data structure (array or db-table) to associate (only) the multi-language countries with their respective languages. If a country is not multi-language (default) you will filter comments just based on " $comment->language === user()->language->id ", otherwise for multi-language countries you will filter the comments matching with all the country's languages (language1 or language2 or …) as per your data structure. Only drawback… if your Canadian user is in business trip in another country ... it will not work as if in Canada! ?

Language detection may remain unchanged to " comment->language = user()->language->id ". But this does not solve the case you described: a Canadian user reading the website in French, but posting a comment in English. Simplest solution could be to add a <select> option in the comment form with the list of multi-language country's languages (only for users ip-identified as being from a multi-language country) to get the information about comment's language directly from the user (why to make it more difficult ... ?). If you do not wish to ask the user, you may think to apply an algorithm of language detection to the comment string (just googled this on the net  https://github.com/patrickschur/language-detection), but this approach may lead to some imprecisions in associating the comment's language.

These are just some ideas, I hope you may find some hint! 
 

[MilenKo]

@Edison Technically I do not need to show the comments of both languages but am just saying that someone might read the comments in one language but post to another if he is capable of etc. In such scenario you might end up having someone read your page in English but then writing the comment in French since he is more used to it or vice versa.

Well in eather way I can live with it if a few comments listed are not with the same language. In theory my goal is not to complicate too much the things but simplify and I think I will be good with your suggested modification ?