Image dropped during API operations (page save)

[Guy Verville]

We have a script that synchronizes data from a CSV file on a daily basis. The data that is synchronized all belong to a page type. However, not all data of this type of template is inspected. For example, we have a field called "sku_image" that is manually filled in by the administrator. 
When there is data to synchronize, the script modifies the relevant fields, then saves the page with the help of a function:

protected function savePage(Page $page, $reportErrors = true) {
		try {
			$page->setOutputFormatting(false);
			$page->save();
            $page->setOutputFormatting(true);
            $this->wire('log')->save('centura-synchro', $page->title);
		} catch(Exception $e) {
			if($reportErrors) {
				$this->error($e->getMessage());
                $this->wire('log')->save('centura-synchro-errors', $page->title . "=> " . $e->getMessage());
			}
		}
		return $page;

However, for some time now, and we don't know why, the entries in the "sku image" field for saved pages have been systematically deleted from the "field_sku_image" table!

The image files remain in each folder, but the reference in the table disappears. The field is a very standard image field and is not processed during synchronization except that when the page object is saved. The other data that is not processed in the synchronization remains in place. It is as if the variable $page forgot the data it is supposed to contain. It's a great mystery.

Synchronization is performed with a special "user" who has editing rights for these pages.

What could cause such behaviour?

[BitPoet]

7 hours ago, Guy Verville said:

When there is data to synchronize, the script modifies the relevant fields

That would probably the relevant part that we would need to see to make an educated guess. Does the script perchance replace the Pageimage object assigned to sku_image with an empty one?

[Guy Verville]

Hi BitPoet,

No, the script load the actual page object and processes it.

if($existingPage = $this->pageExists($query = 'name=' . $itemSku)) {
	$extractedData = "";
			// we compare the previous json data from the page with the actual jsonEncoded csv line
			if($existingPage->p21_json != '') {
				$extractedData = $existingPage->p21_json;
			}
			...
			//These are two strings to compare
			$equal = $this->comparePage($extractedData, json_encode($data));
			if($equal) {
				...
				no change made
				return false
			}
			// there is difference, so we populate the page with the new data.
			// image field and other fields are not affected by $data
			// the function "prepareData" uses $existingPage->set($field, $data [XX]);
			$existingPage = $this->prepareData($existingPage, $data);
			...
			$this->savePage($existingPage,true);
			return true;
}
protected function savePage(Page $page, $reportErrors = true) {
		try {
			$page->of(false);
			$page->save();
            $this->wire('log')->save('centura-synchro', $page->title);
		} catch(Exception $e) {
			if($reportErrors) {
				$this->error($e->getMessage());
                $this->wire('log')->save('centura-synchro-errors', $page->title . "=> " . $e->getMessage());
			}
		}
		return $page;
	}

protected function prepareData($p, $data = null) {

		$fields = $this->generalTools->p21fields;
		...
		foreach($fields as $field) {
				...
					$p->set($field, $data[$ctn]);
				...
		}
		$p->p21_json = json_encode($data);
		...
		return $p;
	}

public function pageExists(string $query) {
		$page = $this->pages->get($query);

		if($page->id === 0) {
			// page do not exist
			return false;
		}
		return $page;
	}

 

[BitPoet]

Hm. sku_image is definitely not contained in p21fields? The line below is apparently the only one that modifies the page's field values, so still the most likely suspect (unless there's more in the code you left out or some hook magic somewhere else). Maybe outputting $field to the log can shed some more light there. Also, check compareData for singe/double equal sign mixup in if conditions.

$p->set($field, $data[$ctn]);

[Guy Verville]

No, p21fields are the reference to csv file we read daily. "sku_image" is an image that must be treated manually.

I queried about the hooks, but nothing relevant here, certainly not around the image.

My programmer colleague suggested me to desactivate the processChangeLog module which was installed recently, which track changes. But I doubt that it can cause problems.

And this table is so simple...

[wbmnfktr]

It stopped working and/or changed its expected and known behaviour without any changes in the code itself, while keeping the exact same workflow?

Check the CSV.

The content in each field, the encoding, the first row/headers, everything.
In Excel, LibreOffice, VS Code, Notepad, whatever.
You, a colleague, someone not involved in programming/code/data management.

[Guy Verville]

Hi, the CSV file is the same, and this field is not involved. The CSV file contains essentially basic data such as sku name, prices for different regions. The price is the data that changes from time to time. The image is not a field in this CSV file.

I made my test on that file and everything operates normally. This is when the $page is saved that something is lost. So my next step is to stop the process and see what's inside the object.

[Guy Verville]

I have perhaps a beginning of an answer. Using Tracy debugger, I checked three times the file object in the $page, before making the changes, after making the changes, and after having saved the page. The $page object changed after the save. The image file is now in an array...

The first, being read before change anything.

The second, the changes have been set. The image object is the same.

I apply the changes. Boom!

While the formatted value should not interfere here, since the output formatting is off there is perhaps something to look here...

On my log, I wrote the filename before and after the save. The second name does not show if I call with $page->sku_image->basename. I have to drop the ->base name part. But this does not explain why the image entry disappear.

if ($existingPage->template->name = "sku-ceramic") {
    $image = $existingPage->sku_image->basename;
}
$this->savePage($existingPage,true);
if ($existingPage->template->name = "sku-ceramic") {
    $this->wire('log')->save('centura-synchro-images', $existingPage->id . "|". $image . "|". $existingPage->sku_image);
}