Jump to content
sjohnson

Odd CSV Import issue that ends up with fatal php error (FieldtypePage bug?))

Recommended Posts

[Posted this in another post, but realized it's doesn't really pertain to it, so I've moved it to this new topic -- again sorry]

Hi Ryan (and everyone else!),

Just starting using PW and I'm loving it... I've got a site build that requires a bunch of CSV loading and would like some help with API Importing, as I am running into some glitches with page fields.

So far I have a bunch of CSVs that work fine with my php code, but I have one that crashes, unless I put in the pageID.

I think it may be something larger as I have tried using both BatchChildEditor & ImportPagesCSV modules with a modified CSV, and both get the same error as my code.

So, here's my code, it does the following:

  • Reads a page field to get CSV filename
  • Reads 1st row of file to figure out what template and parent to use
  • Reads 2nd row for field titles
  • Processes the remaining rows as data, it also echoes out what it is doing -- not elegant, but it's only for my use
//    If no filename exit
    if ($wire->page->getUnformatted('section_slogan') == '') throw new Wire404Exception();
//    File Pointer    
    $myfile = "http://" . $config->httpHost . $config->urls->templates . "loaders/" . $wire->page->getUnformatted('csvfilename');
    $content = "<p>Using Loader File '{$myfile}'</p><hr/>";
//    Read The Data File
    if (($handle = fopen("$myfile", "r")) !== FALSE) {
    //    Get the Setup Info
        if (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
            $myTemplate = $templates->get("$data[0]");
            $myParent = $wire->pages->get("path='$data[1]'");
            if (($myTemplate->id == 'NullPage') || ($myParent->id == 'NullPage')){
                throw new Wire404Exception();                
            }
        } else {
            throw new Wire404Exception();
        }
    //    Setup the parent
        $newItem = new Page();
        $newItem->template = $myTemplate;
    //    Tell User what's going on:    
        $content .= "<p>Adding New Pages based on the '{$newItem->template->name}' template to parent '{$myParent->title}' path: '{$myParent->path}'</p><hr/>";
    //    List out the fields in the template
        $num = count($newItem->fields);
        $content .= "<p> $num fields in template</p><ol>";
        foreach($newItem->fields as $itemfd) {
            $content .= "<li>" . $itemfd->name . "</li>";
        }
        $content .= "</ol><hr/>";
    //    Read header record and match-up fields to template
        $fieldmatchup = '';
        if (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
            $num = count($data);
            for ($c=0; $c < $num; $c++) {
                foreach($newItem->fields as $itemfd) {
                    if($data[$c] == $itemfd->name) {
                        $fieldmatchup[$itemfd->name] = $c;
                    }
                }
            }            
        }
    //    Show the File's Field Array
        $content .= "<p>Display the Field Array</p><ul>";
        foreach($fieldmatchup as $item_key => $item_value) {
            $content .= "<li>['" . $item_key . "'] = '" . $item_value . "'</li>";
        }
        $content .= "</ul><hr/><h4>Reading File</h4><hr/>";
    //    Now add the data
        $row = 0;
        $newItem = '';
        while (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
            $num = count($data)-1;
            $row++;
            $mySel = "parent=" . $myParent->id . ", title='" . $data[$fieldmatchup['title']] . "'";
            $dupPage = $wire->pages->get($mySel);
            if($dupPage->id == 'NullPage') {
                $content .= "<p>newItem (" . $mySel . ")<br/><b>-- Creating from File</b>.</p>\n";
            //    Setup blank WireArray based on a template
                $newItem = new Page();
                $newItem->template = $myTemplate;
                $newItem->title = $data[$fieldmatchup['title']];
                $newItem->parent = $myParent;
                $newItem->save();
            }
            else {
                $content .= "<p>Duplicate Found (" . $mySel . ")<br/><b>-- Updating from File</b>.</p>\n";
                $newItem = $dupPage;
            }
        $content .= "<p> {$num} fields in record {$row}: <br /></p>\n";
            $newItem->of(false);
            foreach($newItem->fields as $itemfd) {
                $key = $itemfd->name;
                if ($key != 'title') {
                    $value = $data[$fieldmatchup[$key]];
                    $content .= "[{$key}] = [{$value}]<br />\n";
                    $newItem->set($key, $value);
                }
            }
            $newItem->save();
            $content .= "<b>-- Saved: </b> <a target='_blank' href='" . $newItem->editUrl . "'>" . $newItem->title . " [" . $newItem->id . "]</a><hr/>";
        }
        fclose($handle);
    }
    $content .= "<b>{$row} records added/updated</b>";    

And here's a sample of data that works: (associated_stat & special_category are page fields)

skill,"/systems/sol/"
title,associated_stat,multiplier,special,special_category,body
Alien Archeology,Int,4,,,"Skill description"
Alien Tech,Tech,4,,,"Skill description"
Alien Weapons,Ref,4,1,Weapons,"Skill description"

And the file that's giving me issues:(pc_role_category, associated_book, sa_skill are page fields, career_skills is a multiple page field)

cp-role,"/systems/sol/"
title,pc_role_category,associated_book,page_no,verified_via,sa_skill,career_skills,body
"Merc","Combat Related","Book Name","8","Book Review","Combat Zen","Athletics|Alien Tech|Drive|Shoot","Role Description"

It's the associated_book field that's giving me the issue, if I blank it or put in the PageID it works, but with anything else it errors out with this:

Fatal Error
Call to a member function __unset() on boolean search

Source File: ...\core\wire\modules\Fieldtype\FieldtypePage.module:439

431: if($value instanceof Page) {
432: // ok
433: } else if($value instanceof PageArray) {
434: $value = $value->first();
435: } else if(is_string($value) || is_int($value)) {
436: $value = $this->sanitizeValueString($page, $field, $value);
437: if($value instanceof PageArray) $value = $value->first();
438: if($value->_FieldtypePage_remove === $value->id) {
439: $value->__unset('_FieldtypePage_remove');
440: $value = null; // remove item
441: }
442: }

So, looking at this field, and the associated page template, everything is setup the same as all the other page fields, except that the page template in question (books) has a field that references a page field (associated_system) that I'm not even referencing, so I'm not sure if that's the culprit or not, but that is the only thing that separated this template from the others is this custom label code and that the pages are outside of the parent:

post-3446-0-12908600-1465320441_thumb.pn

And yes, I've removed the custom label and it still has the same error.

With my luck it's something simple, but I can't see it... Any help would be appreciated :)

Share this post


Link to post
Share on other sites

Hi All,

I've figured out that this error seems to only happen when you are using the "Custom PHP Selection Code" on your Page Input Fields

It seems that PW uses it, but not completely, thereby causing unset errors.

I've done a bunch of rework and now my code works without issue, so in the hopes that this can help others, here's my revised code.

FYI -- This bug still exists, I'm just forcing my code to use the same selections as in the "Custom PHP Selection Code", thereby avoiding the unset errors.  Then again, maybe this is how it's supposed to work and I should have been duplicating the selection code the whole time :)

Also note that I use a folder structure like so:

  • Root
    • System 1
      • Skills
        • Skill 1
      • Roles
        • Role 1
    • System 2
      • Skills
      • Roles

Which allows me to have the same Skills, Roles, etc. names in different systems with different attributes, and still be able to link them properly.

I do this with code like "$myParent->parent->child('name=skills')->id" which allows me to join page fields within the same System together.

//	If no filename exit
	if ($wire->page->getUnformatted('section_slogan') == '') throw new Wire404Exception();
//	File Pointer	
	$myfile  = "http://" . $config->httpHost . $config->urls->templates . "loaders/" . $wire->page->getUnformatted('section_slogan');
	$content = "<p>Using Loader File '{$myfile}'</p><hr/>";
//	Read The Data File
	if (($handle = fopen("$myfile", "r")) !== FALSE) {
	//	Get the Setup Info
		if (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
			$myTemplate = $templates->get("$data[0]");
			$myParent = $wire->pages->get("path='$data[1]'");
			if (($myTemplate->id == 'NullPage') || ($myParent->id == 'NullPage')){
				throw new Wire404Exception();				
			}
		} else {
			throw new Wire404Exception();
		}
	//	Setup the parent
		$errorLog = "";
		$newItem = new Page();
		$newItem->template = $myTemplate;
	//	Tell User what's going on:	
		$content .= "<p>Adding New Pages based on the '{$newItem->template->name}' template to parent '{$myParent->title}' path: '{$myParent->path}'</p><hr/>";
	//	List out the fields in the template
		$num = count($newItem->fields);
		$content .= "<div class='row'>";
		$content .= "<div class='col-sm-6'>";
		$content .= "<p> $num fields in template</p><dl class='dl-horizontal'>";
		foreach($newItem->fields as $itemfd) {
			$content .= "<dt>[" . $itemfd->name . "]</dt><dd><b>Type:</b>  " . $itemfd->type . "</dd>";
		}
		$content .= "</dl>";
	//	Read header record and match-up fields to template
		$fieldmatchup = '';
		if (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
			$num = count($data);
			for ($c=0; $c < $num; $c++) {
				foreach($newItem->fields as $itemfd) {
					if($data[$c] == $itemfd->name) {
						$fieldmatchup[$itemfd->name] = $c;
					}
				}
			}			
		}
		$content .= "</div>";
		$content .= "<div class='col-sm-6'>";
	//	Show the File's Field Array
		$content .= "<p>Display the File's Field Array</p><dl class='dl-horizontal'>";
		foreach($fieldmatchup as $item_key => $item_value) {
			$content .= "<dt>[" . $item_key . "]</dt><dd>" . $item_value . "</dd>";
		}
		$content .= "</dl>";
		$content .= "</div>";
		$content .= "</div>";
		$content .= "<hr/><h4>Reading File</h4><hr/>";
	//	Now add the data
		$row = 0;
		$newItem = '';
		$content .= "<div class='row'>";
		while (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
			$content .= "<div class='col-sm-6'><hr/><dl class='dl-horizontal'>";
			$num = count($data)-1;
			$row++;
			$mySel = "parent=" . $myParent->id . ", name=" . $sanitizer->pageName($data[$fieldmatchup['title']]) . "";
			$dupPage = $wire->pages->get($mySel);
			if($dupPage->id == 'NullPage') {
				$content .= "<dt>New Page:</dt><dd>[" . $mySel . "] -- Creating<br/> </dd>";
			//	Setup blank WireArray based on a template
				$newItem = new Page();
				$newItem->template = $myTemplate;
				$newItem->title = $data[$fieldmatchup['title']];
				$newItem->parent = $myParent;
				$newItem->save();
			} 
			else {
				$content .= "<dt>Duplicate Page:</dt><dd>[" . $mySel . "] -- Updating<br/> </dd>";
				$newItem = $dupPage;
			}
			$content .= "<dt>Record [{$row}]:</dt><dd>Field List<br/> </dd>";
			$newItem->of(false);
			$errorList  = "";
			foreach($newItem->fields as $itemfd) {
				$key = $itemfd->name;
				$value = $data[$fieldmatchup[$key]];
				if ($key != 'title') {
					if ($value == ""){
						$content .= "<dt>- Field:</dt><dd>[{$key}] = [{$value}] -- Clearing</dd>";
						$newItem->set($key, $value);
					} else {
						switch ($itemfd->type) {
							case 'FieldtypePage':
								$pageFilters = "";
								switch ($key) {
									case 'pc_role_category':
										$pageFilters = ", has_parent={$myParent->parent->child('name=categories')->id}";
										break;
									case 'sa_skill':
										$pageFilters = ", has_parent={$myParent->parent->child('name=skills')->id}";
										break;
									case 'career_skills':
										$pageFilters = ", has_parent={$myParent->parent->child('name=skills')->id}";
										break;
									case 'associated_book':
										$pageFilters = ", has_parent={$myParent->parent->child('name=books')->id}";
										break;
									default:
										$pageFilters = "";
								}
								$pageList = explode('|', $value);
								unset($pageIDs);
								$pageItem   = "";
								if (count($pageList) > 1) {
									$cnt = 0;
									foreach($pageList as $arrItem) {
										$cnt++;
										$value2 = $sanitizer->pageName($arrItem);
										$pageItem = wire('pages')->get("name={$value2}{$pageFilters}");
										if ($pageItem->id != 'NullPage') {
											$content .= "<dt>- Page Ref:</dt><dd><b>{$itemfd->label}</b> [{$key}][{$cnt}]: {$arrItem} -- Added</dd>\n";
											$pageIDs[] = "{$pageItem->id}";
										} else {
											$content   .= "<dt>- Page Ref:</dt><dd><b>{$itemfd->label}</b> [{$key}][{$cnt}]: {$arrItem} -- Not Found</dd>\n";
											$errorList .= "Page Ref: <b>{$itemfd->label}</b> [{$key}]: {$arrItem} -- Not Found<br/>";;
										}
									}
								} else {
										$value2 = $sanitizer->pageName($value);
										$pageItem = wire('pages')->get("name={$value2}{$pageFilters}");
										if ($pageItem->id != 'NullPage') {
											$content .= "<dt>- Page Ref:</dt><dd><b>{$itemfd->label}</b> [{$key}]: {$value} -- Added</dd>\n";
											$pageIDs[] = "{$pageItem->id}";
										} else {
											$content   .= "<dt>- Page Ref:</dt><dd><b>{$itemfd->label}</b> [{$key}]: {$value} -- Not Found</dd>\n";
											$errorList .= "Page Ref: <b>{$itemfd->label}</b> [{$key}]: {$value} -- Not Found<br/>";
										}									
								}
								if (isset($pageIDs)) {
									$newItemVal = implode('|', $pageIDs);
									$newItem->set($key, $newItemVal);
								}
							break;
							default:
								$content .= "<dt>- Field:</dt><dd>[{$key}] = [{$value}]</dd>";
								$newItem->set($key, $value);
						}
					}
				}
			}
			if ($errorList <> "") {
				$errorLog .= $errorList;
				$errorList = "<hr/><b>Loading Errors</b><hr/><p>" . $errorList . "</p>";
				$tempBody = $newItem->getUnformatted('body');
				if ($tempBody !== Null) {
					$tempBody = $tempBody . $errorList;
					$newItem->set('body', $tempBody);
				}
			}
			$newItem->save();
			$content .= "<dt> </dt><dd> </dd>";
			$content .= "<dt>-- Saved:</dt><dd><a target='_blank' href='" . $newItem->editUrl . "'>" . $newItem->title . " [" . $newItem->id . "]</a></dd>";
			$content .= "</dl>";
			$content .= "<hr/></div>";
		}
		$content .= "</div>";
		fclose($handle);
	}
	$content .= "<hr/><b>{$row} records added/updated</b>";
	if ( $errorLog !== "" ) $content .= "<hr/><b>Error Log</b><br/>{$errorLog}";

Hope this helps someone else!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Rodd
      Hi everyone!
      I have a website in a production environment and I want to duplicate it in a local environment. I exported the content of the website (with the 'Site Profile Exporter' module) but I cannot use it actually. I've got an issue with the database. I imported this one in MAMP then.

      I also exported the pages (with the 'ProcessPagesExportImport' module), but I cannot import it to my local website because the fields don't exist. So I created this fields, but I have this error :
      How can I use the elements that already exist and are presents in my database? How can I duplicate correctly the templates, fields and pages?
      Thanks by advance
      PS: Sorry if my english is bad
       
    • By hellerdruck
      Hi all
      I need help with something. Situation: We have let's say 2'000 Files (Excel) that should be displayed (list with links) on a page. We'd need to filter these files by given Keywords or a tree structure or both. Now, I'm looking for a solution whereas our customer can synchronise the files from his local computer with the folder on the webserver. They will update and upload files on a daily basis. Therefore, it would need to synchronise rather than load the files manually in pages or repeaters. Maybe indexing would be an idea, too.
      Are there any modules for Processwire that would help achieving this? Could anyone point me in the right direction?
      Thanks in advance.
    • By benbyf
      Hello, and welcome to what I though was either my client being silly and changing things, or some evil doer. Turns out its reproducible and therefore something in Proceswire (I checked my templates and modules but couldnt find anything that would be doing this...). So what is it doing? Check out the video for evidence.
      A repeater field is interacting with a page template and another repeater field somehow to swap the fields in the template and repeater over...
      I have a template called team, and a repeater field called team_repeater with label Team. Some how and for some reason, when I change my fields on repeater called main_menu_links my team template gets those fields and when I try and revert the team template fields to the fields it should have, they get given to the repeater main_menu_links. Also this to say HELP!!!!!
      video: https://www.dropbox.com/s/exkdhc6n7x0xpsa/strange-repeater-PW-mega-bug.mov?dl=0
    • By ICF Church
      Hi 👋
      Anyone else having this problem?
      Requirements:
      - Repeater (matrix & normal) with mutlilanguage fields (text, textarea…) 
      - Backend language set to something other than default (ie. German) 
      Reproduce:
      - Add a new repeater Item (ajax, I found no way to possible to disable it with matrix)

      (Notice how the default language tab is active instead of the backend language…)
      - Write something into the (default language) field
      - Try to save, if field is required, this will not work. If not required, then when reloading, the content will be inside the backend language field, instead of the default language field who was (presumably) active
      Analysis:
      When  loading  a new repeater element with ajax, the default langue tab is active, but the backend language inputfield is visible (with no visual indication). When writing into the field, it will populate the backend language. When manually clicking on the default language tab (which is already active), the field will switch to the actual default language field (which is [now] empty) (that can now be populated…)
      Also Notice, the labels of the elements to be added are in default language as well instead of the translated label (images instead of Bilder)…
      ProcessWire 3.0.148, Profields 0.0.5…
      Is it my system configuration, or does anyone else have the same issue? This is a screen recording of the problem:
      Issue: https://github.com/processwire/processwire-issues/issues/1179

      Screen Recording 2020-02-25 at 14.18.31.mov
    • By humanafterall
      I'm using some Custom fields for images: 
      https://processwire.com/blog/posts/pw-3.0.142/#custom-fields-for-files-images

      When I save the page, and return the fields are blank. When I re-add the text to the fields and save again then the fields save as expected.

      I know this is stated as being quite experimental but it's super useful feature I'd love to get working correctly.
      I have fields that are CKEditor fields but have overidden this on the image specific template. I've also tried it with regular text fields and I get the same bug.
      (currently using Processwire 3.0.155)
      **UPDATE**
      I've found this issue is specific to editing on pages using the PageTable fieldtype. The fields are not saving when I save the page in the PageTable.
×
×
  • Create New...