Jump to content
sjohnson

Odd CSV Import issue that ends up with fatal php error (FieldtypePage bug?))

Recommended Posts

[Posted this in another post, but realized it's doesn't really pertain to it, so I've moved it to this new topic -- again sorry]

Hi Ryan (and everyone else!),

Just starting using PW and I'm loving it... I've got a site build that requires a bunch of CSV loading and would like some help with API Importing, as I am running into some glitches with page fields.

So far I have a bunch of CSVs that work fine with my php code, but I have one that crashes, unless I put in the pageID.

I think it may be something larger as I have tried using both BatchChildEditor & ImportPagesCSV modules with a modified CSV, and both get the same error as my code.

So, here's my code, it does the following:

  • Reads a page field to get CSV filename
  • Reads 1st row of file to figure out what template and parent to use
  • Reads 2nd row for field titles
  • Processes the remaining rows as data, it also echoes out what it is doing -- not elegant, but it's only for my use
//    If no filename exit
    if ($wire->page->getUnformatted('section_slogan') == '') throw new Wire404Exception();
//    File Pointer    
    $myfile = "http://" . $config->httpHost . $config->urls->templates . "loaders/" . $wire->page->getUnformatted('csvfilename');
    $content = "<p>Using Loader File '{$myfile}'</p><hr/>";
//    Read The Data File
    if (($handle = fopen("$myfile", "r")) !== FALSE) {
    //    Get the Setup Info
        if (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
            $myTemplate = $templates->get("$data[0]");
            $myParent = $wire->pages->get("path='$data[1]'");
            if (($myTemplate->id == 'NullPage') || ($myParent->id == 'NullPage')){
                throw new Wire404Exception();                
            }
        } else {
            throw new Wire404Exception();
        }
    //    Setup the parent
        $newItem = new Page();
        $newItem->template = $myTemplate;
    //    Tell User what's going on:    
        $content .= "<p>Adding New Pages based on the '{$newItem->template->name}' template to parent '{$myParent->title}' path: '{$myParent->path}'</p><hr/>";
    //    List out the fields in the template
        $num = count($newItem->fields);
        $content .= "<p> $num fields in template</p><ol>";
        foreach($newItem->fields as $itemfd) {
            $content .= "<li>" . $itemfd->name . "</li>";
        }
        $content .= "</ol><hr/>";
    //    Read header record and match-up fields to template
        $fieldmatchup = '';
        if (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
            $num = count($data);
            for ($c=0; $c < $num; $c++) {
                foreach($newItem->fields as $itemfd) {
                    if($data[$c] == $itemfd->name) {
                        $fieldmatchup[$itemfd->name] = $c;
                    }
                }
            }            
        }
    //    Show the File's Field Array
        $content .= "<p>Display the Field Array</p><ul>";
        foreach($fieldmatchup as $item_key => $item_value) {
            $content .= "<li>['" . $item_key . "'] = '" . $item_value . "'</li>";
        }
        $content .= "</ul><hr/><h4>Reading File</h4><hr/>";
    //    Now add the data
        $row = 0;
        $newItem = '';
        while (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
            $num = count($data)-1;
            $row++;
            $mySel = "parent=" . $myParent->id . ", title='" . $data[$fieldmatchup['title']] . "'";
            $dupPage = $wire->pages->get($mySel);
            if($dupPage->id == 'NullPage') {
                $content .= "<p>newItem (" . $mySel . ")<br/><b>-- Creating from File</b>.</p>\n";
            //    Setup blank WireArray based on a template
                $newItem = new Page();
                $newItem->template = $myTemplate;
                $newItem->title = $data[$fieldmatchup['title']];
                $newItem->parent = $myParent;
                $newItem->save();
            }
            else {
                $content .= "<p>Duplicate Found (" . $mySel . ")<br/><b>-- Updating from File</b>.</p>\n";
                $newItem = $dupPage;
            }
        $content .= "<p> {$num} fields in record {$row}: <br /></p>\n";
            $newItem->of(false);
            foreach($newItem->fields as $itemfd) {
                $key = $itemfd->name;
                if ($key != 'title') {
                    $value = $data[$fieldmatchup[$key]];
                    $content .= "[{$key}] = [{$value}]<br />\n";
                    $newItem->set($key, $value);
                }
            }
            $newItem->save();
            $content .= "<b>-- Saved: </b> <a target='_blank' href='" . $newItem->editUrl . "'>" . $newItem->title . " [" . $newItem->id . "]</a><hr/>";
        }
        fclose($handle);
    }
    $content .= "<b>{$row} records added/updated</b>";    

And here's a sample of data that works: (associated_stat & special_category are page fields)

skill,"/systems/sol/"
title,associated_stat,multiplier,special,special_category,body
Alien Archeology,Int,4,,,"Skill description"
Alien Tech,Tech,4,,,"Skill description"
Alien Weapons,Ref,4,1,Weapons,"Skill description"

And the file that's giving me issues:(pc_role_category, associated_book, sa_skill are page fields, career_skills is a multiple page field)

cp-role,"/systems/sol/"
title,pc_role_category,associated_book,page_no,verified_via,sa_skill,career_skills,body
"Merc","Combat Related","Book Name","8","Book Review","Combat Zen","Athletics|Alien Tech|Drive|Shoot","Role Description"

It's the associated_book field that's giving me the issue, if I blank it or put in the PageID it works, but with anything else it errors out with this:

Fatal Error
Call to a member function __unset() on boolean search

Source File: ...\core\wire\modules\Fieldtype\FieldtypePage.module:439

431: if($value instanceof Page) {
432: // ok
433: } else if($value instanceof PageArray) {
434: $value = $value->first();
435: } else if(is_string($value) || is_int($value)) {
436: $value = $this->sanitizeValueString($page, $field, $value);
437: if($value instanceof PageArray) $value = $value->first();
438: if($value->_FieldtypePage_remove === $value->id) {
439: $value->__unset('_FieldtypePage_remove');
440: $value = null; // remove item
441: }
442: }

So, looking at this field, and the associated page template, everything is setup the same as all the other page fields, except that the page template in question (books) has a field that references a page field (associated_system) that I'm not even referencing, so I'm not sure if that's the culprit or not, but that is the only thing that separated this template from the others is this custom label code and that the pages are outside of the parent:

post-3446-0-12908600-1465320441_thumb.pn

And yes, I've removed the custom label and it still has the same error.

With my luck it's something simple, but I can't see it... Any help would be appreciated :)

Share this post


Link to post
Share on other sites

Hi All,

I've figured out that this error seems to only happen when you are using the "Custom PHP Selection Code" on your Page Input Fields

It seems that PW uses it, but not completely, thereby causing unset errors.

I've done a bunch of rework and now my code works without issue, so in the hopes that this can help others, here's my revised code.

FYI -- This bug still exists, I'm just forcing my code to use the same selections as in the "Custom PHP Selection Code", thereby avoiding the unset errors.  Then again, maybe this is how it's supposed to work and I should have been duplicating the selection code the whole time :)

Also note that I use a folder structure like so:

  • Root
    • System 1
      • Skills
        • Skill 1
      • Roles
        • Role 1
    • System 2
      • Skills
      • Roles

Which allows me to have the same Skills, Roles, etc. names in different systems with different attributes, and still be able to link them properly.

I do this with code like "$myParent->parent->child('name=skills')->id" which allows me to join page fields within the same System together.

//	If no filename exit
	if ($wire->page->getUnformatted('section_slogan') == '') throw new Wire404Exception();
//	File Pointer	
	$myfile  = "http://" . $config->httpHost . $config->urls->templates . "loaders/" . $wire->page->getUnformatted('section_slogan');
	$content = "<p>Using Loader File '{$myfile}'</p><hr/>";
//	Read The Data File
	if (($handle = fopen("$myfile", "r")) !== FALSE) {
	//	Get the Setup Info
		if (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
			$myTemplate = $templates->get("$data[0]");
			$myParent = $wire->pages->get("path='$data[1]'");
			if (($myTemplate->id == 'NullPage') || ($myParent->id == 'NullPage')){
				throw new Wire404Exception();				
			}
		} else {
			throw new Wire404Exception();
		}
	//	Setup the parent
		$errorLog = "";
		$newItem = new Page();
		$newItem->template = $myTemplate;
	//	Tell User what's going on:	
		$content .= "<p>Adding New Pages based on the '{$newItem->template->name}' template to parent '{$myParent->title}' path: '{$myParent->path}'</p><hr/>";
	//	List out the fields in the template
		$num = count($newItem->fields);
		$content .= "<div class='row'>";
		$content .= "<div class='col-sm-6'>";
		$content .= "<p> $num fields in template</p><dl class='dl-horizontal'>";
		foreach($newItem->fields as $itemfd) {
			$content .= "<dt>[" . $itemfd->name . "]</dt><dd><b>Type:</b>  " . $itemfd->type . "</dd>";
		}
		$content .= "</dl>";
	//	Read header record and match-up fields to template
		$fieldmatchup = '';
		if (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
			$num = count($data);
			for ($c=0; $c < $num; $c++) {
				foreach($newItem->fields as $itemfd) {
					if($data[$c] == $itemfd->name) {
						$fieldmatchup[$itemfd->name] = $c;
					}
				}
			}			
		}
		$content .= "</div>";
		$content .= "<div class='col-sm-6'>";
	//	Show the File's Field Array
		$content .= "<p>Display the File's Field Array</p><dl class='dl-horizontal'>";
		foreach($fieldmatchup as $item_key => $item_value) {
			$content .= "<dt>[" . $item_key . "]</dt><dd>" . $item_value . "</dd>";
		}
		$content .= "</dl>";
		$content .= "</div>";
		$content .= "</div>";
		$content .= "<hr/><h4>Reading File</h4><hr/>";
	//	Now add the data
		$row = 0;
		$newItem = '';
		$content .= "<div class='row'>";
		while (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
			$content .= "<div class='col-sm-6'><hr/><dl class='dl-horizontal'>";
			$num = count($data)-1;
			$row++;
			$mySel = "parent=" . $myParent->id . ", name=" . $sanitizer->pageName($data[$fieldmatchup['title']]) . "";
			$dupPage = $wire->pages->get($mySel);
			if($dupPage->id == 'NullPage') {
				$content .= "<dt>New Page:</dt><dd>[" . $mySel . "] -- Creating<br/> </dd>";
			//	Setup blank WireArray based on a template
				$newItem = new Page();
				$newItem->template = $myTemplate;
				$newItem->title = $data[$fieldmatchup['title']];
				$newItem->parent = $myParent;
				$newItem->save();
			} 
			else {
				$content .= "<dt>Duplicate Page:</dt><dd>[" . $mySel . "] -- Updating<br/> </dd>";
				$newItem = $dupPage;
			}
			$content .= "<dt>Record [{$row}]:</dt><dd>Field List<br/> </dd>";
			$newItem->of(false);
			$errorList  = "";
			foreach($newItem->fields as $itemfd) {
				$key = $itemfd->name;
				$value = $data[$fieldmatchup[$key]];
				if ($key != 'title') {
					if ($value == ""){
						$content .= "<dt>- Field:</dt><dd>[{$key}] = [{$value}] -- Clearing</dd>";
						$newItem->set($key, $value);
					} else {
						switch ($itemfd->type) {
							case 'FieldtypePage':
								$pageFilters = "";
								switch ($key) {
									case 'pc_role_category':
										$pageFilters = ", has_parent={$myParent->parent->child('name=categories')->id}";
										break;
									case 'sa_skill':
										$pageFilters = ", has_parent={$myParent->parent->child('name=skills')->id}";
										break;
									case 'career_skills':
										$pageFilters = ", has_parent={$myParent->parent->child('name=skills')->id}";
										break;
									case 'associated_book':
										$pageFilters = ", has_parent={$myParent->parent->child('name=books')->id}";
										break;
									default:
										$pageFilters = "";
								}
								$pageList = explode('|', $value);
								unset($pageIDs);
								$pageItem   = "";
								if (count($pageList) > 1) {
									$cnt = 0;
									foreach($pageList as $arrItem) {
										$cnt++;
										$value2 = $sanitizer->pageName($arrItem);
										$pageItem = wire('pages')->get("name={$value2}{$pageFilters}");
										if ($pageItem->id != 'NullPage') {
											$content .= "<dt>- Page Ref:</dt><dd><b>{$itemfd->label}</b> [{$key}][{$cnt}]: {$arrItem} -- Added</dd>\n";
											$pageIDs[] = "{$pageItem->id}";
										} else {
											$content   .= "<dt>- Page Ref:</dt><dd><b>{$itemfd->label}</b> [{$key}][{$cnt}]: {$arrItem} -- Not Found</dd>\n";
											$errorList .= "Page Ref: <b>{$itemfd->label}</b> [{$key}]: {$arrItem} -- Not Found<br/>";;
										}
									}
								} else {
										$value2 = $sanitizer->pageName($value);
										$pageItem = wire('pages')->get("name={$value2}{$pageFilters}");
										if ($pageItem->id != 'NullPage') {
											$content .= "<dt>- Page Ref:</dt><dd><b>{$itemfd->label}</b> [{$key}]: {$value} -- Added</dd>\n";
											$pageIDs[] = "{$pageItem->id}";
										} else {
											$content   .= "<dt>- Page Ref:</dt><dd><b>{$itemfd->label}</b> [{$key}]: {$value} -- Not Found</dd>\n";
											$errorList .= "Page Ref: <b>{$itemfd->label}</b> [{$key}]: {$value} -- Not Found<br/>";
										}									
								}
								if (isset($pageIDs)) {
									$newItemVal = implode('|', $pageIDs);
									$newItem->set($key, $newItemVal);
								}
							break;
							default:
								$content .= "<dt>- Field:</dt><dd>[{$key}] = [{$value}]</dd>";
								$newItem->set($key, $value);
						}
					}
				}
			}
			if ($errorList <> "") {
				$errorLog .= $errorList;
				$errorList = "<hr/><b>Loading Errors</b><hr/><p>" . $errorList . "</p>";
				$tempBody = $newItem->getUnformatted('body');
				if ($tempBody !== Null) {
					$tempBody = $tempBody . $errorList;
					$newItem->set('body', $tempBody);
				}
			}
			$newItem->save();
			$content .= "<dt> </dt><dd> </dd>";
			$content .= "<dt>-- Saved:</dt><dd><a target='_blank' href='" . $newItem->editUrl . "'>" . $newItem->title . " [" . $newItem->id . "]</a></dd>";
			$content .= "</dl>";
			$content .= "<hr/></div>";
		}
		$content .= "</div>";
		fclose($handle);
	}
	$content .= "<hr/><b>{$row} records added/updated</b>";
	if ( $errorLog !== "" ) $content .= "<hr/><b>Error Log</b><br/>{$errorLog}";

Hope this helps someone else!

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Guy Incognito
      I added some custom styles to the CKeditor menu bar using the example mystyles.js and the PW tutorial. This worked fine for fields when editing on the frontend. But none of our custom styles showed in the backend editor dropdown unless we edited the core copy of mystyles.js in wire/modules.
      Is this correct behaviour, a bug or a mistake on my part? Tried clearing cache, logging in/out etc but the backend ignores our custom styles in the site/modules path.
    • By karian
      I don't know why multiple instances (repeater_repeat_columns1, repeater_repeat_columns2, ...) of my repeater field are displayed inside Template field (see image).
      Is there a way to clean/reset it ?
       

    • By psy
      I'm combining two PW sites into one, Site A into Site B.
      At each step, I did it bit by bit as the 'all at once' approach failed.
       
      First, I exported all the fields from Site A and imported into Site B. Any field types not supported by import/export, eg FieldtypeOptions I manually recreated. All good.
      Next I exported all the templates from Site A and imported them into Site B and copied across their associated template files. All good.
      Finally I exported the pages I needed from Site A into Site B - again, bit by bit to ensure it all went smoothly.
      From the admin side, it all looked and worked perfectly.
      Front end was a totally different story. All existing pages in Site B worked as expected. NONE of the pages imported from Site A displayed. They all ended in a redirect loop with no errors in the PW logs or Tracy Debugger.
      After some trial-and-error, I finally got it working with:
      - create a new template in Site B admin with no associated template file and just a title field
      - import the fields from the imported Site A template into the newly created template (both on Site B)
      - copy the Site A php template file into a new file that matched the new PW Site B template name and save in Site B site/templates
      I can deal with the above workaround. Just curious to know if I did something wrong or if the template import/export feature is problematic?
       
      ### Solution:
      While the export/import was a slow process, turned out the front end redirecting issue was unrelated. For reasons unknown, all templates marked as HTTPS only were the ones redirecting, ie all templates from Site A. Finally solved it by changing the $config->https to true in site/config.php
      Now the pages display correctly as https whether the template forces the issue or not.
       
    • By rareyush
      i am receiving and error whenever I try to run my processwire on localhost,

       
       
      sql code
       
      -- -- Table structure for table `field_fieldset_meta_end` -- CREATE TABLE `field_fieldset_meta_end` ( `pages_id` int(10) UNSIGNED NOT NULL, `data` int(11) NOT NULL ) ENGINE=MyISAM DEFAULT CHARSET=utf8; -- -------------------------------------------------------- -- -- Table structure for table `field_fieldset_meta_END` -- CREATE TABLE `field_fieldset_meta_END` ( `pages_id` int(10) UNSIGNED NOT NULL, `data` int(11) NOT NULL ) ENGINE=MyISAM DEFAULT CHARSET=utf8; anyone ?
      whenever I make a new database and upload it there, database get imported without errors.
×
×
  • Create New...