Jump to content
jrtderonde

Adding pages through the API

Recommended Posts

Hey,

I'm working around a module that adds pages in my ProcessWire installation. The pages are added based on a JSON array that will be imported through a page save. Everything works fine except for when the pages are added. I get the following error

Integrity constraint violation: 1062 Duplicate entry '3e215ecd6774fd99c2b0eb5cadf36a07-1269' for key 'name_parent_id'

I'm using the following code/loop to generate the pages.

// Loop through the files
foreach ($p->importFile as $file)
{
    // Set the file location
    $name = $file->data["basename"];
    $path = $p->importFile->path;
    $location = $path . $name;

    // Get the file
    $json = file_get_contents($location);
    $json = json_decode($json);

    // Loop through the rows of the import
    foreach ($json->ttEntityDelAddrLink as $client)
    {
        // Create new page
        $new = new Page();

        // Create unique hash
        $unique = md5(date("Y-m-d H:i:s") . "-" . $client->CustomerCode);

        // Set some variables for the new page
        $new->setOutputFormatting(false);
        $new->template = "_client";
        $new->parent = $p;

        // Create hash
        $new->title = $client->DelAddressName;
        $new->name = $unique;

        // Page specific fields
        $new->company = $client->DelAddressName;
        $new->companyId = $client->CustomerCode;
        $new->city = $client->DelAddressCity;
        $new->address = $client->DelAddressStreet;
        $new->postcode = $client->DelAddressZipCode;
        $new->country = $countries[$client->DelAddressCountryCode];

        // Save the page
        $new->save();
    }

    // Exit for debugging
    exit;
}


Does anybody know what's wrong?

Share this post


Link to post
Share on other sites

If the CustomerCode can be identical, you're likely creating identical names within a second. In that case, I'd add a simple counter to the page name seed.

 

  • Like 4

Share this post


Link to post
Share on other sites

Ok, but that's why I'm trying to prevent with the hash as $page->name. I'm using the ID of the customer combined with a timestamp to generate a md5 hash that's "unique". (See the code, I'm pretty sure it should work). 

Also, the next entry in the array has a complete different customer id. So it should be "impossible" to be duplicated.

I'm using this code within a pageSaveHook (module). Could this be a problem? Furthermore, the array counts 11000+ entries, could it be that this is a problem? The platform is hosted on a stable and fast VPS and I cranked up the PHP execution time to 10 minutes, so it shouldn't cause any serious trouble.

Share this post


Link to post
Share on other sites
1 hour ago, jrtderonde said:

Ok, but that's why I'm trying to prevent with the hash as $page->name. I'm using the ID of the customer combined with a timestamp to generate a md5 hash that's "unique". (See the code, I'm pretty sure it should work). 

What @BitPoet said. In other words, PHP is fast, MySQL is fast. So, there is no guarantee that you will have only 1 page created per second (your date), or even per microsecond! That leaves you with uniqueness dependent only on the customer ID ($client->CustomerCode). If you can guarantee that customer IDs are unique and that they only appear once per data set, (your JSON array) then you would be fine. The fact that you are getting a SQL error tells us that either your customer IDs are not unique and/or one customer ID is appearing more than once in your data set (i.e., it may be uniqe to the customer, but not the data set). If the database says so, then it must be so ;). Let me illustrate (non-unique customer ID and/or replicated customer ID):

 

$clientCustomerCode = 12345;// replicating repeated use of customer ID
$p = $pages->get(1722);

for ($i=0; $i < 30; $i++) { 
	// Create new page
	$new = new Page();
	// Create unique hash {no guarantee this will be unique if same customer ID used}
	$unique = md5(date("Y-m-d H:i:s") . "-" . $clientCustomerCode);
        // even these (microseconds) do not guarantee uniqueness throughout
	#$unique = round(microtime(true) * 1000) . "-". $clientCustomerCode;
	#$unique = microtime(true) . "-". $clientCustomerCode;
	#$unique = microtime() . "-". $clientCustomerCode;

	echo $i . ': ' . $unique . '<br>';// display 'uniqueness'; you should see duplicates here

	// Set some variables for the new page
	//$new->setOutputFormatting(false);// not needed for a new page (I think)
	$new->template = "basic-page";
	$new->parent = $p;   
	// Create hash
	$new->title = "Title " . rand(5,15);// just for testing
	$new->name = $unique;// not really unique!
	// Save the page
	#$new->save();// you'll get PDO error here within a second

}

Using the above code, you'll be hit by the duplicate entry violation error within a second...

Edited by kongondo
  • Like 6

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By louisstephens
      So I was tinkering around with the "select fields" field type and added it to a repeater. My thoughts were I could have a user select a field (textarea, text, etc etc) that I defined and give it a name (another field in the repeater) and create their own form on the page. To be honest, I am now a little lost with rendering the form and mailing the results as potentially the form will be unique and custom every time.  The only way I know to handle the output is by going about it this way:
      $forms = $page->form_select_fields; foreach($forms as $form) { if($form->name === "form_input") { //output input with custom name } elseif($form->name === "form_textarea") { //output input with custom name } } Is there a better way to go about rendering the elements from the repeater? As far as the custom sending goes, I am really at a loss since it would be pretty dynamic. Has anyone used this type of approach, and if so, how did you handle this without going insane?
    • By Harmen
      I want to add a few pages to an AsmSelect Page field inside a repeater using the following code:
      $trialsPage = wire("pages")->get(28422); // Get the page $trialsPage->of(false); $newTrial = $ordersPage->trial_repeater_orders->getNewItem(); // Add item to repeater foreach ($selectedProducts as $selectedProduct){ $productPage = $pages->get("template=product, reference=$selectedProduct"); $newTrial->trial_selected_products->add($productPage); } $newTrial->save(); $trialsPage->save(); However, when I check the page where the field is located it doesn't have the new values as expected. The selected pages exist, the field is in the right location, made sure that the output formatting is turned off: $page->of(false); But it still doesn't work with a variable. No matter what I try, it doesn't work.
      It only works when I replace $selectedProduct with a hardcoded string. Am I doing something wrong here?
    • By louisstephens
      So I have been diving into hooks lately, and I am enjoying them thus far. However, I guess I am a bit stumped on how to achieve what I want too. I am trying to set up a hook that would create a new child page when the parent page is saved. However, when you save the parent page a second time, I just need to update the child page without creating multiple child pages. What would be the best way to go about this?
      So after rereading my post, I believe it is a bit vague so I will try to explain more. 
      The Goal:
      Create a page with a template "one". Once the page is created/saved => create a new child page with the template of "two" If the parent is saved anytime after, do nothing to the child page (limit the parent page to one child page) The parent page is really just being used to output content, whereas the child page is being used to pull out the some fields from the parent to be used elsewhere. I might have made this too complicated in my head.
    • By David Karich
      The Page Hit Counter module for ProcessWire implements a simple page view counter in backend. Page views of visitors are automatically tracked on defined templates, with monitoring of multiple page views. This gives you a quick overview of how many visitors have read a news or a blog post, for example, without first having to open complex tools such as Google Analytics. This module quickly provides simple information, e.g. for editors. Or, for example, to sort certain news by most page views. For example for "Trending Topics".

       
      Works with ProCache and AdBlockers. With a lightweight tracking code of only ~320 bytes (gzipped). And no code changes necessary! In addition GDPR compliant, since no personal data or IP addresses are stored. Only session cookies are stored without information. 
      In addition, there are some options, for example filtering IP addresses (for CronJobs) and filtering bots, spiders and crawlers. You can also configure the lifetime of the session cookies. Repeated page views are not counted during this period. It is also possible to exclude certain roles from tracking. For example, logged in editors who work on a page are not counted as page views.

      Sort by hits and access page views (hit value)
      Each trackable template has an additional field called phits. For example, you want to output all news sorted by the number of page views.
      // It is assumed that the template, e.g. with the name "news", has been configured for tracking. $news = $pages->find("template=news, sort=-phits"); To output the page views of a tracked page, use:
      echo $page->phits; Example: Tracking a page hit via API and jQuery
      If you want to track a template that does not represent a full page to automatically inject a tracking script, you can define allowed API templates in the module that you can track. Below is an example of how you can track a click on news tag using jQuery. This will allow you to find out which keywords are clicked the most. For example, you can sort and display a tag cloud by the number of hits. Suppose your keywords have the template "news_tag". The template "news_tag" was also configured in the Page Hit Counter Module as a trackable API template.
      Example PHP output of keywords / tags:
      // Required: the data attribute "data-pid" with the ID of the template to be tracked. echo $pages->find("template=news_tag, sort=-phits")->each("<a href='{url}' class='news_tag' data-pid='{id}'>{title}</a>"); Example Tracking Script with jQuery:
      /** * Required: Data attribute "data-pid" with the ID of the news tag template * Required: Send the POST request to the URL "location.pathname.replace(/\/?$/, '/') + 'phcv1'" * Required: The POST parameter "pid" with the ID of the template */ $(function(){ if($('a.news_tag').length > 0) { $('a.news_tag').each(function(){ var tPID = $(this).data("pid"); if(tPID) { $(this).on("click", function(){ $.post(location.pathname.replace(/\/?$/, '/') + 'phcv1', {pid: tPID}); }); } }); } }); So simply every click on a tag is counted. Including all checks as for automatic tracking. Like Bot Filtering, Session Lifetime, etc.
      _______________________________________________________
      Background: This module is the result of a customer requirement, where the editors are overwhelmed with analytics or no tracking tools were allowed to be used. However, a way had to be found to at least count page views in a simple form for evaluations. Furthermore, by using ProCache, a way had to be found to count views of a page without clearing the cache.
      _______________________________________________________
      Pros
      Automatic Page View Tracking Lightweight tracking code, only ~320 bytes (gzipped) No code or frontend changes necessary Works with ProCache! Even if no PHP is executed on the cached page, the tracking works Works with browser AdBlockers No cache triggers (for example, ProCache) are triggered. The cache remains persistent GDPR compliant, session-based cookie only, no personal information Filtering of IPs and bots possible Exclude certain roles from tracking Ability to reset Page Views Works with all admin themes Counter database is created as write-optimized InnoDB API to track events for templates that are not viewable No dependencies on libraries, pure VanillaJS (Automatic tracking script) Works in all modern browsers Pages are sortable by hits Cons
      Only for ProcessWire version 3.0.80 or higher (Requires wireCount()) Only for PHP version 5.6.x or higher No support for Internet Explorer <= version 9 (Because of XMLHttpRequest()) No historical data, just simple summation (Because of GDPR) Planned Features / ToDos
      API access to hit values Since version 1.2.1 Possibility to sort the pages by hits (Request by @Zeka) Since version 1.2.0 Don't track logged in users with certain roles (Request by @wbmnfktr) Since version 1.1.0 Possibility to reset the counter for certain pages or templates (Request by @wbmnfktr) Since version 1.1.0 Better bot filter Since version 1.1.0 Disable session lifetime, don't store cookies to track every page view (Request by @matjazp) Since version 1.2.1 Option to hide the counter in the page tree (Request by @matjazp) Since version 1.2.1 Option to hide the counter in the page tree on certain templates Since version 1.2.1 API to track events for templates that are not viewable Since version 1.2.2 Changelog
      1.2.3
      Bug-Fix: Tracking script triggers 404 if pages are configured without slash (#3) Reported by @maxf5 Enhancement: Reduction of the tracking script size if it's gzipped (~320 bytes) Enhancement: Documentation improvement Enhancement: Corrected few typos 1.2.2
      New feature: API to track events for templates that are not viewable Enhancement: Documentation improvement 1.2.1
      API access to hit values Use $page->phits Bug-Fix: No tracking on welcomepage (Reported by wbmnfktr; Thx to matjazp) Bug-Fix: Tracking script path on subfolders (Reported by matjazp) Bug-Fix: Tracking on pages with status "hidden" Enhancement: Change database engine to InnoDB for phits field Enhancement: Option to disable session lifetime set session lifetime to 0, no cookies Enhancement: Better installation check Enhancement: AJAX Request asyncron Enhancement: Reduction of the tracking script size by ~20% Enhancement: Option to hide the counter in the page tree You can output the counter with the field name "phits" Enhancement: Option to hide the counter in the page tree on certain templates Enhancement: Option for activate general IP validation Enhancement: Reduction of tracking overhead up to ~30ms Enhancement: Better bot list for detection 1.2.0
      New feature: Sort pages by hits – New field phits Migrate old counter data to new field 1.1.0
      New feature: Exclude tracking of certain roles New feature: Reset Page Views Better bot filter and detection 1.0.0
      Initial release Notes
      By default, the page views are stored as INT in the database. This allows a maximum counter value of 4.2 billion views (4,294,967,295) per page. If you need more, change the type to BIGINT directly in the database. But I recommend to use Google Analytics or similar tools if you have such a large number of users.
      _______________________________________________________
      Download GitHub: ProcessWire Page Hit Counter (Version 1.2.3)
      PW Module Directory: ProcessWire Page Hit Counter (Version 1.2.3)
      Install via ProcessWire (Classname): PageHitCounter
      _______________________________________________________
      Update information
      If you have used version 1.2.1 from the DEV branch, please replace it completely with the new master version.
×
×
  • Create New...