Jump to content

Importing data - skip duplicate records


Recommended Posts

I'm importing data from an external system into ProcessWire. Each of the imported records have a unique ID. How might I use the unique ID to prevent duplicate records in ProcessWire?

What I want:

  • When importing data
    • find the foreign record ID
    • use it to check for duplicates
    • if a duplicate record is detected, skip it
  • import data into ProcessWire by creating a page in /import/
  • continue to next record
  • ProcessWire: Later I'll move each page to another location
    • ...  meaning that '/import/' will be empty the next time, I run an import job

Looking into possible solutions:

  • Unfortunately, ProcessWire Unique Text Fieldtype doesn't do the trick. (ProcessWire Unique Text Fieldtype will allow the page to be created - only the field with a duplicated field value will be blank. In my case, the duplicate record needs to be blocked / *not* created.)
  • Could it be solved by modifying the code for ProcessWire Unique Text Fieldtype?
Link to comment
Share on other sites

Why just not create a text field like 'externa_id' and then check it while import like

foreach ($import_records as $import_record) {
    $exists = $this->wire()->pages->has("template=some-template, external_id=" . $import_record['unique_id']);
    if($exists) {
        // update
    } else {
        // create a new page


  • Thanks 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Create New...