Jump to content


Photo

Module: Import Pages from CSV file

Module

  • Please log in to reply
143 replies to this topic

#41 apeisa

apeisa

    Hero Member

  • Moderators
  • 3,161 posts
  • 1713

  • LocationVihti, Finland

Posted 01 February 2012 - 11:16 AM

I think that process should be right after install, so there have been something strange in your situation raydale.

#42 diogo

diogo

    Hero Member

  • Moderators
  • 2,900 posts
  • 2304

  • LocationPorto, Portugal

Posted 01 February 2012 - 11:17 AM

I assumed it would work by doing this. But it should be working by default...

#43 raydale

raydale

    Distinguished Member

  • Members
  • PipPipPipPip
  • 118 posts
  • 64

Posted 01 February 2012 - 11:26 AM

I think that process should be right after install, so there have been something strange in your situation raydale.


Ah, so I'm not being quite such a noob :)

In my case I wasn't taken through any extra steps and just left at the standard module overview screen (the one with 'Module Information').

#44 ryan

ryan

    Reiska

  • Administrators
  • 7,810 posts
  • 6604

  • LocationAtlanta, GA

Posted 01 February 2012 - 12:17 PM

I'm having problems using this module in PW 2.2. I have installed the module and then under 'Setup' and 'Import Pages From CSV' - I get the following message: "This page has no Process assigned."


Testing here in 2.2, but I can't duplicate. Can you think of any other factors involved? Has your site been through a profile export/import (like 2.0-2.1 upgrade)? Have any admin-related pages been renamed? Let me know if you can think of anything else that might be different from a stock installation.

Tahnks,
Ryan

#45 raydale

raydale

    Distinguished Member

  • Members
  • PipPipPipPip
  • 118 posts
  • 64

Posted 01 February 2012 - 02:25 PM

Hi Ryan,

The only thing I can think of is that I have renamed the admin pages url from 'processwire' to 'admin'. I will probably be doing a bit of work on this over the next few days with different localhost installations - so I'll see if I can reproduce it then.

#46 ryan

ryan

    Reiska

  • Administrators
  • 7,810 posts
  • 6604

  • LocationAtlanta, GA

Posted 02 February 2012 - 08:40 AM

It shouldn't matter if you've renamed the admin page, but thanks for mentioning it. Those are the kind of things that are good to know. Let me know if you find that you can reproduce it.

Thanks,
Ryan

#47 Michael Murphy

Michael Murphy

    Distinguished Member

  • Members
  • PipPipPipPip
  • 101 posts
  • 66

  • LocationBasel, Switzerland

Posted 11 February 2012 - 03:13 PM

I am trying to import some events which have two date fields - a start and end date.
I enabled the FieldtypeDate in the module as suggested by Ryan :

protected $fieldtypes = array(
  'FieldtypePageTitle',
  'FieldtypeText',
  'FieldtypeTextarea',
  'FieldtypeInteger',
  'FieldtypeFloat',
  'FieldtypeEmail',
  'FieldtypeURL',
  'FieldtypeCheckbox',
  'FieldtypeFile',
  'FieldtypeDate' // add this line
  );

However there seems to be some odd results after importing. Some event pages have imported both start and end date. others have only a start or end date. and then some have no dates at all. I have tried various times and made sure that the dates are in the correct format (d/m/Y) but no luck.

Here is the CSV file I was using

http://db.tt/h0fGuBHU


Was there any progress with the page field type import?

#48 Michael Murphy

Michael Murphy

    Distinguished Member

  • Members
  • PipPipPipPip
  • 101 posts
  • 66

  • LocationBasel, Switzerland

Posted 12 February 2012 - 04:08 AM

After some further testing, I may have worked out part of the issue.

Both the date fields were changed to use European style dates d/m/Y
However despite changing this in the field settings (output and input tabs) it was still only accepting the US style m/d/Y - hence why it was importing only valid US style dates and skiping the rest.

I have tried lots of variations for European style dates in the field settings and csv but it only accepts US style m/d/Y

After changing the main wire config file

$config->dateFormat = 'd-m-Y H:i:s';

it accepted the dates, but in my scripts it still seems to be calculating with the US style (I am adding / subtracting dates to work out time differences) - any date like 28/05/2012 is calculated wrong - this is still confusing me.

I am using the latest PW 2.2 from Github download.

Thanks,
Michael

#49 Nico Knoll

Nico Knoll

    The Boss.

  • Members
  • PipPipPipPipPip
  • 814 posts
  • 404

  • LocationBerlin, Germany

Posted 12 February 2012 - 10:03 AM

Maybe you should first rebuild a timestamp with "strtotime()". Then you can use it in every format you want/need.

#50 ryan

ryan

    Reiska

  • Administrators
  • 7,810 posts
  • 6604

  • LocationAtlanta, GA

Posted 13 February 2012 - 02:20 PM

Michael, have you tried changing your dates in the CSV to the d-m-Y format? I believe it should import then. The FieldtypeDatetime module sanitizes any string input with PHP's strtotime() function. That function accepts d-m-Y format if you use dashes in the input (rather than slashes).

#51 Michael Murphy

Michael Murphy

    Distinguished Member

  • Members
  • PipPipPipPip
  • 101 posts
  • 66

  • LocationBasel, Switzerland

Posted 13 February 2012 - 06:06 PM

I could not use the date format of d/m/Y that I wanted no mater how hard I tried. As an example of the strange results I was getting here is the code I used on the event page - the results are in the comments :

echo $config->dateFormat; // Y-m-d H:i:s
echo '<br />';
echo $page->event_start;  // 21/01/2012
echo '<br />';
echo $page->event_end;  // 15/04/2012
  
$date1 = strtotime($page->event_start);
$date2 = strtotime($page->event_end);
  
echo $date1;  // empty
echo '<br />';
echo $date2; // empty
echo '<br />';
  
$diff = $date2 - $date1;
  
$no_days = floor($diff / (60*60*24)) ;
  
echo 'duration : ' .$no_days;  // 0

This may have been a result of changing the date field formats too many times.

I deleted all the fields and created them again with default US settings. I then changed all my dates in the CSV to US style m/d/Y re-imported and now it works as expected!

I did not try the d-m-Y format yet (I could not find any country that uses this format in the OSX dates preferences) - it was easier to change to US date settings.

On a side note, I always thought the US date format was odd - the logical way would be small > medium > large (day > month > year). Are they are the only country that starts with a month ?

http://en.wikipedia....rmat_by_country


Thanks,
Michael

#52 Michael Murphy

Michael Murphy

    Distinguished Member

  • Members
  • PipPipPipPip
  • 101 posts
  • 66

  • LocationBasel, Switzerland

Posted 13 February 2012 - 06:23 PM

Just discovered there is a simple way to do the d-m-Y dates (or any format) in Apple Numbers without switching your main format settings - you just need to create a custom date format - see attached screenshot :

http://db.tt/SnvwLoA7

#53 ryan

ryan

    Reiska

  • Administrators
  • 7,810 posts
  • 6604

  • LocationAtlanta, GA

Posted 14 February 2012 - 09:40 AM

I don't know why the US date format is the way it is. The d/m/Y format does seem more logical. Maybe the US date format is meant as some kind of calendar hierarchy? The date format I prefer the most is Y-m-d, but mainly because it's string-sortable. :)

You don't necessarily need to go changing settings in your spreadsheet if you don't want to. Another way to convert your d/m/Y dates to d-m-Y (dashes rather than slashes) is to load the CSV into a quality text editor and then use a regexp to swap the slashes for dashes. Here's the search and replace I'd use in TextWrangler (BBEdit):

(\d{2})/(\d{2})/(\d{4})	 =>	\1-\2-\3

PHP's strtotime() function treats dates with dashes as European format and dates with slashes as US format. So once you convert those dates in your CSV to use dashes rather than slashes, they should import just fine with the CSV import module.

#54 Michael Murphy

Michael Murphy

    Distinguished Member

  • Members
  • PipPipPipPip
  • 101 posts
  • 66

  • LocationBasel, Switzerland

Posted 14 February 2012 - 01:19 PM

Thanks for the regexp tip. I am trying to keep a simple workflow for business users where all data preparation can be done in spreadsheets (excel / numbers / google docs) and then imported to Processwire. I'm a big fan of using spreadsheets as a middle ground or starting point for building applications especially where lots of data is involved.

I've not tested yet, but will the import work with the new multilingual fields? In the past, spreadsheets have been a useful way to translate content with multiple people.

#55 raydale

raydale

    Distinguished Member

  • Members
  • PipPipPipPip
  • 118 posts
  • 64

Posted 15 February 2012 - 01:04 PM

It's been a while - sorry for that!

I have just tested this module again in a live setup (I was previously testing on a localhost setup). I am still getting this 'problem' where upon installing the module 'Import Pages from CSV' I am simply returned to the module information screen with a highlighted message saying: 'Click to your 'Setup' page to start using the CSV Importer'. I am given no clue as to having to change the 'Process' dropdown for that module setup page in the page tree.

I take it this isn't the designed behaviour?

My localhost testing environment is (if any of this helps) a MAMP installation using PHP 5.3.2 and my live site setup is an Apache installation using using PHP 5.2.17

#56 ryan

ryan

    Reiska

  • Administrators
  • 7,810 posts
  • 6604

  • LocationAtlanta, GA

Posted 15 February 2012 - 04:42 PM

It sounds like we're using the exact same localhost testing environment.

This module creates a page: Admin > Setup > Import pages from CSV

Looking at the install function, it won't assign the process if there is already a page there named 'import-pages-csv'. If you've already got a page there with the same name, you'll have to edit it and select 'ImportPagesCSV' for the 'process' field and then hit save. However, assuming you don't have a page there named 'import-pages-csv', then it should take care of it all for you and be ready to use right after you've installed.

#57 raydale

raydale

    Distinguished Member

  • Members
  • PipPipPipPip
  • 118 posts
  • 64

Posted 16 February 2012 - 10:55 AM

Thanks Ryan,

I did copy over a lot of the content from my localhost - so that could explain a page already being there (assuming that's the case). Does uninstalling the module remove it's setup configuration (removing the page and processes)?

It's not a big enough issue to warrant spending much time on it - especially as I now know how to get around it.

#58 ryan

ryan

    Reiska

  • Administrators
  • 7,810 posts
  • 6604

  • LocationAtlanta, GA

Posted 16 February 2012 - 09:31 PM

Uninstalling should remove the page and process. Also there's no harm in manually removing the page either. But if you just want to add the CSV import process, then I would just edit the existing page that's already there and select 'ImportPagesCSV' and save, and that'll be the same as if it had installed it.

#59 wes

wes

    Jr. Member

  • Members
  • PipPip
  • 23 posts
  • 1

Posted 20 February 2012 - 02:36 AM

Thanks a lot for making this module, ryan. I've already used to to import over 400 pages... huge time saver.

The one suggestion I have was touched on by one or two people earlier in the thread: the importer doesn't work when the page is missing a 'title' field, presumably because it's used to generate the page name. It's pretty easy to get around by making a dummy title field, populating it with a CSV column, and then deleting it after importation, but it would be really great if during Step 2 we could specifiy a column from the CSV to use for generating a page name/slug.

#60 AlexV

AlexV

    Jr. Member

  • Members
  • PipPip
  • 21 posts
  • 1

  • LocationMoscow

Posted 13 March 2012 - 09:35 AM

Idea of ​​such exports is great! And thanks to Ryan for this!
As for me: I'm trying and trying! What is the problem, doc?
What I'm doing wrong or is this a known problem?
If i try to import returns: «Unable to import page because it has no required 'title' field or it is blank.»
My file structure is as follows:
"title"
"Yakimanka"
"Yaroslavl"
"Yasenevo"
What could it be?





Also tagged with one or more of these keywords: Module

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users