Switching from Typo3 to ProcessWire (will be continued)

[Nico Knoll]

Well, if you're a active user in this forum you may know, that I had to import a lot of data from an old typo3 page to ProcessWire. And maybe you're in the same situation right now, so I'll try to explain how I did it.

(In this part I'll only explain how to import the main data like pages and site elements to ProcessWire).

Preparation

---

The first step you have to do is to create a template with all the fields you want to have and a field called "typo3_uid", a field called "typo3_pid" and a field called "typo3_author". Later I'll explain the reason of this. Here is a collection of the data types typo3 returns and for which you should create fields for (you don't have to create fields for the red ones and the ones that are not highlighted):

"uid", "pid", "t3ver_oid", "t3ver_id", "t3ver_wsid", "t3ver_label", "t3ver_state", "t3ver_stage", "t3ver_count", "t3ver_tstamp", "t3ver_swapmode", "t3_origuid", "tstamp", "sorting", "deleted", "perms_userid", "perms_groupid", "perms_user", "perms_group", "perms_everybody", "editlock", "crdate", "cruser_id", "title", "doktype", "TSconfig", "storage_pid", "is_siteroot", "php_tree_stop", "tx_impexp_origuid", "url", "hidden", "starttime", "endtime", "urltype", "shortcut", "shortcut_mode", "no_cache", "fe_group", "subtitle", "layout", "target", "media", "lastUpdated", "keywords", "cache_timeout", "newUntil", "description", "no_search", "SYS_LASTCHANGED", "abstract", "module", "extendToSubpages", "author", "author_email", "nav_title", "nav_hide", "content_from_pid", "mount_pid", "mount_pid_ol", "alias", "l18n_cfg", "fe_login_mode", "tx_rlmptmplselector_main_tmpl", "tx_rlmptmplselector_ca_tmpl", "t3ver_move_id"

That are the database columns of typo3 pages. Not the contents of them (only the title).

The next step is about the contents of the pages (same game like above):

"uid", "pid", "t3ver_oid", "t3ver_id", "t3ver_wsid", "t3ver_label", "t3ver_state", "t3ver_stage", "t3ver_count", "t3ver_tstamp", "t3_origuid", "tstamp", "hidden", "sorting", "CType", "header", "header_position", "bodytext", "image", "imagewidth", "imageorient", "imagecaption", "imagecols", "imageborder", "media", "layout", "deleted", "cols", "records", "pages", "starttime", "endtime", "colPos", "subheader", "spaceBefore", "spaceAfter", "fe_group", "header_link", "imagecaption_position", "image_link", "image_zoom", "image_noRows", "image_effects", "image_compression", "altText", "titleText", "longdescURL", "header_layout", "text_align", "text_face", "text_size", "text_color", "text_properties", "menu_type", "list_type", "table_border", "table_cellspacing", "table_cellpadding", "table_bgColor", "select_key", "sectionIndex", "linkToTop", "filelink_size", "section_frame", "date", "splash_layout", "multimedia", "image_frames", "recursive", "imageheight", "rte_enabled", "sys_language_uid", "tx_impexp_origuid", "pi_flexform", "l18n_parent", "l18n_diffsource", "tx_gsttopcontent_abstract", "tx_gooffotoboek_function", "tx_gooffotoboek_path", "tx_gooffotoboek_webpath", "tx_dirlisting_path_to_rep", "t3ver_move_id", "crdate", "cruser_id", "tx_bktfegraphiccounter_visits", "tx_bktfegraphiccounter_align", "tx_bktfegraphiccounter_animate", "tx_bktfegraphiccounter_alt_tag", "tx_bktfegraphiccounter_mindigits", "tx_bktfegraphiccounter_html_before_img", "tx_bktfegraphiccounter_html_after_img", "tx_bktfegraphiccounter_checkreload", "tx_bktfegraphiccounter_istext"

Ok, you only need the "bodytext" and the "header" I guess (and of course "uid" and "pid" to identify the right page for this content).

Export data from Typo3

---

The next problem I hab was to export data from Typo3 (as a readable file!). And I figured out, that the best way is to use the CSV exporter extension because you can export single tables. And that is what we need.

If you installed this extension in your Typo3 install you just can follow this steps:

1. Open the modules page ("user tools" -> "CSV Exporter")

2. Now you can select the table you want to export. In our case this is "pages". In the checkboxes select "all". And don't forget to check the "export table titles as first line" checkbox! And set "split symbol" (or so) to "," not ";". At last press "next".

3. On the next page check "* (all fields)" and press the "create CSV and download it" button.

4. Save this file as "typo3_pages.csv".

Now repeat this for the contents. Only difference is that you have to select "tt_content" and not "pages" and save it as "typo3_content.csv".

The import process

---

(will be continued)

[MarionRaven]

Nico, I have no need for porting a typo3-site to pw right now.

I'm only curious. What happened since November?

[Nico Knoll]

I couldn't continue it because I wasn't sure anymore what I did exactly... But maybe I try to reconstruct this and add further information here.

[lisandi]

Hi Nico

Currently lots of people are actually switching from TYPO3 to other CMS and IMHO this will be also be the case in future as they simply made to many codebreaking changes to extensions where they kept the name, but after updates your complete site was blank etc. So people get frustrated. Like us too by the way.

We were looking long time for another CMS which has most or evenmore benefits than TYPO3 and I ampretty sure we found it in Processwire.

it is faster
very modularized
multidomain capable (i.e. we made a site for 400 churches in one TYPO3 - and they now need to update from TYPO3 4.5 to 6.2.6 and are also not happy that many things no more work like before)
multilingual and has multilingual SEO (if you adjust your module which you just released - it's great thanks)
you can have profiles and very easy backups of your sites and profiles using the profile exporter (a think that was missing in TYPO3 all those years!)

the core is separate and it can be updated much easier than in TYPO3

In TYPO3 6.2 you can now create profiles and import them >>> IMHO one way to build a migrator to processwire! and to move customers over,
another way is the .t3d export of complete branches or only single pages. It is an internal format for TYPO3 where you can afterwards all data back into another TYPO3 page. Also this could be a great way to use for an importer from TYPO3 to Processwire as it imports all kind of data and files and relations.

--
Like said we have lots of customers and usually they ask us to build a TYPO3 website (since 2002 we are doing that) but meanwhile we suggest that they switch toprocesswire. The main problem for all those companies and people is that they ran their sites often in TYPO3 for many many years and are now afraid that it will be very difficult to import their current site data from TYPO3 to processwire. If there would be an importer like your wordpress importer than it would be quite easy to switch customers to processwire.

Another 2 problems - (need to discuss that in another thread) is the flexible templating for editors what is available in TYPO3 and now wordpress and drupal but still not in Processwire. and the Digital Asset management which id great in TYPO3 as you can integrate dropbox, googledrive using a File Abstraction Layer and the build in fileadmin is great to with his Meta Data management (similar to DAM). Those things are still missing Processwire but I hope that the Gallery Extension will be released soon as it looks great. Let's see.

The biggest hurdle to convert customers from TYPO3 to Processwire is actually their data. When I did the coding of the UNESCO Bangkok website 2007-2010 they had about 550 pages with 16.000 Content Elements and 25.000 Digital Assets (in DAM Digital Asset Manager). East West Center (2008-2010) was very similar to that, but had additionally lots of user data coming from their editors and every day lots of news articles and their categories and tags and their specific permissions. This site was also TYPO3 and is now Drupal (also done with an importer to Drupal by Srijan Technology a former TYPO3 and now Drupal supporter) - http://www.srijan.net/work/typo3-drupal-east-west-center/

Those are sites we name here only as an example as we had worked before with it, but to be able to convert those sites incl their user rights management to processwire would open up very new perspectives to Processwire. This is Enterprise Level!

Beside this of course there are tons Universities, NGOs, Schools, Governmental, SME and lots of small TYPO3 websites which meanwhile often complain that TYPO3 in Version 6.2 is to slow, getting to instable and that with each new update you don't know if your site perhaps will turn blank/will be inaccessible. In other words NOW is the right time to crack that nut and convert more and more of the still existing customers to Processwire. A module which would do that would be just ideal for that purpose.

Perhaps you can "reconstruct it" ;-) even it's already 2 years ago you did give it a try.

Perhaps those two links canhelp you to get such a module done:

https://www.drupal.org/project/migrate
https://www.drupal.org/project/typo3_migrate
https://www.drupal.org/node/1188446

Similar to the processwire approach they are using a general Module and a CMS specific (in this case TYPO3) module
the latest update is 2013-Oct-19
 

[Nico Knoll]

Sounds like an important topic. And shouldn't be to hard to find a solution (maybe there's even a better exporter for typo3 now - I have to do some research).

Do you have  kind of an old dump of a site I could use for testing purpose? (Don't need it now but maybe later in that progress).

I can't promise that this will be finished really soon as it is a complex module and typo3's structure is often really complex, too, and I don't have a lot of time at the moment because of university and client work for continuing module development which is not client related. But I'll at least take a look on it now and we'll see. Maybe there's someone out there how wants to financially support (and therefore speed the development up) this project.

[adrian]

Nico - are you thinking of a Typo3 plugin for Migrator?

[Nico Knoll]

yes - but I'm not sure how to export the data from typo3...

[lisandi]

Hi Nico

Best is probably to use one of those introduction packages. And I pm you tomorrow with some example sites so you don't have the hazzle to need to setup TYPO3.

Export is possible via [.t3d typo3] - attention when you search in Google as t3d is also used for other stuff not TYPO3 related.

T3D is basically an XML file. Choose the uncompressed version if you do not intend to import it back to TYPO3
You can specify very exactly what needs to get exported and also what you wantto modify before importing or while importing i.e. some names or paths or email etc.

Than this .t3d puts all together db data, files from fileadmin/uploads and exports the complete stuff and you can import it elsewhere.

In TYPO3 6.2 you can now even make a backup from only one specific extension data tables and only export those.
this would be anotherway but it exports only db Data so related images or relations to other files and records might be missing but they are not missing when using .t3d!

The good thing on .t3d export is that you can export only a Page or one record , a branch or severeal specific pages or branches or a complete site etc. It is very very flexible. Sometimes it is necessary to do several exports because in the past ithad problems when the .t3d was getting much to big.

Die extension die diesen export gewaehrleistet heist impext. (import/export)
https://forge.typo3.org/projects/typo3cms-core/repository/revisions/master/show/typo3/sysext/impexp
http://typo3forum.hosting-agency.de/vorgefertigte-typo3-templates-als-t3d-import-t3004.html
da hast du quasi einblick in die Erzeugung dieser Datei .t3d.

---------

Bezueglich des einlesen woanders wuerde ich bei dem Drupal migrator und typo3_migrator von Drupal (ab)schauen.
https://www.drupal.org/project/TYPO3_migrate
but it can actually only export tt_news /tt_news categories and standard content elements like text/text with image / image etc

---------

how to import a site via .t3d

since TYPO3 6.2. there are distributions. also they work with .t3d files
distributions are more or less similar to profiles in processwire or distributions in drupal.
http://wiki.typo3.org/Distributions
https://forge.typo3.org/projects/typo3v4-distribution/wiki
http://docs.typo3.org/typo3cms/CoreApiReference/ExtensionArchitecture/CreateNewDistribution/Index.html

Some distributions are here:
https://github.com/Ecodev/typo3-cms-speciality-distribution
http://wiki.typo3.org/Introduction_Package
http://typo3.org/extensions/repository/view/introduction

------

In real sites (beside the introduction package) you usually have also extensions installed. I wil setup some examples tomorrow for you so you can use them.

-------

TYPO3 is facing currently lots of challenges as they started to abandon the very userfriendly templaVoila templating engine in favor of less friendly stuff like
flux
https://fluidtypo3.org/

or Gridelements
http://t3sbootstrap.de/t3sbootstrap/

with Fluid Flux
http://t3bootstrap.de/de/typo3-bootstrap-template/

with TemplaVoila in 6.1.*
http://t3bootstraptv.de/de/typo3-bootstrap/

or a combination of in TYPO3 Themes
http://typo3-themes.org/ (it is very very flexible as it orientates on the framework for templaVoila but uses now gridelements to store the data as a real record)

Framework for templaVoila - many US Church websites have been build with that and they are now without that package anymore as the webempowerd church which made all this great userfriendly stuff in TYPO3 possible abandonned TYPO3. Here would be a great resource for conversion projects in US and Germany.
https://github.com/webempoweredchurch
http://www.hopoffacloud.com/webempoweredchurch/

Lots of chiurch websites have been build on the wec starter package
https://github.com/webempoweredchurch/starter_package

with one click more or less you have a complete and complexe website for churches up and running incl example data - with calendar, events management, jobboard, podcast extension, userrights management, news, blog, guestbook, gallery and much more -  all those are now actually potential processwire customers if we could provide them an easy way to switch and a similar church package but based on processwire.

The Framework for TemplaVoila whichwas the base for that package in its version 1.* has also been used by lots of companies all around the world.
http://templavoila.busynoggin.com/

They all are facing now the same problem.
TemplaVoila is working in 6.2. but the newest framework for templavoila extension which is 2.1.3 is not publicly available in TER. so many don't even kmnow how to get it (it's actually still free but you ned to contact the developer) To be honest most customers like all that hazzle anymore which ahrmed TYPO3 a lot since years. So they are looking for alternatives.

As TYPO3 6.2. is also slower than 4.7. and many extensions don't work anymore so they arefrustrated and woudl need to convert their old TYPO3 site (many based on templaVoila) to the new version 6.2.6.

The time needed to do that is probably much much more than simply starting using Processwire ;-) and converting their data and content to Processwire. The great thing is that Processwire already has more features which are usefull for enusers than TYPO3 IMHO - it has separate core, is multilingual and multidomain capable (we currently have to convert a multilingual church website for 400 churches in one TYPO3 from TYPO3 4.5 LTS to TYPO3 6.2.6 LTS and it is also using that framework of Ron Hall and we finally will convert the content to be used with typo3-themes. CRAZY if you ask me but there is currently no other way to get the site again working in TYPO3.

More and more customers with actually the same problems will come.

From a marketing perspektive I would suggest that Processwire needs an import tool to import TYPO3 sites from TYPO3 4.5 LTS / TYPO3 4.7.* / TYPO3 6.1.* and TYPO3 6.2.5+ mainly for sites using TemplaVoila!!! Those are the customers sitting on a sinking ship with a huge whole in it! ;-) Prosesswire can be the safe groud for all of them.

Afterthat migration tool exists we start promoting it in the German and English Onlline Magazines i.e. http://t3n.de - this will open up the eyes of lots of customers. Additionally we can ask the former manager of the WEC to help promotinga Processwire Church Pakcage and of course also the Company Package which includes that migrator. I am pretty sure that lots of people will switch as especially the saftey and security record of processwire is definitley better than The one of TYPO3 and a huge huge difference to stuff like those insecure applications since years i.e. wordpress, joomla, and drupal.

Checkout the secunia security index.

http://secunia.com/community/advisories/search/?search=processwire
http://secunia.com/community/advisories/search/?search=typo3
http://secunia.com/community/advisories/search/?search=joomla
http://secunia.com/community/advisories/search/?search=drupal
http://secunia.com/community/advisories/search/?search=wordpress

I hope that helps a bit to make clear what treassure is actually out there.

Andi

[Nico Knoll]

Wow, thanks for the long answer! 

I think if you prepare the site and examples Adrian (if he wants to) and I can make something good out of it

[FrancisChung]

Hi @adrian, @Nico Knoll 

Is there anything happening with this project?

[jensweigel]

Are there still somebody who is interested in this topic? I made some silly solutions for my personal purpose. Maybe we can exchange know how.

For the base I structure my PW sites similar to Typo3. I have pages like Sites and pages like pagesections. SImilar to Typo3 they have different kinds (text, pictext, pic, …) 

The current task is to rewrite the tt_news urls. This is so crappy if you see how easy PW handle it.