Running a Daily Newspaper website with Process Wire.

[Crssp]

Is the Process Data module in beta or documented in the download:

http://modules.processwire.com/modules/process-data-import/

Not really trying to get off track here, with beginning a project and seeing what I can do.

Beyond the data, the prospect of how to wire it all together, to dynamically get the 7 most recent issues is another thing to get my head around.

We are a daily with the articles being published daily except sundays at around 1am in the morning.

During the day we publish breaking news that usually works it's way into the following days articles queue.

Even to get the data into a PW database and bootstrapping it elsewhere would be cool, such as being able to call it in another system, if nothing else.

[apeisa]

Process Data import expects data to be either in xml or json, so no help here. Custom format, custom import.

[Crssp]

Superfast on the draw with that reply, thanks mate!

I see your solution is sort of in the works, popped in to the forum thread a little bit on that to learn more:

http://processwire.com/talk/topic/237-module-twitter-puller/#entry1647

[Crssp]

The thing is, the current issues we would probably try to pull in just the RSS version, which would be XML possibly.

I'm way out of my element on anything data related

KISS, and keeping it simple is the best way for me to go.

But there's no way a project like this would ever fall quite into the simple category, lol.

My best strategy would be to go AGILE and then add features as I'm able.

[Crssp]

The current site, creates a menu and an index per story type to access those.

This screenshot sort of explains how it was previously done in the navigation, for past days.

There is an index template, and menu list of articles per category, accessed this way from the drop down.

[Pete]

...

Beyond the data, the prospect of how to wire it all together, to dynamically get the 7 most recent issues is another thing to get my head around.

...

Even to get the data into a PW database and bootstrapping it elsewhere would be cool, such as being able to call it in another system, if nothing else.

Well the first bit depends on how you set up your templates and fields in ProcessWire, then you can easily do something like this to get the last 7 issues and output a simple title for each of them with a link:

<?php
foreach ($pages->get('/issues/')->children('sort=-date, limit=7) as $issue) {
    echo "<a href='$issue->url'>$issue->title</a><br />";
}
?>

As several of us have said, everything's really easy once it's in ProcesWire - it's getting it there that will be the big hurdle. As you can see above, I've assumed we have everything under a page called "issues", then sorted the child pages by date descending and limited the results to 7, looped through them and output the issue title and link. I realise this isn't the structure you have, but it's really easy to build any site structure you want in ProcessWire.

The second bit I quoted from that particular post of yours was more so I could ask why you would do that? ProcessWire is perfect for building this kind of site so I don't understand why you would make it work alongside something else when this is what it's designed to do?

[Crssp]

Cool Pete...

Hey I hit @processwire up on Twitter, the link on the sites download page, needs to have the .zip extension added for it to work...

Page:

http://processwire.com/download/

Link should be for the Master.zip file:

https://github.com/ryancramerdesign/ProcessWire/archive/master.zip

Might confuse some folks thought i'ld add it here, since it's high-priority community item of a headsup nature

[Crssp]

I'm noticing some of the major news site, have the same structure, at least the virtual path shows in a similar fashion:

NY times for example:

http://www.nytimes.com/2012/11/27/followed-by-a-title (fake link)

Then to get to the current version they show with another path:

http://www.nytimes.com/pages/todayspaper/index.html

But stories still the same path.

Also they maybe have categories or something in the paths:

http://www.nytimes.com/2013/01/14/world/africa/french-story-etc

What field etc. should I look at as far as importing stories to the database, will really help me out, just with moving forward with learning the system.

For example in the Tutorial about planets, does it matter, it's just a field, and I can then relate to that field.

 

Working through the tutorial, and maybe it will all click, page, pages, page array wire array, sort of muddled now....

thanks for all the awesome help.

[Joss]

In ProcessWire, before you try to do anything tricksy with it, your Tree Structure will be reflected in your menu and consequently in your URLs

Which is nice and simple. So for instance, you might have a page under home called Sports and under that you would have all your sports pages.

Of course, in your case, with so many pages, that would make your menu unusable!

However, the system allows you to list pages by their parent, so, you could create an index listing page called Sport that sits in the top menu.

Then, somewhere else on your page tree, you could create a Hidden Page (published, but it wont show up on search or the menu), called Sport-Articles.

Then, all your articles would be children of that page.

Back on your index listing page you can use a very short piece of code that basically says: Return all pages that are children of Sports-Articles and Display the Title of the page. (I am leaving out the code at this point just so that you understand the workflow for the moment).

Now, suddenly, you have a very simple listings page, that you can add pagination to, that will work its way through those articles. From that point, you can get the URL of the article which, when clicked, will display that page using the bespoke template for the Sports Articles that you have designed.

You will probably have a more complex system than that with the pages sorted in a more complicated way, but the principle will remain the same.

Note: The difference between $page and $pages is that $page is used to get fields from the current page and $pages is for getting fields from another page or a group of pages (using a foreach loop, normally).

It can get a little confusing - the difference between a template and a template file, for instance - but it is less to understand than you think.

Once you have done the planets, have a go at  http://wiki.processwire.com/index.php/Basic_Website_Tutorial

It is a much more detailed tutorial which takes you through a few more of the techniques that you commonly use. It is longer, but not too bad, and has a little bit of scripting for you to copy and paste.

The tutorial is very new and though it has been tested by a couple of people, it may still have some pitfalls for very new users, so I apologise in advance if it gets complicated.

On the upside, it is written by me, and I am not technical - so it is aimed at other non-tech people!

What is going to be essential for your project, is going to be to decide how you want your site to break down. What sections will you have, how will the articles be listed, how will they be displayed and so on.

For instance, you may want a different feel to the sports page than the politics page, each with their own submenu of sections and search - but have a common navigation at the top, or similar. With the amount of articles and folders, it sounds like you may need something like that!

[Crssp]

Hey guys... sorry for muddying the waters on the archived stories/articles data.

Turns out the current .asp tool that publishes the story data, is pretty flexible.

The output can be modified, and the stories can be FTP'ld.

Given these new ideas to the mix, how do I get the FTP stories to the processwire db on a daily basis, is there anything built-in or add-on you would suggest there. I am talking about the new stories then on this, moving on from the whole archival stories issue.

Beyond this one I wish I knew more about database structure, etc. to figure out the best and efficient way to deal with the individual stories vs what is a days issue. I have an idea of what templates I will need to start based on the story type.

Also started looking at PHPquery vs the simpleHtmlDOM resource for working with the archived stories.
http://code.google.com/p/phpquery/

There is plenty of info in this thread for me to dive back through already.

Gracias'

[ryan]

Given these new ideas to the mix, how do I get the FTP stories to the processwire db on a daily basis, is there anything built-in or add-on you would suggest there. I am talking about the new stories then on this, moving on from the whole archival stories issue.

I don't think you'll need any add-ons specifically. But if you are grabbing stories via FTP, you'll want to use PHP's built-in FTP functions to pull them. But other than that, the rest should be similar to the process from earlier in the thread. 

[Crssp]

@Pete I've been reading all sorts of geeky stuff, there's a couple goodies where you mentioned, in your code post:

"// Some code to iterate through the folder structure needs to happen, but when we have a file we do something like this:"

There's the PHP5 directoryIterator:
http://www.techrepublic.com/blog/programming-and-development/how-do-i-recursively-scan-directories-with-phps-directoryiterators/417

And also there's a TreeIterator, I've not yet found a applicable copy paste, beg borrow steal script.

Something like that might be on the right track though, are you familiar with DirectoryIterator?

[Pete]

A few of the comments on the scandir() function are probably your best bet: http://php.net/manual/en/function.scandir.php

It's allfine using other scripts and copying and pasting from examples, but nothing beats understanding how it's actually accomplished, and in this case a recursive scandir() function is probably all you need.

[OrganizedFellow]

@Crssp

How's your project coming along?

This topic popped up while I was searching for a similar solution.