Using page fields like a database?

[h4r4ld]

Hi,

I'm pretty new to processwire, but I'm digging in really fast. My actual project is an invoice management site where one can store data like in a document managment tool, organizing invoices, searching, keywording and so on.

My actual problem is that I expect about 10.000 documents a year (based on what I bring myself  ) and I'm not sure on how to do so.

As far as I know yet, every fieldmap is stored in one page, which means I could try to handle every document in a hidden page, which would mean my pages will probably exceed all given limits?

I need to work with those fields, so I need direct access to them aswell as a good speed fetching all of them (at once, filtered, ...).

So I'm a bit aware what could be the best way to do this with PW (and if so, how?) or handle a new database for this without PW?

Best,

Harri

[LostKobrakai]

…, which would mean my pages will probably exceed all given limits?

Exactly the opposite will be the case. Pages are build to scale almost infinitly, while each page does automatically retreive all it's fields' data at once, which doesn't scale nearly as well. So in you're case I'd suggest using a single page per document, as you most likely will always need all data associated to a document, but you're not always in the need to get all documents ad once.

[h4r4ld]

So in you're case I'd suggest using a single page per document, as you most likely will always need all data associated to a document, but you're not always in the need to get all documents ad once.

Ok, so there is no limit in having documentpages at all?

How about the speed compared to use a database directly while searching lots of documents?

[kongondo]

Database speed is very fast....Just don't retrieve all your 10K pages at once ....Use 'limit' in your selector and you will be fine.

Welcome to PW and the forums

[h4r4ld]

Thanks for the welcome

Coming to use pages, should I put them into a parent "container" page? 

Every document will get an ID (barcode), I thought about using this as the pagename?

If I'm not totally wrong than I should be able to do something like this:

// fetch one document
$item = $pages->get($barcode);

// fetch all documents
$items = $pages->get("/parentpage/");

Is this the suggested way to do?

How about using RegEx in the get function?

*edit:

Just looked into selectors, just like regex I can use something like this?

// fetch all documents ; limit 10
$items = $pages->get("/parentpage/", limit=10);

[Martijn Geerts]

@h4r4ld, have a good read about selectors  99% is explained in those docs.

[h4r4ld]

@h4r4ld, have a good read about selectors  99% is explained in those docs.

That's gone be my evening meal.

Can anyone give a best practice advice on this:

Should I put my documentpages into a parent "container" page? 

[pwired]

What about concentrating on code that can search quickly through the pages if you want to retrieve an invoice ?

[Martijn Geerts]

@h4r4ld

Every page needs to have a parent except the home page

[h4r4ld]

What about concentrating on code that can search quickly through the pages if you want to retrieve an invoice ?

That, in PW, will be done by selectors if I interpret Martijns post right?

Putting pages in a pagecontainer would be more or less only for organizing the pages more nicely on adminpages from my view yet. I just don't know if I would break up some problems ahead while doing so. And before concentrating on quick search I need a good data storage where I can perform a search.

[Martijn Geerts]

My actual problem is that I expect about 10.000 documents a year (based on what I bring myself   ) and I'm not sure on how to do so.

What is the problem of thousands of pages? PW can handle a multitude of that without problem.

As far as I know yet, every fieldmap is stored in one page, which means I could try to handle every document in a hidden page, which would mean my pages will probably exceed all given limits?

Sorry I don't understand the question.

I need to work with those fields, so I need direct access to them aswell as a good speed fetching all of them (at once, filtered, ...).

So I'm a bit aware what could be the best way to do this with PW (and if so, how?) or handle a new database for this without PW?

I doubt that the browser can handle 10.000's inputfields at ones, so that problem is more about the client side then severside.

 

Handling big data is easily done with ListerPro. Out of experience, managing 500.000 pages with ListerPro is not a problem.

[h4r4ld]

What is the problem of thousands of pages? PW can handle a multitude of that without problem.

Sorry I don't understand the question.

I doubt that the browser can handle 10.000's inputfields at ones, so that problem is more about the client side then severside.

Handling big data is easily done with ListerPro. Out of experience, managing 500.000 pages with ListerPro is not a problem.

Ok got that.

Got this runing now and my form automatically adds a new page with all fields once a document is added.

Is it in this kind of "datasource" possible to make a field unique or like primary? As there seems to be no real relation between each page I was wondering how I can keep track if not use the pagename? Like for example the barcodefield, where every barcode is only allowed to be used once.

[Macrura]

Is it in this kind of "datasource" possible to make a field unique or like primary?

page name should be enforced to be unique already, but you could use this for text fields:

http://modules.processwire.com/modules/fieldtype-text-unique/