Making custom application

[matjazp]

I need an advise. I might need the "custom application" (nothing specific in mind) and I'm not sure how to start. I could create forms by hand (<input name=... type=...>), I could create forms in PW way ($f = $modules->get("InputfieldText"); $form->append($f); etc.) but I would like to do it more "visually". I could use FormBuilder (never used before, just saw the video). I could render the forms from templates, like in the admin area (ProcessPageEdit.module?). I could make my own admin theme as full blown admin would probably be too much. What is you advise? How to start? Thanks!

[Nico Knoll]

Use Form Builder - it's really easy to use and offers a lot of after submit actions like sending a mail and saving the data. And it's really easy and fun to create forms with it. So if you really only need a form you should use it

[Ivan Gretsky]

Tell a bit more about your aplication so we can give some more help. If your application sends email than Form Builder is the way to go for sure.

And you should get Form Builder anyway. I am getting my copy of a developer version this week. It is great!

Edit: Got it!

Edited April 17, 2015 by Ivan Gretsky

[matjazp]

Let say that I need enterprise application for inventory of the equipment. The firm buys 100 laptops. Each laptop would get id number (automaticaly generated), name, location, date of purchase, responsible person etc. The user would need an option to modify & delete the items, make a search base on all fields (eg. what items are located in specific location, what are the items older than 5 years) and make reports as PDF. Emailing is not needed at this point. As I noticed in the video for form builder, collected data from the forms could be put into pages. The amount of data would be in range of 10.000 items per year and would need the archive for at least 10 years so 100.000 pages is bare minimum.

[Beluga]

Let say that I need enterprise application for inventory of the equipment. The firm buys 100 laptops. Each laptop would get id number (automaticaly generated), name, location, date of purchase, responsible person etc. The user would need an option to modify & delete the items, make a search base on all fields (eg. what items are located in specific location, what are the items older than 5 years) and make reports as PDF. Emailing is not needed at this point. As I noticed in the video for form builder, collected data from the forms could be put into pages. The amount of data would be in range of 10.000 items per year and would need the archive for at least 10 years so 100.000 pages is bare minimum.

Just a note that complex reporting seems to be a weak spot performance-wise in PW based on this anecdote.

[Ivan Gretsky]

I think you can find a ready to use solution for this task cheaper than writing your own (as it seems like a really common task). And a new one could be simplier to develop from ground up or some general framework with classical 1 table per object approach.

[matjazp]

@Beluga, I already saw post from jordanlev.

@Ivan, I know I can find a ready to use solution, we already have one, my question was more hypothetical. Thanks for your opinion.

[jordanlev]

@matjazp , for my custom application that I recently made in ProcessWire, I assumed that leveraging the built-in ProcessWire tools would save me a lot of time because I would not have to code up my own "CRUD" forms (dashboard forms for the site owners to add, edit, and delete records). I created "templates" and "fields" instead of creating my own database tables and fields. Many people had suggested that utilizing the PageTable field type would give me a ton of flexibility in how I structure my data relationships and turning those into editable interfaces without requiring any programming on my part.

However, I ran into a few problems. The first big problem is that I did *NOT* want this portion of the application to be managed via the "pages" sitemap tree in the dashboard. I know that it can be very flexible and would save me a lot of development time, but it does not provide the streamlined and intuitive interface for my clients that I like to deliver to them... it just doesn't make sense to them why they would be editing their products in the same exact kinds of forms as their content pages. In addition to the sitemap tree not making sense for this use case, there are just too many tabs of fields all over the place, and it was difficult to understand how (or if it was even possible) to customize this editing interface for my clients.

So, what I wound up doing is creating my own dashboard pages so I could customize the forms exactly the way I wanted. This was okay once I figured out how to "carve out my own niche" in the dashboard, but it took a lot of work. It was difficult to leverage the built-in "input" modules so that I could avoid building my own form logic (see this post for some details on how to make it even possible, because out-of-the-box it doesn't work without some modifications: https://processwire.com/talk/topic/8061-problems-with-image-uploads-when-including-processeditpage-form-on-another-admin-page/ ).

The 2nd problem I ran into is that all of the template and field definitions are stored in the database, as opposed to config files or code itself. I understand the benefits of this in terms of making certain kinds of applications very easy for non-programmers to build because they can "point and click" their way to defining a data structure. But for my team this was a big challenge because it means we can't keep the actual data schema and template/field settings in version control. We had to pass around a database from designer to developers, and back and forth from staging servers and development machines and the live server.

The 3rd problem I ran into is when it came time to do aggregate reporting. As I alluded to in that other thread you linked to, the ProcessWire system is basically a "database within a database", where each field is its own table. I had a fairly complex schema (although not totally crazy in my opinion) where I had customers, delivery locations, delivery schedules, orders, line items, and products all combined into one list (basically "show me all of the products in all of the orders that are to be delivered to customers on day x at location y"). This site has tens of thousands of orders per month, so the query was dog-slow (like, taking 1-3 minutes to run). What I had to do was spend a LOT of time "untangling" the ProcessWire query logic and write my own SQL query with a lot of JOINs in it... after I made this custom SQL query it ran in under 1 second!

So, at the end of this project what I realized is that as a programmer who already has a lot of experience building my own forms and my own "CRUD" interfaces and my own database schemas and SQL queries, as well as my business being based on delivering very customized user interfaces to my clients... ProcessWire had nothing to offer me on the "custom dashboard application" side of things. All it did was put some new layers of abstractions that I had to learn and then work around in my way, when what I really wanted was to build things the tried and true way that has worked well for me for over a decade (HTML forms, PHP form processing via a "controller" of some kind, and a database schema with SQL queries for complex reporting).

All that being said, I did not use the "Form Builder" that others are talking about, but I'm not sure that would offer the functionality you're needing in a dashboard interface (maybe it does, I don't know -- haven't used it). Also I understand that my requirements may be a bit more advanced than others... if someone did need to build out a custom application and was okay leveraging the built-in tools and compromising on a rather clunky editing interface, then ProcessWire would be an amazing tool to use. And of course I'm only talking about custom applications here... in terms of the templating and the front-end portion of the site, working with PW is a dream come true... I gladly await the day when every CMS and platform out there has the API consistency and "stay out of my markup"-ness as ProcessWire

[kongondo]

@jordanlev,

I have read the last couple of posts you've written about your experience with ProcessWire in respect of your 'custom/complex' application and I do understand some of your frustration. However, I think some of your assertions may convey the idea that somehow PW corners you to do things in a certain way. Or that ProcessWire is a one-size-fits-all for all kinds of needs. For instance, PW is not and has never claimed to be an e-commerce application. Whilst you can certainly build and e-commerce site with PW (or even a product - there's one in the mix actually) your end product depends on how you code your application. I am not in any way questioning your coding skills; just trying to point out that PW is mainly a tool....what you do with that tool is really up to you. However, for some jobs, you are better of using tools better suited to those jobs, e.g. foxy cart, etc for e-commerce sites.

Secondly, the adage that everything is a page does not mean that you should always take that approach. Again, there are examples here in the forum. You can use custom tables, designed just the way you want. Although using pages gives you access to the PW API, you are not tied to using them. And here is another strength of PW...you can use custom tables with ease, and either use custom CRUD code or PW $database and $db to access them (though with some limitations) and you are good to go. Not many CMS will allow you to do this...at least not with ease

Thirdly, I disagree with the notion that PW is not up to the job when it comes to custom applications. There's a couple of examples of custom apps built using PW right here in the forums. I don't agree that its only good in terms of the 'templating and the front-end portion of the site'. So this statement "ProcessWire had nothing to offer me on the "custom dashboard application" side of things." is not exactly true. PW allows you to build your own custom dashboards by not getting in your way. Yes, it may not offer a turn-key experience (although there are one or two dashboard modules) but it allows you to build your own without bothering you.

Fourthly, PW does not demand that you use its inbuilt 'input' modules or as you put it 'a rather clunky editing interface' when building custom apps/modules with it. In fact, that's one of the strengths of PW. You can build your custom apps using custom code or extend other libraries and PW will happily let you. The 'input modules' are used internally by PW but are also there for convenience if you need to use them, but you are certainly not tied to using them. Build your complex forms by hand or another library? PW doesn't care nor will it get in your way. 

Anyway, if I misread your posts, I apologise. However, you've made the same assertions twice or thrice and I thought they needed clarification/qualification especially for newbies.

Hope I haven't derailed the original discussion

Edited April 21, 2015 by kongondo

[matjazp]

@jordanlev, thank you for your thoughts, I was facing with similar problems. I quite like how edit pages works, but for end users I needed something just a little bit different. I didn't want to reinvent the wheel so I ended up creating my own ProcessPageEdit module, based on the original. There I hid the tabs (now we have module for that) and changed the logic about saving and redirecting. Then I didn't want header and footer, so again, I just grabbed AdminThemeDefault and made my own theme, where I get rid off unwanted stuff. I can elaborate further if needed.

[qtguru]

First of all Jordanlev thanks for the input, I do understand what you mean, its until i came to the point where i felt It was more easier to build my own dashboard into Processwire than go with the Everything is a page "Mantra" and kongondo has a very valid point, there are situations where everything doesn't have to be a page. Though I think because most of the forum focuses on using pages for nearly everything. I do disagree that Processwire has nothing to offer, because it still gives the ability to easily wire modules and follow your own style. I would say we are both building the same type of applications where viewing the information in Pages would be a bit weird for clients.

@Jordanlev I would be eager to get some of your experiences and practices as it seems we are both dealing with the same type of application. which is why i feel processwire 3.0 should be discussed on a round table so we can delibrate on how to enhance Processwire much more better.

@kongondo thanks lot because of you and Adrian moving from Wordpress to Processwire with all my articles was seamless and quick. 

[jordanlev]

@kongondo - I think we are saying the exact same thing, but perhaps my words have more of a negative spin than yours do. I agree with @Sephiroth that there seems to be a "mantra" on the forums that "everything is a page" and that the built-in tools can handle anything and everything... I think this can hurt newbies like myself who go down the wrong path. I just wanted to have a contrary opinion out there so people can see the "boundaries" or limits of the system and make informed decisions about their projects. (Also the OP specifically asked me for my thoughts via a private message

@Sephiroth - I would be delighted to have a more detailed discussion about these things... I am *very* opinionated about how I think CMS's should work What is the best way to engage in a conversation about this? (I'm not sure I'd be able to just write it all up, but if it was more of a back-and-forth conversation or answering specific questions I can certainly do that).

The last thing I will say is that all of you people here on the forums are wonderful and if ProcessWire continues to grow and become successful, it will be as much because of the helpful and friendly community as it is because of the technical underpinnings. So keep up the great discussions!

Thanks,

Jordan

[LostKobrakai]

The 2nd problem I ran into is that all of the template and field definitions are stored in the database, as opposed to config files or code itself. I understand the benefits of this in terms of making certain kinds of applications very easy for non-programmers to build because they can "point and click" their way to defining a data structure. But for my team this was a big challenge because it means we can't keep the actual data schema and template/field settings in version control. We had to pass around a database from designer to developers, and back and forth from staging servers and development machines and the live server.

For such advanced usecases there's always the possibility to version control the json export for templates/fields. The problem of database schema changes is more a problem of relational databases and not the cms itself.

[jordanlev]

@LostKobrakai - I wouldn't consider "more than one person working on a site" an advanced use case. In my experience, most CMS's suffer from this problem, so it's not something that is unique to ProcessWire -- but it seems to me that for some situations, storing the *definition* of the field structures in a config or code file, while keeping the data in the database is ideal.

I utilize this system with a plugin I offer for the Concrete5 CMS where designers can set up their own content field definitions via a point-and-click dashboard interface, but the results of their pointing-and-clicking is saved to a php array. The data itself is *not* normalized into separate database fields though... so there is definitely a tradeoff (the plugin I make is best suited for page content that is not queried in any context other than "give me all the content on this one page that is being viewed"), and I don't think the way ProcessWire is structured would allow for such a setup. But other CMS's do utilize this approach -- Bolt CMS and October CMS come to mind, as well as the flat-file CMS's such as Statamic and Kirby.

So I'm not suggesting that ProcessWire should (or can) be changed, but it is a limitation of building an application in PW. And another reason that if you're building a "custom application" as part of the larger website, you might want to consider using plain old SQL files or a db migration library and do it the old fashioned way instead of leveraging ProcessWire's "everything can be a page" approach.

[pideluxe]

You are saying that building custom applications in PW with the "everything is page"-mantra has its limitations - and that's right! No one has said, that every possible application can and should be built with PW-pages. PW is a Content Managment Framework, and for most of the contents you need for the web, its possibilities are more than enough.

I think it is not fair by judging over PW while developing an ecommerce-application as your first application. Look at most of online-shopping suites like Magento or Prestashop, how difficult they were to develop just with php(-frameworks). Some applications are not best suited for PW (or any other CMS), at least with not building some helping modules or components, which could make it easier for plug the application together.

Just another thought: would you have considered any other CMS with its built-in editing & API to develop an ecommerce solution on top of that?

[Martijn Geerts]

It's rarely I build 'websites' with ProcessWire, most projects I've done this year are of the type application, tool or generator. It has taken a while before I recognised that the admin area is full of potential and could be easily used as starting point. The starting point in the admin is really well designed. You have the ability to hook, to build process modules where you want it, gain access access to or forbid. Easy implementation of Ajax, the ability to use Inputfields, Fieldtypes breadcrumbs tables etc etc and a wide variety of caching possibilities...

The thing about everything is a page is a plus. Pages are API wise very well designed, you can use it and take advantage of it or drop it when you don't want to use them. When I have the option to use pages I will do it the pages way. If it makes more sense to go the plain SQL way you could go that way. But I do remember I mistakenly chosen SQL ones where pages were a way better fit.

[Beluga]

@jordanlev would you consider creating a site profile with pre-populated data of that dog-slow reporting tool? Then it could be used as a benchmark and perhaps even lead to improvements

I mean, who knows what sort of Neo4j solutions will appear in the future!

[jordanlev]

@pideluxe - I think we are in agreement. I am not judging the entire PW system on the ability to create custom application functionality... all I did was explain where the limitations are when trying to do so (again, because the OP asked me to, not because I'm actively trying to give PW a bad name or anything).

@Beluga - I unfortunately cannot do this because the data contains customer names and contact info. And I unfortunately don't have enough spare time to go through and anonymize the data or recreate it with dummy data (sorry, I agree it would be a good case study though).

[Beluga]

@Beluga - I unfortunately cannot do this because the data contains customer names and contact info. And I unfortunately don't have enough spare time to go through and anonymize the data or recreate it with dummy data (sorry, I agree it would be a good case study though).

Ok. I guess CSV import could be used or Batcher together with this to provide dummy content: http://modules.processwire.com/modules/process-field-generator/

[Beluga]

@jordanlev: I wonder, if this recent improvement would make your reporting use-case fast: http://processwire.com/blog/posts/processwire-3.0.6-brings-pages-upgrades-and-link-abstraction/