Distinct Pages (API): Split Topic

[joer80]

We really need an api way to pull a distinct list of pages, or one that is similar to mysql group by.   The way processwire joins every column makes it a bit hairy to do this manually when you step outside the api.

ie.  Say you need to get pages using template model returning only unique model names and titles.   Right now you would loop through every page adding them to a unique list array.

When you get up to a certain size of database, it doesn't make sense to loop through every product page to make your list of unique models for say a select box when databases can just return a unique list.

Needing to step outside the api at times is a big turn off I think.

[joer80]

I also think we need to have a modules type section for front end design.  Processwire excels at backend information, but we need to excel at making and sharing good looking field/row/template styles.

[LostKobrakai]

I also think we need to have a modules type section for front end design.  Processwire excels at backend information, but we need to excel at making and sharing good looking field/row/template styles.

ProcessWire is a PHP cms. It's inherently backendy and even built to be as decoupled from actual markup as possible. For css / js things you'll probably need to look elsewhere.

[joer80]

The cms should be decoupled, but we should make it easy for people to share design as well. Even if you are not sharing the cms but layers that fit on top.

[Pete]

We really need an api way to pull a distinct list of pages, or one that is similar to mysql group by.   The way processwire joins every column makes it a bit hairy to do this manually when you step outside the api.

ie.  Say you need to get pages using template model returning only unique model names and titles.   Right now you would loop through every page adding them to a unique list array.

When you get up to a certain size of database, it doesn't make sense to loop through every product page to make your list of unique models for say a select box when databases can just return a unique list.

Needing to step outside the api at times is a big turn off I think.

I think this only usually comes up in more advanced use cases. I can only think of a handful of times I've needed to do this and it's always outside of "simple" websites.

I see your point though - something simpler than having to use $database->query() etc would be nice, like a helper for the more common things like getting just a title and ID in an array without loading pages into memory, something like $pages->queryArray('template=name, fields=f1:f2:f3, group=whatever, sort=f2'); I mean, you're nearly at a normal query there, so not sure how much merit there is in that, but there might be some value in having that automatically return your array or being able to iterate over it nice and cleanly might be worthwhile.

I'm not sure though - the "long way" is only 1 or 2 lines longer, but maybe something that looks more familiar to the rest of the API for simpler things might be better

[joer80]

The number of lines is not the issue.  Just one line of code can either be neat and readable or a mess.  

Saying the api works for simple websites is not good enough either.  We need to be able to work for medium difficulty websites and custom intranets in an elegant way.

Check out this difference in one of my websites.  At the shear number of manual joins I had to do just to return 3 columns manually of one template.

Lets say you are working with cars and want to build a drop down that pulls all models attached to an inventory, the model id, model name, and make name so you can pass that id to another page in a form submit, and show the user what that id is.  ie.  '123'  "BMW" 3 Series

You could do a group by or a distinct select.  This is what it would look like if you drop to sql outside the api:
 

$SQL = "SELECT Distinct modelPage.id as modelPageID, field_livefeed_model.data as modelTitle, field_livefeed_make.data as modelMake FROM field_livefeed_condition, pages, field_livefeed_model, field_livefeed_make, field_livefeed_status, pages as modelPage, field_title as fieldtitleModel WHERE (field_livefeed_condition.pages_id = pages.id) and (field_livefeed_model.data = fieldtitleModel.data) and (fieldtitleModel.pages_id=modelPage.id) and (pages.id = field_livefeed_model.pages_id) and (field_livefeed_make.pages_id = pages.id) and (pages.id = field_livefeed_status.pages_id) and (pages.templates_id='53') and (field_livefeed_status.data = '1') ORDER BY field_livefeed_condition.data ASC";

I did template id 53 because I didnt want to add yet another join for the template name.


Or I could have just done something like:

$pages->queryArray('template=Inventory, status=1, groupby=Model, sort=title'); 

 

or distinct could be something like

 

$pages->queryArray('template=inventory, distinctfields=model->id|model->title|make->title, sort=make->title');


I think its worth adding even if its no extra lines of code.
 

[teppo]

The cms should be decoupled, but we should make it easy for people to share design as well. Even if you are not sharing the cms but layers that fit on top.

Currently we're catering for this need with Markup modules, site profiles, and to some extent Textformatters. In my opinion these are all "easy ways to share design".

If these are not enough, you could also share template files and smaller snippets of code, but considering that each site may have different fields in place, that only works as long as you also standardize those fields. There has been some discussion about this too, but more as a community project than something the core should take part in.

If you're using the devns branch, there are also field-level template files. These would be much easier to share between different projects, so I'd suggest taking a closer look at them.

Saying the api works for simple websites is not good enough either.  We need to be able to work for medium difficulty websites and custom intranets in an elegant way.

I've built quite a few sites using ProcessWire, ranging from very simple one-page sites to news and intranet sites with (tens of) thousands of pages. I've never felt that the API "only works for simple websites". Admittedly I've used SQL on some extremely rare occasions, but that has mostly been due to the fact that I've either had a custom database structure too, or I've needed to optimize a potentially slow query for just one very specific task.

Assuming that the API is awesome for *at least* 95% of use cases (that's my impression anyway), I don't really think we can state that it's "not good enough". For use cases that a few people here and there might face in some rare situations it's entirely acceptable to figure out a custom solution.

Or I could have just done something like:

$pages->queryArray('template=Inventory, status=1, groupby=Model, sort=title');

or distinct could be something like

$pages->queryArray('template=inventory, distinctfields=model->id|model->title|make->title, sort=make->title');

Just checking, but you do realize that you could easily add a custom queryArray() method to $pages API var and even publish your solution as a module if you feel that it has potential for wider use, right?

That being said, I'm not against this feature per se. It's just that every added feature also adds complexity and the core absolutely shouldn't contain features that are known to be used extremely rarely (not entirely sure if that's the case here, but based on my experience so far it very well might be.)

[Ivan Gretsky]

This is all too far from the original post, which is about marketing) But PW API is so great you just can't stop talking about it everywhere. When I am reading something in paper format, the "everything is a page" concept keeps distracting me from the contents .

[joer80]

I really think we can do just a few things and convert almost all wordpress websites over.

Site profiles serve a great role, and are perfect for providing a turn key website starting point as is, but they are not specific enough to take processwire mainstream.   They only start a project, they can not add to it and make it custom.

From what I have seen, Markup modules and Text formaters will set your html bones in a spesific way and set the class names for you, but they are not bringing style/css options to users.  As far as I know, we don't serve the need I am talking about.  

We need to have a styles section for sharing design that styles those html tags and bones the markup modules generate.  But they need to be tied to modules they are made for.

Field level template files are in the same boat.  It will set the html bones automatically, and give you say a bootstrap naming scheme for a field, but it wont style it various ways.

We need some way to export/package up and share, and preview the look and feel of a section of a page.   This is what I suggest:

We should be able to pack up a repeater group, and upload it to the processwire website as a repeater group module, and let people import the whole repeater group at once.   

Now, in addition, people should be able to attach style options/css to that repeater group module listing also.  CSS already set for that repeaters class names.  

They could import the repeater group with or without a selected sub style into their existing project.  That is a wordpress killer.
 

So you pull up the repeater group, and there would be a tab for compatibile styles attached to that module.  Then you could preview them and choose to package that in with the module download.

Also, people could upload new css design options to a repeater group module without needing to be the author of it.  Just name the style, and provide the css file and images needed to lay on top of that repeaters bones.

Users would be able to download a "contact us" repeater group, and then preview and select which css style they want to download and run with it.  

Or a user can select a Homepage carousel repeater group, with a, b, or c design option included in the download.

I have never had luck moving things like repeaters from one project to another.  This should be exportable and importable.

The design section should let you preview the design/group before you download it.  Once installed, the css and js snippets would just get merged into your existing files.  No matter how many design modules, it doesnt add requests.

To do this, the design module would need its own assets folders to hold any images, js, css, etc. that were used.  The cms would loop through the design module folders and merge them into one file when a new module was added. 

We are a good niche for people that hand code their own design, but something like this would take the project mainstream into people that dont do their own design, but want to have something better than wordpress to build on top of.  

I really think that is why wordpress is growing so fast.  You have a lot of exiting design to choose from.

I think expanding your audience is key to growing processwire in 2016.  Meeting more peoples needs.

The lack of group by in the api just kinda drives me nuts because I use it often and have seen people suggest looping through every page using the api and adding to another array and I dont think that is a good solution.

[WillyC]

thats.sound like hole diffrnt cms no ? WirePress 

why sholud pw main stream ?

pw is seaworthy.vessel

not canooe for stream

[joer80]

I don't think allowing repeater groups in the modules section makes it a new cms, or a canooe, or take away the power of the api.  

It would however expand your audience.

The repeater group should be decoupled from css style by default, but we also should also allow people to upload and share compatible styles to that group as an option.

Les wordpress and moar processwire everwear is a good thing for earth.

[teppo]

I really think we can do just a few things and convert almost all wordpress websites over.

Our main audience are currently web designers/developers that require a platform that does most of the heavy lifting for them, but allows them to freely modify data structures, markup, and even back-end functionality of their sites.

WordPress, on the other hand, is famous for trying to cater for all kinds of users – even those who don't ever want to touch or see any code *or* markup at all. As a result they're not particularly good at anything and just trying to keep their current codebase in good shape seems to be nearly impossible task.

 

I'm not saying that ProcessWire is already as good as it can get – quite the contrary. We've taken huge steps in so many ways in just a few short years and will no doubt continue doing so in the foreseeable future. The point I'm trying to make is that "converting all WordPress websites over" isn't necessarily a goal worth striving for.

At the very least that's not what motivates me personally.

 

Site profiles serve a great role, and are perfect for providing a turn key website starting point as is, but they are not specific enough to take processwire mainstream.   They only start a project, they can not add to it and make it custom.

Correct. Site profiles provide a starting point, and after installing one of those you can customize your site as you see fit.

 

While I have no idea what you mean by "making it custom", the thing is that site profiles in general are not interchangeable like WordPress themes. This is what I was referring to in my previous post: we've had some discussion about agreeing on specific fields, based on which we could develop interchangeable themes. Not entirely sure what's up with that project right now, but it sounds like this might be something you'd be interested in pursuing too.

 

From what I have seen, Markup modules and Text formaters will set your html bones in a spesific way and set the class names for you, but they are not bringing style/css options to users.  As far as I know, we don't serve the need I am talking about.

We need to have a styles section for sharing design that styles those html tags and bones the markup modules generate.  But they need to be tied to modules they are made for.

Field level template files are in the same boat.  It will set the html bones automatically, and give you say a bootstrap naming scheme for a field, but it wont style it various ways.

Actually you could make any module output styles, and some modules already do that. Some existing options include using the $config->styles or $config->scripts arrays (though they also need to be included in your markup somewhere), altering generated markup with a page render hook, and simply outputting styles or scripts in the middle of generated markup.

I'll have to agree that these may not be exactly clean solutions at all times, but on the other hand, have you ever debugged a WordPress site with a ton of plugins adding their own styles/scripts? Trust me, "clean" is not how one would describe that either

 

We need some way to export/package up and share, and preview the look and feel of a section of a page.   This is what I suggest:

[...]

 

We are a good niche for people that hand code their own design, but something like this would take the project mainstream into people that dont do their own design, but want to have something better than wordpress to build on top of.

In my opinion this is a very cool idea, but absolutely not something the core should do. Without going into specifics, nothing you've mentioned so far can't be done with a third party module. If this is something you feel is missing from ProcessWire, feel free to give it a try. You might even find some like-minded people here who'd be willing to help.

 

I really think that is why wordpress is growing so fast.  You have a lot of exiting design to choose from.

In some ways I'll have to agree, but it's really not that simple. WordPress a) was around at the perfect time, b) decided relatively early to cater for the needs of everyone out there, c) doesn't care about how they're perceived by the technically adept audience, and d) doesn't care how much technical debt they amass in the process. I don't think that any platform can really take them head-on when it comes to number of users: at this point their popularity is also their biggest asset (assuming that "being the most popular platform out there at any cost" is really a sensible goal.)

 

I think expanding your audience is key to growing processwire in 2016.  Meeting more peoples needs.

Agreed, and we *are* already expanding our audience. You haven't missed the Composer stuff we've been adding lately, have you? It's just that we've been aiming for expanding our audience to a slightly different direction than you had in mind

 

The lack of group by in the api just kinda drives me nuts because I use it often and have seen people suggest looping through every page using the api and adding to another array and I dont think that is a good solution.

I hate repeating myself, but I'll do it anyway: build a module. Publish it. If it's awesome and a lot of people love it, it has a good chance of becoming a core feature one day.

[DaveP]

Competing with WP is a dead duck. The things 'missing' from PW are completely absent from CodeIgniter, CakePHP and Laravel as well, but those three have waaay more 'market share' than PW.

[joer80]

Core already offers field imports and exports.  When you export a repeater field that has fields in it, it should bring the sub fields with it.  It would save so much time.  That should be in core in my opinion.   

Lets forget about why other cms are so popular.  It would make our lives easier to group fields and export them as a group.  The side effect of the automation and faster dev times is more people using the system.

My suggestion of allowing people to upload style tweaks to existing repeater groups to the processwire website is not something I can do through a module.  Unless modules can edit the processwire website.   It is something that would get less experienced developers involved in the project.

This is pretty detailed, I created a feature request post for this.

[teppo]

Core already offers field imports and exports.  When you export a repeater field that has fields in it, it should bring the sub fields with it.  It would save so much time.  That should be in core in my opinion.   

Lets forget about why other cms are so popular.  It would make our lives easier to group fields and export them as a group.

This doesn't sound completely unreasonable. I'm sure Ryan will take it into consideration.

In the meantime you might want to give ProcessMigrator a try. I get that it's not the built-in solution you're asking for, but for sharing those repeater fields between projects it might be the easiest approach in the short term.

My suggestion of allowing people to upload style tweaks to existing repeater groups to the processwire website is not something I can do through a module.  Unless modules can edit the processwire website.

It's entirely possible to create a module that adds a repeater field, or any other field for that matter. We don't currently have a specific module category for "field-creating modules", but quite a few modules create fields or even templates as a part of their install procedures.

If you've got questions about creating a module that adds fields or does any other specific task, you can always post them to the module/plugin development subforum.