JSON structure with direct access to key

[jploch]

Hey folks,

for a module (a pagebuilder based on PageTable) I need to save some settings as JSON. The values are saved for each page table item (a pw page). It's working well, but I am looking for ways to improve the structure I have. As I'm not that experienced with JSON, maybe someone more experienced can take a look and tell me if my approach is good practice. 

My goal is to make all the items accessible by page id, without looping over them (using objects instead of arrays):

// access from template with pw page var
$jsonObject->items->{$page}->cssClass;

Her is an example of my JSON structure:

{
  "items": {
    "3252": {
      "id": "3252",
      "cssClass": "pgrid-main",
      "breakpoints": {
        "base": {
          "css": {
            "grid-column-end": "auto",
            "grid-row-end": "auto",
            "grid-column-start": "auto",
            "grid-row-start": "auto",
            "align-self": "auto",
            "z-index": "auto",
            "padding-left": "60px",
            "padding-right": "60px",
            "padding-top": "60px",
            "padding-bottom": "60px",
            "background-color": "rgb(255, 255, 255)",
            "color": "rgb(0, 0, 0)"
          },
          "size": "@media (min-width: 576px)",
          "name": "base"
        }
      }
    },
    "3686": {
      "id": "3686",
      "cssClass": "test_global",
      "breakpoints": {
        "base": {
          "css": {
            "grid-column-end": "-1",
            "grid-row-end": "span 1",
            "grid-column-start": "1",
            "grid-row-start": "auto",
            "align-self": "auto",
            "z-index": "auto",
            "padding-left": "0px",
            "padding-right": "0px",
            "padding-top": "0px",
            "padding-bottom": "0px",
            "background-color": "rgba(0, 0, 0, 0)",
            "color": "rgb(0, 0, 0)"
          },
          "size": "@media (min-width: 576px)",
          "name": "base"
        }
      }
    },
    "3687": {
      "id": "3687",
      "cssClass": "block_editor-3687",
      "breakpoints": {
        "base": {
          "css": {
            "grid-column-end": "span 2",
            "grid-row-end": "span 1",
            "grid-column-start": "auto",
            "grid-row-start": "auto",
            "align-self": "auto",
            "z-index": "auto",
            "padding-left": "0px",
            "padding-right": "0px",
            "padding-top": "0px",
            "padding-bottom": "0px",
            "background-color": "rgba(0, 0, 0, 0)",
            "color": "rgb(0, 0, 0)"
          },
          "size": "@media (min-width: 576px)",
          "name": "base"
        }
      }
    },
    "3696": {
      "id": "3696",
      "cssClass": "block_editor-3696",
      "breakpoints": {
        "base": {
          "css": {
            "grid-column-end": "span 2",
            "grid-row-end": "span 1",
            "grid-column-start": "auto",
            "grid-row-start": "auto",
            "align-self": "auto",
            "z-index": "auto",
            "padding-left": "0px",
            "padding-right": "0px",
            "padding-top": "0px",
            "padding-bottom": "0px",
            "background-color": "rgba(0, 0, 0, 0)",
            "color": "rgb(0, 0, 0)"
          },
          "size": "@media (min-width: 576px)",
          "name": "base"
        }
      }
    }
  },
  "breakpointActive": "base",
  "breakpointActiveSize": "@media (min-width: 576px)"
}

 

[ukyo]

@jploch You can check my Mystique module

 

[jploch]

 @ukyo Thanks! Your module looks great! However I already have a way to build my JSON based on inputfields, wich is just needed to save some internal css of my module. All the real content is saved in the database with regular pw fields. I would like to keep it that way, but was looking to get some feedback on my JSON structure. Specially I'm not sure if it's valid to have numerical keys (page id)? Everything is working fine, so if it's not bad practise for some reason, I will just keep it that way.
 
Doing it all in JSON (including the content) with your module would have the benefit of being able to transfer the templates to other installations without importing the database and also have it version controlled, right? But the downside is that it's not searchable as well as native fields and also that not all fields are supported?
 

[ukyo]

I have builder system with PageTable input field,

Text field is InputfieldText Field (Searchable)

Properties field is Mystique Field (Limited search support)

Current Mystique field version support search limited. Working with next version on my side and it will support searching.

[jploch]

cool! but still can someone look at my code and tell me if the JSON structure is fine?

[MoritzLost]

@jploch You're mixing up storage and public API, which is not a good idea. JSON is a storage / transport format, it's always a string which contains a serialization of data. The purpose of JSON is to store or transmit data in different formats between systems – like your database, your server and the client's browser. JSON is also language-independent – you can decode a JSON-string in JavaScript and in PHP. But as soon as you do that, the result is that language's default representation of that JSON string. For PHP, this means you get either a stdClass object or arrays (depending on the flags you give to json_decode).

So you need to ask two different questions: What is the best way to store my data, and what is the best way to present it to a user of your module?

Data representation

Of course you can just json_decode the JSON string and pass the result to the user of your module. But that's a bad idea:

• You have zero guarantees regarding the structure of your data. You won't notice if your data is missing a specific field, some field is an integer instead of a string, your items are indexed by page name instead of key or whatever else. This will cause errors that are very hard to track down.
• The user has no indication how your data is structured – to figure that out, I will either have to refer to the documentation for everything or have to dump stdClass objects and figure things out from there ...
• You can't have nice things like code completion, proper IDE support etc.

Instead, you want to parse that unstructured data into a representation (usually, a set of classes) that guarantees a specific structure and has a proper public API for access and modification of the data. Using proper objects with type-hinting will also allow you to find any errors in the structure and abort the process at parse time with an appropriate error message (see Parse, don't validate for an in-depth explanation). For example, to represent your data, you might use classes like PageBuilderEntry and PageBuilderItem to represent a list of items with meta data (the breakpoint fields, in your example) and individual items, respectively. The you can provide utility methods for different use cases – for example, methods to get a list of items indexed by key or by any other field.

If you want to build closer to ProcessWire, you can also use a class that extends WireArray to represent a list of items, this will give you many helpful methods (like getting an item by key) out of the box. For the items themselves, you might just reuse the page itself (if the IDs correspond to 'real' pages in your system) or extend the Page class.

Data storage

Separating the representation from the storage makes the specific structure you store your data in less important, since only your module will interact with it. Just build a static helper method to build the appropriate representation based on a JSON string (e.g. PageBuilderEntry::fromJSON) and a method to encode that representation back to JSON (PageBuilderEntry::toJSON). The fromJSON method is responsible for parsing the data and throwing the appropriate error when it's not in the expected format.

That said, personally I usually prefer regular arrays instead of maps for storage, because it's easier to parse and massage into different formats. But again, it depends on how you want to represent the data.

[jploch]

@MoritzLost Thanks for your explanation! Helps me to understand this a little bit better.

2 hours ago, MoritzLost said:

Of course you can just json_decode the JSON string and pass the result to the user of your module. But that's a bad idea:

Iam not sure the data will ever be relevant to the user, for now I just use the JSON internally (creating JSON with js and decoding it with PHP). What I meant with "using objects instead of arrays" here is just how I decode the JSON in php: json_decode($jsonString) instead of (json_decode($jsonString, true). This gives me easy access to the items (which are just pages, so I thought it would be nice to be able to access them by page id like the pw api works). To be able to do this I also need to create the JSON structure like in the example above (with JS). Hope that makes sense. Not sure if the cons you listed matter that much in this case. I will think about it.

2 hours ago, MoritzLost said:

If you want to build closer to ProcessWire, you can also use a class that extends WireArray to represent a list of items, this will give you many helpful methods (like getting an item by key) out of the box. For the items themselves, you might just reuse the page itself (if the IDs correspond to 'real' pages in your system) or extend the Page class.

This sounds interesting. Maybe you can provide an example how to extend WireArray?

 

[jploch]

here is another possible structure

{
  "items": [{
    "id": "3252",
    "cssClass": "pgrid-main",
    "breakpoints": {
      "base": {
        "css": {
          "grid-column-end": "auto",
          "grid-row-end": "auto",
          "grid-column-start": "auto",
          "grid-row-start": "auto",
          "align-self": "auto",
          "z-index": "auto",
          "padding-left": "60px",
          "padding-right": "60px",
          "padding-top": "60px",
          "padding-bottom": "60px",
          "background-color": "rgb(255, 255, 255)",
          "color": "rgb(0, 0, 0)"
        },
        "size": "@media (min-width: 576px)",
        "name": "base"
      }
    }
  }, {
    "id": "3686",
    "cssClass": "test_global",
    "breakpoints": {
      "base": {
        "css": {
          "grid-column-end": "-1",
          "grid-row-end": "span 1",
          "grid-column-start": "1",
          "grid-row-start": "auto",
          "align-self": "auto",
          "z-index": "auto",
          "padding-left": "0px",
          "padding-right": "0px",
          "padding-top": "0px",
          "padding-bottom": "0px",
          "background-color": "rgba(0, 0, 0, 0)",
          "color": "rgb(0, 0, 0)"
        },
        "size": "@media (min-width: 576px)",
        "name": "base"
      }
    }
  }, {
    "id": "3687",
    "cssClass": "block_editor-3687",
    "breakpoints": {
      "base": {
        "css": {
          "grid-column-end": "span 5",
          "grid-row-end": "span 2",
          "grid-column-start": "2",
          "grid-row-start": "2",
          "align-self": "auto",
          "z-index": "98",
          "padding-left": "0px",
          "padding-right": "0px",
          "padding-top": "0px",
          "padding-bottom": "0px",
          "background-color": "rgb(255, 204, 204)",
          "color": "rgb(0, 0, 0)"
        },
        "size": "@media (min-width: 576px)",
        "name": "base"
      }
    }
  }, {
    "id": "3696",
    "cssClass": "block_editor-3696",
    "breakpoints": {
      "base": {
        "css": {
          "grid-column-end": "span 2",
          "grid-row-end": "span 1",
          "grid-column-start": "auto",
          "grid-row-start": "auto",
          "align-self": "auto",
          "z-index": "auto",
          "padding-left": "0px",
          "padding-right": "0px",
          "padding-top": "0px",
          "padding-bottom": "0px",
          "background-color": "rgba(0, 0, 0, 0)",
          "color": "rgb(0, 0, 0)"
        },
        "size": "@media (min-width: 576px)",
        "name": "base"
      }
    }
  }],
  "breakpointActive": "base",
  "breakpointActiveSize": "@media (min-width: 576px)"
}

 

[MoritzLost]

Quote

Iam not sure the data will ever be relevant to the user, for now I just use the JSON internally (creating JSON with js and decoding it with PHP).

Then the user is your own module. All the points in my post above still apply – your module has to make some assumptions about the structure of the JSON data. Even if the data is only consumed internally, passing unstructred data around can cause unexpected errors at any point. So it's still better to parse the data into a representation that makes guarantees a specific data structure, and raise errors during parsing (as opposed to a random error anywhere in your module's lifecycle that's hard to track down). Also, will your module not allow customization through, for example, custom page builder elements or hooks? As soon as that's the case, developers using your module will have to use your data, so it should provide a uniform interface to accessing that data.

Quote

This sounds interesting. Maybe you can provide an example how to extend WireArray?

Basically, any ProcessWire class representing some sort of collection, so you can check out those. For example, check out Pagefiles, FieldsArray and PageArray which all extend WireArray. See how they make use of all the utility methods WireArray provides and override just the methods they need to customize to adapt the WireArray to a specific type of data? For example, check out FieldsArray::isValidKey only allows Field objects to be added to it, so adding anything else will immediately throw an error. You can use this to map your pagebuilder items to a specific class and then have a class like PageBuilderItems extending WireArray that only accepts this class as items. This will go a long way towards having a uniform data structure.

Quote

What I meant with "using objects instead of arrays" here is just how I decode the JSON in php: json_decode($jsonString) instead of (json_decode($jsonString, true). This gives me easy access to the items (which are just pages, so I thought it would be nice to be able to access them by page id like the pw api works).

Again, don't mix up storage and representation. For storage, I would prefer the second example you posted, with items being an array instead of a map and the ID being just a regular property. Simply because arrays are more convenient to create, parse and map (in JS especially, but also in PHP). Though again, it doesn't matter as soon as you parse the data. The representation your module (or others, see above) uses to access the data can provide utility methods, like accessing an item by key.

Regarding the last point specifically, WireArray can help you with this. The documentation is kind of handwaivy about how WireArrays handle item keys – WireArray::add doesn't take the key as an argument, but WireArray::get allows you to get an item by key, so where does the key come from? If you check the source code, turns out WireArray automatically infers a key based on the item – see WireArray::getItemKey. Your custom class can extend this to use the ID property of a given item as the key. For example, FieldsArray::getItemKey does exactly this. This allows you to use page IDs to access items without having to store your data specifically as a map. On top of that, you can use all the other utility methods WireArray provides to access elements in the any way you like.

[jploch]

@MoritzLost Thank you for the detailed answer! I'm still at a very beginner level with php, all I know about it, I basically learned from using PW and it's api. Doing mainly frontend development, this is the first bigger module I'm building. So I'm still new to how classes work. Now that I have another look at it, what you sad makes a lot of sense. I will try to implement the class approach with my own utility methods.

I will also have a look at WireArray, this seems like a nice solution.
Thanks again for your help. I might come back at you if I hit another roadblock.

[jploch]

I'm still a little lost on how to implement my JSON decode method and use WireArray as a representation of my data.
My module builds a dynamic stylesheet, based on the JSON file. This dynamic stylesheet is just a php file that the user can include in his/her main template file. The JSON is saved in one field, that live on the page holding all the PageTable items.

Since I need the CSS to be rendered  in the backend and frontend and I wanted to give the user more freedom on where to include the CSS I thought this is a good approach. Not sure if it's really clever to have this in a separate file instead of my module file. 

This is my working code I have in the dynamic stylesheet file:

<style>
<?php 
namespace ProcessWire; 
$backend = 0; 
$p = $this->page;

if($isAdmin = $this->page->rootParent->id == 2) {
$p = $this->pages->get((int) wire('input')->get('id'));
$backend = 1; 
}

$css='';
$settingsArrayPage=json_decode($p->pgrid_settings);

  if( !(empty($settingsArrayPage))) {

       foreach($settingsArrayPage->items as $item) { 
           
           foreach($item->breakpoints as $breakpoint) { 
             
             if ($backend) {
               $css.='.breakpoint-'. $breakpoint->name . ' ';
             } else {
               $css .= $breakpoint->size . '{ ';
             }
            
            $css .= '.' . $item->cssClass. '{ ';
          foreach($breakpoint->css as $style=> $val) {
                   $css .= $style . ': ' . $val . '; ';
               }
             
           $css .= ' } ';
             if (!$backend) {
           $css .= ' } ';
             }
         }
            
       }
  
  echo $css;
    }
 ?>

</style>

First of all where would you put the WireArray code? Is it better to have this in my module file or in the dynamic stylesheet file?
Here is my code so far.

 class PageGridData extends WireArray {
    
   public function getItemKey($item) {
// here I would place the JSON decode method from above and return the items?
		return $item->id;
	}
    
    }

Sorry this is a bit outside of my comfortzone ?

[MoritzLost]

@jploch Ok, so currently you have a procedural approach, which is fine for a fixed use-case. My proposed solution might be overkill for what you're trying to achieve. And it will definitely be more work - and maybe the more general application for your module (which I don't even know the scope or precise purpose of) that I'm imagining isn't even what you're trying to achieve. So before you start refactoring, consider if it will actually be of benefit to your module! And of course, if you have to use code that you don't fully understand the purpose of, it won't help you in the end. If what you have is working fine for your use-case, maybe you don't need all that stuff. So just take my approach with a grain of salt ?

I would start by creating classes that represent the building blocks of your page builder. That would probably include a class that represents a list of CSS rules (that can use WireArray), a class for individual items (with a reference to a list of CSS rules) and a class for a list of items (this again can use WireArray). Decoding JSON would probably be the responsibility of the latter. Though of course you can also go with a more eleborate approach to transforming JSON to classes like the builder pattern.

Here's a start (I wrote this in the browser, so it will need some polish and might include some errors, but hopefully it communicates the idea behind it):

// this class represents a list of CSS rules, as included in your JSON data
class PageGridCSS extends WireArray {
	/** This class gets methods to set, get and remove CSS rules, and render them to CSS. */
}

class PageGridItem {
	// PHP 7.4+, for older PHP versions use a protected property with getters and setters
	public int $id;

	// each item holds a reference to a set of rules
	public PageGridCSS $cssRules;

	/** This class gets properties and methods corresponding to the data it represents */
}


class PageGridData extends WireArray {
    // guarantee that this can only hold PageGridItem objects
    public isValidItem ($item) {
        return $item instanceof PageGridItem;
    }

    // make PageGridItems accessible by their ID
    public function getItemKey($item) {
        return $item->id;
    }

    // convert json to the appropriate classes
    public static fromJSON (string $json): PageGridData {
        $items = json_deocde($data)['items'];
        $dataList = new self();
        foreach ($items as $item) {
            $dataItem = new PageGridItem();
			$dataItem->id = $item->id;
			/** Add all the properties to the item, catching missing or malformed data */
            $dataList->add($dataItem);
        }
        return $dataList;
    }

    public renderCSS() {
        // render the CSS for the items this instance contains
    }
}

Why even do all this? Currently, your dynamic template makes a couple of assumptions that I, as a user of your module, may not agree with. For example:

• It always includes style tags. What if I want to save the output in a CSS file and serve it statically?
• You infer the context (backend / frontend) which doesn't always work reliably. I also can think of some reasons why I would like to get the code for the frontend in the backend - for example, to save it to a field.
• Your ".breakpoint-{...}" classes may interfere with my own stylesheets, so I could use a way to rename them.
• Maybe I want only a part of the stylesheet, like the rules for a specific breakpoint.
• ...

Making all this modular and giving users access to the functionality through a well-defined API with utility methods allows me to customize all this and access the parts that I want to use, changing those that I need to.

Though again, maybe I'm totally wrong regarding the intent and scope of your module. This way a fun exercise to think through anyway, hopefully it will be helpful to you or others looking for something similar ? Hm, maybe that stuff would make a good article for processwire.dev ... ?

[jploch]

@MoritzLost Thanks again! This seems to be the best way to handle it. I will further investigate if it complicates my current setup too much. But I get the benefits now. Hopefully this will help others too. 

I'm generally ok with making some assumptions in my case, because you can change the CSS stuff inside the backend for each item (and the breakpoints you want) anyway, also its easy enough to overwrite some css rules with plane css/scss. Maybe I provide some api options and helper functions along the way, so this is definitely helpful. Here is a preview of the module I'm developing to give you a little more context:

 

[jploch]

One more question to finally understand php classes a little bit better:
If I have this simple class:

class PageGridCSS {

    public $cssClasses = 'test classes';

    public function getCssClasses() {
        return $this->cssClasses;
    }
}

How can I call the function getCssClasses from outside the file it is declared in?
So far I was able  to call it from within the same file only by doing this:

$PageGridCSS = new PageGridCSS();
$PageGridCSS->getCssClasses();

 

[MoritzLost]

You need a reference to the instance, so the $PageGridCSS variable in your case. That's because there can be different instances of the same class with different states and holding different data. So yes, if you create a new instance in a specific context, only that context has access to the variable. If you want to pass it around, you need to provide utility methods or pass specific instances of a class as parameters. Usually, the instances of a set of classes are linked together by properties referencing each other. For example, in my example code above, the PageGridData contains a list of PageGridItem instances, which can be accessed by key or iterated over using the utility methods provided by WireArray. Each PageGridItem then has a reference to the PageGridCSS instance belonging to it (that's the $cssRules property in my example code). For example:

function doStuff(PageGridData $data) {
	$key = 1234;
	// $item will be an instance of PageGridItem
	$item = $data->get($key);
	// $cssRules will be an instance of PageGridCSS
	$cssRules = $item->cssRules;
	// ...
}

Does that help? The best way to pass around data depends on the application flow. For ProcessWire modules, the main module class will usually have helper methods to access specific classes of the module. For example, my module TrelloWire contains a separate class (ProcessWire\TrelloWire\TrelloWireAPI) that handles all requests to the Trello API. The main module file has a utility method to get an instance of that class with the API key and token configured in the module settings. Another example is the TrelloWireCard class, which holds all the data used to create a Trello card through the API. This one is also a module, so you can access it using $modules->get() to get an empty instance. But the main module class also has a utility method to create an instance populated with data from a regular page and the module configuration. This method is used by the module internally, but can be hooked to overwrite specific fields or used to build your own application or business logic on top of the module.

[jploch]

thx! it's working now calling the function like this. Cool!  

$CssClasses = $modules->get('InputfieldPageGrid')->getCssClasses();