Update page list based on JSON endpoint response

[a-ok]

Hi folks,

I'm going to be using a combination of Shopify's JS Buy SDK (to display the products and add to cart etc) and their API endpoint response to generate the pages required on PW (to match the products on Shopify).

The idea is that I'll have two PW templates; one for the Shop overview and then one for the individual detail pages. These will both use the JS Buy SDK to either fetchAll products or fetch a specific product. Obviously the pages on PW need to be created from an endpoint response and thus updated (delete or added to) when new products are removed/added on Shopify.

My plan was to do something similar to this forum post: https://processwire.com/talk/topic/16064-create-pages-from-json-feed/ where I would "use a lazy cron function to automatically query the API endpoint every 24 hours. To delete pages that are no longer contained in the JSON response you can collect all of the unique identifiers in the response and then use a PW selector to get pages not matching any of those identifiers. Those pages you delete."

I think this sounds good and I have the understanding of it... but I have a couple of questions that I am currently stuck on.

Using the PW API... what would be the best way to update the pages based on the returned result? Would it be to loop through the products and check if they exist... if not then create it? I'm unsure how to reverse this to then check if a page exists in the feed that's not on PW then delete it?

function myHook(HookEvent $event) {

	$http = new WireHttp();

	$host = 'XXX';
	$shopify_api = 'XXX';
	$shopify_password = 'XXX';

	$shopify_base = 'https://' . $shopify_api . ':' . $shopify_password . '@' . $host . '/admin/products.json';

	$products = $http->getJSON($shopify_base);

	foreach ($products['products'] as $product) {

		 $productID = $product['id'];
		 $productHandle = $product['handle'];

		 $p = wire('pages')->get("shop_detail_shopify_id=$productID");

		 if (!$p->id) { // If it doesn't exist, create it

			 $p = new Page();
			 $p->template = 'shop-detail';
			 $p->parent = '/shop/';

			 $p->name = $productHandle;
			 $p->title = $product['title'];
			 $p->shop_detail_shopify_id = $productID;

			 $p->of(false);

		 }

		 $p->save();

	}

}
wire()->addHook('LazyCron::every30Minutes', null, 'myHook');

1.  

Edited December 18, 2017 by oma

[a-ok]

Anyone have any thoughts on this? Just need to work out how to delete the pages that aren't in the JSON but on PW...

[netcarver]

The easy way to do this is to compare the current JSON reply to the previous one. If an entry exists in the previous JSON reply that doesn't exist in the current JSON then you know that you need to delete the corresponding entry in PW. 

Storage is easy - just write the JSON to disk. Compare is easy - convert the two JSON feeds to arrays and just do an array diff. Delete is easy, as long as there is a unique field in the JSON that maps through to a unique field value in PW. Once you've processed the current deletions, just store the JSON ready for the next cycle.

[elabx]

Why don't you rather use Shopify's webhooks?? I think it would simplify the problem, you can save the product id in a field and just look for that every time the hook get's called, no need to compare JSON, just decode the payload into an array and grab the product/variant id, Selector API to look for the product, and decide what to do depending on the event of the webook.

[a-ok]

On 21/12/2017 at 11:28 PM, elabx said:

Why don't you rather use Shopify's webhooks?? I think it would simplify the problem, you can save the product id in a field and just look for that every time the hook get's called, no need to compare JSON, just decode the payload into an array and grab the product/variant id, Selector API to look for the product, and decide what to do depending on the event of the webook.

Thanks for the reply.

It is currently saving the product ID in a field (in order to load the product detail data)  – it's creating the pages fine it's just I cannot work out how to delete the pages that no longer exist in the endpoint list of products (once they remove it from Shopify).

[a-ok]

As I am saving the product ID of the Shopify product to each page... I ended up building an array of all the pages that currently exists on Shopify, with the product ID as the key, then I imploded that array by "|" then created a nonActiveProducts pageArray using

$nonActiveProducts = wire('pages')->find("shop_detail_shopify_id!=$activeProducts");

What this does is return all the pages that exist in PW that's shopify ID doesn't match the active products... so this returns all the pages that need to be deleted.

So, my final question is... is there a method to delete all the pages returned by a pageArray (rather than looping through each and deleting?)

[elabx]

8 hours ago, oma said:

Thanks for the reply.

It is currently saving the product ID in a field (in order to load the product detail data)  – it's creating the pages fine it's just I cannot work out how to delete the pages that no longer exist in the endpoint list of products (once they remove it from Shopify).

Oh maybe I didn't make myself understood, but my idea with hooks also involves checking for deletion, you can check for a header in the hook to see if it's a product getting deleted, the header is supposed too look like this!

X-Shopify-Topic: products/delete

Docs: https://help.shopify.com/api/reference/webhook#events

I think this is the type of problems hooks solve very well!

This way you wouldn't have to query the /products.json endpoint  every 24 hours, every action registered as webhooks will trigger the event when create/delete/update products happen, which you can mimic on ProcessWire on the handler you set on the route asignet to the webhook.

Personally I haven't tried webhooks myself, but I do have some experience using the shopify API and I believe this would be the way to go to make a one way store sync.

 

 

[a-ok]

@elabx 4 years ago you posted this and it's still tripping me up!

I now have a Shopify webhook firing to /api/ in my PW app when a product is updated. I have verified that PW is getting the response with the following...

$log->save('shopifyApi', 'Webhook triggered from Shopify');

Great! The request is a POST request so I'm assuming I use `$input->post()` but I am unsure how to check the headers to see what the X-Shopify-Topic is and also to grab the data?

Attached below is the webhook response tested on request catcher...

[a-ok]

I used `php://input' to test with and it's returning the data from the Shopify hook of the product updated etc.

$test = json_decode(file_get_contents("php://input"), true);

Is there any way to do this using the PW API and if not is it possible to get the headers so I know what X-Shopify-Topic is being requested?

[elabx]

Have you tried PHP's get_headers?

3 hours ago, a-ok said:

Is there any way to do this using the PW API and if not is it possible to get the headers so I know what X-Shopify-Topic is being requested?

What's wrong with the way you are doing it now? I understand from your replies that it is actually working?

Also, the request's JSON seems to have a topic field that you can check too, in case you want to skip the header check.

[a-ok]

16 hours ago, elabx said:

Have you tried PHP's get_headers?

What's wrong with the way you are doing it now? I understand from your replies that it is actually working?

Also, the request's JSON seems to have a topic field that you can check too, in case you want to skip the header check.

Yes but I thought the headers `X-Shopify-Topic` etc would be included in the response. but it's not? `file_get_contents("php://input")` returns the raw data, which is good, but no way of checking the headers on this? Sorry I'm just a bit confused.

[elabx]

3 hours ago, a-ok said:

but no way of checking the headers on this?

You are right, I got a bit confused with the documentation page, the response doesn't have info about the header. 

Did you try get_headers?

[kongondo]

4 hours ago, a-ok said:

but no way of checking the headers on this?

Have you tried this?

<?php

$_SERVER['X-Shopify-Topic'];