Transferware Collectors Club Database of Patterns and Sources

[Jonathan Lahijani]

Over the course of 2021, I re-developed the Transferware Collectors Club Database of Patterns and Sources.  It is a private, members only database that contains nearly 20,000 patterns of various types of transferware, which in layman's terms is basically art on plates, bowls, cups, etc.  (Wikipedia)  Many archaeologists and transferware enthusiasts use this database to learn more about the historical nature of their collections or archaeological finds.

The system had a long history and there were two versions prior.  I believe the first iteration was made with a PHP framework of some sort, then it was re-developed around 2015 in ProcessWire.  One special aspect of the previous version was that it utilized Elastic Search.  Advanced search capability of the system, both in terms of free-form text entry as well as filters were the crux of the system.

In 2020, I was introduced to this project as the previous developer decided to retire.  While doing some various maintenance type work, I realized that there was much that could be done to improve the system, from re-imagining the user interface to utilizing all the latest features in the newer versions of ProcessWire, as it had been locked to version 3.0.135 or something at least 3-4 years old.  Also, Elastic Search had become deeply outdated and unreliable since the entire site had been moved to a new server.  Finally, I saw various aspects of the data model that could be improved as well as reduce several dependencies, while also using new techniques like HTMX.

I redeveloped the system over the course of 2021 and the end result has greatly impacted the capabilities of the system as well as its ability to be maintained with little complication years into the future.

Search:

First, it became apparent that the way in which Elastic Search was being used had little benefit, became unstable and was outdated in the time since it was implemented.  I don't like relying on 3rd party solutions, especially for something so critical and I thought about how I could develop a great search experience with ProcessWire alone.  To do this, I utilized the WireWordTools module to handle lemmatization and such.  Then, I advocated for this feature to be developed into the core, which could allow the results from multiple search queries to be easily stacked one after another and not affect order or pagination.  This was a key change allowed for "ranked" search results.   Finally, I integrated the BigHugeThesaurus API so that entered search terms can have their synonyms automatically generated.

Filters:

The previous version of the system had filtering capability, but it was too cumbersome and in my opinion, just didn't feel right.  A significant amount of thought went into how to improve it while untangling some issues with the data model (and just understanding what Transferware is in the first place).  The end result has has been an extensive set of filters with a special user interface so that users can easily select what they need.

Frontend:

I take the "classic", server-rendered approach when it comes to web-apps, and during this time, HTMX started becoming a favorite among such developers.  By using HTMX, searches can be conducted without having to hit the submit button every time.  It gives it that polished ajax-y feel without having to write any JavaScript whatsoever and allowing the use of ProcessWire in the expected way as opposed to it acting as some API end-point.  To be able to do this is somewhat of a game-changer and I feel like I'm not "missing out" on these features which in the past were a pain to have to invent as there really wasn't a clean, straight-forward way to do it.  HTMX is a gem and it couples very nicely with ProcessWire.  I'm also using Alpine.js lightly for toggling things here and there.  My goal was to make this project jQuery-free.  UIkit 3 is being used as the frontend CSS framework.

User Authentication:

As mentioned, the database must be accessed after a membership has been purchased.  TCC already has a 3rd party membership system, so I developed a handshake that allows a user to log in with the ProcessWire admin form using their credentials from that 3rd party system.  If it's the member's first time logging in, it will then automatically create a ProcessWire account for the member so they can bookmark patterns, save searches, etc.  The previous version had a different type of handshake that ultimately only logged the users into one master "member" account, so users wouldn't be able to save any information since everyone technically shared the same account.

New Server:

I put the site on its own dedicated droplet to squeeze out even more performance and be independent of their main marketing site (based on Drupal).  We had a couple instances of configuration changes causing some clashes, so this move isolated it from any potential breaking changes that may occur on the main site.  A 2-core droplet is all that is needed for now, which is cost effective.

Demo Video and Article:

While the site can't be visited given it requires an account, I did create an in-depth search tutorial video which shows off quite a bit which I can post here since it's publicly accessible:

https://player.vimeo.com/video/666912411?h=30a94132db

Feel free to read more about the TCC DB in this interview with Dr. Loren Zeller, who I worked closely with the entire time and who provided invaluable feedback and guidance along the way:

https://www.transferwarecollectorsclub.org/sites/default/files/pdf/special-interest-pdf/0225_d_001_analink.pdf

More details on my personal website:
https://jonathanlahijani.com/projects/transferware-collectors-club/

[Macrura]

Awesome write up and thanks so much for this post and the details – very inspiring. I also recently started using HTMX for a lot of things on PW sites, and am similarly impressed and loving it.

<off-topic> have a site that I need to develop a subscription/membership system for. I'm looking at amember, memberful, and a special setup using Foxycart. All have their pros and cons, but I see that this site uses amember. I wonder if you have any insights about working with amember... </off-topic>

 

 

[Jonathan Lahijani]

19 minutes ago, Macrura said:

<off-topic> have a site that I need to develop a subscription/membership system for. I'm looking at amember, memberful, and a special setup using Foxycart. All have their pros and cons, but I see that this site uses amember. I wonder if you have any insights about working with amember... </off-topic>

The TCC site actually uses aMember Pro which is what I integrated with (the handshake).  I didn't set up aMember Pro originally, but I believe TCC likes the system, although there has been some concern with the recent Russia/Ukraine situation as there are developers from both countries that are involved in the project (and could lead to potential support issues should they arise given the displacement of many people).

Creating the handshake was straight-forward and didn't involve too much code.  I could give you some tips as to how I did it should you go that route, but in short, when a member logs in via ProcessWire's admin form, a hook it hits a script on the main site (via a get request) that bootstraps aMember's system and runs a check on the credentials entered.

[Macrura]

Thanks - yeah I'm thinking about what is best to use - this is a paywall type of situation, and I like that aMember pro has been around for a long time, and is a 1-time payment not an ongoing subscription... I think i will need to use PW users because this client wants a lot of stuff to change for subscribers, like no google ads, etc..