Hi,
I am building a site containing around 50000 products which will be updated on a regular basis. We are far from putting this site into production. I am actually devising the future functionalities.
The actual update script uses an AJAX process so to avoid timeout. However, the update process will have to be fired automatically, something with either curl or phantom.js.
What I have coded at the moment isn't probably not the best solution, albeit it works pretty well. There is a PW template called from an address : http://example.com/start-ajax which starts on load a simple jQuery Ajax process. The 50000 products are listed in a CSV file that is previoulsy broken down into small pieces (200 lines per file). The process is smooth and takes 40 minutes to read the data and update/or create Processwire pages accordingly.
I would like to know your advices, your experience in that matter. What would you do to create an automated batch file?