Search the Community

Which of the following are the minimum MySQL database user privileges required by ProcessWire? ALTER CREATE CREATE ROUTINE CREATE TEMPORARY TABLES CREATE VIEW DELETE DROP EXECUTE INDEX INSERT LOCK TABLES REFERENCES SELECT SHOW VIEW TRIGGER UPDATE

Here are the step by step to install and setup ProcessWire with the help of a server management tool - https://cloudstick.io/ 1. Create your Vultr compute. 2. Select the Operating system Ubuntu 16.04/18.04/20.04 LTS >> Enter your server root password then click on Deploy now! 3. Create an account in CloudStick and connect your server: Click on connect server >> Enter your server login details >> Add this server. 4. Your server setup will be done in couple of minutes - The setup will finish less than 8 minutes. Then select your server: 5. Create an account to host/upload ProcessWire: Click on Accounts >> Create an account >> Create Custom Web application: 6. Enter the web application details, such as the email address which you would like to receive the SFTP login details >> web application name >> Domain name >> username then >> select the web application stack >> nginx + apache >> then create web application. 7. Now select the web application then install SSL: 8. Open your email account, and find the login details to connect the server over SFTP >> then upload the source code of ProcessWire: 9. Let us open the domain in browser once the upload finish. 10. Select the profile and click next: 11. Click next to proceed further: Now, you will see an incompatibility issue with PDO-Mysql which can be install in 2 clicks. 12. Go back to the summary page >> Click on easy PHP >> Select the PHP version of your web account: 13. Then it is time to enable PDO_Mysql, scroll down and enable it: 14. Go back to to the ProcessWire installation URL and click on check again >> You can see no incompatibility issue after enabling PDO_Mysql: 15. Click on Next and now it is time to enter the database credentials: 16. let us create the database, db user and grant privilege's to the db user - it is just matter of few clicks and very easy! Click on the menu Accounts >> Select your web account of ProcessWire >> Click on App Database then create the a database: 17: Click on create database and enter the database name: 18. Create the database user: 19: Go back to the database page and click on Grant user then grant permission: 20. Go back to the ProcessWire installation URL and enter the database credentials you have created in CloudStick dashboard. Now, it is time to setup your admin user credentials and setup admin area URL: 21. Then you are done:

Hello, Does anyone have experience with migrating content from Django to Processwire? Or are there any suggestions for achieving this?

I have built a system in processwire, which has more than 600K pages. A team of 40 people is using the system, DB is from AWS with 16GB Ram. But still, select queries are getting locked. It would be great if someone suggests how to solve this problem. Also note: We have around 48 fields in one template. Thanks in advance.

Hi, I hope this is the correct section for my problem. All I need is a connection to an external Database and a query gettings some data. I do this in a processwire Page-Template. I am honestly not sure if it is a problem with processwire or my code: $host = ‚XXXXX’; $user = ‚XXXXX‘; $pass = ‚XXXXX‘; $db = ‚XXXXX‘; $port = ‚3306‘; $mydb = new Database($host, $user, $pass, $db , $port); $result = $mydb->query("SELECT * FROM char“); while($row = $result->fetch_assoc()) { print_r($row); } Produces the following error: Error: Exception: DB connect error 2002 - Connection timed out (in /customers/9/4/e/XXXX.de/httpd.www/wire/core/Database.php line 79) I also tried connecting without the $port variable but got the same error.

Hey Ryan, hey friends, we, Mobile Trooper a digital agency based in Germany, use ProcessWire for an Enterprise-grade Intranet publishing portal which is under heavy development for over 3 years now. Over the years not only the user base grew but also the platform in general. We introduced lots and lots of features thanks to ProcessWire's absurd flexibility. We came along many CMS (or CMFs for that matter) that don't even come close to ProcessWire. Closest we came across was Locomotive (Rails-based) and Pimcore (PHP based). So this is not your typical ProcessWire installation in terms of size. Currently we count: 140 Templates (Some have 1 page, some have >6000 pages) 313 Fields ~ 15k Users (For an intranet portal? That's heavy.) ~ 195 431 Pages (At least that's the current AUTOINCREMENT) I think we came to a point where ProcessWire isn't as scalable anymore as it used to be. Our latest research measured over 20 seconds of load time (the time PHP spent scambling the HTML together). That's unacceptable unfortunately. We've implemented common performance strategies like: We're running on fat machines (DB server has 32 gigs RAM, Prod Web server has 32gigs as well. Both are running on quadcores (xeons) hosted by Azure. We have load balancing in place, but still, a single server needs up to 20 sec to respond to a single request averaging at around about 12 sec. In our research we came across pages that sent over 1000 SQL queries with lots of JOINs. This is obviously needed because of PWs architecture (a field a table) but does this slow mySQL down much? For the start page we need to get somewhere around 60-80 pages, each page needs to be queried for ~12 fields to be displayed correctly, is this too much? There are many different fields involved like multiple Page-fields which hold tags, categories etc. We installed Profiler Pro but it does not seem to show us the real bottleneck, it just says that everything is kinda slow and sums up to the grand total we mentioned above. ProCache does not help us because every user is seeing something different, so we can cache some fragments but they usually measure at around 10ms. We can't spend time optimising if we can't expect an affordable benefit. Therefore we opted against ProCache and used our own module which generates these cache fragments lazily. That speeds up the whole page rendering to ~7 sec, this is acceptable compared to 20sec but still ridiculously long. Our page consists of mainly dynamic parts changing every 2-5 minutes. It's different across multiple users based on their location, language and other preferences. We also have about 120 people working on the processwire backend the whole day concurrently. What do you guys think? Here are my questions, hopefully we can collect these in a wiki or something because I'm sure more and more people will hit that break sooner than they hoped they would: - Should we opt for optimising the database? Since >2k per request is a lot even for a mysql server, webserver cpu is basically idling at that time. - Do you think at this point it makes sense to use ProcessWire as a simple REST API? - In your experience, what fieldtypes are expensive? Page? RepeaterMatrix? - Ryan, what do you consider as the primary bottleneck of processwire? - Is the amount of fields too much? Would it be better if we would try to reuse fields as much as possible? - Is there an option to hook onto ProcessWires SQL builder? So we can write custom SQL for some selectors? Thanks and lots of wishes, Pascal from Mobile Trooper

All of a sudden, with nothing changed on the database or server, a website was getting error when doing a search: Error: Exception: SQLSTATE[HY000]: General error: 23 Out of resources when opening file './your-database-name/pages_parents.MYD' (Errcode: 24 - Too many open files) (in /home/forge/example.com/public/wire/core/PageFinder.php line 413) #0 /home/forge/example.com/public/wire/core/Wire.php(386): ProcessWire\PageFinder->___find(Object(ProcessWire\Selectors), Array) #1 /home/forge/example.com/public/wire/core/WireHooks.php(723): ProcessWire\Wire->_callMethod('___find', Array) #2 /home/forge/example.com/public/wire/core/Wire.php(442): ProcessWire\WireHooks->runHooks(Object(ProcessWire\PageFinder), 'find', Array) #3 /home/forge/example.com/public/wire/core/PagesLoader.php(248): ProcessWire\Wire->__call('find', Array) #4 /home/forge/example.com/public/wire/core/Pages.php(232): ProcessWire\PagesLoader->find('title~=EAP, lim...', Array) #5 /home/forge/example.com/public/wire/core/Wire.php(383): ProcessWire\Pages->___find('title~=EAP, lim...') #6 /home/forge/example.com/public/wire This error message was shown because: you are logged in as a Superuser. Error has been logged. I tried several things, listed in this thread: https://serverfault.com/questions/791729/ubuntu-16-04-server-mysql-open-file-limit-wont-go-higher-than-65536 But for some reason, MySQL was not getting its limit increased, but in the end, the one that did the trick was this: This worked for me on Ubuntu Xenial 16.04: Create the dir /etc/systemd/system/mysql.service.d Put in /etc/systemd/system/mysql.service.d/override.conf: [Service] LimitNOFILE=1024000 Now execute systemctl daemon-reload systemctl restart mysql.service Yes indeed, LimitNOFILE=infinity actually seems to set it to 65536. You can validate the above after starting MySQL by doing: cat /proc/$(pgrep mysql)/limits | grep files

I've never encountered this issue before. My local installing is using AMPPS 3.7, setup about 3 months ago. Everything was working fine last night, and I think possibly this morning, before it complete stopped with these errors. Please see below. I have not changed any config setttings, access settings. The only 2 other things I did yesterday was log into phpmyadmin and export the DB, then I think an Xcode update was installed yesterday. Can anyone help? I know people have run into these issues when they migrate, but I haven't migrated, setup, changed anything. Some people have said to just disable to do something like: $cfg['Servers'][$i]['AllowNoPassword'] = true; In phpMyAdmin's config.inc.php file. I tried but that didn't work. I even tried changing: $config->dbHost = 'localhost'; to 127.0.0.1, in site/config.php but no good.

I'm displaying a list of products which are found by their templates, but the pages are taking a very long time to load. At first, I blamed it on my image rendering (using PIM2), but even with all those images now stored in the file tree, the page is taking abysmally long to load. ProCache seems to help but I don't feel as though what I'm trying to do should be gnawing the bones of my resources quite so long. The variable for the selector is defined in my header include: $productCatList="prod_series|prod_series_ethernet|prod_series_access|prod_series_accessories|prod_series_fiber|prod_series_pwr_supplies|prod_series_pwr_systems|prod_series_wireless"; $getCurrentProdOptions="template=$productCatList, prod_status_pages!=1554|1559|1560|4242"; Then in the template for the page upon which the directory loads: $products = $pages->find("$getCurrentProdOptions"); include_once("./prod-list-row.inc"); echo $out; And the prod-list-row.inc foreach (which is on every page that's exhibiting the slowdown): <?php $sum = 0; $out =""; $out .= "<div class='span_12_of_12'>\n"; foreach($products as $p){ $sum += 1; if ($sum % 2 == 0) { $bgcolor = '#fff'; } else { $bgcolor = '#e4e4e4';} $par = $p->parent; $out .="<div class='section group' style='background: $bgcolor ; min-height: 110px'>\n"; $img = $p->prod_image; $thumb = $img->pim2Load('squarethumb100')->canvas(100,100,array(0,0,0,0),'north',0)->pimSave()->url; $out .="<div data-match-height='{$p->title}' class='col span_2_of_12 hide'>"; $out .="<a href='{$p->url}'><span class='product-image-box'><img src='{$thumb}' alt='{$p->title}' title='{$p->title}'></span></a>"; $out .= "</div>"; $out .= "<div data-match-height='{$p->title}' class='col span_6_of_12'>"; $out .= "<div class='prod-list-name-label'><a href='{$p->url}'>{$p->title}</a></div>"; if($page!=$par) { $out .= "<div class='prod-list-category-label' style='font-size: .7em;'>Category: <a href='{$par->url}'>{$par->title}</a></div>"; } $out .= "<div class='list-headline' style='font-size: .8em;'>{$p->headline}</div>"; $out .="<div class='learn-more-buttons-sm'>"; $out .="<a href='{$p->url}' title='Product Specs and Documentation'><span class='find-out-more-button' style='font-size: .8em;'><i style='font-size: .8em;' class='fa fa-lightbulb-o' ></i> &nbsp; Learn More</span></a>"; $out .="</div>"; $out .="</div> \n"; $out .= " <div data-match-height='{$p->title}' class='col span_4_of_12'>"; if(count($p->prod_feat_imgs) >0 ){ $out .= "<div class='featured-icons-list' margin: 2em .5em;'>"; foreach($p->prod_feat_imgs as $feat){ $icon = $pages->get("$feat->prod_featicon_pages"); if($icon->image) { if($feat->prod_feat_textlang) { $icontitle = $feat->prod_feat_textlang;} else {$icontitle = $icon->title;} $out .= "<img src='".$icon->image->size(35,35,$imgOptions)->url . "' alt='" . $icontitle . "' title='" . $icontitle . "' class='listing-feat-icon' style='margin-right: .5em;' />"; } } $out .= "</div>"; if($p->prod_product_line){ foreach($p->prod_product_line as $pline) if($pline->image) { $out .= "<div style='height: 35px;'>\n"; $out .= "<img src='{$pline->image->size(75,35,$imgOptions)->url}' alt='{$pline->title}' />"; $out .= "</div>"; } } } $out .= "</div>"; $out .="</div>"; } $out .= "</div>"; Is there a clear culprit here of what I'm doing that's so stressing the system? I turned off TracyDebugger because I saw another thread about that causing slowdown (even though I'm using the latest), but that had no effect. Every time I thought I found the culprit and commented it out, nothing changed. Would appreciate some more eyes on this. Thank you! ETA: prod_feat_imgs is a repeater field which contains a Page reference field (from which I pull the image and title) and a multilanguage textfield (to override the page reference title if it exists). Could that be the problem?

With Microsoft Azure recently releasing proper Linux and native Mysql hosting, it is becoming very easy to get processwire hosted on that platform. By default, they force SSL connection to the managed Mysql server. This causes processwire to fail when trying to connect. I can turn off SSL for my MySql server, but I wish there was a $config setting like $config->useSSL = true that would allow processwire to include the appropriate property in the connection string (like ssl=true or whatever). Here is the Azure link that explains what I would like to do: https://docs.microsoft.com/en-us/azure/mysql/concepts-ssl-connection-security Am I missing this config setting somewhere? Or would it be easy to add? Thanks.

Hey! Quick question, Should I limit how many find() methods I use on a single page? Is it fairly resource-intensive? I am getting consistent 2006 MySQL server has gone away errors and have followed all the suggestions I can find, but wonder if I am just "overloading" my server resources with database requests or something like that? Thank you!

With any website, there is the possibility of db issues - overloaded server, network connectivity if the db is on another machine in the hosting network, etc. I would love to see a feature where if there is any reason the db fails or cannot be accessed, then pw displays a dedicated page that is stored in the filesystem - instead of displaying nothing, or an ugly mysql error. Obviously it would be good to log the error, and possibly send a notification to the admin (email?). This gives us the opportunity to still present a professional front (albeit with no functionality) while problems are resolved behind the scenes. I cannot think of a company I have worked for that hasn't had db errors at times What are your thoughts?

Today I have been running in mysql errors using @renobird s Module MarkupActivityLog which still uses mysqli Driver. @all developers Although mysqli is still supported PDO driver is the default database driver in PW since https://processwire.com/about/news/introducing-processwire-2.4/ and its strongly recommend to all module authors to use/change-to PDO driver instead of mysqli. @renobird I have sent a pull request. A list of other affected modules not updated until now. (I try to keep them up to date. Please help) @apeisa ProcessTrashman, ProcessRedirects @netcarver ProcessDiagnostics

I'd like to read up on importing content from Textpattern into a fresh ProcessWire installation. I know there are more than a few Textpatrons here: Is there a tutorial or thread or blog post covering this? I've used Textpattern for years. I'm very familiar with it's database schema and pretty comfortable exporting and manipulating it with MySQL. ProcessWire's schema uses a totally different paradigm, and I'm not confident that I could simply dump the data and import it into ProcessWire with the same facility. I'm happy with Textpattern for most of the sites in which I use it, but there are a few that I think using ProcessWire would be a significant boon. Thanks in advance!

Hi, since i updated my local mysql to v5.7.9, the Processwire SystemUpdater fails (on all local PW installations) with the following message: SystemUpdater: ERROR: Update #12 ERROR: SQLSTATE[42000]: Syntax error or access violation: 1067 Invalid default value for 'created' I suppose PW is not prepared for this mysql version. Sources: https://dev.mysql.com/doc/refman/5.7/en/upgrading-from-previous-series.html#upgrade-system-table-changes, https://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_sql_mode As my database knowledge is limited,- can someone confirm this? regards Olaf

I'm just wondering exactly what MySQL permissions are required for ProcessWire? From a security point of view i would prefer to lock down the MySQL user that ProcessWire uses to the least amount of privileges needed. I guess it's SELECT, INSERT, UPDATE, DELETE, but do any modules create additional indexes? Any help on this would be great.

Hi all I have big troubles with my DB backups and phpMyAdmin. I can't manage to import a backup. The phpMyAdmin export settings are left to the defaults. The error message I get: #1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INSERT INTO `caches` (`name`, `data`, `expires`) VALUES ('ModulesUninstalled.in' at line 11 When I delete the complete 'cache' section there are almost for all tables the same errors. (part of the dump attached) I had this problem before and back then (couple months ago) I think I copied almost every table individually. I thought it was specific to that DB. But I can't do a dump/import of any of my PW installations. MySQL: 5.6.27 PHP version: 7.0.11 phpMyAdmin: 4.5.5.1 Anybody knows that problem? Or what to do? phpMyAdmin SQL Dump.sql

Hi all One of my sites (running PW since 2014, currently v. 2.5.3) had problems with crashed tables. Last week the body field and and image field crashed – noteworthy the latter crashed in a separate DB in my working copy. A couple hours ago another field table crashed (production system again). Funnily, today's table fixed itself. The site has nearly 20,000 pages, about 100 users a day and is installed on a managed server. Unless the logs in sites/assets/logs/errors.txt I have no idea how to research the problem. The hosting company says 'system is fine' and 'mostly it's cased by the CMS'. I have seen others here having that problem, but unless fixing it immediately with repair table there wasn't much info to see what is wrong. Perhaps you guys have some tips how to debug that kind of problem. I even don't know if it is possible to fix this on my own … if it's a problem with the filesystem for example. Any ideas? My code or system failure?

Hey all, For my website I had to make a connection with an external DB. In that DB there are categories of products in different languages, and I want to import these categories in the languages that are available in processwire, I can do that with a very long code but I want to make it shorter. So I wrote a function to get the categories, but the different languages of the category names not, only if I insert the ISO-code hard-code. So here is my code: protected function get_category() { $languages = wire("languages"); try { $prestashp = $this->get_dbPrestashop(); foreach($languages as $language) { $iso_code= $language->name ; $categories = $prestashp->query( "SELECT l.id_lang, l.iso_code, cl.description, cl.name AS category_title, cl.id_shop, c.active, c.id_category, c.id_parent, c.position, cl.link_rewrite FROM ps_category c INNER JOIN ps_category_lang cl ON c.id_category=cl.id_category LEFT JOIN ps_lang l ON cl.id_lang=l.id_lang WHERE c.active='1' AND cl.id_shop='1' AND c.id_parent = '2' ORDER BY c.position "); } }catch(Exception $e) { $this->message('Failed to show categories. Try again later.'); return false; } return $categories; } So in the DB there is Deutsch, Dutch, English(default), Spanish, Italian. And when I edit the template the different languages of the category names need to be imported in the right tab of the field. https://gyazo.com/52a3549a8adaed2d5f44cac6dfb6d9d3 As you can see, there are language tabs above the field, and the names of the category in that language need to be insterted there. Anyone have an idea how I can fix this?

Redgate is giving out non commercial licenses for its MySQL Compare and MySQL Data Compare tool. I've used their SQL Server Compare tools and the ToolBelt extensively many years ago, and it saved my back side time and time again. I only happen to come across it because I was looking out for a MySQL Compare tool to work out the difference between my Test & Live Servers. The unfortunate caveat is that it only runs on Windows ...... perhaps it will still be of use to someone. I will try running it on Parallels and see if it can access a MySQL Instance running on the OS X Parent .... http://www.red-gate.com/products/mysql/mysql-comparison-bundle/

I'm not sure that I really follow this page; http://imgur.com/a/hCxqg I've created a database when setting up the server, so I'm presuming that this is what's being asked for here? The page linked for getting setup http://webdesign.tutsplus.com/tutorials/how-to-install-and-setup-processwire-cms--cms-25509 doesn't really mention anything about these fields. After trying to use the user name on the server, and the password that I entered when setting up the mySQL to start with I got the following error (so I had filled in "DB Name"={some name that I thought up }, "DB user" = {name of user on server}, "DB pass" = {pass i set up for mySQL}); http://imgur.com/a/cgFKq After this i tried using db name; {whatever} db user; {name of user on server} db pass; {password for user on server} This didn't work either, so I'm not sure what I'm meant to put here.

Hi guys I'm trying to duplicate a ProcessWire site on a different server so I can work on it. The good news is the live and the test servers are identical VPS. I already have multiple PW sites running on both. I can export the DB from the Live server but when I try to import to Test server I get the following error. Am I correct in thnking this error in the SQL syntax is being generated at the export stage? IE my Live PW site wouldn't be working with a syntax error so I must be messing up the export of the .sql file There is nothing at line 48 but a find command shows CREATE TABLE `fieldtype_options` ( `fields_id` int(10) UNSIGNED NOT NULL, `option_id` int(10) UNSIGNED NOT NULL, `title` text, `value` varchar(255) DEFAULT NULL, `sort` int(10) UNSIGNED NOT NULL ) ENGINE=MyISAM DEFAULT CHARSET=utf8; BTW, my server setup is: Version Plesk v12.5.30_build1205150826.19 os_CentOS 6 OS CentOS 6.3 (Final) MySQL: 5.5.41 PHP: 5.6.19

I'm sure there may have been a topic talking about this previously, but ironically i couldn't find it when using the search box in the forum. Recently i've been using Processwire as more of a CMF really, which i've found it really easy to mould and shape to the application needs. The one thing that concerns me though is the fact it only uses the MyISAM DB Engine. These days InnoDB seems to be the standard, and seems to fit better with the way PW stores things, for example Foreign keys help the DB understand links of data across different tables (PW stores each field in a separate table). This would also benefit greatly from transactions, making sure that every SQL operation needed to store an item and its field data would either all be successful, or wouldn't happen at all. This gives guarantees that no data went missing during the save due to DB issues and crashes. Row locking (InnoDB) rather than Table Locking (MyISAM) is a huge advantage, take the situation where i want to save an item that has a common field, like field_body, if i understand it correctly, then the field_body table would be locked on every read as MyISAM uses table locking, so other queries to read or write would have to wait until the table lock is released. InnoDB on the other hand only locks the row in question so other operations can happen to the table at the same time. Another feature that goes along with the ACID compliance is The commit log, InnoDB keeps a commit log of transactions, so in the event of a crash it can recover to a consistent state. MYISAM however does not so it can be hard to know what state the data should be in, when recovering. I think i read previously that Ryan chose MyISAM at the time for it's Full text search capabilities, which InnoDB only introduced in MySQL 5.6, but i think unless full text search is the key part to the internal PW system, then is it really that necessary a trade off? I would rather the reliability of InnoDB storing data than to have full text search, for that kind of functionality i would use a separate system built for specifically with this feature, such as ElasticSearch. So i was wondering, will InnoDB become the default DB engine for PW ??

Hi all! this is more of a general MYSQL Schema discussion but i noticed that PW does a pretty good job of handling flexible objects (pages) by creating a new table for each object property (field). I was wondering what other options there are out there for storing flexible but efficient objects in MYSQL. I noticed that Wordpress uses it's wp_postmeta table extensively for storing flexible schema. Also according to Facebooks engineering there's a simplified schema overview of it's TAO based system that uses MYSQL as it's underlying datastore for it's graph data. (https://www.facebook.com/notes/facebook-engineering/tao-the-power-of-the-graph/10151525983993920) So as i say i was just curious really as to different setups and indexing options anyone has used to be able to handle flexible objects within MYSQL.

http://www.adminer.org/ is a small php file to manage SQL databases I received this on Smashing Magazine newsletter and decided to try it because of all those PhpMyadmin securiy concerns. It's very easy to use, so i decided to install it on the server, inside my PW install. I droped the PHP file on the templates folder, created a new template, and a page with this template without Guest access. Works great! And I have two layers of security, the PW admin login, and the DB login. Just wanted to share