Beluga

Edit: silly stuff here, nothing to see.

Note that I only took into account the code path which did not deal with "field pairings". This is because I was unable to locate any field pairing feature in the BCE interface.

Well, my custom naming thing is something that you should not incorporate ? So I think the naming probably needs some work. Re: parsecsv - maybe it was my inability to use it without Composer. I have not run into issues with PHP 7.2.

I modified Batch Child Editor to work with my use case of not requiring title field for templates. Commented out: if(!in_array('title', $convertedFieldPairings)) { if(in_array('add', $this->data['pageSettings'][$pp->id]['editModes']) || in_array('replace', $this->data['pageSettings'][$pp->id]['editModes'])) { $this->error($this->_("You must include a \"title\" field in the CSV Field Pairings, unless the only available modes are \"Edit\" and \"Update\".")); } } To create a new naming scheme, changed row 1560 to: $this->setPageTitleOrName('name', trim($childTemplate . "-" . $this->wire('sanitizer')->text($childFieldValue) . "-" . $x), $np, $newPage); After //populate numeric array of field names $fieldsArray = array(); foreach($np->fields as $f) $fieldsArray[] = $f->name; I added $this->populateField($np, $fieldsArray[$i], $childFieldValue); so it would not leave the first field blank. Btw. there are new releases of ParseCsv: https://github.com/parsecsv/parsecsv-for-php/releases It has received a big renovation, PHP 7.2 support etc. I was unable to make it work, but I did not sacrifice much time on it. Another library to consider is https://github.com/thephpleague/csv

Result with MyISAM: Trash with 29108 pages emptied: 1 min 25 s

Eh, I added autocommit=0 to my cnf override, but it turns out PW is unable to create any pages (failed when I tried to create the parent page). Btw. another perf data point with an InnoDB install: Trash with 7232 pages emptied in 6 min 35 s. I don't think any of this is related to the CSV import module used.

I have found that InnoDB performs badly when importing pages via CSV. I really should have tested this 1,5 years ago when @adrian suggested it (I guess I needed a rest). All the other testers back in 2017 obviously used MyISAM as their import results were so performant. My results from today: MyISAM: 30729 pages imported, 120 s timeout (actual time 4 min 42 s) InnoDB: 7232 pages imported, 120 s timeout (actual time 9 min 10 s) Used a stopwatch for the actual time. Machine is i7-6700K, 32GB memory, SSD. Dev environment running under Docker. PW 3.0.98. DB charset utf8mb4. PHP Version 7.2.5 Batch Child Editor used for importing. Test case included as attachment. It has importable JSON for fields and the template and a CSV file. CSV is tab separated and all fields are enclosed with the character ^ My MySQL cnf override is: [mysqld] innodb_buffer_pool_size = 16G # (adjust value here, 50%-70% of total RAM) innodb_log_file_size = 256M innodb_flush_log_at_trx_commit = 1 # may change to 2 or 0 innodb_flush_method = O_DIRECT innodb_file_per_table=ON innodb_stats_on_metadata = OFF innodb_buffer_pool_instances = 8 # (or 1 if innodb_buffer_pool_size < 1GB) query_cache_type = 0 query_cache_size = 0 innodb_autoinc_lock_mode = 2 innodb_io_capacity_max=6000 innodb_io_capacity=3000 key_buffer_size = 1024M max_allowed_packet = 1024M table_open_cache = 2000 It seems to me we can't affect the performance by simply tweaking InnoDB settings. I assume PW would need core changes to adapt to InnoDB here. Pinging @ryan I did some digging and ran into an article, which details the bugginess of InnoDB FULLTEXT indexes. I went through the list of bugs and only a couple were fixed (some were closed as wontfix). Kind of depressing ? csv-testcase.zip

I answered you on IRC, but then you quit. I will just share my docker-compose.yml in case you want to try it out (for local development): version: '3.0' services: caddy: image: abiosoft/caddy:latest volumes: - /path/to/local/directory/Caddyfile:/etc/Caddyfile - /path/to/local/directory/processwire:/srv ports: - "2015:2015" links: - mariadb - php-fpm restart: always mariadb: image: bianjp/mariadb-alpine:latest environment: MYSQL_DATABASE: mydb MYSQL_USER: mydb MYSQL_PASSWORD: mypass MYSQL_ROOT_PASSWORD: mypass volumes: - /path/to/local/directory/mariadbdata:/var/lib/mysql - /path/to/local/directory/override.cnf:/etc/mysql/conf.d/override.cnf ports: - "3306:3306" restart: always php-fpm: image: wodby/php:latest environment: PHP_FPM_USER: wodby PHP_FPM_GROUP: wodby PHP_MAX_EXECUTION_TIME: 180 PHP_MEMORY_LIMIT: 4096M PHP_UPLOAD_MAX_FILESIZE: 4096M PHP_POST_MAX_SIZE: 4096M PHP_MAX_INPUT_TIME: 180 PHP_MAX_INPUT_VARS: 10000 volumes: - /path/to/local/directory/processwire:/srv restart: always I use that separate wodby php, because I can define the running user/group to be "wodby", which has ID 1000, so I don't have to fiddle with my local file ownership (same ID as my host user). As you see, I have PW's files on local disk and also the database + Caddy and MariaDB configs.

It is not the law yet

So a pretty cool thing happened

Why push these proprietary and closed source slacks and discords, when we have an IRC channel? https://kiwiirc.com/nextclient/#irc://irc.freenode.net/processwire We will soon see some very cool announcements from the Kiwi IRC camp. IRC is stepping up its game. Also, if you use Vue.js, it is easy to get on board and contribute to Kiwi IRC.

Finally got around to watching this recent documentary about Microsoft's abuse of monopoly and its lobbying practices in the EU: MS lobbyists using government email addresses and other fun stuff revealed!

If this sort of thing gets implemented in GitLab and the whole self-hosted sphere, it will be interesting: Implement cross-server (federated) merge requests (+ see related issues) Btw. if someone is wondering why people are alarmed: https://jacquesmattheij.com/what-is-wrong-with-microsoft-buying-github

Mapbox itself seems to be using Leaflet: https://www.mapbox.com/help/define-leaflet/ I don't know how to use your studio map with the Leaflet module.

Can you describe, how exactly would you use a mapbox map? Why is the default map not enough?

The newest versions of Leaflet.js (>= 1.2) do not fully work with the PW module. A couple of undocumented plugins were needed to get to the point where multi-marker maps show the clusters, but single markers will not show up: https://github.com/madebymats/FieldtypeLeafletMapMarker/issues/15 https://github.com/madebymats/FieldtypeLeafletMapMarker/issues/16

Cool, I saw you swing by #libreoffice-dev to give thanks just now

When I said "you naturally can't use "grid in grid" for navigation layout etc.." I was referring to the simple fallback solution of all unsupported browsers falling back to mobile layout. I was not referring to what CSS Grid can or cannot support.

Another framework from China: https://github.com/walkor/Workerman

You posted the correct link, but Adrian missed it

...aaand another new version: http://leafletjs.com/2018/01/15/leaflet-1.3.0.html I assume it is still blocked by this, should probably fire up Tracy and do more digging: https://github.com/madebymats/FieldtypeLeafletMapMarker/issues/16

Note that the Flot library listed here is essentially orphaned. Pure D3 sounds like a solid choice. I really like where this thread has gone so far I hope I can make use of this and buy bernhard dozens of drinks in the future (his liver better be made of steel).