PW3 - Custom Upload Method for Data Archive

[ridgedale]

Reference: PW 3.0.62 and uikit3 based site using the Regular-Master profile.

I was wondering if it is possible to create an upload method that is not directly associated with any particular Processwire page so that a custom folder can be used from within the admin frontend to upload for example newsletters so they are stored in one hierarchical location to allow the latest edition to be automatically displayed using a single code call.

What I'd like to be able to do is provide an upload method that allows a user to create year folders and upload newsletters to the relevant folders, making sure the uploaded files are named as required.

Any input would be greatly appreciated.

[flydev]

Hi,

As far as your webserver has write permission on a given folder outside the web directory, you can write your logic in a module which handle your upload thing - everything is possible inside ProcessWire.

1 hour ago, ridgedale said:

What I'd like to be able to do is provide an upload method that allows a user to create year folders and upload newsletters to the relevant folders, making sure the uploaded files are named as required.

I personnaly would go with pages. Make a parent page "newsletters-upload" in your page tree then create a child page for each "year", then create a child page inside "year" for each uploaded newsletter. Each newsletter page have a fieldtype-secure-file field which give you the possibility to store your uploaded files outside the webfolder. 

In the process module, the author could create/select a "year" page (folder) and upload file by drag&drop in the admin.  You can also pull files from an FTP server where authors create their folder tree, then do the logic in the module.

 

This is just an idea from what I understand, more details needed  ?

[Robin S]

2 hours ago, ridgedale said:

What I'd like to be able to do is provide an upload method that allows a user to create year folders and upload newsletters to the relevant folders, making sure the uploaded files are named as required.

PW does not really use the concept of folders. I agree with @flydev - you should use pages in place of folders.

I've used a repeater for doing something quite similar to what I think you are describing. Create a page that will be the central place where all files are uploaded. Add a repeater to that page's template, containing an integer field for year and a files field to hold the uploads. Users will upload their files to the relevant year item. Using API code you can get the most recent year (i.e. the highest integer) and the most recently added file in that year (the last file in the field).

For automatic upload renaming there is @adrian's Custom Upload Names module.

[LMD]

I am doing something very similar to this and am just using pages.

I use two templates:  'newsletters_parent' and 'newsletters_child'.

The 'newsletters_parent' is the container and onyl has a title, but its settings on the "Family" tab are:

• Can have children: YES
• Can be used for new pages: ONE
• Allowed template(s) for children: newsletter_child
• Name format for children: Y/m/d H:i:s  (the page creation date/time -- this will get overwritten at a later stage)
• Sort settings for children:  pub_date (a field that will be added to the newsletter_child page)
• Reverse sort direction: CHECKED (yes)

The 'newsletters_child' template has the following fields:

• title (default and required by all templates)
• pub_date (a datepicker field  set to YYYY-MM-DD  -- the publication date might not necessarily be the same as the date the file was uploaded!)
• pdf_file (file-upload field, single file only, and also set to only allow PDF files)

The settings ("Family" tab) for the child template are:

• Can have children: NO
• Can be used for new pages: YES
• Allowed template(s) for parents: newsletter_parent
• Show in the add-page shortcut menu: YES

To make the process of adding a new newsletter as easy as possible for users -- all they have to do is select the file to upload and the pick the pub date -- I use a couple of hooks hook in the site's ready.php file:

The first hook adds dummy text to the page title field on page creation so that the user doesn't have to enter anything (if it is left blank, it will throw an error) -- the proper title, along with the page name will be created from the pub_date field when the page is saved.

The second hook also creates the actual page title and the sets page name based on the supplied 'pub_date'. [NOTE: the newsletters only get published once a month, so we only use the Year/Month in page names/titles.]

/**
 * 1. Set a dummy title on page save at creation, to prevent the required field showing an error if it is blank.
 *    (Requires the parent template option 'Name format for children' to be set)
 * 2. Modify newsletter title and page name from pub_date before saving.
 */
$pages->addHookAfter('saveReady', function(HookEvent $event) {
	$page = $event->arguments(0);

	// Only on newsletter_child templates where the title is blank (on page creation)
	if ($page->template->name == 'newsletter_child' && $page->title == '') {
		$page->title = 'Newsletter'; // a dummy title, will be replaced at a later stage
	}

	// Only on newsletter_child templates with a pub_date
	if ($page->template->name == 'newsletter_child' && $page->pub_date) {
		$newName = wire('sanitizer')->pageName('newsletter-' . date('Y-M', $page->pub_date), true);
		$newTitle = date('F Y', $page->pub_date);
		if ($page->title != $newTitle) {
			$page->title = $newTitle;
		}
		if ($page->name != $newName) {
			$page->name = $newName;
		}
	}
});

 

Thiis then makes it easy to search the system for whatever you want:

// All newsletter files (no limit applied)
$allNews = $pages->find('template=newsletter_child, sort=pub_date'); // ascending order (earliest first)
$allNews = $pages->find('template=newsletter_child, sort=-pub_date'); // descending order (most recent first)

// All newsletters from a particular year (e.g., 2017):
$yearStart = strtotime('2017-01-01 00:00:00');
$yearEnd = strtotime('2018-01-01 00:00:00');
$yearNews = $pages->find("template=newsletter_child, pub_date>={$yearStart}, pub_date<{$yearEnd}, sort=-pub_date");

// The most recent newsletter:
$mostRecent = $pages->findOne('template=newsletter_child, sort=-pub_date');
 
// Accessing the file itself (using $mostRecent)
if ($mostRecent->pdf_file) {
	$file = $mostRecent->pdf_file;
	$downloadLink = "<a href='{$file->url}'>Download the newsletter: {$mostRecent->title}</a> ({$file->filesizeStr})";
}
  

 

That's all there is, really.  There is lots you can do with the system.

 

Edited March 21, 2018 by LMD
Realised I had two hooks, when I only needed one!

[ridgedale]

Thank you for all your feedback, flydev, Robin S and LMD.

I should have clarified that I have already setup a Parent page for the newsletters and the Children pages to each hold 10-years' worth of monthly newsletters - the newsletters go back to 1975 (the ones I've scanned and have yet to process) and possibly well beyond. See the screen grabs attached.

The thinking behind a single folder hierarchy for the newsletter store is that, firstly, it would allow straightforward portability should the archive ever need to be moved, and, secondly, to facilitate the automatic display of the current newsletter based upon today's yyyy-mm date, provided it has been uploaded, or otherwise display the previous month's newsletter.

The problem I foresee with storing the files as media attached to the Pages is that the paths to the files will not be predictable making the automatic displaying of the latest newsletter in the sidebar less than straightforward, if not impossible, and the newsletters will end up being scattered among many different folders meaning the portability of the archive would be less than ideal.

Any further thoughts or suggestions would be welcome. Thanks again.

[Robin S]

@ridgedale, the essence of your scenario is that you have some newsletters, and for each newsletter you have:

1. A file
2. Some metadata about that file (its month and its year)

You want to associate the metadata with the file. Using a file naming scheme and a folder structure is one way to associate the metadata with the files, but it's not the only way, nor is it the "ProcessWire way".

The ProcessWire way is to use pages and/or the metadata fields that are built into File fields.

If you have a lot of metadata to associate with a file then the most powerful approach is the "one page per file" approach, which is what @LMD is suggesting. This way you can use all the available PW fields in your file page to store different kinds of metadata about each file.

But the downside to this approach is that the UI/workflow can be a bit less than ideal for clients, especially if they have a lot of files that they want to upload at once and populate metadata for. That's because, generally speaking, they will have to edit each file page individually (although there can be solutions to this such as Lister Pro inline editing, Batch Child Editor, and automatically creating a page for each file similar to what is done in AutoImagePages).

But you only have a little bit of metadata per file so I don't think you need to go this sophisticated. Instead I suggest you use a repeater for the years, and file tags for the months. See these screenshots...

Then you can get the latest newsletter like this...

$newsletters_page = $pages->get("template=newsletters");
$latest_year = $newsletters_page->newsletters->find("newsletter_files.count>0, sort=-year")->first();
$latest_newsletter = $latest_year->newsletter_files->sort("-tags")->first();
$month = substr($latest_newsletter->tags, 3);
echo "<a href='$latest_newsletter->url'>$month $latest_year->year</a>";

 

13 hours ago, ridgedale said:

I should have clarified that I have already setup a Parent page for the newsletters and the Children pages to each hold 10-years' worth of monthly newsletters

The pages that visitors browse the newsletters through can be completely decoupled from the storage of the newsletter files. You just get the relevant repeater items by year according to the page being viewed, e.g.

$years = $newsletters_page->newsletters->find("year>=1980, year<1990, sort=year");

You could even do it without actual pages for each decade, and use URL segments to get any range of years.

[ridgedale]

Hi Robin S,

Your suggestion looks like it should fit the bill. Is it possible to reverse sort the repeater fields so the most recent year appears at the top of the list?

Thanks again for all your input.

[flydev]

5 hours ago, ridgedale said:

Is it possible to reverse sort the repeater fields

from @Robin S

$this->addHookAfter('FieldtypeRepeater::wakeupValue', function($event) {
    $field = $event->arguments('field');
    if($field->name !== 'myrepeater') return; // repeater field name
    $pa = $event->return;
    $pa->sort("-created"); // or whatever sort you want
    $event->return = $pa;
});

 

Will return :

• 2018
• 2017
• 2016
• ...

[Robin S]

9 hours ago, ridgedale said:

Is it possible to reverse sort the repeater fields so the most recent year appears at the top of the list?

Another approach besides the one @flydev mentioned:

$pages->addHookAfter('saveReady', function(HookEvent $event) {
    $page = $event->arguments(0);
    if($page->template != 'newsletters') return; // Only for specific template
    $page->newsletters->sort('-year'); // Sort the repeater how you want
    // Sequentially set the sort property of the repeater items
    $i = 0;
    foreach($page->newsletters as $item) {
        $item->sort = $i;
        $i++;
    }
});