Jump to content
hellerdruck

Filter folder structure, download, file-import in PW

Recommended Posts

Hi all

I need help with something. Situation: We have let's say 2'000 Files (Excel) that should be displayed (list with links) on a page. We'd need to filter these files by given Keywords or a tree structure or both. Now, I'm looking for a solution whereas our customer can synchronise the files from his local computer with the folder on the webserver. They will update and upload files on a daily basis. Therefore, it would need to synchronise rather than load the files manually in pages or repeaters. Maybe indexing would be an idea, too.

Are there any modules for Processwire that would help achieving this? Could anyone point me in the right direction?

Thanks in advance.

Share this post


Link to post
Share on other sites
10 hours ago, hellerdruck said:

that should be displayed (list with links) on a page

  1. Displayed in the backend?
  2. Links to whats? 
  3. Are the files pages? or stored as files in file fields in multiple or a single page?
10 hours ago, hellerdruck said:

with the folder on the webserver

Hmm. So, seems like 'loose' files stored on the system? Not ProcessWire managing the files?

10 hours ago, hellerdruck said:

Are there any modules for Processwire that would help achieving this?

I don't think so. If the files were pages, then maybe Lister or (Lister Pro). There could be others I have forgotten about.

Depending on your responses to the above questions, there could be other approaches/solutions.

Share this post


Link to post
Share on other sites

Hey @kongondo, thanks for replying.

1. no, displayed on a front-end page
2. Links to the files, the files should be downloadable by the visiting user
3. not sure how to store them best. it's a load of files, and it would be great, if they shouldn't be uploaded one by one. More like batch processing or a folder with the files on the server. I guess it would be great if PW could manage the files, but the customer does not want to go to the backend to upload the new versions of the files individually. rather some sort of folder synchronisation. if possible.

does that make sense?

Share this post


Link to post
Share on other sites
5 minutes ago, hellerdruck said:

does that make sense?

It does. So, there's clearly two main groups of issues. One, the frontend bit and two the backend.

Frontend
 

12 hours ago, hellerdruck said:

We'd need to filter these files by given Keywords or a tree structure or both.

This is for frontend as well? If yes, rather than re-invent the wheel, you could use on of the plethora of 'data tables' out there. 

Backend

I am assuming this is where you want the client to do the uploading (albeit not individually) or...

9 minutes ago, hellerdruck said:

but the customer does not want to go to the backend to upload the new versions of the files individually. rather some sort of folder synchronisation.

do you mean you want to give them some sort of FTP access? I hope not.

Bottom line is the client will need to upload the files, somewhere so best if they login, get authenticated and upload the files. The easiest thing is for them to upload a zip file. Obviously, size of file matters as well as for synchronisation, you need identical file names. You may also need to store backups of older files for a while just in case client uploaded the wrong files and everything got overwritten (basically some sort of versioning system). 

For the upload area, you could go for a custom Process module that allows ajax upload (works nicely with batch file uploads if you need that sort of thing). The batch Processing would be all PHP. Somewhere in the forums there's a module that can run tasks at set times, but not sure if that is what you want. The Process module could either have pages, for uploading and for viewing current files and maybe for doing backups (that only you can access). Or, you can have one page split into n tabs.

Pages or Not

This depends on your needs, especially the frontend ones. For instance, if you want human readable titles to be displayed in the frontend together with the files, then pages would server you well. This is because ProcessWire would sanitize the file names and you end up with lots of underscores or hyphnes in file names. On the other hand, if you manage the files yourself, you will have to write the code (but $files() can help), to track files. If you are worried about the visibility of 2000+ pages with files in the backend, you can 'hide' these under /admin/some-parent/.

Just thinking out loud here.

22 minutes ago, hellerdruck said:

sort of folder synchronisation

There are tools for that, but not ProcessWire, although you could achieve some sort of synchronisation yourself using ProcessWire based on a few simple rules.

Share this post


Link to post
Share on other sites
32 minutes ago, kongondo said:

do you mean you want to give them some sort of FTP access? I hope not.

Actually, I thought this could be an option. Include a folder structure somewhere in "templates" and have them synchronise with something like Cyberduck.

34 minutes ago, kongondo said:

This is for frontend as well? If yes, rather than re-invent the wheel, you could use on of the plethora of 'data tables' out there. 

yes. Their customers need to access this page front-end and filter by type, classes, keywords, whatever. Jquery Data Table looks good at first glance. I'll definitely check that out.

1 hour ago, kongondo said:

For the upload area, you could go for a custom Process module that allows ajax upload (works nicely with batch file uploads if you need that sort of thing). The batch Processing would be all PHP. Somewhere in the forums there's a module that can run tasks at set times, but not sure if that is what you want. The Process module could either have pages, for uploading and for viewing current files and maybe for doing backups (that only you can access). Or, you can have one page split into n tabs.

Could the Media Manager work here for the batch processing? Something else that would make things even more complicated: is it possible to index the filex by keywords in a batch upload?

 

1 hour ago, kongondo said:

There are tools for that, but not ProcessWire, although you could achieve some sort of synchronisation yourself using ProcessWire based on a few simple rules.

Could you specify this a little?

Sorry for all the questions..

Share this post


Link to post
Share on other sites
10 hours ago, hellerdruck said:

Actually, I thought this could be an option. Include a folder structure somewhere in "templates" and have them synchronise with something like Cyberduck.

OK. Won't they get confused by the interface though?

10 hours ago, hellerdruck said:

Jquery Data Table looks good at first glance. I'll definitely check that out.

Also check out Tabulator

10 hours ago, hellerdruck said:

Could the Media Manager work here for the batch processing?

Yes, in a sense, though it depends on whether you want automatic  (i.e. no user interaction, like an auto module or cron thing) versus manual (press a button and it gets synchronized.  The latter can be done, easily in Media Manager. The former would need a bit of coding, essentially calling Media Manager API via some automated process. 

Media Manager has a 'Scan' uploads mode. How it works is you upload files to a set folder in /site/MediaManager. You can upload files to that folder via FTP, or other method. Inside Media Manager. The files can be loose or zipped. If zipped, Media Manager would unzip them. The files are then presented in a list. You can then select the ones you wish to upload to the Media Manager library. Those added to the library are then deleted from the folder where they were uploaded. There are settings for what to do when Media Manager encounters duplicates: rename/replace/skip. In your case, you would go for replace (synchronization). 

10 hours ago, hellerdruck said:

is it possible to index the filex by keywords in a batch upload?

In Media Manager? Yes, but post upload (not during). Each media is a page. Say you uploaded a file called My End of Year Report.xlsx. Media Manager would create a page with that file name (sanitized) as the title. That will end up as My End of Year Report. The file name itself would be sanitized as my-end-of-year-report.xlsx. Post-upload, you can manually add keywords, e.g. via tags, or a custom field or fields (e.g. options, page reference fields, etc). 

If you want automatic indexation, that would require some custom code. If this is something you want to pursue alongside a Media Manager option, we can discuss further via PM (as a custom paid option).

10 hours ago, hellerdruck said:

Could you specify this a little?

I meant tools like rsync 😄 

  • Like 1

Share this post


Link to post
Share on other sites
6 hours ago, kongondo said:

In Media Manager? Yes, but post upload (not during). Each media is a page

Is there a demo version i can try? would love to play around and find out if it works for me..

Share this post


Link to post
Share on other sites
17 minutes ago, hellerdruck said:

Is there a demo version i can try?

There isn't, I am afraid. Now that I think about it, maybe I should set up an online demo. Otherwise, you can purchase a copy (fully refundable within 7-days' of purchase date) to test.

Share this post


Link to post
Share on other sites
25 minutes ago, kongondo said:

maybe I should set up an online demo

you should 😀

25 minutes ago, kongondo said:

you can purchase a copy (fully refundable within 7-days' of purchase date) to test.

will do that right away, thanks!

  • Like 1

Share this post


Link to post
Share on other sites
17 minutes ago, hellerdruck said:

will do that right away, thanks!

Please let me know how it goes. Post questions related to MM in its support forum here:

 

Share this post


Link to post
Share on other sites

You could also look into implementing Dropzone.js on the frontend and pass the files via ajax to a function that would loop through the files and create pages for them. It has been awhile since I last did it (over a year) and I would need to hunt down the code, but it wasn't too much of a pain to get sorted.

However, Media Manager is a solid option in the backend.

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By donatas
      Hi,
      I am trying (wondering if even possible) to set a very different file folder path for my PW install. The path I want to set exist in the same server, but in another domain. Can the PW API see that far into file system? It is a shared hosting environment so no permission management options available.
      Also can this be achieved through .htaccess redirection? If you request a file (images mostly, but translation json files are important too) from `domain-A.com/site/assets/files/` to grab them from `domain-B.com/site/assets/files/`? I would prefer to do this through PW API, I can see the function `$config->setPath()` but it didn't work when set in  config.php like `$config->setPath("files", "/domains/domain-B/public_html/site/assets/files/")`.
      Is there some other option to do this?
      My ultimate goal is to have two PW installs on different domains but one is just a "mirror" that is using the same database as the other and should use the same files structure if editors upload any images to the main (domain-B.com) website. I could use domain parking function but it then needs a more expensive SSL certificate for two domains , which I am trying to avoid if possible 🙂 (I'd like to use single domain Let's Encrypt certificates, thus I need to PW installs).
      Would appreciate any insight! Thanks!
    • By Vineet Sawant
      Hi,
      I'm trying to import some heavy data into Processwire, but I'm not sure what would be the best way to do it.
      Usually I use CSV to Pages plugin, but this time the data is too heavy(~40k rows with 10+ columns of excel sheet), thus this plugin can't help.
      I also tried Tasker plugin but I can't seem to go through the setup itself, it requires some template setup but I'm totally clueless about how to do it, so that plugin is not of any use either.
      I wanted to know from you guys how you do it and in future what would be the best way to migrate thousands of rows of data in to PW.
       
      Thanks.
       
       
    • By dotnetic
      Hi folks, I published "Simple file downloads with ProcessWire tutorial"  today which explains how to make a simple download function with ProcessWire (tested with version 3.0+).
      Basically this is based on my post here in the forums 
       
    • By Rodd
      Hi everyone!
      I have a website in a production environment and I want to duplicate it in a local environment. I exported the content of the website (with the 'Site Profile Exporter' module) but I cannot use it actually. I've got an issue with the database. I imported this one in MAMP then.

      I also exported the pages (with the 'ProcessPagesExportImport' module), but I cannot import it to my local website because the fields don't exist. So I created this fields, but I have this error :
      How can I use the elements that already exist and are presents in my database? How can I duplicate correctly the templates, fields and pages?
      Thanks by advance
      PS: Sorry if my english is bad
       
×
×
  • Create New...