Question about "Assets/Files" folder

[Zahari M.]

Hi Guys

Looking into the site/assets/files folder, there is a folder there for perhaps every single post.

Coming from WordPress and Drupal, they tend to group such assets into sub folders, say by way of month/year folders...

Of course the ProcessWire way has a simplicity and directness to it. But it looks like it can get pretty big!

My questions are.... if we have say 10,000 posts,

1. In the case of Mac users here, does our Macs finder have any difficulty displaying 10,000 folders at once?

2. For those of you who have been there , any tips or suggestions to be mindful off when working with a large number of folders?

Cheers guys!

[kongondo]

The folders contain images uploaded on a page. The folder names are the IDs of the pages. See the following topic for a discussion about this issue:

http://processwire.com/talk/topic/992-problem-with-assetsfiles-folder/

And an empty folder cleaner

http://processwire.com/talk/topic/1585-module-clean-empty-directories-from-siteassetsfiles/

http://modules.processwire.com/modules/page-clean-empty-dirs/

[Martijn Geerts]

Large amounts of sub folders is still a little concern I think. ( on ext3 max 32000 subdirectories or something )

I can imagine sites with user generated content easily grow very big in terms of subdirectories. I'm not worried about the PW core.

[k07n]

what about /assets/files/1/4/5/8/

instead of /assets/files/1458/

for post id 1458?

[Martijn Geerts]

I don't see a good option for this. How it is now is realy clean & I don't know if there's a site right now that reaches that limit.

And normal pw installs, and modules should not suffer i think.

Sorry bringing it up again. Somewhere i knew there was an old topic. But had forgotten it.

[Zahari M.]

Thanks for that link kongondo!

I was thinking a bit about this after reading that link..... If i had a blog and each post had a featured image, then the "practical limit" for the maximum number of posts would be ~ 30,000 posts due to file system limits.

Be interesting to see how this one gets overcome eventually.

Fortunately for me, Im not a prolific blogger

[DaveP]

I am absolutely confident that if anyone actually had > 32,000 (ext3) or > 64,000 (ext4) pages with images or other files and came to the forum asking for help, Ryan or someone else here would have a solution available in a couple of days at most.

At the moment however, that hasn't yet happened, so the problem is moot.

[teppo]

what about /assets/files/1/4/5/8/

instead of /assets/files/1458/

for post id 1458?

That's essentially what Antti suggested on the linked thread, and as you can see from Ryan's reply it'll probably be added at some point. Like @DaveP pointed out above, I wouldn't worry too much about this.. though I do know of some sites that are steadily approaching that ext4 limit and wouldn't mind if this got tweaked soon(ish)

[k07n]

That's essentially what Antti suggested on the linked thread, and as you can see from Ryan's reply it'll probably be added at some point. Like @DaveP pointed out above, I wouldn't worry too much about this.. though I do know of some sites that are steadily approaching that ext4 limit and wouldn't mind if this got tweaked soon(ish)

Dunno how i've missed 1st link from kongondo. I'm very glad to know, what this problem on queue.

[k07n]

I don't see a good option for this. How it is now is realy clean & I don't know if there's a site right now that reaches that limit.

Maybe we have not such sites because of the limit?

What about e-shops with 100k+ goods or blogs with open registration ("Oh, I'm in shop with 70% sale, look what I have bought"... photo-photo-photo.. "oh, I'm so tired of this, take a look on my dinner".. click-click-click... =)))?

[Martijn Geerts]

Im sorry for the panic k07n. As you can see the community will be there.

[ryan]

I think it really depends on the file system and version of it. If I recall correctly, many servers have no practical limit nowadays. It sounds like it used to be a bigger consideration than it is now. But it's one of those things that I'm not sure is worth the effort of "solving" until we have an instance of it being a problem. So far, nobody has ever run into any issues as a result of this, and we certainly have many installations with more than 30k pages. Though I don't know of any installations yet with 30k pages that each contain file assets, so always keeping an eye open to it. 

[nfil]

I had an inodes limit on LAMP server, the hosting company blocked the cpanel file manager and limited my ssh access, no compress files etc, I used ssh to remove all the cache/temp files by type and folder, it was a nightmare to backup and move the contents to another server.

This happened due to clients uploading/changing lots of contents before the sites went live, modX revo cache folder and a two shops cache had more than 60k inodes, Now I'am really careful if the client wants a very content dynamic site, I host those on a reseller acount or a VPS server where I customise the limitations and its easier to backup. 

On Process Wire, I also use the module mentioned above,  http://modules.processwire.com/modules/page-clean-empty-dirs/ and from a PW noob point of view does the job quite well.