Jump to content

Custom Fieldtype for Amazon S3


dfunk006
 Share

Recommended Posts

  • 3 years later...

Hey @joer80,
I've been developing a module that mirrors every (or ones you specify) upload to S3 bucket. I've started with S3 and Cloudfront support and I'm planning to support other CDNs in the future. Backblaze seems like a great provider to include as well. Once it is finished, it will be a near complete CDN solution.

Right now basic uploads are working fine, but it's not ready for production yet. However, I hope to complete and release it in a few weeks. Is there any feature that you think would be nice?

Some features I'm planning to include before releasing:

  • Full Amazon S3 support
  • Origin pull support
  • Process module/page for managing buckets and objects, such as clearing buckets, force uploading files & directories, maintenance mode etc.
  • Configuration page
  • Caching uploaded file paths in DB to keep track of uploaded files and fallback to local to local urls to prevent 404 errors.
  • Textformatter module for rewriting local urls to CDN urls. Optional custom domain support etc.
  • Form integration with AJAX for quick upload/upload all/force upload/delete/delete all etc buttons for file/image fields while editing a page/image.
  • Hooks for custom file uploads
  • Extendable/hookable API for new services

Future considerations

  • Other providers. Backblaze, MaxCDN etc.
  • Wireshell integration, (not sure how extensible WS is, but it'd be nice to have)
  • CSS rewrites? (for imports, images etc)

 

Edited by abdus
Added plans
  • Like 3
Link to comment
Share on other sites

With a custom field type you are not required to store the file locally.  You can choose to have it only on s3 or b2.  Is that the case with your module?  Or do you need to keep it both places?

With so many vps being small ssd, it would be great to offline files to cloud. 

Link to comment
Share on other sites

29 minutes ago, joer80 said:

Or do you need to keep it both places?

There's no need. I'll provide an option to remove local copy right away on successful upload, and a maintenance function to remove mirrored files from local drive as well.

The module doesn't even need to be a custom fieldtype, it hooks into built-in modules to get file paths then upload to cloud.

  • Like 3
Link to comment
Share on other sites

  • 3 weeks later...
On 5/1/2017 at 1:06 PM, abdus said:

Hey @joer80,
I've been developing a module that mirrors every (or ones you specify) upload to S3 bucket. I've started with S3 and Cloudfront support and I'm planning to support other CDNs in the future. Backblaze seems like a great provider to include as well. Once it is finished, it will be a near complete CDN solution.

Right now basic uploads are working fine, but it's not ready for production yet. However, I hope to complete and release it in a few weeks. Is there any feature that you think would be nice?

Some features I'm planning to include before releasing:

  • Full Amazon S3 support
  • Origin pull support
  • Process module/page for managing buckets and objects, such as clearing buckets, force uploading files & directories, maintenance mode etc.
  • Configuration page
  • Caching uploaded file paths in DB to keep track of uploaded files and fallback to local to local urls to prevent 404 errors.
  • Textformatter module for rewriting local urls to CDN urls. Optional custom domain support etc.
  • Form integration with AJAX for quick upload/upload all/force upload/delete/delete all etc buttons for file/image fields while editing a page/image.
  • Hooks for custom file uploads
  • Extendable/hookable API for new services

Future considerations

  • Other providers. Backblaze, MaxCDN etc.
  • Wireshell integration, (not sure how extensible WS is, but it'd be nice to have)
  • CSS rewrites? (for imports, images etc)

 

How is your project coming along?

Link to comment
Share on other sites

Hey @joer80,

I've been dealing with finals/graduation/masters applications lately and cannot spend as much time coding as I would like. Because of this I haven't been active in the forum for the last two weeks as well.
However, hopefully tomorrow, after submitting a project which filled my last three weeks, I'll be finished with school (for now) and be able to go back on coding. Once I get a usable beta going, I'll open a topic for the updates and feedback.

Thanks a lot for your interest.
Abdus.

  • Like 3
Link to comment
Share on other sites

 Share

×
×
  • Create New...