Jump to content

How to export all user emails to a CSV file?


Vineet Sawant
 Share

Recommended Posts

Hello,

I've a website with over 40k registered users who sign up using their email IDs. We've a ticket booking service and we wish to alert users about certain functionality changes in the site.

Whenever I'm trying to print all the email using foreach, page is going blank due to memory issue I suppose (it takes ages to respond).

Is there any way or any module which will export all the emails to a single CSV file?

Thanks.

Link to comment
Share on other sites

I don't think there's a module for this, but it's kinda easy to circumvent timeouts. You'd build a php file that calls a reasonable number of users (1000 or so) using the pagination feature of processwire. The results are then written / appended to a file somewhere in the system. And the php then returns the next pagenumber, at best as simple json string. Now you'd just build some js, which calls the same page via ajax and iterates on as soon as the response is returned. After 40 calls you've your full csv. 

Another way would be using some SQL, to get all user id's and select the emails directly from the email fields table. 

  • Like 1
Link to comment
Share on other sites

Another option is to call $pages->uncacheAll() at a regular interval. This helps you avoid memory issues in most cases, and allows you to run the whole query at once. For a rough example, see how it's done in VersionControl.module.

If you do take this route, make sure that your maximum execution time isn't an issue here either. If your script takes a long time to run, you'll usually want to increase the max execution time using set_time_limit() first. The default value is typically 30 seconds, and a script running longer than that gets killed.

First things first, though: you mentioned that the page goes blank supposedly because of a memory issue. If you're not 100% sure of the reason for the issue, the very first thing you should do is checking why it actually happens. Do the same thing in a test environment with $config->debug = true, or see your error logs on the live site. Assuming is never a good strategy :)

  • Like 2
Link to comment
Share on other sites

This worked really great for me in a site with thousands of pages

$selector = "template=pages_template"; // as an example

while (1) {
    $p = wire('pages')->get("{$selector}, id>$id"); // get page with id bigger than previous
    if(!$id = $p->id) break; // assign current page's id to $id or break the loop if it doesn't exist

    // do stuff using $p as the current page

    wire('pages')->uncacheAll();
};

Like this you never have an array in memory, only one page at each time, and you can do all the operation in one go. Of course you would probably have to constantly write to the csv file (like, open,write,close,open,write,close,open,write,close  edit: this doesn't make any sense you can open, write, write, write,... close), but I don't think that would be a problem.

Edit: Ideally you would do this from the terminal by bootstrapping PW, to prevent overloading apache.

  • Like 3
Link to comment
Share on other sites

The code on the post above is not full proof, if the process is interrupted for any reason, you would have to start over. The ideal would be to, on each iteration, read the last line of the file, identify the user, find the next user, and write the next line.

  • Like 1
Link to comment
Share on other sites

I ended up using limit with foreach & pagination.

I also found out that start=n in the selectors breaks pagination module. It stops working. Pagination renders page numbers but after clicking the page, e.g., 3 still shows data from page 1. I guess it happens cause I'm using start=n. When I removed, it started working just fine.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By prestoav
      Hi all,
      I've noticed that, when creating new users for clients, the chosen admin theme is not saved. For example, I create a new user select Reno or UIKit as the theme at point of creation and save. When logging in that user for the first time the admin theme is back to default. Changing it again and saving corrects this.

      Anyone else noticed this or is it a known bug at the moment?

      CMS V 3.0.156.

      Thanks!
    • By AndZyk
      Hello,
      we have many projects as pages containing images inside regular image fields and repeater matrix elements. Now we would like to organize our assets.
      ProcessWire saves assets with id as folder name and makes separate folders for repeater matrix elements.
      Is there a way to collect all assets of one page and export them in a folder with the page name/title?
      I think I could achieve this by using the command line wget for this website or maybe by a hook. But maybe someone has done this before. 😉
      Regards, Andreas
    • By hellerdruck
      Hi all
      I need to export all the texts from a website to a translation company (as json or csv or txt...). How can this be done? Of course manually, but this website is huge and it would take me years...
      Also, as a second step, importing the translation ...
      Any ideas anyone? Tutorials? Plugins?
      Thanks for your help.
    • By Rodd
      Hi everyone!
      I have a website in a production environment and I want to duplicate it in a local environment. I exported the content of the website (with the 'Site Profile Exporter' module) but I cannot use it actually. I've got an issue with the database. I imported this one in MAMP then.

      I also exported the pages (with the 'ProcessPagesExportImport' module), but I cannot import it to my local website because the fields don't exist. So I created this fields, but I have this error :
      How can I use the elements that already exist and are presents in my database? How can I duplicate correctly the templates, fields and pages?
      Thanks by advance
      PS: Sorry if my english is bad
       
    • By Greg Lumley
      Hi! I've just discovered Processwire and I'm absolutely loving how clean it is. Especially with the full control over what it outputs. 
      Excuse me if this is a really stupid question.
      What happens if a user wants to insert multiple images into the blog field? In other words Shift+Click all the images attached to a specific page in and insert. It seems I can insert one at a time but not multiple.

      Based on the above, I want to rebuild my website. I'm a photographer (ex php dev 2004) and plan also to target websites at other photographers. So inserting multiple images into blog posts is a key part. I'd like to output that to something like UIkit or some kind of Masonry Gallery script. Any thoughts would be fantastic please. 
      Thank you.
      Greg. 
      I'm an ex PHP developer now photographer, now thanks to Corona and not photo work, a rusty PHP developer 😁
×
×
  • Create New...