Proposing to add SSE to the core

[bernhard]

I've built this over and over in several of my modules (RockDevTools for Livereload, RockCalendar for creating events). Always a lot of work. Always a lot of boilerplate code. Always a lot of issues to fix. I think it's time to bring SSE (Server Sent Events) to everybody.

Simple example (Empty Trash):

Client-Side:

// create stream
const stream = ProcessWire.Sse.stream(
  'ssedemo-empty-trash',
  (event) => {
    const textarea = document.querySelector('textarea[name="empty-trash-status"]');
    try {
      let json = JSON.parse(event.data);
      stream.prepend(textarea, json.message, 100);
    } catch (error) {
      stream.prepend(textarea, event.data);
    }
  }
);

// update progress bar
const progressBar = document.querySelector('#empty-trash-progress');
stream.onProgress((progress) => {
  progressBar.value = progress.percent;
});

// click on start button
document.querySelector('#empty-trash').addEventListener(
  'click',
  (e) => {
    e.preventDefault();
    stream.start();
  });

// click on stop button
document.querySelector('#stop-empty-trash').addEventListener(
  'click',
  (e) => {
    e.preventDefault();
    stream.stop();
  });

Server-Side:

  public function __construct()
  {
    parent::__construct();
    /** @var Sse $sse */
    $sse = wire()->modules->get('Sse');
    $sse->addStream('ssedemo-empty-trash', $this, 'emptyTrash');
  }

  public function emptyTrash(Sse $sse, Iterator $iterator)
  {
    $user = wire()->user;
    if (!$user->isSuperuser()) die('no access');

    $selector = [
      'parent' => wire()->config->trashPageID,
      'include' => 'all',
    ];

    // first run
    if ($iterator->num === 1) {
      $iterator->max = wire()->pages->count($selector);
    }

    // trash one page at a time
    $p = wire()->pages->get($selector);
    if ($p->id) $p->delete(true);
    else {
      $sse->send('No more pages to delete');
      return $sse->stop();
    }

    // send message and progress info
    $sse->send(
      $iterator->num . '/' . $iterator->max . ': deleted ' . $p->name,
      $iterator
    );

    // no sleep to instantly run next iteration
    $sse->sleep = 0;
  }

Code + Readme: https://github.com/baumrock/SSE/tree/dev

What do you think? Would be nice if you could test it in your environments and let me know if you find any issues!

[poljpocket]

Hi Bernhard, very cool module! I can imagine some interesting use-cases in combination with RockDaemon.

I have one question: Why are you using token-based authentication? What's wrong with the session PW provides?

Edited August 11 by poljpocket

[bernhard]

3 hours ago, poljpocket said:

Why are you using token-based authentication? What's wrong with the session PW provides?

From my research/testing this was necessary to make the SSE stream non-blocking. When using regular url hooks after the session has been started, then you can't open any other tabs while the stream runs. That's one of the caveats that you'll have to fight when developing something from scratch.

3 hours ago, poljpocket said:

Hi Bernhard, very cool module! I can imagine some interesting use-cases in combination with RockDaemon.

Thx 🙂 For example?

[poljpocket]

36 minutes ago, bernhard said:

For example?

• Cleaning up image variations
• Warming caches
• ...

e.g. with a clickable button and a progress bar in the PW Admin. Using RockDaemon to manage the process doing it.

[bernhard]

Nice!! Yeah, absolutely 🙂 

[dotnetic]

Great addition @bernhard. I would love to see this in the core. Async and streaming operations are everything, if you do custom programming with ProcessWire. Did you forget to provide a link to the PR or issue on github? I would give a thumb up.

[poljpocket]

I think this being a module for now is just fine. It can easily be added to the core's module folder sometime down the road (as it was the case for example with the TinyMCE module). We absolutely should follow the project's guidelines about additions to the core:

Quote

Please note that we generally avoid adding features that aren't going to be used by at least 30% of the ProcessWire audience. Often new features can be better accommodated with modules.

Time will tell though :)

[bernhard]

@poljpocket so you think the current solution for "empty trash" where after clicking it the user gets no feedback at all about what is going on and about the progress is a good user experience? And then if the chunk was not enough you have to click again and again until the trash is empty...

And the solution for that is to install the Sse module from baumrock's github profile?

[bernhard]

1 hour ago, dotnetic said:

Great addition @bernhard. I would love to see this in the core. Async and streaming operations are everything, if you do custom programming with ProcessWire. Did you forget to provide a link to the PR or issue on github? I would give a thumb up.

I would prefer to give it more real world testing before I do that. In my projects I seem to get some blocking behaviour if a stream is running. It has not been an issue with the sse based livereload but I'm working on an sse based page edit lock and I'm seeing some strange behaviour unfortunately.

[poljpocket]

Dear @bernhard, there is no reason to get defensive if people don't get you. I am sorry if I misunderstood you.

Of course, the answer to your questions would be no and no. But your proposal is about adding server-sent events code to the core, not about improving the empty-trash functionality of the core. Although the latter might be a consequence of the former. Sure, you added a demo module which demonstrates a custom "empty trash" function. But that doesn't actually replace the current one and thus, you wouldn't ever add something like this to the core. And I stand by my opinion about things like this: They don't have to be in the core if the core doesn't make use of them (as I also mentioned in the JS hooks discussion).

Wouldn't it be nice to turn this around? Make a proposal for an improved experience when handling the trash using SSE to accomplish it. This means, actually changing the code of the core. This, as a consequence, could mean that the core adds SSE functionality for everyone to use. And this is where your code for sure is a very good place to start.

Did I understand you correctly now?

[dotnetic]

SSE (Server-Sent Events) would be a game-changer for ProcessWires core, and the trash functionality example perfectly illustrates why:

The Current Problem: As you mentioned, the current trash system is painful - deleting hundreds of pages requires multiple confirmations, long waits, and often timeouts. Your clients shouldn't need API knowledge just to empty their trash efficiently.

How SSE Solves This:

• One-click bulk operations: Start deleting 1000+ pages with a single confirmation
• Real-time progress: "Deleting page 234 of 1000..." instead of a frozen screen
• No more timeouts: SSE keeps the connection alive, bypassing PHP execution limits
• Graceful interruption: Users can safely stop/pause operations if needed

Beyond Trash - Core Benefits:

I have written several converter or importer modules for my clients, that import data and transform them into pages or a payment matcher module, which compares bank statements with invoice pages inside of ProcessWire. For all of these modules I wrote custom SSE Event handlers myself which is much boilerplate and duplicated code as Bernhard mentioned.

Imagine every ProcessWire installation having this built-in:

• Import/export operations with live feedback
• Batch page operations (move, publish, unpublish)
• Asset processing (image optimization, file management)
• Search index rebuilding
• Module installations and updates

Why Core Integration Matters:

• Standardized approach: All modules can use the same SSE implementation
• Better UX across the board: Every long-running operation becomes transparent
• Developer-friendly: No need to reinvent the wheel for each module
• Professional feel: Matches modern user expectations from enterprise CMS

Addressing Adoption Concerns: I can't predict what percentage of users would use this feature - nobody can provide statistics for something that doesn't exist yet. But consider this: every ProcessWire user who has ever dealt with timeouts, batch operations, or large imports would benefit immediately. The feature would be invisible to those who don't need it, while being invaluable to those who do.

Your modules already prove SSE works brilliantly with ProcessWire. Making it core functionality would elevate the entire ecosystem.