Need help making SSE work

bernhard · January 23, 2022

Hi community,

last week I've tried to get PHP SSE working. Unfortunately without success. When using laragon it works, but on my DDEV setup and also on my live server it does not. I think there is some kind of compression and/or proxying going on, but I don't know how to debug or disable that.

@kongondo and @netcarver have mentioned SSE is "is so simple it is unbelievable at first." Maybe you can help me?

This is what I've tried:

<?php
header("Cache-Control: no-cache");
header("Content-Type: text/event-stream");

$i = 0;
while(++$i<=10) {

  echo "event: ping\n";
  $curDate = date(DATE_ISO8601);
  echo 'data: {"time": "' . $curDate . '"}';
  echo "\n\n";

  file_put_contents(__DIR__."/dump.txt", "triggered@$curDate\n", FILE_APPEND);

  ob_end_flush();
  flush();

  if ( connection_aborted() ) break;

  sleep(1);
}

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Document</title>
</head>
<body>
  <script>
  const evtSource = new EventSource("sse.php", { withCredentials: true } );
  evtSource.onmessage = function(event) {
    console.log(event.data);
  }
  </script>
</body>
</html>

The file_put_contents works as expected, just the javascript does not get the messages from SSE:

The eventstream is sent after 10 seconds at once and not on each event:

Then the request is sent again and the same process starts from the beginning...

You can try the example here: http://sse.baumrock.com/sse.html

I'd be very happy if anybody could help me making this work ?

netcarver · January 23, 2022

You're misquoting me, @bernhard :) Could you try this...

<script>
   var evtSource = new EventSource("sse.php")
   evtSource.addEventListener('ping', function(event) {
       var data = JSON.parse(event.data)
       console.log(data)
   }, false)
</script>

bernhard · January 23, 2022

Thx @netcarver sorry I've quoted @kongondo and linked you in the same sentence ?

http://sse.baumrock.com/sse2.html

--> the event is received every 10s which is wrong as far as I understood how SSE should work. I'd expect a console.log() happen each second and have one open stream as long as I click on "close the connection"

3fingers · January 23, 2022

Hi @bernhard, this is what worked for me:

<?php

namespace ProcessWire;

$wire->addHook('/emit', function ($event) {
    header('Content-Type: text/event-stream');
    header('Cache-Control: no-cache');
    $reference = $event->pages->get('/')->page_selector->title;
    $selector = $event->pages->getRaw("title=$reference, field=id|title|text|image, entities=1");
    $result = json_encode($selector);
    while (true) {
        echo "event: ping" . PHP_EOL;
        echo "data: " . $result . PHP_EOL;
        echo PHP_EOL;
        ob_end_flush();
        flush();
        sleep(1);
    }
});

In the frontend then:

    // Init SSE and listen to php page emitter
    let source = new EventSource("/sse/emit/");
    source.addEventListener("ping", function(event) {
        getData(event)
    });

    // Retrieve ping data, parse it and pass the result to the resolveData function
    function getData(event) {
        return new Promise((resolve, reject) => {
            if (event.data) {
                let result = JSON.parse(event.data);
                resolve(result);
            }
        }).then((result) => {
            resolveData(result)
        });
    }

    // Resolve data and update alpine data
    function resolveData(result) {
        console.log(new Date().getSeconds()); // It logs correctly every 1 sec
        Object.assign(Alpine.store('sse'), {
            fragmentShow: true,
            id: result.id,
            title: result.title,
            text: result.text,
            image: result.image[0].data,
            imageDescription: result.image[0].description
        });
    }

bernhard · January 23, 2022

Hi @3fingers

thx but I think the problem is my server setup and I'm not good at debugging network stuff. See http://sse.baumrock.com/sse3.html where I have this code:

<?php
date_default_timezone_set("America/New_York");
header("Content-Type: text/event-stream");

$i = 0;
while(true) {
  $i++;

  // manual limit of 10 runs for development
  if($i>=10) return;

  echo "data: value of i = $i\n\n";

  while (ob_get_level() > 0) ob_end_flush();
  flush();

  if(connection_aborted()) break;

  sleep(1);
}

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Document</title>
</head>
<body>
  <h1>Open DevTools Console + Network Tab!</h1>
  <script>
  const evtSource = new EventSource("sse.php", { withCredentials: true } );
  evtSource.onmessage = function(event) {
    console.log(event.data);
  }
  </script>
</body>
</html>

It works locally: https://calip.io/HkCWbYAB#ueAgNM5v

But it does not work on my server: https://calip.io/DzhmJQo2#4mfu1aP6

I bet this has something to do with gzip but I can't make it work ? I thought maybe you guys know more about networking and http/tcp stuff and can help me debugging or making it work...

Jan Romero · January 23, 2022

What does ob_get_status(true) output?

bernhard · January 23, 2022

23 minutes ago, Jan Romero said:

What does ob_get_status(true) output?

data: value of i = 1

array(1) {
  [0]=>
  array(7) {
    ["name"]=>
    string(22) "default output handler"
    ["type"]=>
    int(0)
    ["flags"]=>
    int(112)
    ["level"]=>
    int(0)
    ["chunk_size"]=>
    int(4096)
    ["buffer_size"]=>
    int(8192)
    ["buffer_used"]=>
    int(22)
  }
}
data: value of i = 2

array(0) {
}
data: value of i = 3

array(0) {
}
data: value of i = 4

array(0) {
}
data: value of i = 5

array(0) {
}
data: value of i = 6

array(0) {
}
data: value of i = 7

array(0) {
}
data: value of i = 8

array(0) {
}
data: value of i = 9

array(0) {
}

bernhard · January 23, 2022

A user in another forum found a solution!!!

<?php
header("Cache-Control: no-cache");
header("Content-Type: text/event-stream");

$i = 0;
while(true) {
  $i++;

  echo "data: value of i = $i\n\n";
  echo str_pad('',8186)."\n";
  flush();

  if(connection_aborted()) break;
  sleep(1);
}

Reading the manual about flush() finally broucht me to the correct settings for my setup:

<?php
header("Cache-Control: no-cache");
header("Content-Type: text/event-stream");
@apache_setenv('no-gzip', 1);
@ini_set('zlib.output_compression', 0);
@ini_set('implicit_flush', 1);
ob_implicit_flush(1);

$i = 0;
while(true) {
  $i++;
  echo "data: value of i = $i\n\n";
  flush();
  if(connection_aborted()) break;
  sleep(1);
}

I've tried several settings before regarding gzip, but nothing worked. For me the ob_implicit_flush(1) made the difference ?

https://www.php.net/manual/de/function.flush.php

Jan Romero · January 23, 2022

Ah, interesting! Do you still need the explicit flush() at all with implicit_flush on?

bernhard · January 23, 2022

Ok I did some more testing and found out that using str_pad is the most reliable solution. At least in the environments I tested (laragon, DDEV, ubuntu+apache). So I can't say anything about ob_implicit_flush()

bernhard · January 23, 2022

yeeha!! ?

<?php
  /**
   * Center Buttons
   * @return Buttons
   */
  public function getButtonsCenter() {
    $buttons = parent::getButtonsCenter();
    $buttons->add([
      'name' => 'markClaimed',
      'icon' => 'check',
      'tooltip' => 'Mark selected items as claimed',
      'hidden' => '!selected',
      'appendMarkup' => '<div id=markClaimedConfirm hidden>...</div>',
      'callback' => [
        'name' => 'sendIDs',
        'confirmmarkup' => '#markClaimedConfirm',
        'loading' => 'Saving items...',
        'reload' => true,
        'execute' => function($input, self $grid) {
          $ids = $this->wire->sanitizer->intArray($input->ids);
          $note = $this->wire->sanitizer->text($input->note);
          foreach($this->wire->pages->findMany(["id" => $ids]) as $p) {
            $p->setAndSave(Effort::field_claimed, 1);
            $p->setAndSave(Effort::field_claimedwith, $note);
            $grid->sse("Saved page $p");
            sleep(1); // just for the screencast
          }
          $grid->sseDone();
          return ['success' => true];
        },
      ],
    ]);
    return $buttons;
  }

dragan · August 13, 2023

On 1/23/2022 at 5:46 PM, bernhard said:

found out that using str_pad is the most reliable solution.

Same here. I've been tinkering with SSE on and off, and that one-liner made the crucial difference.

teppo · March 8

@bernhard, out of curiosity (and since I'm currently also testing out SSE): did you figure out why exactly this was needed?

Found a Stack Overflow post that pointed to nginx docs, as in that particular case it appeared that proxy buffering was the culprit. It has by default 4K or 8K buffer size, so it would seem to make sense that sending 8K+ spaces before real value solves the issue by effectively overriding the buffer.

What I'm wondering is if that was also your problem, or if there are other potential causes for this as well? Seems kind of hacky having to send (relatively speaking vast amounts of) extra data with each response 😄

bernhard · March 10

Hey @teppo no I didn't dig deeper to be honest 😇 But your buffering note makes sense and when working with SSE on RockCalendar with DDEV I think it worked without any strpad or such!

Sign In

Need help making SSE work

Recommended Posts

bernhard

netcarver

bernhard

3fingers

bernhard

Jan Romero

bernhard

bernhard

Jan Romero

bernhard

bernhard

dragan

teppo

bernhard

Recently Browsing 0 members

Browse

Activity

My Activity Streams

Support

Store

My Details