Skip to content
Streamlordv0.23.0

Operations

Reconnects, heartbeats, proxies and threads: what a long-lived stream needs before it goes to production.

A stream that stops when no one is watching was never a stream. It was a message.

Gorvek of Bonereach

Datastar’s documentation does not describe what its client does when a connection drops, and no SDK describes it either. Everything here was read out of the Datastar 1.0.5 client source, measured against a deployed service or in this repository’s own tests, and each claim says which; the few that are neither say so. The ones marked observed are behaviour of the client, not promises of the protocol. Check them against the version you ship.

Your stream dies when the reader switches tabs

@get is the only action whose connection closes while the page is hidden:

createHttpMethod('get', 'GET', false)   // fetch.ts:241, openWhenHidden defaults to false
createHttpMethod('post', 'POST')        // every other method defaults to true

A long-lived stream is almost always a @get, so this is almost always your case. The client aborts the request on visibilitychange and builds a fresh one when the page comes back (fetch.ts:507-513). Observed.

Turn it off where you open the stream:

div {
    dataInit(get("/feed") { openWhenHidden = true })
}

Leaving the default is a defensible choice, since a reader who has tabbed away costs you a connection for nothing. But then every return to the tab is a new request, and your handler has to answer it from the top.

When Datastar reconnects, and when it does not

The default is retry: 'auto', which reconnects only on a transport failure. Not on a 4xx or 5xx (fetch.ts:594), and not when a 200 stream ends cleanly (fetch.ts:680). Returning 503 from a handler under load does not buy you a retry; it buys you a dead stream.

ValueReconnects on
auto (default)network errors only
errornetwork errors, and 4xx/5xx
alwaysanything that is not 204 or a redirect, including a clean end
nevernothing

'never' gives up at the first failure: the catch around the request rejects without a retry (fetch.ts:689), and the action fails with that error. Datastar 1.0.4 read the setting only for an answer that was not a 200, so there never still reconnected after a dropped connection, as auto does. If your pages may still load 1.0.4, do not count on never to stop a stream.

Backoff is retryInterval (1000 ms) multiplied by retryScaler (2) up to retryMaxWait (30 000 ms), for at most retryMaxCount (10) retries over the whole action: a 200 in between does not start the count again, and the wait goes back to retryInterval only after a stream that ended. A retry that is waiting when the page is hidden is dropped, and the request is sent anew when the page shows again. All five are typed:

div {
    dataInit(
        get("/feed") {
            openWhenHidden = true
            retry = FetchOptions.Retry.ALWAYS
            retryInterval = 2.seconds
            retryMaxCount = 20
        },
    )
}

Resuming: id: and last-event-id

Two SSE fields the SDK specification requires and Datastar’s own reference never mentions.

retry: sets the client’s base retry interval, overriding retryInterval (fetch.ts:668). Observed. It holds for the rest of the action, across reconnects, until a later frame sets another. The first reconnect waits the full value; retryMaxWait only caps the backoff steps after it (fetch.ts:541-544). Streamlord omits the line when it writes as the protocol default of 1000 ms, as the specification says to, so a stream that has changed the interval goes back to it with 999 or 1001 ms:

patchSignals("phase" to "busy", retry = 5.seconds)
patchSignals("phase" to "idle", retry = 1001.milliseconds)

A retry is a whole number of milliseconds from 1 to Int.MAX_VALUE, since the line carries whole milliseconds. Under one it would read retry: 0: a reconnect with no wait, whose backoff never grows, since zero times retryScaler is zero. Above the limit the browser’s timer overflows. An event with either fails when it is built. A short retry has a cost of its own: a server that answers 200 and then drops the connection is reconnected to on that schedule retryMaxCount times, and then the client gives up for good, since the count runs over the whole action and a 200 does not start it again. A stream meant to live for hours needs a retryMaxCount to match.

id:, when you send one, is kept by the client and put on the next attempt as a last-event-id request header (fetch.ts:661). Observed. An event without an id leaves the header as it was: the client changes it only when an id: line arrives, and drops it only on an empty one (fetch.ts:664). Observed. An event cannot carry an empty id; it is refused when the event is built. An SseFrame built by hand can, to drop the header on purpose. Streamlord also keeps an id to printable ASCII with no space at either end, which comes back as it was written: fetch cannot send a character above U+00FF, Node’s fetch refuses a control character, Tomcat and Netty answer one with 400, which ends the client’s retries, and Ktor’s CIO reads the header as UTF-8, so a Latin-1 character arrives as U+FFFD. A space at either end is stripped on the way. A tab inside an id would come back, and is refused all the same.

So once a stream has sent an id, every retry and every reopen after a hidden tab carries the last id the client received. The header goes only when the action runs again. A new @get starts without it, unless its own headers option passes one.

That is what lets the sample below resume across any number of retries, not only the first. The sample gives every event an id. Events without one that follow the last id are sent again, on a retry or a reopen, by a server that starts after the header.

The client takes the id as soon as its line arrives (fetch.ts:421), and Streamlord writes id: before the data: lines. Observed. A connection that drops partway through a frame leaves the client holding the id of an event it never applied, and a server that starts after it skips that event. Where losing one matters, keep frames small, or resume from one id earlier and let the page take a repeat.

Number your events, read the header, start after it: that is the whole of a resumable stream.

get("/feed") {
    val resumeAfter = call.request.headers["last-event-id"]?.toLongOrNull() ?: 0L
    call.respondDatastar {
        for (entry in 1L..20L) {
            if (entry <= resumeAfter) continue
            patchElements(
                "<li>entry $entry</li>",
                selector = "#feed",
                mode = ElementPatchMode.APPEND,
                eventId = entry.toString(),
            )
        }
    }
}

The header name is lower-case on the wire. Ktor and the servlet adapter both look headers up case-insensitively, so either spelling finds it.

Heartbeats

An idle proxy closes a connection that has been silent too long. An SSE comment is the cheapest thing you can put on the wire: the client ignores it, the proxy sees traffic. Every stream Streamlord opens has one: a stream that has written nothing for 30 seconds gets a : keep-alive comment, and any frame starts the wait again, so a busy stream carries none. The heartbeat runs alongside your handler and stops when it returns.

The first comes sooner, after five seconds of silence from the start. Some servers send no headers until the first byte of the body and give up on a response that has written nothing for a while: Netty does after ten seconds. A stream that writes within those five seconds carries no early comment.

Thirty seconds sits under the idle timeout of most proxies, but a proxy that closes after exactly thirty silent seconds races it. Set the interval below your proxy’s, once, or turn it off with null:

install(StreamlordPlugin) { heartbeat = 15.seconds }

On Spring it is a parameter of the bean you already declare, Streamlord(codec = JacksonSignalsCodec(mapper), heartbeat = 15.seconds).

Four things it does not do:

  • It does not keep Datastar’s client alive. The client has no idle timeout of its own: its fetch runs without a timer, and a frame that arrives does not touch its count of retries. The heartbeat is for what sits between you and the browser, and for finding out on the server that the reader has gone: the comment is a write, and a write to a closed connection fails. It takes a few: the first still land in the operating system’s buffer. Over a local socket, the stream ended four to five intervals after the client left on Ktor’s CIO, about two minutes at the default, and two intervals on Netty, about one minute.
  • It does not run on the request’s thread on WebMVC. The block runs inside runBlocking on that thread, and the heartbeat on Dispatchers.IO, so a JDBC call or a Thread.sleep in the block does not hold the comment back. Two things follow. An authorisation’s allows() and onRefused, when the heartbeat asks, run on an IO thread without the request’s thread-locals, Spring Security’s context among them: read what they need before the stream opens. And a comment stuck in a write to a slow reader holds one of the threads Dispatchers.IO shares with the rest of the application, at least 64 by default. On Ktor, suspend rather than block, as anywhere in a handler.
  • It does not stop a blocking call. When the stream ends early, because the client left, allows() refused or Spring’s timeout fired, its coroutine is cancelled, and cancellation waits for the next suspension point. A call that blocks inside withContext(Dispatchers.IO) runs to its end first. runInterruptible(Dispatchers.IO) { … } interrupts it instead, which stops a Thread.sleep or a blocking queue at once. Two exceptions: an NIO channel that is interrupted closes, so keep pooled connections out of it; and a read from a plain java.net.Socket, which most JDBC drivers use, ignores the interrupt. Give a query its own timeout (Statement.setQueryTimeout, or your pool’s), since nothing else will stop it.
  • It does not reach WebFlux. A Flow handed to WebFlux never passes through a Streamlord stream, so a heartbeat on the bean does nothing there. Merge a ticker into the flow after asServerSentEvents(), each tick a ServerSentEvent.builder<String>().comment("keep-alive").build(), or raise the proxy’s idle timeout above your quietest stretch.

With a StreamAuthorisation on the stream, the comment asks the question like any other write once every has passed, so an idle stream is checked on the heartbeat’s schedule and a refusal ends it there.

The interval is set once per Streamlord, and one stream can take its own: heartbeat on respondDatastar or datastarStream, or null for none. A report that goes quiet for two minutes after its first patch, behind a proxy stricter than the rest, can keep its own connection warm without changing anyone else’s:

get("/report") {
    call.respondDatastar(heartbeat = 5.seconds) {
        patchElements("""<div id="report">Working</div>""")
        delay(120.seconds)
        patchElements("""<div id="report">Done</div>""")
    }
}

On Spring it is the same parameter:

@GetMapping("/report")
fun report(
    request: HttpServletRequest,
    response: HttpServletResponse,
): org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody =
    response.datastarStream(request = request, heartbeat = 5.seconds) {
        patchElements("""<div id="report">Working</div>""")
        delay(120.seconds)
        patchElements("""<div id="report">Done</div>""")
    }

One that is not positive throws where you call it, before the response starts.

Compression

A view that re-renders a whole region sends most of the same markup again on every change. Turn gzip on once:

install(StreamlordPlugin) { compress = true }

On Spring it is the bean, Streamlord(codec = JacksonSignalsCodec(mapper), compress = true), and datastarStream needs the request to see what the browser accepts. A stream is gzipped only when the request’s Accept-Encoding takes it, and then carries Content-Encoding: gzip; with compress on, every stream carries Vary: Accept-Encoding, on Ktor and on WebMVC when the request is given and nothing was written to the response before the stream opened. A response something has already written to through its output stream is streamed without gzip, since those bytes would sit in front of the gzip member. It works over plain HTTP too; browsers ask for Brotli only over HTTPS, which is how they behave in general and is not measured here.

Each event is flushed through the compressor whole, a sync flush per event, and the compressor keeps its 32 KiB window from one event to the next. Measured in GzipTest, through the Ktor sink with the JDK’s gzip at its default level, on a table of 200 rows, each <tr id="row-N"><td class="name">Item N</td><td class="qty">…</td></tr>, re-rendered whole:

Bytes
One render, as text14,733
The first render, gzipped1,476
The next render, one row changed251
A heartbeat comment20

The second render is cheap because the first is still in the window. A render larger than 32 KiB no longer finds the last one there, and costs about what the first did; that follows from the window’s size and is not measured here. That browsers decode a gzipped fetch body as it arrives, event by event, is how they behave in general and is not measured here.

Four things to know:

  • Ktor’s own Compression plugin stays off a stream gzipped here, because the content already names its encoding; a test installs the plugin to show it. Ktor 3.6.0’s plugin does compress a bare event stream it is pointed at, so leave text/event-stream out of its types. nginx’s gzip on is documented to leave an encoded response alone, not measured here.
  • WebFlux is not covered. A Flow handed to WebFlux never passes through a Streamlord stream. Spring Boot’s server.compression leaves text/event-stream out of its default types, by its documentation; whether it flushes per event when you add it is not measured here.
  • Mind what shares a compressed stream. Compression leaks how much two pieces of text have in common through the size of the result, and the window carries across events, so a secret in one event and text an attacker chose in another, up to 32 KiB later, compress together. Each event is flushed on its own, so its size shows through TLS one event at a time. An attacker who can trigger the stream, from a cross-site request or by writing what another user’s stream shows, and who can watch the sizes of encrypted traffic, can recover the secret a little at a time. Keep tokens and other secrets out of streams that carry text anyone else can influence, or leave those streams uncompressed.
  • A proxy has to pass it through as it comes. X-Accel-Buffering: no, below, tells nginx. A CDN that decompresses and re-encodes may hold events back, not measured here; the Railway measurement below was taken without gzip.

Through the proxy

Streamlord sets two headers on every stream it opens, which is respondDatastar on Ktor, datastarStream on WebMVC and asDatastarResponse on WebFlux:

HeaderWhy
Cache-Control: no-cacherequired by the SDK specification
X-Accel-Buffering: nonginx buffers a response body by default, which holds your events back until the stream ends

Connection: keep-alive is added only for HTTP/1.1, because the header is forbidden over HTTP/2 and the specification says as much. Ktor reads the request’s protocol version by itself. The servlet adapter reads it from the request you hand it, or else from the one Spring MVC is handling for that response:

@GetMapping("/feed")
fun feed(
    request: HttpServletRequest,
    response: HttpServletResponse,
): org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody =
    response.datastarStream(request = request) {
        patchElements("""<div id="feed">Open</div>""")
    }

With neither, the header is left out: outside a request Spring MVC is handling, and in a suspend controller after it has suspended, since Spring’s request holder does not follow it to the thread it resumes on. Pass the request there. Tomcat drops connection headers from requests over HTTP/2 but not from responses, and curl and Node refuse a response that carries one.

On Spring the servlet engine has a say too. The adapter sets the header only when the request asked neither for keep-alive nor to close. Tomcat answers a request that asked for keep-alive with its own Connection: keep-alive, and one that asked to close with Connection: close. It also closes a connection on its own, on the 100th request on it by default (server.tomcat.max-keep-alive-requests); a stream that is that request carries both Connection: keep-alive and Connection: close, and Tomcat closes the connection after it. So does an error answer from a stream that failed before its first event, when its status is one Tomcat closes the connection after: a 400, 413, 500 or 503, among others. Jetty never sends Connection: keep-alive on a stream, whatever the request asked.

On WebFlux the headers belong to the response and a Flow is only its body. Return the bare flow from asServerSentEvents and Spring sets the content type and nothing else, so behind nginx the events are held back until the stream ends. asDatastarResponse returns the same flow with both headers on it; see Spring WebFlux.

One proxy, measured

Railway does not buffer Server-Sent Events. That is not a guess. The CI run that deploys this project’s demo holds a stream open against the live service for six seconds and fails the build unless two counter values arrive inside that window. A proxy that buffered the body would hand back nothing at all. The step is Check that the deployment answers in .github/workflows/ci.yml.

That is one proxy. nginx needs X-Accel-Buffering: no, which Streamlord sends, or proxy_buffering off; in your own configuration. For anything else, that six-second check is the shape of the test worth running against it.

Threads

Ktor and Spring WebFlux hold no thread per stream. Ktor writes into the response channel from the handler’s own coroutine, and asServerSentEvents is a Flow mapping that Spring drives reactively.

Spring WebMVC does hold one. datastarStream returns a StreamingResponseBody and bridges into the suspending core with runBlocking, inside datastarStream, which occupies the executing thread for as long as the stream is open. Which thread that is belongs to Spring unless the stream asks for its own: it is the MVC async task executor, or the executor passed to datastarStream (see Spring WebMVC). Boot’s default is a pool of eight platform threads with an unbounded queue behind it, so the ninth stream does not get a thread. It waits in the queue with its response open and nothing written, for as long as the first eight stay open, and nothing logs it. Twenty streams opened against a default Boot 4.0 application: eight were running after two and a half seconds, and twelve were waiting. Measured.

Make the threads virtual instead, and all twenty run:

spring.threads.virtual.enabled=true

runBlocking does not pin a carrier, measured

That only helps if runBlocking releases its carrier thread while the stream waits. If it pinned one, virtual threads would buy nothing and a thousand streams would still be a thousand carriers.

It does not pin. Two hundred virtual threads, each running runBlocking { delay(500.milliseconds) }, on eight carriers:

What each virtual thread didWall time for 200
Thread.sleep(500)517 ms
runBlocking { delay(500) } 554 ms
synchronized(ownLock) { Thread.sleep(500) }12 675 ms

The third row is the control, and it is the one that proves the measurement can see pinning at all: synchronized around a blocking call does pin on JDK 21, so 200 tasks over 8 carriers take 200 ÷ 8 × 500 ms. Each task holds a lock nobody else wants, so that number is pinning alone and not contention. runBlocking lands with plain Thread.sleep, which means all 200 ran at once. Measured on JDK 21.0.2.

What this does not measure is your container’s write. runBlocking releases the carrier while the stream waits; whether the servlet output stream blocks in a way that pins is Tomcat’s business and depends on the JDK, since synchronized stopped pinning in JDK 24. The part that belongs to Streamlord is clear.

What a deploy does to an open stream

A rolling deploy sends SIGTERM and starts a new instance. With a stream open, measured against this project’s own service on Ktor CIO:

  • the process was gone 1.1 seconds after the signal;
  • the client saw a severed connection, not a finished stream. curl ended with exit 18, CURLE_PARTIAL_FILE: the body stopped mid-response with no clean end.

Adding a shutdown hook does not fix it, which is the part worth knowing before you write one:

server.addShutdownHook { server.stop(gracePeriodMillis = 3_000, timeoutMillis = 10_000) }

The hook runs. stop() returned after 1026 ms of a three second grace period and the stream was severed exactly as before. The grace period drains requests that are about to finish; a response that intends to stay open for an hour is not something it waits for.

So a deploy cuts your streams, and that is the behaviour to design around rather than configure away. The good news is in the table further up: a severed connection is a transport failure, and a transport failure is the one case the default retry: 'auto' does reconnect from. The first time a stream is cut, its reader comes back on the new instance after the one second base interval, and so do all the others, together. If that herd is a problem, raise retryInterval where you open the stream. Each later cut of the same stream waits twice as long as the one before, up to thirty seconds, and the tenth is the last, since the count runs over the stream’s whole life: raise retryMaxCount for a stream that is to outlive more deploys than that.

To end streams cleanly instead, the handler has to know the server is going down. Watch Ktor’s ApplicationStopPreparing event and finish your flows from there; nothing in Streamlord can do it for you, because only your handler knows what a half-finished stream owes its reader.

Six streams, and the page stalls

Browsers cap connections per origin on HTTP/1.1, and an open stream holds one for its whole life. The number is exactly six in Chrome, and the cost of exceeding it is not subtle. Twelve EventSources against one origin, then an ordinary fetch of a trivial endpoint:

HTTP/1.1HTTP/2
Streams open, of 12612
Still connecting60
fetch("/health") while they run21 350 ms34 ms
The same fetch with nothing open3 to 7 ms

Measured, in Chrome, against this project’s service: the HTTP/1.1 column locally, the HTTP/2 column against the deployed instance. The twenty-one seconds is not the server being slow. It is the request sitting in a queue behind six streams that have no intention of ending, on a server that answers in three milliseconds when asked.

So over HTTP/1.1 a page can starve itself with its own streams, and nothing warns it. Two or three concurrent streams is the honest ceiling there. Over HTTP/2 and HTTP/3 they multiplex onto one connection and the limit stops mattering, which is the strongest argument on this page for terminating HTTP/2 in front of your service.

Asking again while the stream runs

A request is authorised once and is over in milliseconds. A stream is authorised once and then lives for minutes or hours, patching the whole time, while the reader logs out, an administrator revokes access, a subscription lapses or a role changes. Nobody asks again, so the stream carries on. How much that matters depends on what you are pushing: public dashboard numbers, little; another team’s documents after someone left that team, rather more.

Pass a StreamAuthorisation where you open the stream:

get("/feed") {
    val session = call.request.headers["Authorization"]

    call.respondDatastar(
        authorisation = StreamAuthorisation(every = 10.seconds) { session != null },
    ) {
        ticks.collect { tick -> patchSignals("tick" to tick) }
    }
}

The check is your own Kotlin, compiled with the rest of your application. It never reaches the browser and is never named in markup. Look in a database, read a cached token, ask your session store; Streamlord fixes the contract and not the logic.

Four things it does, each for a reason:

  • It asks at the open, before your handler runs. The cheapest place to refuse a connection is before it starts, and a refusal there writes nothing of your handler’s; only what onRefused writes goes out. On Ktor the 200 and its headers are already on their way.
  • It asks again on an interval, not before every patch. A chatty stream would otherwise cost one database round trip per patch. The verdict stands for every, five seconds by default. The write that finds the verdict stale waits for the new one; other writes go out on the old one until it arrives.
  • A refusal ends the stream rather than letting it fall silent, so the client sees a finished response instead of an idle server. Your handler stops where it stood; nothing after the refused write runs. A handler that catches the exception and carries on gains nothing by it: every later write is refused again, so nothing more reaches the reader.
  • You get the last word. onRefused runs on the still-open stream, so the reader can be told rather than dropped:
get("/feed") {
    call.respondDatastar(
        authorisation =
            StreamAuthorisation(
                onRefused = {
                    patchElements("""<p id="notice">Your session expired</p>""")
                },
            ) { false },
    ) {
        ticks.collect { tick -> patchSignals("tick" to tick) }
    }
}

redirect("/login") works there too, and so does anything else the stream can send. It has StreamAuthorisation.LAST_WORDS_LIMIT, five seconds, to say it.

What the client does next is the rule from further up this page: a 200 stream that ends cleanly does not reconnect under the default retry: 'auto', which is what you want. A reader who should be let back in reloads or follows your redirect; one who should not is not hammering your endpoint in the meantime.

On Spring the parameter is the same, on datastarStream(streamlord, request, authorisation) { }.

On WebMVC the heartbeat asks from a thread of Dispatchers.IO, without the request’s thread-locals, Spring Security’s context among them, while the first answer and the block have them. Read what allows needs before the stream opens; see Heartbeats.

Not covered here yet

One thing is still unmeasured, so it is not written down as advice: Micrometer metrics and tracing across a long-lived stream. The open question is narrow and worth naming. A request timer records the whole call, and a stream that is open for an hour is one call, so a handful of streams can move a latency percentile more than every other endpoint combined. Whether the Ktor and Spring Micrometer integrations let you exclude a streaming route from that histogram, and what a tracing span across an hour does to a backend that expects milliseconds, is what someone has to sit down and measure.

What went over the wire

The frames your last search produced, encoded by the same SseEncoder the golden-file tests check. Not a description of them. The frames.

Nothing yet. Search from the top of the page, and what the server sends will appear here.