Operations
Reconnects, heartbeats, proxies and threads: what a long-lived stream needs before it goes to production.
A stream that stops when no one is watching was never a stream. It was a message.
Gorvek of Bonereach
Datastar’s documentation does not describe what its client does when a connection drops, and no SDK describes it either. Everything here was read out of the Datastar 1.0.5 client source, measured against a deployed service or in this repository’s own tests, and each claim says which; the few that are neither say so. The ones marked observed are behaviour of the client, not promises of the protocol. Check them against the version you ship.
Your stream dies when the reader switches tabs
@get is the only action whose connection closes while the page is hidden:
createHttpMethod('get', 'GET', false) // fetch.ts:241, openWhenHidden defaults to false
createHttpMethod('post', 'POST') // every other method defaults to true
A long-lived stream is almost always a @get, so this is almost always your case. The client
aborts the request on visibilitychange and builds a fresh one when the page comes back
(fetch.). Observed.
Turn it off where you open the stream:
div {
dataInit(get("/feed") { openWhenHidden = true })
}
Leaving the default is a defensible choice, since a reader who has tabbed away costs you a connection for nothing. But then every return to the tab is a new request, and your handler has to answer it from the top.
When Datastar reconnects, and when it does not
The default is retry: 'auto', which reconnects only on a transport failure. Not on a 4xx or
5xx (fetch.), and not when a 200 stream ends cleanly (fetch.). Returning 503 from
a handler under load does not buy you a retry; it buys you a dead stream.
| Value | Reconnects on |
|---|---|
auto (default) | network errors only |
error | network errors, and 4xx/5xx |
always | anything that is not 204 or a redirect, including a clean end |
never | nothing |
'never' gives up at the first failure: the catch around the request rejects without a retry
(fetch.), and the action fails with that error. Datastar 1.0.4 read the setting only for
an answer that was not a 200, so there never still reconnected after a dropped connection, as
auto does. If your pages may still load 1.0.4, do not count on never to stop a stream.
Backoff is retryInterval (1000 ms) multiplied by retryScaler (2) up to retryMaxWait
(30 000 ms), for at most retryMaxCount (10) retries over the whole action: a 200 in between
does not start the count again, and the wait goes back to retryInterval only after a stream that
ended. A retry that is waiting when the page is hidden is dropped, and the request is sent anew
when the page shows again. All five are typed:
div {
dataInit(
get("/feed") {
openWhenHidden = true
retry = FetchOptions.Retry.ALWAYS
retryInterval = 2.seconds
retryMaxCount = 20
},
)
}
Resuming: id: and last-event-id
Two SSE fields the SDK specification requires and Datastar’s own reference never mentions.
retry: sets the client’s base retry interval, overriding
retryInterval (fetch.). Observed. It holds for the rest of the action, across
reconnects, until a later frame sets another. The first reconnect waits the full value;
retryMaxWait only caps the backoff steps after it (fetch.). Streamlord omits
the line when it writes as the protocol default of 1000 ms, as the specification says to, so a
stream that has changed the interval goes back to it with 999 or 1001 ms:
patchSignals("phase" to "busy", retry = 5.seconds)
patchSignals("phase" to "idle", retry = 1001.milliseconds)
A retry is a whole number of milliseconds from 1 to Int., since the line carries whole
milliseconds. Under one it would read retry: 0: a reconnect with no wait, whose backoff never
grows, since zero times retryScaler is zero. Above the limit the browser’s timer overflows. An
event with either fails when it is built. A short retry has a cost of its own: a server that
answers 200 and then drops the connection is reconnected to on that schedule retryMaxCount
times, and then the client gives up for good, since the count runs over the whole action and a
200 does not start it again. A stream meant to live for hours needs a retryMaxCount to match.
id:, when you send one, is kept by the client and put on the next attempt as a
last-event-id request header (fetch.). Observed. An event without an id leaves the
header as it was: the client changes it only when an id: line arrives, and drops it only on an
empty one (fetch.). Observed. An event cannot carry an empty id; it is refused when
the event is built. An SseFrame built by hand can, to drop the header on purpose. Streamlord
also keeps an id to printable ASCII with no space at either end, which comes back as it was
written: fetch cannot send a character above U+00FF, Node’s fetch refuses a control
character, Tomcat and Netty answer one with 400, which ends the client’s retries, and Ktor’s CIO
reads the header as UTF-8, so a Latin-1 character arrives as U+FFFD. A space at either end is
stripped on the way. A tab inside an id would come back, and is refused all the same.
So once a stream has sent an id, every retry and every reopen after a hidden tab carries the last
id the client received. The header goes only when the action runs again. A new @get starts
without it, unless its own headers option passes one.
That is what lets the sample below resume across any number of retries, not only the first. The sample gives every event an id. Events without one that follow the last id are sent again, on a retry or a reopen, by a server that starts after the header.
The client takes the id as soon as its line arrives (fetch.), and Streamlord writes id:
before the data: lines. Observed. A connection that drops partway through a frame leaves
the client holding the id of an event it never applied, and a server that starts after it skips
that event. Where losing one matters, keep frames small, or resume from one id earlier and let
the page take a repeat.
Number your events, read the header, start after it: that is the whole of a resumable stream.
get("/feed") {
val resumeAfter = call.request.headers["last-event-id"]?.toLongOrNull() ?: 0L
call.respondDatastar {
for (entry in 1L..20L) {
if (entry <= resumeAfter) continue
patchElements(
"<li>entry $entry</li>",
selector = "#feed",
mode = ElementPatchMode.APPEND,
eventId = entry.toString(),
)
}
}
}
The header name is lower-case on the wire. Ktor and the servlet adapter both look headers up case-insensitively, so either spelling finds it.
Heartbeats
An idle proxy closes a connection that has been silent too long. An SSE comment is the cheapest
thing you can put on the wire: the client ignores it, the proxy sees traffic. Every stream
Streamlord opens has one: a stream that has written nothing for 30 seconds gets a : keep-alive
comment, and any frame starts the wait again, so a busy stream carries none. The heartbeat runs
alongside your handler and stops when it returns.
The first comes sooner, after five seconds of silence from the start. Some servers send no headers until the first byte of the body and give up on a response that has written nothing for a while: Netty does after ten seconds. A stream that writes within those five seconds carries no early comment.
Thirty seconds sits under the idle timeout of most proxies, but a proxy that closes after exactly
thirty silent seconds races it. Set the interval below your proxy’s, once, or turn it off with
null:
install(StreamlordPlugin) { heartbeat = 15.seconds }
On Spring it is a parameter of the bean you already declare,
Streamlord(.
Four things it does not do:
- It does not keep Datastar’s client alive. The client has no idle timeout of its own: its
fetchruns without a timer, and a frame that arrives does not touch its count of retries. The heartbeat is for what sits between you and the browser, and for finding out on the server that the reader has gone: the comment is a write, and a write to a closed connection fails. It takes a few: the first still land in the operating system’s buffer. Over a local socket, the stream ended four to five intervals after the client left on Ktor’s CIO, about two minutes at the default, and two intervals on Netty, about one minute. - It does not run on the request’s thread on WebMVC. The block runs inside
runBlockingon that thread, and the heartbeat onDispatchers., so a JDBC call or aIO Thread.in the block does not hold the comment back. Two things follow. An authorisation’ssleep allows()andonRefused, when the heartbeat asks, run on an IO thread without the request’s thread-locals, Spring Security’s context among them: read what they need before the stream opens. And a comment stuck in a write to a slow reader holds one of the threadsDispatchers.shares with the rest of the application, at least 64 by default. On Ktor, suspend rather than block, as anywhere in a handler.IO - It does not stop a blocking call. When the stream ends early, because the client left,
allows()refused or Spring’s timeout fired, its coroutine is cancelled, and cancellation waits for the next suspension point. A call that blocks insidewithContext(runs to its end first.Dispatchers. IO) runInterruptible(interrupts it instead, which stops aDispatchers. IO) { … } Thread.or a blocking queue at once. Two exceptions: an NIO channel that is interrupted closes, so keep pooled connections out of it; and a read from a plainsleep java., which most JDBC drivers use, ignores the interrupt. Give a query its own timeout (net. Socket Statement., or your pool’s), since nothing else will stop it.setQueryTimeout - It does not reach WebFlux. A
Flowhanded to WebFlux never passes through a Streamlord stream, so a heartbeat on the bean does nothing there. Merge a ticker into the flow afterasServerSentEvents(), each tick aServerSentEvent., or raise the proxy’s idle timeout above your quietest stretch.builder<String>(). comment( "keep-alive"). build()
With a StreamAuthorisation on the stream, the comment
asks the question like any other write once every has passed, so an idle stream is checked on
the heartbeat’s schedule and a refusal ends it there.
The interval is set once per Streamlord, and one stream can take its own: heartbeat on
respondDatastar or datastarStream, or null for none. A report that goes quiet for two
minutes after its first patch, behind a proxy stricter than the rest, can keep its own connection
warm without changing anyone else’s:
get("/report") {
call.respondDatastar(heartbeat = 5.seconds) {
patchElements("""<div id="report">Working</div>""")
delay(120.seconds)
patchElements("""<div id="report">Done</div>""")
}
}
On Spring it is the same parameter:
@GetMapping("/report")
fun report(
request: HttpServletRequest,
response: HttpServletResponse,
): org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody =
response.datastarStream(request = request, heartbeat = 5.seconds) {
patchElements("""<div id="report">Working</div>""")
delay(120.seconds)
patchElements("""<div id="report">Done</div>""")
}
One that is not positive throws where you call it, before the response starts.
Compression
A view that re-renders a whole region sends most of the same markup again on every change. Turn gzip on once:
install(StreamlordPlugin) { compress = true }
On Spring it is the bean, Streamlord(, and
datastarStream needs the request to see what the browser accepts. A stream is gzipped only when
the request’s Accept-Encoding takes it, and then carries Content-Encoding: gzip; with
compress on, every stream carries Vary: Accept-Encoding, on Ktor and on WebMVC when the request
is given and nothing was written to the response before the stream opened. A response something
has already written to through its output stream is streamed without gzip, since those bytes
would sit in front of the gzip member. It works over plain
HTTP too; browsers ask for Brotli only over HTTPS, which is how they behave in general and is
not measured here.
Each event is flushed through the compressor whole, a sync flush per event, and the compressor
keeps its 32 KiB window from one event to the next. Measured in GzipTest, through the Ktor
sink with the JDK’s gzip at its default level, on a table of 200 rows, each
<tr id="row-N"><td class="name">Item N</td><td class="qty">…</td></tr>, re-rendered whole:
| Bytes | |
|---|---|
| One render, as text | 14,733 |
| The first render, gzipped | 1,476 |
| The next render, one row changed | 251 |
| A heartbeat comment | 20 |
The second render is cheap because the first is still in the window. A render larger than 32 KiB
no longer finds the last one there, and costs about what the first did; that follows from the
window’s size and is not measured here. That browsers decode a
gzipped fetch body as it arrives, event by event, is how they behave in general and is not
measured here.
Four things to know:
- Ktor’s own
Compressionplugin stays off a stream gzipped here, because the content already names its encoding; a test installs the plugin to show it. Ktor 3.6.0’s plugin does compress a bare event stream it is pointed at, so leavetext/out of its types. nginx’sevent-stream gzip onis documented to leave an encoded response alone, not measured here. - WebFlux is not covered. A
Flowhanded to WebFlux never passes through a Streamlord stream. Spring Boot’sserver.leavescompression text/out of its default types, by its documentation; whether it flushes per event when you add it is not measured here.event-stream - Mind what shares a compressed stream. Compression leaks how much two pieces of text have in common through the size of the result, and the window carries across events, so a secret in one event and text an attacker chose in another, up to 32 KiB later, compress together. Each event is flushed on its own, so its size shows through TLS one event at a time. An attacker who can trigger the stream, from a cross-site request or by writing what another user’s stream shows, and who can watch the sizes of encrypted traffic, can recover the secret a little at a time. Keep tokens and other secrets out of streams that carry text anyone else can influence, or leave those streams uncompressed.
- A proxy has to pass it through as it comes.
X-Accel-Buffering: no, below, tells nginx. A CDN that decompresses and re-encodes may hold events back, not measured here; the Railway measurement below was taken without gzip.
Through the proxy
Streamlord sets two headers on every stream it opens, which is respondDatastar on Ktor,
datastarStream on WebMVC and asDatastarResponse on WebFlux:
| Header | Why |
|---|---|
Cache-Control: no-cache | required by the SDK specification |
X-Accel-Buffering: no | nginx buffers a response body by default, which holds your events back until the stream ends |
Connection: keep-alive is added only for HTTP/1.1, because the header is forbidden over
HTTP/2 and the specification says as much. Ktor reads the request’s protocol version by itself.
The servlet adapter reads it from the request you hand it, or else from the one Spring MVC is
handling for that response:
@GetMapping("/feed")
fun feed(
request: HttpServletRequest,
response: HttpServletResponse,
): org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody =
response.datastarStream(request = request) {
patchElements("""<div id="feed">Open</div>""")
}
With neither, the header is left out: outside a request Spring MVC is handling, and in a
suspend controller after it has suspended, since Spring’s request holder does not follow it to
the thread it resumes on. Pass the request there. Tomcat drops connection headers from requests
over HTTP/2 but not from responses, and curl and Node refuse a response that carries one.
On Spring the servlet engine has a say too. The adapter sets the header only when the request
asked neither for keep-alive nor to close. Tomcat answers a request that asked for keep-alive
with its own Connection: keep-alive, and one that asked to close with Connection: close. It
also closes a connection on its own, on the 100th request on it by default
(server.); a stream that is that request carries both
Connection: keep-alive and Connection: close, and Tomcat closes the connection after it.
So does an error answer from a stream that failed before its first event, when its status is one
Tomcat closes the connection after: a 400, 413, 500 or 503, among others.
Jetty never sends Connection: keep-alive on a stream, whatever the request asked.
On WebFlux the headers belong to the response and a Flow is only its body. Return the bare flow
from asServerSentEvents and Spring sets the content type and nothing else, so behind nginx the
events are held back until the stream ends. asDatastarResponse returns the same flow with both
headers on it; see Spring WebFlux.
One proxy, measured
Railway does not buffer Server-Sent Events. That is not a guess. The CI run that deploys this
project’s demo holds a stream open against the live service for six seconds and fails the build
unless two counter values arrive inside that window. A proxy that buffered the body would hand
back nothing at all. The step is Check that the deployment answers in
.github/.
That is one proxy. nginx needs X-Accel-Buffering: no, which Streamlord sends, or
proxy_buffering off; in your own configuration. For anything else, that six-second check is the
shape of the test worth running against it.
Threads
Ktor and Spring WebFlux hold no thread per stream. Ktor writes into the response channel
from the handler’s own coroutine, and asServerSentEvents is a Flow mapping that Spring drives
reactively.
Spring WebMVC does hold one. datastarStream returns a StreamingResponseBody and bridges
into the suspending core with runBlocking, inside datastarStream, which occupies the executing thread
for as long as the stream is open. Which thread that is belongs to Spring unless the stream asks
for its own: it is the MVC async task executor, or the executor passed to datastarStream (see
Spring WebMVC). Boot’s default is a pool of eight platform threads with an
unbounded queue behind it, so the ninth stream does not get a thread. It waits in the queue with
its response open and nothing written, for as long as the first eight stay open, and nothing logs
it. Twenty streams opened against a default Boot 4.0 application: eight were running after two and
a half seconds, and twelve were waiting. Measured.
Make the threads virtual instead, and all twenty run:
spring.threads.virtual.enabled=true
runBlocking does not pin a carrier, measured
That only helps if runBlocking releases its carrier thread while the stream waits. If it pinned
one, virtual threads would buy nothing and a thousand streams would still be a thousand carriers.
It does not pin. Two hundred virtual threads, each running runBlocking { delay(,
on eight carriers:
| What each virtual thread did | Wall time for 200 |
|---|---|
Thread. | 517 ms |
runBlocking { delay( | 554 ms |
synchronized( | 12 675 ms |
The third row is the control, and it is the one that proves the measurement can see pinning at all:
synchronized around a blocking call does pin on JDK 21, so 200 tasks over 8 carriers take
200 ÷ 8 × 500 ms. Each task holds a lock nobody else wants, so that number is pinning alone and
not contention. runBlocking lands with plain Thread., which means all 200 ran at once.
Measured on JDK 21.0.2.
What this does not measure is your container’s write. runBlocking releases the carrier while the
stream waits; whether the servlet output stream blocks in a way that pins is Tomcat’s business and
depends on the JDK, since synchronized stopped pinning in JDK 24. The part that belongs to
Streamlord is clear.
What a deploy does to an open stream
A rolling deploy sends SIGTERM and starts a new instance. With a stream open, measured against
this project’s own service on Ktor CIO:
- the process was gone 1.1 seconds after the signal;
- the client saw a severed connection, not a finished stream.
curlended with exit 18,CURLE_PARTIAL_FILE: the body stopped mid-response with no clean end.
Adding a shutdown hook does not fix it, which is the part worth knowing before you write one:
server.addShutdownHook { server.stop(gracePeriodMillis = 3_000, timeoutMillis = 10_000) }
The hook runs. stop() returned after 1026 ms of a three second grace period and the stream
was severed exactly as before. The grace period drains requests that are about to finish; a
response that intends to stay open for an hour is not something it waits for.
So a deploy cuts your streams, and that is the behaviour to design around rather than configure
away. The good news is in the table further up:
a severed connection is a transport failure, and a transport failure is the one case the default
retry: 'auto' does reconnect from. The first time a stream is cut, its reader comes back on
the new instance after the one second base interval, and so do all the others, together. If that
herd is a problem, raise retryInterval where you open the stream. Each later cut of the same
stream waits twice as long as the one before, up to thirty seconds, and the tenth is the last,
since the count runs over the stream’s whole life: raise retryMaxCount for a stream that is to
outlive more deploys than that.
To end streams cleanly instead, the handler has to know the server is going down. Watch Ktor’s
ApplicationStopPreparing event and finish your flows from there; nothing in Streamlord can do it
for you, because only your handler knows what a half-finished stream owes its reader.
Six streams, and the page stalls
Browsers cap connections per origin on HTTP/1.1, and an open stream holds one for its whole life.
The number is exactly six in Chrome, and the cost of exceeding it is not subtle. Twelve
EventSources against one origin, then an ordinary fetch of a trivial endpoint:
| HTTP/1.1 | HTTP/2 | |
|---|---|---|
| Streams open, of 12 | 6 | 12 |
| Still connecting | 6 | 0 |
fetch( while they run | 21 350 ms | 34 ms |
| The same fetch with nothing open | 3 to 7 ms |
Measured, in Chrome, against this project’s service: the HTTP/1.1 column locally, the HTTP/2 column against the deployed instance. The twenty-one seconds is not the server being slow. It is the request sitting in a queue behind six streams that have no intention of ending, on a server that answers in three milliseconds when asked.
So over HTTP/1.1 a page can starve itself with its own streams, and nothing warns it. Two or three concurrent streams is the honest ceiling there. Over HTTP/2 and HTTP/3 they multiplex onto one connection and the limit stops mattering, which is the strongest argument on this page for terminating HTTP/2 in front of your service.
Asking again while the stream runs
A request is authorised once and is over in milliseconds. A stream is authorised once and then lives for minutes or hours, patching the whole time, while the reader logs out, an administrator revokes access, a subscription lapses or a role changes. Nobody asks again, so the stream carries on. How much that matters depends on what you are pushing: public dashboard numbers, little; another team’s documents after someone left that team, rather more.
Pass a StreamAuthorisation where you open the stream:
get("/feed") {
val session = call.request.headers["Authorization"]
call.respondDatastar(
authorisation = StreamAuthorisation(every = 10.seconds) { session != null },
) {
ticks.collect { tick -> patchSignals("tick" to tick) }
}
}
The check is your own Kotlin, compiled with the rest of your application. It never reaches the browser and is never named in markup. Look in a database, read a cached token, ask your session store; Streamlord fixes the contract and not the logic.
Four things it does, each for a reason:
- It asks at the open, before your handler runs. The cheapest place to refuse a connection is
before it starts, and a refusal there writes nothing of your handler’s; only what
onRefusedwrites goes out. On Ktor the200and its headers are already on their way. - It asks again on an interval, not before every patch. A chatty stream would otherwise cost
one database round trip per patch. The verdict stands for
every, five seconds by default. The write that finds the verdict stale waits for the new one; other writes go out on the old one until it arrives. - A refusal ends the stream rather than letting it fall silent, so the client sees a finished response instead of an idle server. Your handler stops where it stood; nothing after the refused write runs. A handler that catches the exception and carries on gains nothing by it: every later write is refused again, so nothing more reaches the reader.
- You get the last word.
onRefusedruns on the still-open stream, so the reader can be told rather than dropped:
get("/feed") {
call.respondDatastar(
authorisation =
StreamAuthorisation(
onRefused = {
patchElements("""<p id="notice">Your session expired</p>""")
},
) { false },
) {
ticks.collect { tick -> patchSignals("tick" to tick) }
}
}
redirect( works there too, and so does anything else the stream can send. It has
StreamAuthorisation., five seconds, to say it.
What the client does next is the rule from further up this page:
a 200 stream that ends cleanly does not reconnect under the default retry: 'auto', which is what
you want. A reader who should be let back in reloads or follows your redirect; one who should not
is not hammering your endpoint in the meantime.
On Spring the parameter is the same, on datastarStream(.
On WebMVC the heartbeat asks from a thread of Dispatchers., without the request’s
thread-locals, Spring Security’s context among them, while the first answer and the block have
them. Read what allows needs before the stream opens; see Heartbeats.
Not covered here yet
One thing is still unmeasured, so it is not written down as advice: Micrometer metrics and tracing across a long-lived stream. The open question is narrow and worth naming. A request timer records the whole call, and a stream that is open for an hour is one call, so a handful of streams can move a latency percentile more than every other endpoint combined. Whether the Ktor and Spring Micrometer integrations let you exclude a streaming route from that histogram, and what a tracing span across an hour does to a backend that expects milliseconds, is what someone has to sit down and measure.