Caching and CDN for OTT delivery¶
An appendix for operating large installations: how caching of Perfect Streamer OTT responses works and how to place a caching reverse proxy or a CDN in front of the node. Delivery modes, URL formats and authorization are described in the main section OTT and DVR.
The server produces responses of three categories that differ in content lifetime and in their suitability for caching by intermediate nodes (reverse proxy, CDN, client cache).
1. Caching model¶
1.1. Resources and HTTP headers¶
Resource |
URL |
Content-Type |
Cache-Control |
|---|---|---|---|
TS segment (HLS) |
|
|
|
VTT segment (subtitles) |
|
|
|
fMP4 segment (DASH / LL-HLS) |
|
|
|
DASH MPD |
|
|
|
HLS master |
|
|
|
HLS media |
|
|
|
302 Redirect |
|
— |
|
Raw TS |
|
|
not set; not cached |
1.2. Segment properties¶
The hexadecimal segment identifier in the URL (<keyHex> in paths /h<sess>/<keyHex>.ts) is computed as a CRC64 of the segment start time and the stream ID, and is globally unique. A segment URL addresses immutable content — repeated requests for the same URL return an identical byte stream (as long as the segment stays within the sliding window).
The immutable directive suppresses conditional revalidation by the client (If-None-Match, If-Modified-Since). The value max-age=60 provides compatibility with a typical timeShiftBufferDepth=40s.
CMAF fMP4 segments (.m4s) and the shared init.mp4 for DASH / Low-Latency HLS are addressed in the same way and cached under the same model (immutable, max-age=60). LL-HLS partial segments (parts) are byte-range requests into the same .m4s, so they do not create a separate cache entry.
1.3. Manifest properties¶
max-age=1 bounds the staleness of cached content at one second. Combined with proxy_cache_lock on (nginx), bursts of manifest requests are coalesced into one request to the origin per second.
1.4. Content variability¶
With absPath=0 (the default value, no a URL parameter) the HLS media manifests and the DASH MPD contain no session identifier in the body. The manifest content is identical across sessions belonging to the same (stream, param) combination. This allows a reverse-proxy cache to reuse an entry across sessions when the cache key is normalized.
With absPath=1 (URL parameter a=1) the manifest body contains absolute URLs that include the scheme, the host and the session identifier. The content becomes session-specific, and cross-session cache reuse is not possible.
2. Client behaviour¶
Client |
Manifest refresh URL |
Effect on the number of sessions |
|---|---|---|
HLS players using the media playlist |
|
One session per playback session |
DASH players using the root URL |
|
Handled by session reuse (see 3.2) |
Browser players (MSE) |
|
One session per playback session |
3. Special mechanisms¶
3.1. HTTP 302 Redirect for DASH¶
A request of the form /dash/<stream>/<login>/<pass>/index.mpd returns a 302 Found response with the header Location: /h<sess>/index.mpd. The response body is empty. Authorization and session allocation are performed while the redirect is processed.
Clients that support redirect caching address the session URL directly in subsequent requests. Clients that do not support it repeat the redirect request. The cost of reprocessing the redirect is limited to the authentication check and the session reuse operations.
3.2. Session reuse for DASH¶
When processing a /dash/.../index.mpd request from the same login to the same stream (with the same adaptive flag), the server finds an already existing DASH session and returns its identifier again. No new session is created and no slot of the concurrent-connection limit is consumed. Only sessions in the same playback mode are reused: a live session and an archive VOD session (parameters t/d/epg) are not merged.
Applies to DASH only. HLS requires no separate reuse mechanism: HLS clients refresh the media playlist via the session URL and do not create a new session on every refresh.
3.3. Segment reuse across sessions¶
The path /h<sess>/<keyHex>.ts does not depend on <sess> when <keyHex> is resolved to content: <keyHex> globally and unambiguously identifies a TS segment within the stream. Nginx with a normalized cache key (one that strips the /h<sess>/ prefix) serves all requests for the same <keyHex> from a single cache entry, regardless of which clients issued them.
Deduplication applies only to content-addressed names — segments with a 16-character hexadecimal identifier (.ts, .m4s with <keyHex>, .vtt). Numbered live-DASH segments (<number>.m4s), init.mp4 and manifests are unique only within their own session; their cache key must keep the /h<sess>/ prefix, otherwise the cache will serve the content of one stream to the viewers of another. For DASH, manifest deduplication between clients of the same login is provided by session reuse (see 3.2).
4. Request parameters¶
Parameter |
Default value |
Effect |
|---|---|---|
|
|
|
|
|
|
|
|
Minimum window length required to serve a manifest |
|
|
|
|
absent (off) |
opt-in for HTTP/3 (QUIC): makes the server emit |
Changing a parameter through the query string updates the values stored in the session the next time it is reopened. The archive playback parameters t, d and epg are described in Archive playback (VOD).
5. Load characteristics¶
The load on the origin scales with the number of distinct streams being watched concurrently. Increasing the number of clients watching the same stream does not increase the number of requests to the origin when a reverse-proxy cache and a normalized cache key are in place.
Scenario |
Origin request rate (ref.) |
|---|---|
1 client on stream X |
MPD: 0.4 req/s, segment: 0.2 req/s |
N clients on a single stream X (cache enabled) |
MPD: 1 req/s, segment: 0.2 req/s |
N DASH clients in refresh loop on a single stream |
MPD: 1 req/s (with |
N clients on N distinct streams |
MPD: 0.4·N req/s, segment: 0.2·N req/s |
Segments are deduplicated globally by their content-addressed names; manifest deduplication between clients is provided by session reuse (DASH) and by caching within the session.
6. Nginx as a caching reverse proxy¶
6.1. Basic configuration¶
proxy_cache_path /var/cache/nginx/pss_segments
levels=1:2 keys_zone=pss_segments:100m
max_size=20g inactive=30m use_temp_path=off;
proxy_cache_path /var/cache/nginx/pss_manifests
levels=1:2 keys_zone=pss_manifests:10m
max_size=256m inactive=5m use_temp_path=off;
upstream pss_backend {
server 127.0.0.1:43972;
keepalive 64;
}
map $uri $pss_cache_key {
"~^/h[0-9a-f]{16}(?<tail>(/[0-9]+)?/[0-9a-f]{16}\.(ts|m4s)|/sub/[0-9]+/[0-9a-f]{16}\.vtt)$" "stream:$tail";
default $uri;
}
server {
listen 80;
server_name stream.example.com;
location ~* "^/h[0-9a-f]{16}(/[0-9]+|/sub/[0-9]+)?/([0-9a-f]+\.(ts|m4s|vtt)|init\.mp4)$" {
proxy_cache pss_segments;
proxy_cache_key $pss_cache_key;
proxy_cache_valid 200 60s;
proxy_cache_valid 404 403 0s;
proxy_cache_lock on;
proxy_cache_use_stale updating error timeout;
proxy_cache_revalidate on;
proxy_ignore_headers Vary;
add_header X-Cache-Status $upstream_cache_status;
proxy_pass http://pss_backend;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_buffering on;
}
location ~* "(^/h[0-9a-f]{16}(/[0-9]+|/sub/[0-9]+)?/index\.(m3u8|mpd)$|^/(hls|dash|llhls)/.*\.(m3u8|mpd)$)" {
proxy_cache pss_manifests;
proxy_cache_key $pss_cache_key;
proxy_cache_valid 200 1s;
proxy_cache_valid 404 403 0s;
proxy_cache_lock on;
proxy_cache_lock_timeout 2s;
proxy_cache_use_stale updating;
add_header X-Cache-Status $upstream_cache_status;
proxy_pass http://pss_backend;
proxy_http_version 1.1;
proxy_set_header Connection "";
}
location / {
proxy_pass http://pss_backend;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_buffering off;
proxy_read_timeout 3600s;
}
}
The origin port in the example is the default streaming HTTP server port (43972; HTTPS — 43982). Only content-addressed segment names are normalized (see 3.3); manifests, init.mp4 and numbered segments are cached under the full URI of their own session. The proxy_ignore_headers Vary directive in the segment location is justified: the server accompanies OTT responses with the header Vary: Origin, and without it the cache would be split into variants by the client Origin value (see 7).
6.2. Purpose of the directives¶
Directive |
Purpose |
|---|---|
|
Serializes upstream requests on concurrent cache misses for the same key |
|
Returns the stale copy to concurrent requests while the cache is being updated |
|
Uses |
|
Disables caching of authorization errors and 404 |
|
Maintains a pool of persistent connections to the origin |
|
For segments; enables response buffering in nginx |
|
For the |
6.3. Calculating max_size of the segment cache¶
Approximate value: bitrate × timeShiftBufferDepth × distinct_streams × 2
Example: 10 streams × 8 Mbps × 40s × 2 ≈ 800 MB. A 10x margin is recommended to account for bitrate variability.
6.4. TLS termination¶
The Perfect Streamer server accepts connections on the HTTP and HTTPS ports. When TLS is terminated on nginx, the upstream uses the HTTP port. Forwarding the X-Forwarded-Proto and X-Forwarded-Host headers is mandatory for correct construction of absolute URLs when absPath=1.
server {
listen 443 ssl http2;
server_name stream.example.com;
ssl_certificate /etc/letsencrypt/live/stream.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/stream.example.com/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 1d;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
location ... {
proxy_pass http://pss_backend;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header Host $host;
# + caching directives from 6.1
}
}
server {
listen 80;
server_name stream.example.com;
return 301 https://$host$request_uri;
}
When HTTPS is used between nginx and the origin, the directives proxy_ssl_verify and proxy_ssl_trusted_certificate apply. For loopback connections encryption is redundant.
6.5. Multi-host¶
When several server_name values are served from a single nginx process, $host is added to the cache key to isolate the content:
map $uri $pss_cache_key {
"~^/h[0-9a-f]{16}(?<tail>(/[0-9]+)?/[0-9a-f]{16}\.(ts|m4s)|/sub/[0-9]+/[0-9a-f]{16}\.vtt)$" "$host:stream:$tail";
default "$host:$uri";
}
The keys_zone size is calculated as 8000 keys/MB. For multi-host installations with thousands of streams, keys_zone=...:300m or higher is recommended.
7. Client-side caching¶
Cache-Control: immutable is honoured by modern browsers. The client cache returns the segment without a conditional request on repeated access (including a backward seek within the player buffer).
Service Workers may apply a cache-first strategy based on the Cache-Control contents. DASH players use MSE through SourceBuffer; a segment placed into the buffer remains available without a repeated HTTP request until it falls outside the sliding boundary.
For cross-origin requests the server returns Access-Control-Allow-Origin: * together with the header Vary: Origin, Access-Control-Request-Headers. A caching proxy that honours Vary splits the entries into variants by the client Origin value (browser players send Origin, console clients do not), which lowers the HIT rate. Since the ACAO is the universal “*”, it is safe to add proxy_ignore_headers Vary in the segment location (see 6.1).
8. Delivery through a CDN¶
Perfect Streamer is compatible with CDNs in pull-from-origin mode.
Origin shield. Placing one or several shield nodes between the CDN edge and the origin is recommended in order to reduce the origin request rate when clients are distributed globally.
Purge. Content-addressed segments require no purge. When the stream metadata changes (codec, resolution), the manifests are refreshed within max-age=1 without an explicit purge.
Cache warming. When a load increase on a particular stream is expected, warming the CDN from several geographic locations before the broadcast starts is acceptable.
Geographic distribution. Segments (max-age=60) are well suited to geographically distributed caching. Manifests (max-age=1) tolerate a delivery delay of up to one second — acceptable for live broadcasting without low latency.
9. Monitoring¶
9.1. X-Cache-Status¶
Add add_header X-Cache-Status $upstream_cache_status; to every location with caching. Values:
Value |
Description |
|---|---|
|
Response served from the cache |
|
Not present in the cache, fetched from the origin and stored |
|
Expired, refreshed |
|
A stale copy was served to a concurrent request during the update |
|
|
|
The origin returned 304 Not Modified |
|
|
9.2. Access log format¶
log_format pss_cache '$remote_addr $status $request_method "$request" '
'$body_bytes_sent rt=$request_time ut=$upstream_response_time '
'cache=$upstream_cache_status key=$pss_cache_key';
server {
access_log /var/log/nginx/pss.log pss_cache;
}
9.3. Metrics¶
The nginx-vts module exports per-zone metrics in Prometheus format:
GET /status/format/prometheus
Recommended alert thresholds:
Metric |
Threshold |
Possible cause |
|---|---|---|
Segment HIT rate |
< 90% over 5 minutes |
Cache key normalization is broken; |
Manifest MISS rate |
> 50% over 1 minute |
|
Upstream response time p95 |
> 500 ms over 1 minute |
Origin overload |
Cache zone fill |
> 90% over 10 minutes |
Approaching |
Delivery quality on the node side is visible in the session metrics on the “Clients” screen (Clients).
10. Diagnostics¶
Symptom |
Probable cause |
Resolution |
|---|---|---|
Low segment HIT rate |
|
Add |
404 on segments after they leave the window |
A cached 404 for a segment that has fallen out of the sliding window |
Add |
Playback start delay of 2–5 s |
|
Reduce to 1–2 s; enable |
Manifest is not refreshed |
|
Specify |
Growth of TIME_WAIT on the upstream |
No |
Add |
403 on |
The client resolves relative URLs against the pre-redirect URL |
The server emits |
Stutter and frequent rebuffering on remote clients |
Low effective TCP throughput caused by slow start and idle restart at high RTT (300 ms and above) |
Tune the Linux network stack on the origin: see 10.1 |
10.1. Origin TCP tuning for high-RTT clients¶
The problem appears on clients with a high RTT to the origin (for example, 300 ms and above) when the stream bitrate is close to the channel capacity. The symptoms in the player are frequent rebuffering and dropouts; on the server the client looks normal and no errors are reported.
Cause:
TCP slow start. Every new TCP connection starts with a congestion window of about 14 KB and grows it over several RTTs. At an RTT of 300 ms, reaching the full window takes 2–3 seconds. During that time an HLS/DASH segment of 5 s duration (4–6 MB) is downloaded noticeably slower than real time.
TCP idle restart. Between segment requests, a client following the HLS pull model pauses for 4–5 s. By default, after such a pause the Linux kernel resets the connection congestion window back to the initial cwnd (the behaviour of
net.ipv4.tcp_slow_start_after_idle=1). As a result, on the next GET the keep-alive connection starts transmitting from slow start again — even on an already warmed-up session.
An aggravating factor is that the default CUBIC congestion control copes poorly with long RTTs and packet loss on intermediate network segments.
The solution is two sysctl parameters on the origin:
# Keep congestion window across idle pauses inside keep-alive sessions.
sysctl -w net.ipv4.tcp_slow_start_after_idle=0
# Use BBR instead of CUBIC: better behaviour on long-RTT paths
# with mild packet loss; paces sending instead of bursting.
modprobe tcp_bbr
sysctl -w net.ipv4.tcp_congestion_control=bbr
To make the change permanent:
cat > /etc/sysctl.d/99-pss-net.conf <<EOF
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_congestion_control = bbr
EOF
sysctl --system
The main effect comes from the first parameter (tcp_slow_start_after_idle=0) — it directly eliminates the repeated slow start between segment requests within a single keep-alive connection. The second one (BBR) provides additional robustness and applies to all new connections. The tuning does not require a restart of Perfect Streamer. These parameters do not apply to HTTP/3 (QUIC) clients: QUIC runs over UDP with its own congestion control.
11. Security¶
11.1. Session URL¶
A URL of the form /h<sess>/... acts as a session token — it requires no repeated authentication. Its lifetime is bounded by an inactivity timeout of 60 seconds; with no activity the session is deleted automatically.
Requirements:
HTTPS for all OTT paths (
/hls/,/dash/,/h<sess>/) in production;The session ID in the
Locationheader of the 302 response is not cached (no-cache, no-store).
11.2. Rate limiting¶
limit_req_zone $binary_remote_addr zone=dash_top:10m rate=5r/s;
limit_req_zone $binary_remote_addr zone=hls_top:10m rate=5r/s;
limit_req_zone $binary_remote_addr zone=llhls_top:10m rate=5r/s;
server {
location /dash/ {
limit_req zone=dash_top burst=20 nodelay;
proxy_pass http://pss_backend;
}
location /hls/ {
limit_req zone=hls_top burst=20 nodelay;
proxy_pass http://pss_backend;
}
location /llhls/ {
limit_req zone=llhls_top burst=20 nodelay;
proxy_pass http://pss_backend;
}
}
The session URL (/h<sess>/) requires no rate limiting — processing is cheap and the responses are cached.
11.3. Caching of error responses¶
proxy_cache_valid 200 60s;
proxy_cache_valid 301 302 0s;
proxy_cache_valid 404 403 0s;
proxy_cache_valid any 1s;
Disables caching of redirects (a unique sess in Location) and of responses reporting authorization errors or a missing resource.
11.4. Restricting network access to the origin¶
The streaming HTTP server port (43972 by default; 43982 for HTTPS) must be closed to external traffic. Acceptable configurations:
Bind Perfect Streamer to
127.0.0.1(when nginx is local)Firewall rule:
iptables -A INPUT -p tcp -m multiport --dports 43972,43982 ! -s 10.0.0.0/8 -j DROP
12. Middleware integration¶
12.1. The prefix-login model¶
Perfect Streamer supports delegating user identification to a middleware system through the prefix-login mechanism — a built-in alternative to full integration with external billing (accounts verified by an external system are configured on the “Users / Logins” screen, the “External (billing)” tab, Users / Logins).
The prefix flag is enabled on a local login, after which the server accepts URLs with a login of the form <prefix><subscriber_id>:
/dash/test1/sub42/xxx/index.mpd
/hls/test1/sub43/xxx/index.m3u8
Here sub is the configured login prefix, and 42/43 are the middleware subscriber identifiers; the password is shared.
12.2. Per-subscriber statistics¶
In the client list each session carries both the configured login and the actual subscriber URL login (match-login), by which the middleware maps sessions to its own subscribers. The data is visible on the “Clients” screen (Clients).
12.3. Limitations of prefix-login¶
Shared password. All subscribers of a prefix pool use a single password value. Compromising the password grants access to any
<prefix><string>.Access granularity. The list of permitted streams applies to the entire prefix pool. Per-subscriber access control is provided by integration with external billing.
Password rotation. Changing the password disconnects all active subscribers. A gradual replacement requires the temporary use of two prefix logins.
