Monitoring¶

The Monitoring group in the “Node” area collects the screens for monitoring the current node: the state of streams, clients, hardware and services. The screens refresh automatically. Most of them only display data, but some also let you manage the object: “Streams” to create and configure streams, “DVB adapters” to add, configure and scan an adapter, “EPG Database” to edit a channel. Services, accounts and storages are configured in the Configure and Administration groups.

The central screen of the group, “Streams”, is covered in a separate section, Streams.

Node overview¶

The start dashboard of the node. The top bar holds the key indicator tiles: the number of streams and how many of them are “On air”, the “Problems” counter, active “Alerts” (in the administrator role), CPU and memory load. Every tile is a link to the screen its figure is taken from: the first three lead to Streams, “Alerts” to Alerts, “CPU” and “Memory” to System Monitor. Two tiles also set up where they lead, so that the list matches the figure that was clicked: “Streams” returns the list to the “All” mode, and “Alerts” sets the “Own node” scope. “On air” and “Problems” deliberately leave the filter alone: it divides the streams by pause, and that is not what either figure counts. The “Alerts” tile counts the alerts of the node itself only, so in a domain its number is smaller than that of the top bar bell if the bell has the “All nodes” scope selected; going to the list from the tile makes the two numbers equal. Below are the panels of the problem subsystems; each one lists the faulty objects or, when there are none, shows a short summary of how many objects are working. “Problem streams” is always there. “Transcoders” appears when the node has transcoder instances or an alert about the node’s transcoder has been raised, “DVB adapters” when a tuner has been detected or at least one adapter is configured. The conditions for the panels are wider than for the menu items of the same name (Interface layout): a subsystem that has dropped out does not take its panel with it, so a configured adapter keeps the panel even when the tuner has gone, and an alert keeps the transcoders panel even when not a single instance is left. In that case the menu item is already hidden while the panel is still in place, and the “Open →” link in its heading remains the only way into the section. When a failure leaves nothing to list, the transcoders panel reports it on a line with a counter of the failed instances. That is why on a node without transcoders and without a tuner only one subsystem panel remains, and the overview offers no way into those two sections. “Recent logs” follows a rule of its own: the panel is visible to an administrator whose log database is working, and only while it holds records of the “warning” level and above from the last five minutes. On a quiet node it is absent, however many records have piled up over the years, and it goes away by itself when the last warning leaves that window. In the subsystem panels the name of an object is a link that opens the statistics window of exactly that stream, transcoder instance or adapter on its own screen, while the “Open →” link in the panel heading leads to the whole list. In “Recent logs” the rows are not links: go there from the heading or through the “+N more →” link at the bottom of the panel. At the very bottom is the “Favorites” bar with the pinned signal charts of DVB adapters and mosaic tiles (they are pinned on their own screens); while nothing is pinned, there is no bar. The operator sets the order in the bar: an item is moved by dragging or with the Alt+↑ and Alt+↓ shortcuts. The bar draws the charts and the tiles as two groups, so a move stays within its own group: a chart takes its place among the charts, a tile among the tiles. The order is kept in the browser together with the favourites list itself and survives a page reload, but another machine and another operator have their own. Rearranging works only on a wide screen; the bar has no separate reset button, and an item is removed with the same star that pinned it. The System Monitor screen gives a more detailed view of the node load.

Pipeline¶

The processing graph of a single selected entity. The core of the graph is three lanes: “INGRESS”, “STREAM” and “EGRESS”. If the stream is linked to a transcoder or to a multiplex, the link is unfolded right in the picture. Three more lanes are added on that side: the neighbouring streams, the processing stage between them (“TRANSCODER”, “DEMUX” or “MUX”) and the neighbours’ own transport points (“INPUTS” and “OUTPUTS”). A chain unfolded on both sides occupies nine lanes and shows the whole path, from the input socket of the neighbouring stream to its own output socket. One stage is unfolded on each side, and only where a link actually exists; there is no switch for this, and what has been unfolded is taken out of the core, so the same entity never appears twice in the graph.

The view of the graph depends on the type of the entity. An SPTS stream and an MPTS multiplex are drawn in the same way: for a multiplex, “MEMBERS” unfolds upstream (the card of each one shows its PNR) and “CONSUMERS” downstream; for the stream list, see Streams. An ABR bundle (Adaptive bundles), a transcoder (Transcoders) and a DVB adapter (DVB adapters) each have their own fixed view.

The neighbouring streams are given as two lists, above the diagram and below it; the focus is moved by clicking a name or with the mouse wheel over the list, one position per step. The focus also changes with the search by stream name in the header and with transitions from other screens; the current focus is kept in the address, so a direct link to it can be shared. After a transition the diagram returns to the centre of the screen. The diagram area works as a camera: dragging pans it, the wheel zooms it, and the “−”, “+” and “Fit” buttons are in the top right corner. “Fit” only shrinks a chain that is too wide to the size of the window; the zoom is not adjusted by itself, and a new focus opens at natural size.

A stream card opens its statistics. A click on a neighbouring stream, on a programme of the multiplex or on the “TRANSCODER” stage moves the focus to it; as long as the transcoder chain has not been unfolded, the transcoder card in “INGRESS” or “EGRESS” does the same, and for an input this requires a decoder to be set. The inputs and outputs in the transports of the shared domain catalogue have a “Show in the library” button (Library — this feed); the lanes of the neighbouring streams do not. A separate “Node relays” panel gathers the local relays of the node and is not shown on a node that has none, while the “doors” to the adjacent nodes open their admin UI at the same focus.

Selecting several rows on the “Streams” screen and pressing “Open in pipeline” gives a group view. The streams in it are drawn as one shared diagram: a row for each participant, while what they have in common (a transcoder decoder that two encoding variants work from, or a multiplex that several of the selected streams belong to) is drawn once, with links running from it to the participants’ rows. The lists of adjacent streams, the focus counter and the search are removed here: each of them moves the focus to a single stream and would break up the group. The group view does not show the live state of the inputs, and the bar above the diagram warns about this: the diagram is built from a single snapshot of the configuration, the time of the snapshot is stated there as well, the “Refresh” button reads it again, and “Back to the stream list” returns to the screen. The state of each stream itself is live, as usual. No more than twelve participants are drawn; more can be selected, but the bar will then say how many out of how many are shown. A participant that is no longer on the node shows “This stream is no longer on the node.” instead of a diagram. Selected ABR bundles do not share the common diagram: each gets its own card with an “Open separately” button. The address of the group view can be used as a link and opens on any device: a wide window is needed only to make the selection.

The graph does not draw paused inputs and outputs, and paused streams get neither into the list of neighbours nor into the composition of a multiplex; a direct link still opens such a stream, and a paused transcoder variant in the “VARIANTS” lane stays on the diagram. The transmission model is described in Transmission model: Stream, Sender, Receiver, Peer, stream processing on a node in Streamer: implementation details, and MPTS multiplexes in MPTS streams.

Clients¶

The list of connected receivers, split across tabs for the delivery protocols: “HTTP / OTT”, “SRT”, “PS1” and “RIST”; each tab shows a live counter of active sessions. The columns “Client” (with a mark of its origin: “Local login” or “External billing”), “Address”, “Stream”, “Login” and “Uptime” are present on every tab; the protocol columns are inserted between them rather than appended at the end: on “HTTP / OTT” they are “Type”, “Session” and “Quality”, on “SRT” “Mode” (caller or listener), “Loss %”, “TSBPD” and “RTT”, on “PS1” “Loss %” and “RTT”, on “RIST” “Status”, “RTT”, “Bandwidth” and “Quality”. Search works over the list (name, login, address), and long lists are split into pages.

The operator sets the row order: a column header is a button, and clicking it sorts the list by that column. The icon beside the caption shows the state of the column: ↕ means the column can be sorted but the list is not sorted by it now; ▲ and ▼ show the direction chosen. A column that does not sort has no icon at all. Clicks cycle through three states: first the direction useful for that column, then the reverse, then the sorting is dropped and the order in which the node sent the rows returns. The useful direction for numeric columns is descending (“Uptime”, “Loss %”, “RTT”, “Bandwidth”, “Quality”), so that the outliers land at the top on the first click; for text columns it is A to Z. “Session” is ordered not alphabetically but by the kind of delivery: “direct” first, then “OTT”. “TSBPD” does not sort at all: the column holds “Yes” and “No”, and grouping by it says nothing. Rows with no value in the column being sorted go to the bottom in both directions rather than to the top; the exception is “Uptime”, where an unknown running time is shown as 0s and sorts together with the zeros. Numbers inside text values are compared as numbers rather than character by character, anywhere in the string: 192.168.1.9 comes before 192.168.1.10, port :80 before :5000, and stb2 before stb10. The whole list found is sorted, not the page on display, and the counter of rows shown does not change with the sorting, though the list returns to the first page. Rows with equal values always line up in the same order, so the next refresh of the list does not swap them around. The chosen order is not remembered: the screen always opens in the node’s order. On switching to another protocol tab the order is kept, but it takes effect only if the same column is present there as well; if it is not, the list is in the node’s order, and on returning the sorting applies again.

Expanding a row opens the full set of diagnostic counters of the session, updated in real time. No more than one row is expanded at a time: expanding the next one closes the previous one, so two sets of counters cannot be compared side by side, and the readings are taken in turn. A row stays expanded as long as its session is alive: refreshing the list does not close it, even when an OTT session changes its address. It is closed by clicking again, switching tabs, changing the search string, changing the sort order or paging; closing the row also clears the selection of the session for “Kick”.

Two columns of the “HTTP / OTT” tab divide the receivers by the way of delivery: on the same port the node serves both direct MPEG-TS and OTT. “Type” shows what the node has reported: the protocol and the scheme separated by a slash (HLS/https, LL-DASH/quic). The protocol is HTTP (direct MPEG-TS delivery), HLS, LL-HLS or LL-DASH, the scheme is http, https or quic. “Session” reduces this to two values: “direct” for the HTTP protocol and “OTT” for the rest; a dash would mean that the node did not report the kind of session, which does not happen on a healthy build. In practice the values differ as follows. Direct delivery is a row per connection: as many rows as the “Max connections, HTTP/HLS” of this login allows (User: add and edit), and they are disconnected separately; the row stays as long as the connection is alive, since the node does not close it for inactivity. An OTT session is a row per player session, not per request, and it disappears by itself about a minute after the player’s last request in that session: any request counts, not only a chunk download, so a player that keeps refreshing the manifest keeps the row. With DASH, several players of one login watching one stream share a single session (3.2. Session reuse for DASH). “Quality” is computed over chunks, so for the rows of direct delivery this column is always empty.

The administrator has the “Kick” action. It ends a single session, the one whose row is expanded; the other sessions of the same login keep working. The expanded row is the selected target, so the button is enabled the whole time at least one row is expanded, and it moves with the selection from session to session. When nothing is expanded, it is disabled, and the text next to it says what is missing: “Expand a session row to select it, then kick.” Two more reasons for unavailability are permanent: on the “RIST” tab kicking is not supported at all (“RIST sessions can’t be kicked.”), and on “SRT” the connections in caller mode cannot be kicked (“Caller-mode sessions can’t be kicked (display-only).”). The selection is confirmed in the “Kick session” window. The command goes to the node asynchronously: the reply “Kick sent.” means that the node has accepted it, not that the connection is already closed. The delivery protocols are described in Peer protocols for reliable transmission, OTT delivery in OTT and DVR, and the receivers’ accounts on the Configure screen (2. Client behaviour).

DVR monitor¶

Monitors DVR recording per storage (read-only). For each storage the screen shows the fill level relative to the cleanup target (a bar with the target mark), the volumes (used, free, total, archive size broken down into TS/MP4), the “Disk pressure” flag, the current background task and the number of bound streams; the active alerts of the storage are shown alongside. Expanding shows the archive streams line by line (recording / idle, retention depth, accumulated volume, read/write latencies) and the maintenance diagnostics (collection of orphaned directories, trimming under disk pressure). For a stream, the Stream statistics window opens; the administrator can also clear the archive of an individual stream and run the general “Clean up orphaned archives” command. The storages themselves are configured on the DVR storages screen; archive recording and playback are described in OTT and DVR and DVR: the stream archive.

Transcoders¶

The list of the node’s active transcoder instances with an “All / Active” filter. The header shows the total load, “Total CPU” and “Total memory”, and the actions on the selected instance: “Statistics” and “Open in the pipeline” (Pipeline). Each row shows the state, the decoder (the source), the type, the number of encoders, the running time, CPU and memory; expanding it shows the encoders with their output parameters (video codec, resolution, frame rate/scanning, audio codecs, actual bitrate). The decoder and every encoder have their own actions on the stream named in that row: for the decoder it is the source stream, for an encoder the stream it produces. The pipeline label opens the statistics of that stream, and an administrator also has a settings button beside it that opens the stream’s editor over the list (Configure stream). Both keep the operator here; the “Show <stream name> in the stream list” button, by contrast, leads away from the screen: it opens Streams, with the row of that stream expanded and the list scrolled to it, and opens nothing over the list. Arriving by such a link clears the filters on that screen and, in particular, returns the “All / Active” switch to “All”, otherwise the stream could be hidden. Unlike the settings button, the jump is available in any role: it is navigation, not a change. Going back in the browser returns to the transcoders screen, but it is reloaded, and the expanded row of the instance closes. Transcoding capabilities and configuration are described in Transcoders, monitoring in Monitoring, and installing packages and drivers in Transcoders.

Transcoder instance¶

The card of an individual transcoder: a statistics window opened from the screen header, from the “Transcoders” panel of the node overview or from the central node of the transcoder graph. It shows the source (decoder), the list of encoders (1 → N), the resources (CPU, uptime, PID, memory) and the process log for diagnostics: the last exit code, the current and the previous run logs (with copying). What of this to capture, and when, for a support request is described in Transcoder logs. When opened by a link from another screen, the window also expands and highlights the row of this instance in the list behind it, so that after the window is closed the operator stays on it; if the list was in the “Active” mode, it returns to “All”, otherwise a stopped instance (and the links from the node overview lead precisely to such instances) would be hidden. A window opened from the header of the list itself leaves the filter alone.

DVB adapters¶

A single DVB reception screen: the adapter configuration together with the live state (signal lock, quality, bitrate). The rows are grouped by mode: “Stream mode” (reception into a stream) and “FE Monitor” (monitoring of a frontend tuned by another application); the columns are name, transponder, signal, bitrate, alerts and state; each row has a pause toggle. The actions on the selected adapter are in the header: “Statistics”, “Open in Pipeline” (Pipeline), and also (for the administrator) “Configure adapter” and “Delete adapter”. The header also holds the “All / Active” filter, “Add adapter” and “Scan…” (for the administrator; the scan button also requires a window at least 768 pixels wide, so there is none on a phone). Arriving at the screen by a link that names a particular adapter returns the filter to “All”, so that the adapter does not stay hidden; opening the statistics from the list itself leaves the filter alone. DVB reception is described in DVB receiver, and adapter monitoring in Signal monitoring.

DVB adapter editor¶

The window for adding and configuring a DVB adapter. “Mode” and “Broadcast system” are set when the adapter is added and cannot be changed afterwards, because they determine the set of settings. The hardware is selected from the list of detected frontends (see Hardware). Stream mode provides the tuning parameters (frequency, symbol rate, FEC, modulation and so on, depending on the broadcast system), the LNB section for satellite reception, “T2-MI”, “BISS descrambling” and “CI/CAM descrambling”. Muting the adapter alerts and the “Advanced” section (extended parameters) are available in both modes; FE Monitor mode reads the signal without retuning the tuner. Reception and descrambling are described in Adapters, Descrambling and T2-MI decapsulation.

The dialog requires some fields to be filled in and does not let an adapter be created without them. The hardware is mandatory in both modes, FE Monitor included: while the list still reads “— select hardware —”, the “Create and configure” button is locked and the text under the field says “Select the hardware — without it the adapter cannot be saved.”. The frontend monitor also watches a particular tuner; it only does not retune it. On a node where no frontend is listed there is nothing to choose from, and the same requirement passes to the manual “Adapter index” field. For DVB-S, DVB-S2 and DVB-C in Stream mode, “Symbol rate, ksym/s” is also mandatory: while it is zero, creation is locked and the field explains “State the symbol rate — without it the adapter will not be configured.”. The frequency and the symbol rate are checked not only by the creation dialog but by the editing dialog as well: a zero value is no longer saved there halfway.

The requirement comes from the node: it declares these keys with a default value below its own lower bound and validates them when the adapter is created, so the operator has to supply the value, and the default cannot be used. Before the symbol rate field appeared, a satellite or cable adapter in Stream mode could not be created by hand at all (the node refused), and only the scanning wizard worked, which takes the rate from the multiplex it has found.

DVB scan¶

A channel search wizard (for the administrator). It goes through steps: choosing the tuner and the broadcast system, then choosing the source, either “Catalog” (satellite, cable or terrestrial with LNB parameters) or “Blind scan” over a frequency range; then the transponders are swept with a progress indicator, collecting the multiplexes found, each with its list of services.

The catalogues of satellites and of cable and terrestrial regions are searched by string: a “Search” field sits above the list, the list itself is sorted by name, and numbers within the names are compared as numbers. The words entered are looked for in any order and across the whole row (including the orbital position of a satellite and the country code of a cable or terrestrial region), so the query hot 13 finds Hotbird at 13.0°E. A row already selected does not disappear from the list even if it does not match the query: a search cannot clear the selected source. The counter under the list counts the rows displayed, not the matches, and as long as there is at least one row in the list, “Nothing found.” does not appear; an empty catalogue has no counter at all, and the message above it states the reason. The query is cleared when moving between satellite, cable and terrestrial and when switching “Catalog” ↔ “Blind scan”; changing DVB-T to DVB-T2 or DVB-S to DVB-S2 keeps it.

The satellites no longer follow the arc: the name of each begins with its own position, so sorting by name lines them up by degree, alternating eastern with western. It is quicker to type the position wanted into the search than to find it by scrolling.

For a terrestrial scan the first step gets a “Probe PLP up to” field, which sweeps the PLPs on every DVB-T2 transponder (Scanning). The default is 0: there is no sweep, and the node receives the same request as before. A value outside the range 0…255 is not sent; the start button is disabled instead. A node older than 2.0.2.291 does not understand such a request, and the field has no effect there.

Multiplexes found on different PLPs of one frequency are shown as separate rows, each with its own programme list, its own PLP mark and its own selection checkbox; a row read with no filter is marked “PLP auto”. The “PLP auto” mark relates to the whole result list rather than to a single transponder: it appears on the rows without a number once at least one row in the list has one. The PLP number also arrives without a sweep, if the catalogue gives it.

The tuner list also shows the busy tuners (they cannot be selected), with the state of each beside it: “free”, “busy · scanning” or “busy” with an English token for the reason: kernel means an outside process holds the tuner, pss-id-<number> means an adapter of this same node holds it, and such an adapter can be paused to release the tuner. The sweep is stopped with the “Stop” button; while the command is on its way, the button reads “Stopping…”. The node itself never refuses a stop and always accepts one, but the command may fail to reach it: a timeout expired, the link broke, the session ran out. A message then appears in the window starting with the words “Could not stop the scan — the tuner is still busy.”, followed by a description of the failure; the sweep carries on, and “Stop” can be pressed again. The interface will not pass an unstopped scan off as stopped. There is one scan per node, shared by all adapters, so starting a new pass ends the previous one and takes its place. The node can refuse a start, and that is the only case where it really refuses: the previous pass has not released the tuner within the time allowed (“Could not start the scan — the tuner is still busy with the previous pass. Try again in a few seconds.”). Closing the window by any means sends the node the same stop command, but now there is no window in which a failure could be reported; the node’s own timeout covers a command that does not arrive. The multiplexes found are added one at a time or in a batch. One at a time: the “Add DVB adapter” window opens with the parameters pre-filled; it holds only the essentials, and the settings window with all the parameters opens once the adapter has been created. In a batch: the selected rows are created with no windows, and only a warning about matching names can interrupt it. Names are formed automatically and checked for duplicates; for a row with a PLP number, the number goes into the name. In both cases the PLP number is written into the “Stream id (ISI/PLP)” of the new adapter at once, so check and correct it in the settings of the adapter created. Adding also closes the wizard and therefore stops the sweep. The channel search is described in Scanning.

Adapter statistics¶

A window with the live parameters of the selected adapter. It contains the state summary and the tuning parameters, the active alerts, the signal history charts (level, SNR, BER, UCB), the “Metrics” section (frontend, lock status, frequency, “PLP / ISI”, SNR, signal level, error counters and, for Stream mode, the bitrate), the list of programmes, “T2-MI carriers”, “BISS descramblers”, “CI/CAM module”, “Profiler” (processing thread load, DVR ring overflow) and the adapter log. The administrator can use “Reset statistics”. It is used for antenna alignment and for checking the stability of reception (Signal monitoring).

In the adapter’s programme list and in this window (the programme heading and summary, the “Program” line in the media block), a service inside a T2-MI carrier is shown with the composite PNR, the same one by which the programme is addressed everywhere: in the programme selector of the demultiplexer, of BISS and of CAM (T2-MI decapsulation). The programme’s own number inside the carrier cannot be used for addressing: on a transponder with mirrored groups it repeats on every carrier and points to nothing. The programmes of the outer multiplex are shown as before.

The “PLP / ISI” line in “Metrics” appears only when the node reports the number of the selected stream: an adapter on “auto”, and DVB-C and ATSC, do not have it. This is a tuner-level number, set in “Stream id (ISI/PLP)”: the PLP for DVB-T2, the stream identifier for satellite multistream. The PLPs of T2-MI carriers are a different thing, shown by the separate “T2-MI carriers” section.

Test streams¶

The test streams screen, which generates service streams for checking signal paths (Test streams). The section is in preparation.

EPG Database¶

The collected programme guide and the state of its sources. The screen contains these tabs:

Database: Viewing the collected guide: source selection, the channel list with search and filtering by sets, the schedule of the selected channel by day (day and language switching, the current event highlighted). Since version 2.0.2.419, a programme with an age rating shows a rating badge after its title, for example 16+; its tooltip reads “Age rating 16+”, with the rating system in brackets if it is known. The administrator can edit a channel: name, time zone, icon, membership in sets.
Source status: The import status for each source: state, the time of the previous and the next update, the number of channels and events, the error; the administrator can force an update of a source. A source waiting for its turn to fetch (Fetch queue and retries) is shown in the Queued: position N state. Since version 2.0.2.419 such a source is marked with a blue dot, and while fetches of a source keep failing, the “Next update” field is called “Next retry”, and the “Failed fetches in a row” field is shown next to it.

EPG import and processing are described in EPG Database and EPG/XMLTV import, and the sources and their configuration in Source configuration and on the EPG Settings screen.

Mosaic¶

A preview wall: live frame thumbnails, one tile per active SPTS stream and per programme of an active MPTS (multiplex programmes are marked with an icon). The grid adapts to the screen width, and the tile size is switched by the “Size” control (1, 1/2, 1/3). Paused and stopped streams do not appear in the mosaic. Clicking the picture opens the enlarged frame, and clicking the caption opens the statistics window of its stream (Stream statistics); for a multiplex programme these are the statistics of the whole MPTS, not of a single programme. The star pins the tile to “Favorites” on the Node overview screen.

The operator sets the tile order: a tile is moved to the place of its neighbour by dragging or with the Alt+↑ and Alt+↓ shortcuts; the tile’s tooltip says the same: “Drag to rearrange · Alt+↑/↓”. The gesture works only on a wide screen: on a tablet and a phone the wall displays the layout that has been set but does not let you change it. The layout is kept in the browser, not on the node and not in the account, so another operator and another machine have an order of their own. The “Reset layout” button appears in the header next to the “Size” control as soon as a layout of your own has been set, and returns the wall to the usual order: by stream number and, for a multiplex, also by programme number. The tile of a paused stream keeps its place and returns to exactly that place, even if the neighbouring tiles have been rearranged in the meantime. The place is forgotten only when the tile has not appeared on the wall for a month. A new tile is always added at the end.

System Monitor¶

The state of the server as cards: “CPU” (the total load and the share of the PSS process, the number of cores), “Memory” (used and the share of PSS, the amount in GB), “Network” (receive/transmit and link utilisation per interface, with VLANs nested under the physical adapter) and the hardware acceleration cards (NVIDIA: GPU, “Memory bandwidth”, decoder, encoder; Intel VPL). “Memory bandwidth” is the load of the card’s memory controller, not how full the memory is. The fill is shown by the “Video memory” bar, and the card header shows the used and total amount in GB; both appear only if the driver reports these values (they are absent on Intel cards and on nodes before version 2.0.2.416). Video memory fill is tracked by the code 64 alert (Alert code catalogue). In an interface row, the name and, if it is known, the link speed are followed by its IPv4 address: eno3 · 1 Gbps · 10.0.0.5; for a nested VLAN the speed is not reported and the address follows the name directly. Several addresses on one interface are listed separated by commas, and an interface with no IPv4 address shows nothing. Below is the “History” block: graphs of the indicators over time with the same window presets as the stream statistics (Stream statistics); the heading of an interface graph shows the same address. An interface with a known link speed is drawn as a percentage of it, while one whose speed the node does not report (usually a VLAN) is drawn in megabits. Such a graph does not squeeze the scale below 1 Mbit/s, so that an almost idle interface does not look loaded; and if at least one direction carried something over the window but its peak fell short of a megabit, the graph’s readings switch to two decimal places, otherwise an interface carrying tens of kilobits would read as dead. A direction that stayed silent for the whole window does not count towards this, so a completely idle interface stays at a plain zero. On nodes with configured DVB adapters, the signal quality graphs of the frontends are shown in a section of their own (Signal monitoring). The screen helps to judge the load headroom; recommendations for tuning resources are in Node performance tuning, and external metric collection in Connecting external monitoring systems.

Logs¶

A structured event log of the node (available to the administrator) with filters and paginated output. Entries are filtered by severity (“Severity ≥”), by source type and identifier, by message text and by date range; the sort order can be switched (newest / oldest first). The columns are time, severity, source and message; a stream source opens its Stream statistics window, and other sources lead to the screen for that object. It is used for diagnostics and incident analysis.

Every start of the service is marked in the feed by a separator line of its own, “Streamer start”, with the moment of the start and the build the run came up on. The current run has its uptime beside it, past ones show how long the run worked and how long the node stood idle before the next start; a run that ended without a proper stop is marked separately. The separator is placed even when the feed is filtered by severity. The start line itself is a notice and would not pass an error filter, yet the boundary between runs is needed precisely when an incident is being analysed.

A long message in the list is collapsed to two lines; a click on the row (or on the chevron at its start) opens a panel beneath it with the full text, a “Copy message” button, the time, the level, the source of the entry and, if the entry repeated, the number of repetitions. Rows expand independently of each other, and paging or refreshing the feed collapses them: a log entry has no identifier of its own, so there would be nothing to remember.

The source of a record is denoted by the object type and, for streams, also by the number and the name: Stream #43 · orig CBC HD; if no display name is set, the stream name is used. Streams are labelled with the same name and number in the selection list of the “Source ID” filter. Records made by an input or an output of a stream are attributed to the stream itself and show its name, while the particular input or output is named at the start of the message text.

Alerts¶

A summary screen of the active alerts: those of the current node or those of the whole domain; the scope in the header switches the set. The screen shows only the active set: an alert that has been cleared disappears from the list. The columns are severity, source, message and age; a stream source opens its Stream statistics window. An alert about an input or an output is attributed to the stream, while the input or the output itself is appended to the stream name as an English token: input #2 for an input, output #7 for an output. For an input this is its number in the redundancy list, the same one that is visible on screen; for an output it is the internal record number, which is shown nowhere, because outputs have no priority and are given no ordinal number. Counting the outputs down the list does not help: the numbers stay with deleted records. It is simpler to open the expanded row of the stream, where the alert is shown right on the row of its own input or output. In a domain of several nodes, the row shows the originating node (with a link to its admin UI). Alerts at the administrator-action level have a “Reset” button (manual acknowledgement) that acts on their own node. For such alerts the place to remove the cause is shown under the source; when it is an input or an output, the row serves the administrator as a link and opens the editor of the owning stream (Configure stream). An alert that arrived from another node gives no link, because each node has its own stream numbers. The alert model, its analysis and the full list of codes are given in Alerts (alerter) and Alert code catalogue, and threshold configuration and external delivery on the Alerter screen.

The “Grouped” / “Flat” switch in the header, to the left of “Own node” / “All nodes”, sets how the list is shown; the choice is remembered in the browser. By default the alerts are grouped: first by the originating node, and within a node by the alert code. The header of a node group shows the highest severity in the group, the node name and the number of rows; the current node comes first, the others follow by highest severity and group size. If the set has a single node, there is no node header, and rows without a node are gathered in the “Node not reported” group. The header of a code group shows the highest severity, the code name (for example, sync-gap-absorbed) and the number of rows. A group of more than three alerts opens collapsed; clicking its header expands or collapses it. The “Flat” view is the former single list.

In both views a row can carry a tag with a tooltip after the code name, and the group headers show how many such rows they contain:

“sticky resync” — a recommendation to resynchronise the output (codes 58, 59 and 61). Such an alert stays for hours after the last event: the source feed is worth checking, but this is not a current outage;
“backup input” — a backup input is failing while the stream runs on another input: since version 2.0.2.420 this is a code 65 row, one per stream, with the list of faulty inputs. On nodes of earlier versions this mark is put on the error row of an input that is not the first in the redundancy order. The source is worth checking before the backup is needed. On a 2.0.2.420 node a code 1 row for a backup input appears only when the stream runs through no input at all — that is an outage, and the row carries no such mark.

Monitoring¶

Node overview¶

Pipeline¶

Clients¶

DVR monitor¶