Real-Time Dashboards & Analytics
Data becomes decisions on a screen. This course builds the serving layer and live dashboards that turn position, P&L, and exposure into trustworthy, sub-second views, with drill-down and alerting that reconcile to the books.
The pipeline at the heart of this course
What makes a dashboard trustworthy
A trading dashboard is only useful if the desk trusts it, and trust is fragile: one number that fails to reconcile to the book and the whole screen is doubted. This course builds dashboards that earn trust, by getting the serving layer, the semantics, and the reconciliation right, not just the visuals. A beautiful dashboard on untrustworthy data is worse than no dashboard.
The course treats the dashboard as the visible tip of the whole platform: it consumes the positions, P&L, and exposures the earlier courses produced, and its trustworthiness depends on theirs. The engineering focus is the layer between the data and the screen, where correctness and speed are won or lost.
The course's governing principle is that trust is the scarce resource, and every design choice is judged against it. A dashboard the desk does not believe is worse than no dashboard, because it invites either false confidence or wasted effort double-checking, and trust is lost far faster than it is earned. The course therefore spends its effort on the serving layer, the semantics, and reconciliation, the things that make the numbers right, before it spends any on how they look.
The serving layer and semantic model
Between the marts and the screen sits a serving layer that must answer interactive queries fast. The course builds it around a semantic model: single, authoritative definitions of metrics, position, exposure, P&L, so that every view computes them the same way and no two screens disagree. This semantic layer is what prevents the classic failure where two dashboards show two different P&Ls.
Aggregation for drill-down is central: pre-aggregating along the dimensions the desk slices by, book, desk, counterparty, time, so that drilling from a portfolio total down to a single position is instant. The course covers OLAP-style pre-aggregation and how it trades storage for interactivity.
The serving layer and semantic model are where consistency is won, and the course treats them as the foundation. A single authoritative definition of each metric, computed the same way for every view, is what prevents the classic failure of two dashboards showing two different P&Ls, and pre-aggregation is what makes drilling through those metrics instant. You learn to build this layer so that speed and consistency are properties of the platform rather than of each individual screen.
Real-time push to the browser
A trading dashboard must update live, not on a refresh button. The course builds real-time push, streaming updates to the browser over websockets so that positions and P&L move as the market does, fed by the streaming pipeline from the previous course. It weighs push against polling and shows when each is appropriate.
Perceived speed matters as much as actual speed: caching, incremental updates, and careful rendering make a dashboard feel instant even under load. The course covers the latency budget, from event to pixel, and how to stay within it when thousands of positions are on screen.
Real-time push is what makes a trading dashboard live rather than periodically stale, and the course is concrete about the engineering it takes. Streaming updates to the browser, handling reconnection and missed messages, and staying within a latency budget from event to pixel are what keep the screen genuinely current under load. The course also weighs push against polling honestly, so you choose the right approach rather than the fashionable one.
Views, drill-down, and alerting
The course designs the views that matter: position and exposure grids, P&L with attribution drill-down, and the alerting that surfaces breaches without the user hunting for them. Visualization choices are treated as engineering decisions, driven by what risk data actually needs, not decoration, and time-travel views let the desk see the book as of any past moment, using the bitemporal and event-sourced foundations from earlier.
Access control and entitlements ensure each user sees only what they should, a real requirement for regulated trading data, and the course shows how row-level policy flows from the platform through the serving layer to the screen.
The views themselves are designed around what risk data actually needs, and the course treats visualization as an engineering decision rather than decoration. Position and exposure grids, P&L with attribution drill-down, alerting that surfaces breaches, and time-travel views that use the bitemporal foundations from earlier are chosen for what they let a user do, not for how they look. Entitlements ensure each user sees only what they should, which is a real requirement for regulated data.
Reconciling the screen to the books
The course closes on the discipline that makes all of it credible: reconciling the dashboard to the source of record, continuously, so that the live screen and the official books never silently diverge, and instrumenting the pipeline so that when they do, it is caught immediately. You leave with a production trading dashboard: a serving layer, live push, rich views, and reconciliation.
The artifact below shows a serving-layer query and the websocket push that keeps a live position view current.
Reconciliation to the books is the discipline that makes all of it credible, and the course ends on it deliberately. A live screen that does not tie to the source of record is misleading the very people meant to rely on it, so the course shows how to reconcile continuously and instrument the pipeline so a break is caught immediately. You finish with a production dashboard whose trustworthiness is built in rather than hoped for.
The semantic layer and the end of duelling numbers
The classic failure of trading analytics is duelling numbers: two dashboards showing two different P&Ls, each defensible, neither trusted. The cure is a semantic layer, a single authoritative place where each metric, position, exposure, P&L, is defined once, so that every view computes it the same way. This course treats the semantic layer as the foundation of a trustworthy dashboard, not an afterthought.
You learn to define metrics precisely and centrally, so that a number means the same thing wherever it appears, and to serve those definitions to every view through a common layer. This is as much an organizational discipline as a technical one, because it requires agreeing what each metric means, but the payoff is a set of dashboards that never contradict each other, which is the precondition for the desk trusting any of them.
The serving layer built on top of these definitions is engineered for interactivity: pre-aggregating along the dimensions the desk slices by so that drilling from a portfolio total to a single position is instant, and answering from a shape designed for the question rather than recomputing from raw data each time.
Live push, latency, and reconciling the screen to the books
A trading dashboard must be live, and this course builds the real-time push that streams updates to the browser over websockets so that positions and P&L move as the market does, fed by the streaming pipeline from the previous course. It weighs push against polling, budgets the latency from event to pixel, and shows how caching and incremental updates keep the screen feeling instant even with thousands of positions on it.
But liveness without correctness is a trap, so the course closes on the discipline that makes the whole thing credible: continuously reconciling the dashboard to the source of record, so that the fast live screen and the official books never silently diverge, and instrumenting the pipeline so that when they do, it is caught immediately rather than discovered by a trader who notices a number looks wrong.
This is the same reconciliation principle that runs through the position, streaming, and STP courses, applied at the last mile where data meets the human eye. A dashboard that reconciles is one the desk can act on; one that does not is one they will quietly stop trusting, however beautiful it looks.
Where this course sits, and what it unlocks
Dashboards is the eleventh course, the last mile where all the platform's data meets the human eye. It consumes the positions, P&L, exposures, and real-time streams from the earlier courses and turns them into live, trustworthy views, and it depends on the semantic consistency and reconciliation disciplines established throughout the bundle.
It is the course that makes the platform's value visible, and it sets up the STP capstone by demonstrating the serving and reconciliation patterns that the automated flow's monitoring also relies on.
Common dashboard failures, and how to avoid them
The classic dashboard failure is duelling numbers: two views showing two different P&Ls because each computes the metric its own way. A semantic layer, defining each metric once, eliminates this and is the precondition for trust. A second failure is a dashboard that looks live but is subtly inconsistent with its source, so a trader spots a wrong number and stops trusting the whole screen; continuous reconciliation to the books prevents this.
A third failure is sacrificing correctness for speed, showing a fast number that is stale or approximate without saying so, which is worse than a slower correct one on a trading desk. The course teaches how to be both fast and correct through pre-aggregation, incremental push, and careful reconciliation.
By avoiding these patterns you get a dashboard the desk actually acts on, because its numbers never contradict each other and never diverge from the books.
A worked example: drilling from portfolio to position
Consider a trader drilling from a portfolio-level P&L into the single position driving it. Because the serving layer pre-aggregates along the book, counterparty, and instrument dimensions, and because every view computes P&L from the same semantic definition, the drill is instant and consistent: the sum of the positions exactly equals the portfolio total, and the number means the same thing at every level.
The worked example shows the pre-aggregated serving query answering the drill without recomputation, and the websocket push updating the view live as the market moves. The trader can go from a portfolio anomaly to its root cause in a single position in seconds, with confidence that the numbers reconcile all the way down, because the semantic layer guarantees consistency and the reconciliation job guarantees agreement with the books.
This is the payoff of the course's disciplines, a semantic layer and a reconciled serving layer, made tangible in one interaction the desk performs constantly.
Operational realities: entitlements, load, and observability
Operationally, a trading dashboard must enforce data entitlements so each user sees only what they are permitted to, handle the load of many users and thousands of positions without degrading, and be observable so that a problem in the pipeline is caught before a trader notices a wrong number. The course covers row-level access control flowing from the platform through the serving layer to the screen, and the caching and incremental-update techniques that keep the dashboard fast under load.
Observability closes the loop: instrumenting the pipeline from source to pixel so that latency, staleness, and reconciliation breaks are monitored and alerted. The operational goal is a dashboard the desk can rely on without second-guessing, which requires that its correctness and freshness are actively guarded, not assumed.
Design trade-offs: push versus poll, and the cost of trust
A dashboard can stay current by polling, asking the server for fresh data on a timer, or by having the server push updates as they happen. Polling is simple and robust but wasteful and always slightly stale; pushing over a persistent connection is efficient and truly live but more complex to build and operate, and it must handle reconnection and missed messages gracefully. For a trading dashboard where the desk acts on what it sees, the liveness of push usually wins, but only if it is engineered to stay consistent.
The deeper trade-off is between raw speed and trustworthiness. A dashboard can be made to feel instant with aggressive caching and optimistic updates, but if those shortcuts let the screen drift from the books, the speed is worthless because the desk stops believing the numbers. The course teaches to spend the engineering on a semantic layer and continuous reconciliation first, so that the fast, live view is also a correct one. A trustworthy dashboard that is slightly harder to build beats a fast one that no one relies on.
Carrying dashboards into the capstone
The dashboard is the capstone's window: it surfaces the positions, P&L, exposures, and alerts that the flow produces, so that the humans who oversee the automated pipeline can see what it is doing and step in when an exception needs judgment. The reconciliation to the books that you build here is what lets that oversight be trusted, because an operator watching a screen that does not tie to the source of record is being misled about the very flow they are meant to control.
In a straight-through platform, the dashboard is also where the straight-through rate and the repair queue become visible, turning the capstone's operational metrics into something a team can watch and improve. So the dashboard is not the end of the platform bolted on for show; it is the control surface that makes an automated flow observable and governable by the people responsible for it.
Performance and scale considerations
A dashboard's performance is felt directly by the user, so the course treats perceived speed as a real engineering target. Pre-aggregation moves the cost of a drill-down off the interactive path, caching and incremental updates keep the screen responsive under load, and pushing only what changed rather than re-sending whole views is what keeps a live dashboard fast when thousands of positions are on screen. The latency budget from event to pixel is treated as a constraint to design within.
Scale shows up as the number of positions, users, and updates the dashboard must handle at once, and the course shows how to keep it responsive as each grows. Serving from a purpose-built layer rather than querying source systems directly, and bounding the data any single view must render, are what let the dashboard stay fast for a busy desk rather than degrading exactly when it is needed most.
Testing and validating the dashboard
The dashboard's most important test is reconciliation to the books, and the course builds it in as a continuous check rather than a one-time verification. The numbers on the screen must tie to the P&L mart of record, and the course shows how to compare them automatically and raise a break the moment they diverge, so the screen is never trusted blindly. A dashboard that reconciles continuously is one the desk can act on.
The course also teaches testing the serving layer's semantic consistency, confirming that every view computes each metric identically, and validating that the live push delivers updates correctly under reconnection and load. Together these turn the dashboard's trustworthiness into something demonstrated rather than assumed. You finish able to show that the live screen agrees with the source of record, which is what makes it safe to rely on.
How it works, concretely
// Server: push incremental position updates over a websocket.
// The serving layer answers from a pre-aggregated cube for instant drill-down.
import { DuckDBInstance } from "@duckdb/node-api";
async function servePositions(ws, filter) {
const db = await DuckDBInstance.create("serving.db");
// Pre-aggregated gold mart; drill-down is a filter, not a recompute
const sql = `
SELECT book, counterparty,
SUM(net_qty) AS net_qty,
SUM(mtm) AS mtm,
SUM(daily_pnl) AS pnl
FROM position_mart
WHERE desk = ?
GROUP BY book, counterparty
ORDER BY ABS(SUM(mtm)) DESC`;
const rows = await db.runAndReadAll(sql, [filter.desk]);
ws.send(JSON.stringify({ type: "snapshot", rows: rows.getRows() }));
}
// On each streamed position delta, push only what changed (sub-second)
positionStream.on("delta", (d) => {
ws.send(JSON.stringify({ type: "delta", book: d.book, mtm: d.mtm, pnl: d.pnl }));
});
// A reconciliation job compares this serving cube to the P&L mart of record
// every minute and raises an alert on any break, so the screen is never trusted blindly.Chapters 201 to 220
Twenty sequential chapters. Within the full bundle these are chapters 201 through 220 of 240.
- 201What makes a trading dashboard trustworthyWhat makes a trading dashboard something the desk actually trusts. One number that fails to reconcile can cost a dashboard all of the desk's trust.
- 202The serving layer between data and screenDesigning the serving layer between data and screen. The serving layer is where interactive queries are answered fast enough to feel instant.
- 203Semantic models and metric definitionsDefining metrics once in a shared semantic model. A shared semantic model is what stops two dashboards from showing two different P&Ls.
- 204Aggregation for interactive drill-downPre-aggregating for instant interactive drill-down. Pre-aggregation is what makes drilling from a portfolio total to a position instant.
- 205OLAP cubes and pre-aggregationUsing OLAP cubes and pre-aggregation. OLAP cubes trade storage for the interactivity a live desk needs.
- 206Real-time push vs polling architecturesChoosing real-time push over polling. Push versus poll is a real architectural choice with different liveness and complexity.
- 207WebSockets and live data to the browserStreaming live data to the browser over websockets. Websockets are what stream live updates to the browser without a refresh.
- 208Position and exposure view designDesigning position and exposure views that read clearly. Position and exposure views are designed around what risk data actually needs to show.
- 209P&L views and attribution drill-downBuilding P&L views with attribution drill-down. P&L drill-down is what lets a total be explained down to its attributed causes.
- 210Alerting, thresholds, and notificationsAdding alerting, thresholds, and notifications. Alerting surfaces breaches so the user does not have to hunt for them.
- 211Latency, caching, and perceived speedWinning perceived speed with caching and incremental updates. Caching and incremental updates are what make a dashboard feel instant under load.
- 212Consistency between dashboard and sourceKeeping the dashboard consistent with its source. Consistency between screen and source is the property that earns lasting trust.
- 213Time-travel and as-of views in the UIOffering time-travel and as-of views in the UI. As-of views use the bitemporal and event-sourced foundations from earlier courses.
- 214Visualization choices for risk dataChoosing visualizations suited to risk data. Visualization choices are engineering decisions driven by the data, not decoration.
- 215Handling scale: thousands of positionsHandling thousands of positions at scale. Handling thousands of positions is where naive dashboards fall over.
- 216Access control and data entitlementsEnforcing access control and data entitlements. Entitlements are a real requirement for regulated trading data on a shared screen.
- 217Embedding analytics and self-serviceEmbedding analytics and enabling self-service. Self-service and embedding extend the dashboard's reach beyond a fixed set of views.
- 218Observability of the dashboard pipelineInstrumenting the dashboard pipeline for observability. Observability of the pipeline is what lets a broken feed be caught before users notice.
- 219Reconciling the screen to the booksReconciling the screen continuously to the books. Continuous reconciliation to the books is what makes the live screen safe to rely on.
- 220A production trading dashboard end to endAssembling a production trading dashboard end to end. You finish with a production trading dashboard backed by a reconciling serving layer.
What you'll be able to do
- Define metrics once in a semantic layer so numbers never conflict
- Build a serving layer for instant interactive drill-down
- Push live updates to the browser within a latency budget
- Enforce access control and data entitlements
- Reconcile the live screen continuously to the books
Audience & prerequisites
Analytics and front-end engineers turning risk data into live, trustworthy views, and anyone who has watched a desk stop trusting a dashboard.
Prove the module by building it
Build a live position and exposure dashboard backed by a serving layer, with drill-down, alerting, and sub-second refresh that reconciles to the P&L mart.
This course project is one of twelve that culminate in the bundle's end-to-end capstone, a complete working ETRM data platform run as a straight-through pipeline.
Learn it your way
This course is included in the ETRM Data Engineering and Analytics bundle, available self-paced with PDF handbooks, slide decks, video explainers, and hands-on labs and a project, and deliverable as private corporate training.
Explore the full 12-course bundle and its end-to-end capstone project →
Frequently asked questions
What does the Real-Time Dashboards & Analytics course cover?
Turning position, P&L, and exposure data into live, trustworthy views that traders and risk actually use. It runs to 20 sequential chapters (chapters 201 to 220 of the 240-chapter bundle).
Where does this sit in the learning path?
It is course 11 of 12 in the ETRM Data Engineering and Analytics sequence. It builds on the courses before it and prepares for those after it.
Is there a project?
Yes. Build a live position and exposure dashboard backed by a serving layer, with drill-down, alerting, and sub-second refresh that reconciles to the P&L mart.
What technologies are involved?
The course works with React, WebSockets, DuckDB, Plotly, OLAP, chosen to reflect how real ETRM data platforms are built.
Is it available self-paced?
Yes. Every course in the bundle is available self-paced with PDF handbooks, slide decks, video explainers, and hands-on labs and a project, with lifetime access.
Can my team take it as corporate training?
Yes. The whole bundle, or individual courses, can be delivered as private corporate training mapped to your stack and data. Use the contact form to scope it.
Take this course, or the whole platform
Enroll self-paced, or bring the ETRM Data Engineering bundle to your team.