The only program that teaches modern data engineering on a complete, real ETRM trading platform - from canonical trade model to a signed-off P&L, built end to end by you.
Design the trading data platforms global energy companies run on. Build production-grade pipelines for trade capture, market data, forward curves, positions, valuation, settlement, and risk - end to end, on one canonical model that serves a whole trading floor. Twelve sequenced courses and 240 chapters take you from the core data model to a fully automated, straight-through pipeline, and a capstone in which you build a working ETRM data platform yourself. A shared book of seven canonical trades, spanning all seven instrument formats, threads through every course, on a data model that represents all twelve commodity desks.
The whole ETRM data platform, taught as one journey
Most training teaches the pieces of an ETRM data platform in isolation: a course on data modeling here, a course on streaming there, with no thread connecting them. Real platforms are not built that way. A position number depends on the trade model, the reference data, the curves, and the P&L logic all being right and all fitting together, and a straight-through pipeline depends on every one of those pieces being wired into a single flow. This portal is built around that reality: twelve courses that deliberately build on each other, in sequence, toward one working system.
You begin with the foundations, the canonical data model and the governed reference data every calculation trusts. You move through how a trade is represented and lives its lifecycle, including the FpML standard the industry uses to exchange trades. You build the valuation inputs, curves and market data, and turn trades and curves into positions, P&L with attribution, and the credit and limit controls that keep a desk safe. You then learn the platform that holds it all, a governed lakehouse, an event-sourced core, and add the real-time and analytics layers, streaming risk and live dashboards. Finally, you assemble everything into Straight-Through Processing, the capstone course where the platform becomes one automated flow from trade capture to settlement.
The result is not a collection of certificates but a capability. By the end you can build, and have built, the data engineering behind a modern trading platform, and you have an end-to-end project to prove it. And because a shared book of seven canonical trades, spanning all seven instrument formats, runs through every course on a model that represents all twelve commodity desks, what you build is not a toy that handles one instrument on one commodity, but the format-independent foundation a whole trading floor actually needs.
Why this program is unique
ETRM domain knowledge and modern data engineering, in one program
Generic data-engineering courses teach you pipelines, warehouses and streaming on retail, clickstream or IoT data - domains where the hard parts of trading simply do not exist. Trading-and-risk courses, on the other hand, teach the finance but stop at the spreadsheet, never touching the data platform underneath. This program is the rare combination of both: modern data engineering taught on a complete, real ETRM trading platform, where a position must reconcile to the blotter, a curve must be reproducible point-in-time, and a P&L number has to survive an auditor. That intersection - deep commodity-trading domain knowledge plus production data engineering - is exactly what energy majors, banks and trading firms struggle to hire for, and it is what this program is built to produce.
Dimension
Generic data-engineering course
This program
Data domain
Retail, clickstream, IoT, generic logs
Real trades, curves, positions, P&L across 12 commodity desks
Domain knowledge
None - data is abstract
Deep ETRM: trade lifecycle, FpML, valuation, risk, settlement
Correctness bar
Row counts and freshness
Positions reconcile to the blotter; P&L survives an auditor
Data model
Star schemas on toy data
A production canonical model spanning all seven instrument formats
Time handling
Ingestion timestamps
Bitemporal, point-in-time reproducible curves and market data
What you build
Disconnected exercises
12 production components assembled into one end-to-end platform
Proof of ability
A certificate
A working ETRM data platform capstone you built yourself
Who it is for
General data engineers
The trading-data specialist energy firms and banks can't hire enough of
The learning path
Twelve courses, in the order they build
Each course is a standalone 20-chapter program and a step in a deliberate sequence. Straight-Through Processing comes last because it assembles everything before it.
Build the ten components a trading floor actually runs on
This is not a lecture series. Across the twelve courses you build the actual components of an ETRM data platform - a Position & P&L engine that reconciles to the blotter, a forward-curve service with bitemporal, point-in-time valuation, an event-sourced trade core - each as a runnable artifact, then assemble them into one end-to-end system in the capstone.
✓ Trade Capture API
✓ Market Data Platform
✓ Forward Curve Engine
✓ Position Engine
✓ Daily P&L Engine
✓ MTM Valuation Service
✓ Kafka Streaming Pipeline
✓ Snowflake / Lakehouse
✓ Power BI Risk Dashboard
01
Trade Capture Platform
A canonical trade store with enforced integrity, capturing all seven instrument formats across twelve desks.
02
Market Data Platform
Bitemporal ingestion and storage for curves, volatility, FX and reference data, reproducible point-in-time.
03
Forward Curve Service
Bootstrapped forward curves with seasonality and versioning, feeding every downstream valuation.
04
Position Engine
Trades aggregated into live positions as a derived query over the canonical model, never a stored guess.
05
Valuation Pipeline
Positions struck against curves into mark-to-market and P&L with attribution that reconciles to the blotter.
06
Settlement Engine
Post-trade lifecycle to settlement, with exceptions peeling off to a repair queue and reconciliation at every handoff.
07
Risk Data Mart
Counterparty exposure, credit and limits with pre-deal and post-deal checks and audited breaches.
08
Real-time Streaming Pipeline
Trade and market-data streams joined statefully for live positions and limit alerts within seconds, exactly-once.
09
Medallion Lakehouse
Bronze to silver to gold, with catalog, lineage, quality and access control governing every layer.
10
Executive Dashboard
Role-based dashboards turning the gold marts into a live view for trading, risk and management.
Each component is the deliverable of one or more courses, and the straight-through-processing capstone wires all ten into a single automated flow from trade capture to settlement. You finish with a portfolio you can show an employer, not a certificate you can name.
Professional capabilities
After completing this program, you will be able to
These are the capabilities an employer or client is buying - not topics you have seen, but things you can do.
✓ Design enterprise ETRM data models spanning every instrument format and desk
✓ Build streaming market-data pipelines with bitemporal, point-in-time reproducibility
✓ Engineer position-calculation services that reconcile to the trade blotter
✓ Implement valuation and P&L datasets with full attribution
✓ Design medallion lakehouse architectures with governance and lineage
✓ Support risk and limit reporting with audited, trustworthy data
✓ Build event-sourced trade cores you can replay to any past state
✓ Stand up straight-through processing from capture to settlement
✓ Lead ETRM data-modernization projects end to end
How an ETRM data platform works
From a trade to a settled, reconciled number
To see why the sequence is what it is, follow a single trade through the platform the bundle teaches you to build. A trade is captured and immediately enriched with governed reference data, the counterparty, the instrument, the calendar, so that it means the same thing everywhere. If it arrived as an FpML message, it is validated and mapped into the canonical model. Its lifecycle begins: it is confirmed, perhaps amended, and every change is recorded as an immutable event, so the full history is always reconstructable.
To value the trade, the platform strikes it against curves built from market data and stored bitemporally, so the valuation is reproducible as of any moment. The valued trade rolls up into a position, and the change in that position's mark-to-market becomes P&L, decomposed by attribution into price, time, and new-trade effects that reconcile exactly to the blotter. Before the trade was ever accepted, a credit and limit check ran against the counterparty's exposure; after it, the same checks monitor for breaches as the market moves.
All of this lands on a governed lakehouse, bronze to silver to gold, and is driven in real time by streaming, so live positions and limit alerts reach a dashboard within seconds. And the whole flow, capture, enrich, represent, value, position, control, settle, is wired together as straight-through processing, automated end to end, with exceptions peeling off to a repair queue and reconciliation at every handoff. The animations below show the pieces that make this work.
The data model above is the core the whole platform rests on: trades referencing instruments, counterparties, and books, generating cash flows, valued against curves. Every downstream number is a query over this structure, which is why the bundle starts here.
Every trade moves through the lifecycle above as a series of immutable events on an enforced state machine, so its full history is always reconstructable and any past state can be replayed exactly.
Positions are not stored guesses; they are derived from the canonical trade store as of a chosen date and book, netted and aggregated, and reconciled back to the blotter, as the flow above shows.
Valuation strikes those positions against point-in-time curves to produce mark-to-market and a P&L that decomposes into price, time and new-trade effects, every number reconciled and attributable.
Straight-through processing, above, is the flow the bundle builds toward: a trade moving automatically from capture to settlement, with exceptions handled and every handoff reconciled. It is the capstone course precisely because it connects everything else.
The platform, in depth
Lakehouse, event sourcing, and streaming
Beneath the flow sits an architecture the bundle teaches in detail. Data lands and is refined through a medallion lakehouse: bronze for raw feeds exactly as they arrive, silver for conformed canonical data, gold for the business-ready marts that positions, P&L, and dashboards consume, with catalog, lineage, quality, and access control governing every layer.
Underneath the state sits an event-sourced core. Rather than overwriting rows, every change is recorded as an immutable event in an append-only log, and current state, positions, P&L, the audit view, is a projection derived by replaying those events. This is what gives the platform a perfect audit trail and the ability to reconstruct any state as of any past moment, which regulated trading demands.
And to make the desk's view live rather than end-of-day, trade and market-data streams are joined in a stateful streaming layer that maintains live positions and fires limit and risk alerts within seconds, with exactly-once guarantees so nothing is double-counted. These three, lakehouse, event sourcing, and streaming, are the architectural spine the whole bundle assembles.
The engineering, at depth
The data engineering patterns that hold it together
The bundle is opinionated about a small set of engineering patterns that recur across every course, because they are what make a trading data platform trustworthy rather than merely functional. The first is a single canonical data model. Trades reference instruments, counterparties, and books through stable surrogate keys and generate cash flows, and every downstream calculation, position, P&L, exposure, is a query over that structure rather than a bespoke integration. When twelve modules agree on one model, they compose; when they each invent their own, every seam becomes a reconciliation break.
The second pattern is bitemporality. Reference data, curves, and trade state all change over time, and the platform must distinguish when something was true in the world from when the system learned it. Every important fact carries both a valid-time axis and a transaction-time axis, so the platform can answer both what is true now and what we believed last Tuesday. This is what makes revaluation reproducible and audit possible, and it appears in the modeling, reference-data, curves, and event-sourcing courses alike.
The third is event orientation. Rather than overwriting rows, the platform records changes as immutable events in an append-only log and derives current state as a projection. This gives a perfect audit trail, point-in-time reconstruction, and safe reprocessing, and it is the backbone the streaming and straight-through-processing courses build on. Closely related is idempotency: because events may be replayed after a failure, every operation is designed so that applying it twice has the same effect as applying it once, which is what lets the capstone reprocess a trade without double-settling it.
The fourth is the medallion lakehouse. Raw feeds land in a bronze layer exactly as they arrive, are conformed into canonical models in a silver layer, and are served as business-ready marts in a gold layer, with schema enforcement quarantining bad data and open table formats bringing transactions and time travel to cheap storage. Governance, catalog, lineage, quality, and access control, spans every layer. The fifth is reconciliation as a first-class output: at every handoff, and between every fast incremental view and its slow authoritative recomputation, the platform checks that the numbers agree and raises a break the moment they do not. Together these five patterns are the difference between a demo and a platform, and the bundle teaches each one not as theory but as something you build.
Why trading data is different
The data-engineering problems that are specific to trading
Generic data engineering skills transfer only partway into ETRM, because trading imposes constraints that ordinary analytics pipelines never face, and the bundle is built around exactly those constraints. The first is reproducibility under scrutiny. A retail-analytics dashboard that is approximately right is fine; a P&L number that cannot be reproduced exactly, as of the moment it was struck, is a control failure and potentially a regulatory one. This is why bitemporality runs through the whole program: the platform must always be able to answer not just what a number is, but what it was, and what inputs produced it, at any past moment.
The second is that the data is adversarially temporal. Trades are amended and corrected, sometimes with backdated effect; reference data changes and those changes have effective dates that may precede the moment they were entered; market data is revised after the fact. A naive pipeline that overwrites as it goes destroys the history these corrections depend on and produces numbers that cannot be explained. The event-oriented, append-only patterns the bundle teaches exist precisely because trading data refuses to sit still, and because the questions asked of it, who changed this, when, and what did we believe then, demand that nothing is ever silently overwritten.
The third is that latency and correctness are both non-negotiable and in tension. A desk needs its risk live, within seconds of a trade or a market move, but it also needs those live numbers to agree with the official books struck in batch. Meeting one requirement by sacrificing the other is a failure either way, which is why the streaming course spends so long on event time, watermarks, exactly-once processing, and reconciliation against batch truth. The fourth is that the cost of a wrong number is asymmetric and large. A mispriced curve, a double-counted trade, a stale limit, any of these can drive a real trading loss or a breach, so validation gates, reconciliation, and integrity checks are not quality-of-life features but core requirements, present in every course rather than bolted on at the end.
These four properties, reproducibility, adversarial temporality, live-yet-consistent, and asymmetric cost of error, are what separate ETRM data engineering from the general discipline, and they are the reason a dedicated program exists. The bundle does not teach generic pipelines and hope they transfer; it teaches the patterns that these specific constraints force, using the actual entities, calculations, and controls of a trading platform.
At the code level
What "the canonical model" actually looks like
It is easy to talk about a canonical data model in the abstract, so it is worth being concrete about what the bundle has you build. The core is a small set of related tables where the constraints, not the comments, are what keep the data honest. A trade carries a surrogate key and a business reference, points at exactly one instrument, counterparty, and book, and cannot exist if any of those do not, because referential integrity forbids it. Quantities and prices use explicit precision, because rounding error in a trading system is a real financial risk. A cash flow can never outlive its trade. These are not stylistic choices; they are how invalid states are made unrepresentable.
The canonical trade table and a derived position view. Position is a query over trades, so it can never disagree with them.
CREATE TABLE trade (
trade_id bigint PRIMARY KEY, -- surrogate key
external_ref text NOT NULL UNIQUE, -- business identifier
instrument_id bigint NOT NULL REFERENCES instrument(instrument_id),
counterparty_id bigint NOT NULL REFERENCES counterparty(counterparty_id),
book_id bigint NOT NULL REFERENCES book(book_id),
trade_date date NOT NULL,
quantity numeric(20,6) NOT NULL, -- precision matters
price numeric(20,8) NOT NULL,
buy_sell char(1) NOT NULL CHECK (buy_sell IN ('B','S'))
);
-- Position is derived, never stored: a query over trades.
CREATE VIEW position AS
SELECT book_id, instrument_id,
SUM(CASE WHEN buy_sell='B' THEN quantity ELSE -quantity END) AS net_qty
FROM trade
GROUP BY book_id, instrument_id;
From this seed, everything grows. Reference data attaches to the instrument and counterparty entities and is served through a validated golden source. The lifecycle course turns the trade into a stream of immutable events, so the row above becomes a projection of its own history. Curves, stored bitemporally, feed the revaluation that turns positions into P&L. The lakehouse lands this same model in its silver layer; the streaming course maintains that position view live; and the straight-through-processing capstone flows real trades through all of it. Each course adds a layer, but they all speak this one model, which is exactly why they compose into a working platform instead of a pile of disconnected parts. The child pages go down to this level of detail for every course, with the data models and code you can read and learn from directly.
One shared book of deals
Seven canonical trades, seven instrument formats
The bundle ships a shared Canonical Trade Book: seven worked trades, TRD-10001 through TRD-10007, that between them span all seven instrument formats a commodity trading floor deals in. These are not throwaway examples invented fresh in each course. They are one book of deals, carried from the data model in the first course all the way through to the running platform and the straight-through-processing capstone, so you see the same trades take shape, get valued, roll into positions, hit limits, stream, and settle. When a concept appears in course five that you first met in course one, it is the same trade, which is what makes the sequence feel like building one platform rather than taking twelve separate courses.
Trade
Instrument format
Commodity
TRD-10001
Exchange future
Crude oil (WTI)
TRD-10002
OTC swap
Natural gas (Henry Hub / TTF)
TRD-10003
Physical forward
LNG cargo (DES)
TRD-10004
Exchange option
Crude oil (Brent)
TRD-10005
OTC option
Power (PPA-linked)
TRD-10006
Spot physical
Coffee / refined product
TRD-10007
Environmental / credit
Emissions (EUA) / Freight (FFA)
The teaching idea underneath the book is that format is independent of commodity. A future is modeled the same way whether it holds crude, gas, power, carbon, or coffee, so one instrument master and one taxonomy serve a whole trading floor rather than a single desk. Most ETRM courses show one instrument type on one commodity; this bundle builds the abstraction that lets a single canonical model represent all of them, which is exactly what a real platform must do.
That abstraction is what lets the data model represent all twelve commodity desks, grouped into five markets: Core Energy (crude, gas, LNG, power); Physical Fuels and Freight; Energy Transition and Environmental (emissions and carbon, biofuels and SAF, hydrogen); Power Origination and Flexibility (PPAs, battery); and Soft Commodities (coffee, agricultural and CTRM). The seven canonical trades are the shared worked examples; the model beneath them covers the full desk landscape. If you want to go deeper on any individual desk, the energy trading desk programs cover them one by one.
What each course teaches
The twelve courses at a glance
Each course is a full 20-chapter program with its own data models, code, diagrams, and project. Here is the thread that runs through them.
1. Data Modeling for ETRM Systems. The relational and dimensional foundations, entities, keys, normalization, temporal and bitemporal modeling, and the canonical trade model every later course extends, delivered as runnable schema.
2. Reference and Master Data Management. The governed golden source of instruments, counterparties, calendars, and hierarchies, with survivorship, versioning, and a validation gate that keeps bad static data out of trading.
3. Trade Lifecycle and Instrument Modeling. The trade as an ordered sequence of immutable events over an enforced state machine, from capture through amendment, novation, and termination, with a fully queryable history.
4. FpML and Trade Representation. Parsing, validating, and mapping the industry XML standard for derivatives into the canonical model, with a reversible round-trip and a resilient ingestion pipeline.
5. Market Data, Curves and Time-Series. Bootstrapping forward curves, modeling volatility and seasonality, and storing everything bitemporally so every valuation is reproducible point-in-time.
6. Position and P&L Data Models. Aggregating trades into positions, revaluing against curves, and producing P&L with attribution that decomposes every daily move and reconciles to the blotter.
7. Counterparty, Credit and Limit Modeling. Modeling exposure and collateral and building a real-time limit framework that enforces pre-deal and post-deal checks with audited breaches and overrides.
8. Data Lake, Warehouse and Lakehouse. The platform architecture beneath it all, the medallion lakehouse, open table formats, partitioning and performance, and the governance that keeps it from becoming a swamp.
9. Event Sourcing and Audit Trail Design. Recording every change as an immutable event, building projections and snapshots, and reconstructing any state as of any past moment for perfect auditability.
10. Streaming and Real-Time Risk. Processing trades and market data in motion, event-time windowing and late data, and maintaining live positions and alerts with exactly-once guarantees.
11. Real-Time Dashboards and Analytics. The serving layer, semantic models, and live push that turn the data into trustworthy sub-second views, with drill-down, alerting, and reconciliation to the books.
12. Straight-Through Processing (Capstone). Wiring all eleven prior modules into one automated, controlled, exception-managed flow from capture to settlement, the synthesis of the entire program.
The capstone project
The Capstone: a working ETRM data platform, end to end
The bundle culminates in a single end-to-end project that is, in effect, a thirteenth module built on top of the twelve. You assemble everything, the data model, reference data, trade lifecycle, FpML ingestion, curves, position and P&L, credit and limits, the lakehouse, event sourcing, streaming, and dashboards, into one working ETRM data platform, and then run it as a straight-through pipeline from trade capture to settlement.
01
Model & reference
Stand up the core schema and a governed reference-data service (courses 1-2).
02
Capture & represent
Ingest FpML trades into the lifecycle model with full history (courses 3-4).
03
Value
Build curves and revalue positions into a reconciling P&L mart (courses 5-6).
04
Control
Enforce credit and limit checks over the exposures (course 7).
05
Platform
Land it all on a medallion lakehouse with an event-sourced core (courses 8-9).
06
Real-time
Drive live positions and alerts, and surface them on a dashboard (courses 10-11).
07
Automate
Wire the whole thing into one straight-through flow with exception handling (course 12).
You deliver
What the capstone produces
A runnable data platform repository with schema, pipelines, and services
An end-to-end STP run from trade capture to settlement on sample data
A reconciliation and exception report proving straight-through integrity
An architecture and data-model portfolio piece you can show an employer
Why it matters
A portfolio piece, not a certificate
The capstone is, in effect, a thirteenth module built on top of the twelve. It is the difference between having watched twelve courses and having built a working ETRM data platform you can show an employer or apply at work. Each course's own project is a rehearsal; the capstone is the performance.
Project-driven throughout
Every module justified by a project
Learning sticks when it is applied. Every one of the twelve courses ends with a project that puts that module to work, so you finish each course with something built, and the bundle with a complete platform.
01
Data Modeling for ETRM Systems
Design a normalized ETRM core schema (trades, instruments, counterparties, books) with keys, constraints, and a dimensional mart on top, delivered as runnable DDL and an ER diagram.
02
Reference & Master Data Management
Build a reference-data service with a golden-record model, survivorship rules, versioned history, and a validation gate that rejects bad static data before it reaches trading.
03
Trade Lifecycle & Instrument Modeling
Model the full trade lifecycle as an explicit state machine with event transitions, then implement capture, amend, novate, and terminate against your core schema with a queryable history.
04
FpML & Trade Representation
Parse real-shaped FpML trade messages, validate against schema, and map them into your canonical trade model with a reversible round-trip and a validation report.
05
Market Data, Curves & Time-Series
Build a curve-construction pipeline: ingest quotes, bootstrap a forward curve, store it bitemporally, and expose point-in-time curve retrieval for revaluation.
06
Position & P&L Data Models
Compute positions from the trade store, revalue against curves, and produce a P&L attribution mart that decomposes daily change and reconciles to the trade blotter.
07
Counterparty, Credit & Limit Modeling
Build a limit-checking service: model exposures and limits, compute utilization in real time, and enforce pre-deal and post-deal limit checks with breach alerts.
08
Data Lake, Warehouse & Lakehouse
Design a medallion lakehouse for ETRM: bronze ingestion, silver conformed models, and gold marts, with schema enforcement and a governance layer.
09
Event Sourcing & Audit Trail Design
Implement an event-sourced trade store: append-only events, projections for read models, and a full audit trail that can rebuild any state as of any time.
10
Streaming & Real-Time Risk
Build a streaming pipeline that maintains live positions and fires limit and risk alerts within seconds of a trade or market move, with exactly-once guarantees.
11
Real-Time Dashboards & Analytics
Build a live position and exposure dashboard backed by a serving layer, with drill-down, alerting, and sub-second refresh that reconciles to the P&L mart.
12
Straight-Through Processing (Capstone)
Assemble the full STP pipeline end to end, integrating all eleven prior modules into one automated flow from capture through settlement, with exception handling, reconciliation, and observability.
Self-paced learning
Everything you need, in every format
The full bundle is self-paced with lifetime access. Each course comes with written handbooks, presentation decks, video explainers, and hands-on labs and a project, so you can read, watch, and build in whatever way fits you.
Self-paced
Work through all 12 courses and 240 chapters at your own pace, with lifetime access to updates.
PDF handbooks
A written handbook per course, with the data models, code, and diagrams to keep as reference.
Slide decks
Presentation decks for every course, ready for self-study or to teach a team.
Video explainers
Concise videos that walk through the harder concepts and architectures visually.
Labs & projects
Hands-on labs and a graded project per course, plus the end-to-end capstone project.
How each course is built
Read it, see it, build it
Every course follows the same rhythm, so that once you find your footing in the first one, the rest feel familiar. Each opens with the problem it solves and why it sits where it does in the sequence, then works through its twenty chapters in order, each chapter building on the last. The concepts are explained in prose you can actually learn from, not bullet-point summaries, and the harder ideas are shown as animated diagrams, a data model, a pipeline, a flow, so you can see the shape of the thing before you build it. Crucially, each course goes down to the code and data-model level: real schema, real parsing and mapping logic, real streaming topologies, so the page itself teaches rather than just pointing at a lab.
Then you build. Every course carries a project scoped to put that module to work, design the schema, stand up the reference service, model the lifecycle, ingest the FpML, construct the curve, compute the P&L, enforce the limits, and so on, so you finish each course with something concrete rather than a passive sense of having watched it. The written handbook, slide deck, and video explainers give you the same material in the format that suits how you learn, and the labs give you a place to apply it. Because the bundle is self-paced with lifetime access, you can move as fast or as carefully as you need, and return to any course as a reference later.
The twelve projects are deliberately cumulative: the schema you design in the first course is the schema the later courses extend, the events you model in the lifecycle course are the events the streaming course consumes, and so on. By the time you reach the capstone, you are not starting from scratch, you are assembling work you have already done into one end-to-end platform. That cumulative design is what makes the final capstone achievable rather than overwhelming, and what makes the whole bundle feel like building one system rather than taking twelve separate courses.
Who teaches it
Learn from a practitioner, not a lecturer
The bundle is built and taught by Prajesh Jha, founder of Gravitas ETRM and an industry practitioner with 20+ years of real-world experience across global banks, energy trading firms, and enterprise consulting. He has worked with JPMorgan Chase, ADNOC, BP, ENI, Barclays, NatWest, Centrica and Baker Hughes - Fortune 500 and global-major engagements - solving real business and technology problems in trading, risk, data and operations. His work spans front-to-back ETRM implementations, investment-banking trading and risk systems, and modern cloud and data-platform builds, so the material reflects what actually happens inside live trading organizations, and what employers test for in interviews and on the job, rather than textbook theory.
ETRM data engineering is a niche, high-value specialism. The capability you build maps directly to roles that energy and commodity trading firms, banks, and consultancies hire for.
ETRM / CTRM Data Engineer
Own the trade, market-data, position and P&L pipelines behind a trading platform.
Trading Platform / Solution Architect
Design the canonical model, lakehouse and streaming spine a whole desk depends on.
Risk & Analytics Engineer
Build the exposure, limit and risk data marts that keep a book safe and reportable.
Quantitative / Market-Data Developer
Build curve services and bitemporal market-data platforms feeding valuation.
ETRM Implementation Consultant
Lead trading-technology implementations with the data-layer depth to de-risk them.
Trading Technology / Data Lead
Take a team through building or modernizing an ETRM data capability end to end.
Corporate training
Bring the whole platform to your team
The bundle is built for teams standing up or modernizing an ETRM data capability. Delivered as private corporate training, it is mapped to your stack, your data, and your use cases, with optional on-prem or cloud labs, so your engineers learn on the platform they actually work on. A whole team can move through the sequence together and build the capstone against your own architecture.
Mapped to your stack
Tailored to your databases, streaming platform, lakehouse, and ETRM systems, so the learning transfers directly.
Team cohorts
Move a whole team through the 12-course sequence together, with shared projects and a common vocabulary.
Capstone on your data
Build the end-to-end capstone against your own architecture, turning the training into a real internal asset.
This is a professional program, not a standard online course. Choose the format that fits how you or your team need to learn, all built on the same 240-chapter curriculum and end-to-end capstone.
Edition
Self-paced
The full 12-course sequence, all artifacts and the capstone, on your own schedule. Lifetime access and updates. Best for individual engineers building a serious portfolio.
Edition
Cohort
A guided run through the sequence with a group and set milestones, so momentum and peer review carry you to the capstone. Best for those who learn better with structure and a deadline.
Edition
Corporate
Private delivery for a whole team, mapped to your stack, data and use cases, with the capstone built against your own architecture. Best for standing up or modernizing an internal ETRM data capability.
1:1 Mentoring
Optional direct guidance from a practitioner as you work through the projects and capstone.
Certification
A completion path that recognises the capstone platform you build, not just chapters read.
Lifetime updates
The curriculum evolves with the field and with the platform; your access includes what comes next.
The point of the bundle is what you can do at the end of it, so it is worth being specific. You will be able to design a normalized ETRM core schema and the dimensional marts on top of it, and defend every key, constraint, and temporal choice. You will be able to stand up a governed reference-data service with survivorship and a validation gate, and explain why bad static data is the root of most trading incidents and how yours stops it. You will be able to model a trade's full lifecycle as events over an enforced state machine, and reconstruct any trade's state as of any past date.
You will be able to ingest real-shaped FpML, validate it, and map it losslessly into your model. You will be able to build a bitemporal curve service and prove that a past revaluation reproduces exactly against the as-of curve. You will be able to compute positions and a P&L that decomposes into price, time, and new-trade effects and reconciles to the blotter, and to build a real-time limit-checking service that blocks a breaching trade before it is done. You will be able to design a medallion lakehouse with schema enforcement and governance, implement an event-sourced core with projections and snapshots, and run a streaming pipeline that maintains live positions and alerts with exactly-once guarantees.
And you will be able to put a trustworthy live dashboard on top of all of it, and then, in the capstone, wire the whole thing into a straight-through pipeline from capture to settlement with exception handling and reconciliation at every step. That last capability is the one that matters most, because it is the one that proves the rest fit together. It is the difference between knowing the pieces and being able to build the platform.
Who it is for, and how to progress
Built for people who want to build the platform
This bundle is for data engineers, analytics engineers, quantitative developers, ETRM implementers, and trading-technology teams who want to build the data platform behind commodity and energy trading, not merely operate one. It suits an individual engineer assembling a serious portfolio, a specialist moving from generic data engineering into the trading domain, and a whole team standing up or modernizing an ETRM data capability together. No deep trading background is assumed; the courses build the domain context as they go, though comfort with SQL and a programming language will let you get the most from the labs and code.
The recommended way through is the order the courses are numbered, because each one leans on the ones before it. The foundations, modeling and reference data, come first because everything references them. Representation and lifecycle come next, then valuation and the numbers that flow from it, then the platform and real-time layers, and finally the straight-through-processing capstone that only makes sense once the rest exists. That said, an experienced engineer can treat any single course as a standalone 20-chapter program on its subject, dipping in for the specific depth they need, because each course is complete in itself even as it fits the whole.
However you approach it, the destination is the same: a complete, defensible command of ETRM data engineering across all 240 chapters, twelve applied projects, and one end-to-end capstone that stands as a working platform you built yourself. That is what the sequence is designed to deliver, and why it is worth doing in order.
What this program covers
The ETRM data-engineering topics, in depth
Each of these is a first-class subject inside the curriculum, taught to production depth rather than mentioned in passing.
ETRM Data Engineering
The discipline of building the trade, market-data, position and P&L pipelines behind a commodity and energy trading platform.
Energy Trading Data Platform
The end-to-end system that captures trades, values them, computes positions and risk, and settles - taught as one architecture.
Commodity Trading Lakehouse
A medallion lakehouse (bronze/silver/gold) with governance, lineage and quality, purpose-built for trading data.
Trade Capture Data Model
A canonical trade representation spanning all seven instrument formats and twelve desks, with FpML ingestion.
Position Management Data Model
Positions as derived, reconcilable queries over the canonical store, not stored snapshots that drift.
Market Data Engineering
Bitemporal ingestion and storage of curves, volatility, FX and reference data, reproducible point-in-time.
Forward Curve Management
Bootstrapped, versioned forward curves with seasonality, feeding every downstream valuation deterministically.
MTM Data Engineering
The pipelines that turn positions and curves into mark-to-market and attributed P&L that reconciles to the blotter.
ETRM Analytics
Risk, exposure, limit and P&L analytics served from governed gold marts to dashboards and reports.
ETRM Data Architecture
The event-sourced core, streaming spine and lakehouse that hold the whole platform together.
FAQ
Frequently asked questions
What is the ETRM Data Engineering and Analytics bundle?
A one-stop learning portal of 12 sequenced courses and 240 chapters that build, end to end, the data engineering and analytics capability behind a modern ETRM platform, from the core data model through reference data, the trade lifecycle, FpML, curves, position and P&L, credit and limits, the lakehouse, event sourcing, streaming, dashboards, and finally straight-through processing.
How is it sequenced?
The courses build on each other in order. You start with data modeling and reference data, move through the trade lifecycle and representation, then valuation, position, P&L, and risk, then the platform, real-time, and analytics layers, and finish with Straight-Through Processing, the capstone that assembles everything into one automated flow.
Why is Straight-Through Processing last?
STP wires every prior module into a single automated pipeline from capture to settlement. You cannot build it until you have built the model, the reference data, the lifecycle, valuation, position and P&L, credit and limits, the platform, event sourcing, streaming, and dashboards that it connects. It is deliberately the final, integrative course.
How many chapters are there in total?
240: twelve courses of exactly twenty sequential chapters each, numbered 1 to 240 across the bundle.
Is there a project?
Yes, thirteen in effect. Each of the twelve courses has its own applied project, and the bundle culminates in an end-to-end capstone project, a real thirteenth module, in which you assemble all twelve into one working ETRM data platform and run it as a straight-through pipeline.
What formats is the bundle available in?
Fully self-paced with lifetime access: PDF handbooks, slide decks, video explainers, and hands-on labs and a graded project per course, plus the end-to-end capstone. It is also available as guided cohorts and as private corporate training.
Who is it for?
Data engineers, analytics engineers, ETRM implementers, quantitative developers, and trading-technology teams who want to build, not just use, the data platform behind commodity and energy trading. It suits both individuals building a portfolio and teams standing up a capability.
Can we run it as corporate training?
Yes. The whole bundle, or any subset of courses, can be delivered privately, mapped to your stack, your data, and your use cases, with optional on-prem or cloud labs. Use the contact form to scope a corporate engagement.
Do I need prior ETRM experience?
No deep trading background is required, though basic SQL and programming familiarity helps. The bundle builds trading-domain context as it goes and focuses on the data engineering and analytics craft.
What is the Canonical Trade Book?
It is a shared set of seven worked trades, TRD-10001 through TRD-10007, that span all seven instrument formats a commodity floor deals in: exchange future, OTC swap, physical forward, exchange option, OTC option, spot physical, and environmental or credit. The same seven trades are threaded through every course, from the data model in course one to the running platform and the straight-through-processing capstone, so you follow one book of deals end to end rather than disconnected examples.
How many commodity desks does it cover?
The data model represents all twelve commodity desks, grouped into five markets: Core Energy (crude, gas, LNG, power); Physical Fuels and Freight; Energy Transition and Environmental (emissions and carbon, biofuels and SAF, hydrogen); Power Origination and Flexibility (PPAs, battery); and Soft Commodities (coffee, agricultural and CTRM). The teaching idea is that format is independent of commodity, so one instrument master and taxonomy serve the whole floor. The seven canonical trades are the shared worked examples; the model beneath them covers the full desk landscape.
What will I have at the end?
A complete, defensible understanding of ETRM data engineering across all 240 chapters, twelve applied projects, and one end-to-end capstone, a working ETRM data platform, that stands as a portfolio piece.
Build the data platform behind trading
Choose the next step that fits you. Finish with a working ETRM data platform of your own.