Programme 1 of 14 · Foundations

Data Engineering & Pipelines Foundations

1,037 words5 min read

The foundation that ties the whole track together: how data moves through modern systems, ETL and ELT, orchestration, data modeling, and the role of the data engineer, before you specialize in any single platform.

PythonSQLOrchestration (Airflow)ETL / ELTData modeling
Data Engineering & Pipelines Foundations: the syllabus at a glance1How data moves2ETL and ELT3Data modelingfor pipelines4Orchestrationand reliabilityProject
Curriculum · 20 chapters in 4 modules

The full syllabus

Four modules of five chapters each, sequenced so the material builds cumulatively. The brochure carries a note on what each chapter teaches.

M1 How data moves Ch 1-5 · 5 chapters
  1. What a data engineer does and why it matters
  2. Sources, sinks, and the shape of a pipeline
  3. Batch versus streaming, and when to use each
  4. The modern data stack, from a height
  5. Where cloud, warehouses, and processing fit
M2 ETL and ELT Ch 6-10 · 5 chapters
  1. Extract, transform, load, and the ELT shift
  2. Ingestion patterns and change data capture
  3. Transformation and the role of SQL
  4. Data quality, validation, and testing
  5. Idempotency and safe reprocessing
M3 Data modeling for pipelines Ch 11-15 · 5 chapters
  1. Relational modeling and normalization
  2. Dimensional modeling for analytics
  3. The medallion (bronze, silver, gold) pattern
  4. Schema evolution and contracts
  5. Partitioning and file formats
M4 Orchestration and reliability Ch 16-20 · 5 chapters
  1. Orchestration and scheduling
  2. DAGs and dependency management
  3. Idempotent, retryable tasks
  4. Monitoring, alerting, and observability
  5. The end-to-end pipeline project
Where Data Engineering & Pipelines Foundations can leadThis programmeopens roles inData EngineerAnalytics EngineerPipeline DeveloperETL Developer(the foundation for every role above)
FAQ

Frequently asked questions

What is the Data Engineering & Pipelines Foundations programme?

The foundation that ties the whole track together: how data moves through modern systems, ETL and ELT, orchestration, data modeling, and the role of the data engineer, before you specialize in any single platform.

Who is this programme for?

It suits newcomers building a base before choosing a platform, along with others described on this page.

How is it delivered?

Self-paced with hands-on labs; the recommended starting point and the map for the whole track.

Is there a project or capstone?

Build an end-to-end data pipeline from scratch: ingest from a source, transform through a medallion (bronze, silver, gold) layout, model the result for analytics, orchestrate it as a scheduled, retryable, monitored job, and document it as the reference project the rest of the track builds on.

How does this fit the wider journey?

This is the first programme and the map for the whole track. If you are from a non-data background, pair it with the Programming and Data Analytics tracks for coding and SQL. Everything after it, the clouds, warehouses, engines, and databases, slots into the picture this programme builds.

Can my organization run this as a private cohort?

Yes. Every programme can be delivered as a tailored corporate cohort. Contact us to scope it.

How it is delivered

Three ways to take Data Engineering & Pipelines Foundations

Self-paced is a document-and-media programme with lifetime access - no live sessions. Cohort and enterprise add live instructor-led training. Mentorship is not offered in any tier.
Feature Self-paced Cohort Enterprise
Format Written chapters, video explainers and podcasts Everything in self-paced, plus scheduled live sessions Everything in cohort, delivered privately to your team
Live sessions None Scheduled, instructor-led Scheduled, instructor-led, private
Mentorship Not offered Not offered Not offered
Access Lifetime Lifetime Lifetime for every enrolled seat
Pace Entirely your own Guided schedule with a peer group Agreed with your desk
Tailoring Fixed curriculum Fixed curriculum Sequenced to your markets, systems and governance
Best for Individuals learning around a job Individuals who want structure and deadlines Desks building the same capability together
For individuals

Get the programme guide

The full chapter list, what each module covers, and how the tiers compare - sent to your inbox as a PDF.

For teams

Run this for my desk

Private delivery for your desk, sequenced to your markets and systems. Tell us the team and we will scope it.

Run this for my desk