London, UK

Adil
Osman

Data Engineer — Analytics Engineer

I design end-to-end data pipelines, lakehouse architectures, and analytics-ready warehouses that hold up under pressure. From real-time ingestion and CDC to dbt transformations and ML workflows — I build systems you can trust.

View My Work Get in Touch

tfl_realtime_lakehouse — run log

London, UK First Class BSc · Birkbeck, University of London Python · SQL · dbt · Airflow · Spark

10+

Selected Builds

1st

Class Honours BSc

87%

Pipeline SLA Improvement

Years Experience

Adil Osman

Data Engineer · Analytics Engineer

Building pipelines
that hold up.

I'm a London-based data professional with a First Class BSc in Data Science & Computing from Birkbeck, University of London. My work focuses on data engineering and analytics engineering — designing pipelines that are reliable, observable, and deliver data that downstream consumers can actually trust.

I work with real-time ingestion, CDC architectures, local-first lakehouses, and dbt-driven warehouses across domains including finance, sport, urban transport, and environmental data.

More about me

Projects that ship.

Selected projects

Data Engineering · Real-time

TfL Real-Time Lakehouse

Real-time London transport data on your laptop. Airflow ingests TfL arrivals and stores them in Parquet. dbt and DuckDB transform the data, Great Expectations validates it, and OpenLineage tracks data lineage — a fully observable, production-pattern lakehouse without cloud dependency.

PythonApache AirflowdbtDuckDBParquetGreat ExpectationsOpenLineage

Data Engineering · CDC

StreamShop CDC Analytics Stack

Production-style Change Data Capture pipeline: Postgres → Debezium → Redpanda (Kafka API) → ClickHouse. Includes a synthetic e-commerce data generator, Python CDC sink, and dbt models with tests.

PythonDebeziumRedpandaClickHousedbtPostgreSQL

Machine Learning · Full-Stack

Deep Stock Insights

Full-stack ML platform predicting prices across 53 assets using N-HiTS, LightGBM, XGBoost, and Monte Carlo simulations. Live WebSocket streaming, React dashboard, and walk-forward backtesting.

PythonFastAPIReactTensorFlowLightGBM

Data in motion.

Every system I build follows the same discipline — from raw events to decisions people can trust.

01 / Ingest

Sources

APIs · Events · CDC

02 / Orchestrate

Pipelines

Airflow · Debezium

03 / Store

Lakehouse

Parquet · DuckDB

04 / Transform

Modelling

dbt · Tests · Lineage

05 / Serve

Insight

Warehouse · Dashboards

The tools I trust.

Python SQL dbt Core Apache Airflow Apache Spark DuckDB PostgreSQL BigQuery ClickHouse Kafka / Redpanda Debezium Great Expectations FastAPI PyTorch GitHub Actions

Full stack breakdown

AdilOsman

Building pipelinesthat hold up.