Projects that ship.

Data Engineering · Real-time

TfL Real-Time Lakehouse

Real-time London transport data on your laptop. Airflow ingests TfL bus and tube arrivals and stores them in Parquet. dbt and DuckDB handle transformation, Great Expectations validates every batch, and OpenLineage provides full data lineage tracking — a completely observable, production-pattern lakehouse with zero cloud dependency.

Problem

Live TfL API data is high-frequency, inconsistent, and hard to trust without repeatable validation.

Build

Airflow ingestion, Parquet lake storage, DuckDB/dbt modelling, Great Expectations checks, and OpenLineage tracking.

Result

A local-first lakehouse pattern that demonstrates production data engineering without cloud spend.

Architecture

TfL API

Airflow DAGs

Parquet Lake

dbt + DuckDB

Quality + Lineage

PythonApache AirflowdbtDuckDBParquetGreat ExpectationsOpenLineageTfL API

Read case study Source code

Data Engineering · CDC

StreamShop CDC Analytics Stack

Production-style Change Data Capture pipeline: Postgres → Debezium → Redpanda (Kafka-compatible API) → ClickHouse. Includes a synthetic e-commerce event generator, Python CDC sink, and dbt models with tests for analytical queries on captured changes.

Problem

Analytical stores need fresh operational changes without brittle batch exports or manual reloads.

Build

A CDC chain from Postgres WAL events through Debezium and Redpanda into ClickHouse marts.

Result

A reproducible streaming analytics stack with generated events, sink code, dbt tests, and query-ready facts.

Architecture

Postgres

Debezium

Redpanda

Python Sink

ClickHouse + dbt

PythonDebeziumRedpandaClickHousedbtPostgreSQL

Source code

Data Engineering · Environmental

London Air Quality & Weather Lakehouse

Local-first lakehouse ingesting OpenAQ air quality and Open-Meteo weather data. Airflow orchestrates ingestion, raw JSON lands in MinIO (S3-compatible), and dbt applies incremental models with full validation in PostgreSQL.

Problem

Environmental analysis needs consistent station, pollutant, and weather data from separate APIs.

Build

Airflow ingestion into S3-compatible MinIO, PostgreSQL serving tables, and incremental dbt models.

Result

A portable lakehouse that joins air quality readings with weather context for repeatable analysis.

Architecture

OpenAQ

Open-Meteo

Airflow

MinIO + Postgres

dbt Marts

PythonApache AirflowdbtPostgreSQLMinIO (S3)OpenAQ API

Source code

Analytics Engineering · Sports

F1 RaceOps Analytics Warehouse

Formula 1 analytics warehouse built on an Ergast-compatible schema, converting raw race timing data into KPIs for pit stop performance, strategy effectiveness, and reliability. Dimensional modelling with dbt and full CI via GitHub Actions.

PythondbtSQLGitHub ActionsErgast API

Analytics Engineering · Football

Premier League Analytics Warehouse

End-to-end football analytics warehouse on BigQuery, built with dbt and GitHub Actions CI. Covers Premier League seasons 2014–15 to 2024–25 from openfootball JSON, modelled into analytics-ready dimensional tables.

dbtBigQueryGitHub ActionsSQLPython

Data Engineering · Lakehouse

Mini Lake — dbt + DuckDB

Local-first, portable lakehouse demo using dbt and DuckDB. Features versioned seeds, staging-to-marts models, schema tests, auto-generated documentation, and a GitHub Actions CI example — ideal for learning and prototyping analytics engineering patterns.

dbtDuckDBPythonGitHub ActionsSQL

Machine Learning · Full-Stack · Finance

Deep Stock Insights — Financial Prediction Platform

Full-stack platform combining N-HiTS neural forecasting, LightGBM, XGBoost, and Monte Carlo simulations to predict prices across 53 assets (crypto, equities, commodities). Features live WebSocket price streaming, 18+ technical indicators, walk-forward backtesting, AI-powered buy/sell signal ranking, and an interactive React dashboard with JWT authentication.

Problem

Financial ML projects often overfit in notebooks and fail to prove how predictions would behave over time.

Build

A full-stack forecasting system with streaming prices, feature pipelines, ensemble models, backtesting, and a React UI.

Result

A production-style dissertation system showing modelling, serving, evaluation, and user-facing delivery together.

Architecture

Market Data

Feature Pipeline

Forecast Models

FastAPI

React Dashboard

PythonFastAPIReactTensorFlow / KerasLightGBMXGBoostScikit-learnTailwind CSS

Source code

Machine Learning · Reinforcement Learning

AI Trading Agent

Autonomous swing trading system combining NHITS, TFT, and LightGBM forecasting with PPO/DQN reinforcement learning and five rule-based strategies. Includes FinBERT sentiment scoring, ATR-based position sizing, risk controls (daily loss limits, kill switch), and Dockerised deployment with Alpaca paper trading.

PythonPyTorchFastAPILightGBMFinBERTDockerSupabaseAlpaca API

Machine Learning · Finance

Stock Market Prediction with LSTM Networks

End-to-end ML pipeline for financial time-series forecasting using LSTM deep learning. Achieves an R² of 0.95, mean absolute error of $2.4, and 87% directional accuracy. Covers data ingestion, feature engineering (technical indicators, rolling stats, lag features), LSTM architecture design, hyperparameter tuning, and rigorous backtesting methodology.

PythonTensorFlow / KerasLSTMPandasNumPyScikit-learnMatplotlib

Data Science · ML

Missing Data — Imputation Techniques

Interactive notebook clarifying MCAR, MAR, and MNAR missingness mechanics; NumPy masked arrays; and modern imputation using scikit-learn KNN and Iterative/MICE imputation. Includes reproducible experiments and visual analysis.

PythonScikit-learnNumPyJupyter

Data Science · Statistics

Chebyshev, LLN & CLT — Probability Fundamentals

Notebook exploring core probability theory: binomial distributions, the Law of Large Numbers, the Central Limit Theorem, and Chebyshev's inequality with Uniform/Normal/Gamma tail comparisons — all implemented and visualised from scratch.

PythonNumPyMatplotlibJupyter

Data Science · Statistics

Linear Regression from Scratch

Implements OLS linear regression from the ground up — slope/intercept derivation, loss function, gradient descent, and model evaluation with visualisation. Demonstrates core ML foundations without library abstractions.

PythonNumPyMatplotlibJupyter