About

Building pipelines
that hold up.

London-based Data Engineer and Data Scientist. First Class BSc, Birkbeck, University of London.

Adil Osman at his Birkbeck graduation ceremony
Adil Osman
Data Engineer · Analytics Engineer

The work,
the thinking.

I am a data professional based in London, holding a First Class BSc in Data Science & Computing from Birkbeck, University of London. My experience of studying part-time while developing real-world projects has significantly influenced my engineering approach: I prioritise deliberate, methodical processes, and I maintain a strong emphasis on creating systems that are resilient under pressure.

My expertise lies in data engineering and analytics engineering, where I focus on building reliable, observable pipelines and systems that generate trustworthy data for informed decision-making. I specialise in the entire pipeline lifecycle, from raw data ingestion and orchestration (using tools like Airflow and CDC with Debezium/Redpanda) to transformation and warehousing (leveraging dbt, DuckDB, and BigQuery), as well as data quality enforcement (with Great Expectations) and lineage tracking (utilising OpenLineage).

In the realm of data science, I leverage machine learning to deliver genuine predictive insights. My focus includes financial time-series forecasting utilising LSTM networks, statistical modelling, and comprehensive model evaluation. I approach machine learning as an engineering discipline, emphasising reproducible workflows, transparent backtesting, and results that can be confidently endorsed.

My work spans domains such as finance, sports, and urban/environmental data, where data quality and pipeline reliability are paramount. My projects encompass real-time data from Transport for London (TfL), analytics warehouses for the Premier League and Formula 1, air quality monitoring in London, and equity price forecasting.

Data Engineering Analytics Engineering Real-time Pipelines CDC / Streaming dbt & Warehousing Data Observability Machine Learning Financial Data Sports Analytics Urban & Environmental

Top grades from
Birkbeck.

94%
Software & Programming I
2022/23 · Year 2
90%
Database Management
2024/25 · Final Year
84%
Data Structures & Algorithms
2022/23 · Year 2
84%
Artificial Intelligence & Machine Learning
2024/25 · Final Year
84%
Software & Programming II
2023/24 · Year 3

First Class Honours · BSc Data Science & Computing · Birkbeck, University of London

Where I've worked.

Apr 2024 — Jul 2025 · 1 yr 4 mos
Junior Data Engineer & Peer Mentor
Somalis in Tech · Full-time · London, UK · Hybrid

Designed and maintained production-grade end-to-end data pipelines to ingest, clean, and model member and event engagement data using Python, SQL, dbt, and Apache Airflow — improving data freshness from 6 hours to 45 minutes (87% improvement) and ensuring 95% of loads met a 1-hour freshness SLA.

Automated data collection and reporting workflows for workshops, hackathons, and startup events, eliminating ~65% of manual reporting tasks and expediting time-to-insight for weekly stakeholder updates.

Developed self-service dashboards and reporting packages highlighting KPIs — attendance trends, participation rates, and program impact — reducing manual reporting by ~18 hours per month.

Partnered with cross-functional leaders to establish metrics and enforce reporting SLAs; achieved 99% on-time weekly delivery with end-to-end pipeline latency under 60 minutes at the 95th percentile, contributing to a 22% increase in attendance for flagship programs.

Provided structured 1-on-1 mentoring through the Caawi Mentorship Platform, guiding cohorts of ~12 early-career candidates in data engineering, portfolio development, and job readiness. Built the TfL Real-Time Lakehouse as a community teaching tool, giving users and students hands-on experience with production-style data pipelines — improving their confidence and understanding of real-world data engineering.

PythonSQLdbtApache AirflowMySQLData PipelinesMentoring
Jun 2023 — Feb 2024 · 9 mos
Junior Data Analyst
HQ Analytics · Full-time · Dubai, UAE · Remote

Pre-processed large-scale, multi-source datasets using Python and SQL, implementing deduplication and schema validation to reduce data defects by 30% and cut data preparation time from 10 hours to 5 hours per reporting cycle.

Automated data extraction and established daily refresh pipelines, improving stakeholder turnaround from 3 days to same-day delivery — a 67% efficiency gain.

Conducted comprehensive EDA to identify key operational drivers — lead-time variance, fulfilment delays, returns, and margin leakage — producing 12+ actionable recommendations adopted by stakeholders to drive data-driven decision-making.

Applied feature engineering techniques (lateness flags, supplier segmentation) to enhance signal separation between high- and low-performing suppliers by 15%.

Built interactive Power BI KPI dashboards for real-time monitoring, reducing manual reporting by 16 hours per month, growing adoption to 25+ users across operations and commercial teams, and contributing to an 8% reduction in logistics costs and 12% increase in on-time delivery.

PythonSQLPower BIEDAFeature EngineeringMySQLReporting

Education & projects.

2021 — 2025
First Class BSc Data Science & Computing
Birkbeck, University of London
Undergraduate degree (part-time) combining statistical foundations, machine learning, software engineering, and data systems — awarded First Class Honours. Achieved standout results in Database Management (90%), Software & Programming (94%), and Artificial Intelligence & Machine Learning (84%). Final-year dissertation, Deep Stock Insights, applied LSTM deep learning to financial time-series forecasting — an end-to-end data science system built for production quality. Also served as a final-year peer mentor through BBK Peer Mentoring.
First Class HonoursMachine LearningStatisticsDatabasesPythonSQLSoftware Engineering
2024/25 · Final Year Project
Deep Stock Insights — Stock Market Prediction
Birkbeck, University of London · Grade: 67%
Final-year dissertation applying LSTM deep learning networks to financial time-series forecasting. Built an end-to-end data science system covering data collection, preprocessing, model training, evaluation, and prediction — designed with production-quality engineering practices throughout.
LSTMDeep LearningTime-Series ForecastingPythonFinancial Data
Ongoing
Independent Data Engineering Projects
Self-directed — London
Building and publishing production-grade systems: a real-time TfL lakehouse with Airflow, dbt, DuckDB, Great Expectations, and OpenLineage; a CDC analytics stack using Debezium, Redpanda, and ClickHouse; analytics warehouses on BigQuery for Premier League and F1 data; an air quality lakehouse with MinIO; a full-stack financial prediction platform (Deep Stock Insights) combining N-HiTS, LightGBM, and XGBoost across 53 assets; an autonomous AI trading agent using reinforcement learning (PPO/DQN) with FinBERT sentiment analysis; and an LSTM-based financial forecasting pipeline. Focused on production patterns — observability, data quality, CI, and engineering discipline.
dbtAirflowDuckDBBigQueryDebeziumClickHouseFastAPIReactPyTorchGitHub Actions
Community
Member — Somalis in Tech
London
Active member of Somalis in Tech, a community supporting Somali professionals and students in the technology industry. Co-built the community's TfL Travel App — a real tool used by London students and commuters.
Read the case study