Data Engineer

London United Kingdom Ref: 2030

Data Engineer — Supply Chain Analytics (Mid-Level)

ROLE OVERVIEW
We are looking for a mid-level Data Engineer to join a specialist team working on advanced supply chain analytics within a defence and critical infrastructure context. This role focuses on building, running, and maintaining large-scale data pipelines and numerical workflows that support sophisticated supply chain and network models. You will work closely with quantitative researchers, helping turn complex algorithms into robust, scalable production systems. The work is technically challenging, highly applied, and directly supports decision-making at a national level.

KEY RESPONSIBILITIES

• Develop and maintain Spark-based data pipelines using PySpark and Databricks
• Work with large datasets using Spark SQL, window functions, applyInPandas, and Delta Lake
• Implement numerical algorithms in Python, including fixed-point solvers, piecewise-linear interpolation, and binary search and optimisation routines
• Work closely with researchers to operationalise analytical methods
• Handle edge cases in numerical workflows such as convergence issues and tolerance thresholds
• Design fallback strategies when data or models behave unexpectedly
• Apply data cleansing and validation to ensure reliable downstream results
• Write clear documentation explaining data flows, assumptions, and outputs for non-technical stakeholders

PERSON SPECIFICATION

Essential

• 2–5 years’ experience as a data engineer, analytics engineer, or similar
• Strong experience with PySpark and Databricks, including Spark SQL, window functions, applyInPandas, and Delta Lake
• Solid Python skills for data and numerical work, including pandas and numpy
• Experience managing and running pipelines over large datasets
• Ability to reason carefully about numerical stability, edge cases, and data quality
• Clear written communication skills
• Eligible for UK Security Clearance (SC) — British nationality required (dual nationality acceptable; sole British preferred)

Desirable

• Experience working in a defence or defence-adjacent context
• Familiarity with defence procurement data such as NATO Stock Numbers or MOD systems
• Exposure to graph or network analysis

Candidate Profile

• Careful and rigorous in approach to data and numerical work
• Able to communicate technical concepts clearly to non-technical stakeholders
• Comfortable working closely with researchers in a small, expert team
• Motivated by high-impact work with real-world consequences

HOW TO APPLY To apply, please send your CV and a covering letter to Arran Goodridge via [email protected] quoting the role title in the subject line.

Salary: Competitive
Location: UK-based (hybrid working)