Senior ML Systems Engineer - Simulations
We are looking for a Senior ML Systems Engineer to build and validate simulation infrastructure for large-scale machine learning systems. This role focuses on modelling the compute and communication behaviour of systems used for ML training and inference, and using simulation to guide architecture, performance optimization, and capacity planning.
The ideal candidate combines strong systems experience with hands-on experience in measurement, benchmarking, and performance analysis of modern ML systems.
What You’ll Do:
Build simulation models for compute, memory, interconnect, and communication behavior in ML systems.
Develop tools to simulate performance for training and inference workloads.
Model distributed execution across accelerators, hosts, and network fabrics, including collectives, synchronization, and communication bottlenecks.
Use simulation and analytical modelling to evaluate tradeoffs, identify bottlenecks, and guide system design.
Run performance experiments and benchmarks on real ML systems to calibrate and validate simulation models.
Analyze end-to-end performance, including throughput, latency, scaling efficiency, utilization, and cost/performance tradeoffs.
Partner with hardware/software/Networking/ML teams to align simulation with real workloads and constraints.
Create reproducible benchmarking methodologies across models, system configurations, and compare against real system measurements to prove validity.
Communicate findings through technical reports and design recommendations.
Qualifications
Required:
Master’s, or PhD in Computer Science, Electrical Engineering, Computer Engineering, or a related field.
Strong experience in ML systems, distributed systems, performance engineering, computer architecture, or simulation.
Understanding of systems used for machine learning training and inference.
Experience analyzing compute, communication, and memory behavior in large-scale ML systems.
Hands-on experience with performance benchmarking, profiling, and measurement of ML systems.
Experience with distributed training concepts such as data parallelism, tensor/model parallelism, pipeline parallelism, collectives, and synchronization overheads.
Proficiency in one of the following Python, C++, or Rust.
Strong analytical skills and the ability to connect simulation results to real system behavior.
Preferred:
Experience with system performance modelling, network simulation, or architecture evaluation tools. - this background is ideal
Familiarity with accelerator-based systems such as GPUs, TPUs, or custom ML hardware.
Experience with PyTorch, JAX, TensorFlow, NCCL, XLA, CUDA, or similar tools.
Knowledge of interconnect and networking technologies such as InfiniBand, Ethernet/RDMA, NVLink, PCIe, or equivalent.
Experience evaluating both training throughput and inference latency/serving efficiency.
Background in workload characterization, trace-driven simulation, or model calibration.
Ability to work across hardware and software boundaries in a cross-functional environment.
What Success Looks Like:
Build simulation models that accurately predict performance trends and inform architectural decisions.
Identify compute and communication bottlenecks in ML training and inference systems.
Correlate simulation outputs with real-world benchmark data.
Improve system efficiency, scalability, and cost effectiveness through data-driven insights.
Recommended Jobs
Senior Strategist
Who are Coolr? We’re an independent social media agency and team of creatives, social experts, content publishers and change makers. Completely wired into popular culture, our work connects brands …
Science Technician - London
A well-resourced secondary school in London is seeking a reliable and organised Science Technician to support the Science department. The Role You will prepare materials and equipment for pract…
NPL - Senior Investment Manager
Lead the management and optimisation of non-performing loan (NPL) portfolios. Develop and execute investment strategies to maximise portfolio returns. Conduct detailed financial analysis and du…
Live Out Nanny Job in London
My Requirements Babysitter Live Out Nanny
Chemistry ECT Role | Outstanding School in Camden
A prestigious, Outstanding Ofsted-rated secondary school and sixth form in Camden, Central London, known for its academic rigour and strong university placement record in STEM, is seeking an Early Ca…
Subrogation Claims Adjuster (Solicitor)
Our growing London based client have a new opening for a Solicitor to join their subrogation claims team. You will liaise with several prestigious clients predominantly handling Property subrogation c…
Work from home as a GCSE tutor - Part Time, Flexible...
We are looking for an innovative and dedicated GCSE Tutor to join our excellent team of tutors in the UK. This is a great opportunity to provide online GCSE tutoring and help students improve the…
Area Sales Director
About the opportunity The position of the Area Sales Director Mid-Market EMEA (f/m/d) is a critical role in our EMEA Sales team and will be reporting to the Senior Vice President (SVP) of EMEA Sal…
KS2 Teacher — Southwark — January 2026 start
Are you an accomplished KS2 Teacher looking for a stimulating Full-Time role from January 2026? A vibrant Southwark primary seeks a talented KS2 Teacher to join its upper/lower KS2 provision. The KS2…
Experience Butler (Hiring Immediately)
As the Experience Butler onboard the British Pullman, A Belmond Train, you oversee and deliver a seamless, luxury experience for Celia’s private dining guests on board, coordinating personalised arra…