Senior Data Engineer

Luxoft
London

We're seeking a highly skilled and motivated Senior Data Engineer to join our growing data team. In this role, you'll architect and maintain robust, scalable data pipelines and infrastructure that power our analytics, machine learning, and business intelligence initiatives. You'll work with cutting-edge technologies like Python, PySpark, AWS EMR, and Snowflake, and collaborate across teams to ensure data is clean, reliable, and actionable.

Responsibilities

Build and maintain scalable ETL pipelines using Python and PySpark to support data ingestion, transformation, and integration

Develop and optimize distributed data workflows on AWS EMR for high-performance processing of large datasets

Design, implement, and tune Snowflake data warehouses to support analytical workloads and reporting needs

Partner with data scientists, analysts, and product teams to deliver reliable, well-documented datasets

Ensure data integrity, consistency, and accuracy across multiple sources and systems

Automate data workflows and processes to improve efficiency and reduce manual intervention

Monitor pipeline performance, identify bottlenecks, and resolve issues proactively

Apply best practices in CI/CD, version control (e.g., Git), and infrastructure-as-code (e.g., Terraform, CloudFormation)

Enforce data security, compliance, and governance standards in line with industry regulations

Mentor junior engineers, conduct code reviews, and foster a culture of continuous learning and knowledge-sharing

Skills

Must have

5+ years of experience in data engineering or software development

Strong proficiency in Python and PySpark

Hands-on experience with AWS services, especially EMR, S3, Lambda, and Glue

Deep understanding of Snowflake architecture and performance tuning

Solid grasp of data modeling, warehousing concepts, and SQL optimization

Familiarity with CI/CD tools (e.g., Jenkins, GitHub Actions) and infrastructure-as-code

Experience with data governance frameworks and security best practices

Excellent communication and collaboration skills

Nice to have

N/A

Other

Languages

English: C1 Advanced

Seniority

Senior

Posted 2025-09-24

Recommended Jobs

Hospitality Assistant

Restaurant Associates
London

Job Details We're looking for a Hospitality Assistant who will be the face of our business, delivering exceptional customer service for Restaurant Associates on a part time basis, contracted to 20…

View Details
Posted 2025-10-30

Steel Erector

Core Group
London

Core Group are currently seeking experienced Steel Erectors for an ongoing project in the London Area NW6  Job Title: Steel Erectors Job Type: Temporary Duration: Ongoing Location: London (N…

View Details
Posted 2025-11-24

HR ERP Transformation - Consultant/ Senior Consultant - London

Capgemini
London

GBS/ERP Transformation and Adoption Specialist - Senior Consultant - GlasgowReference Code: 346550-en_GBContract Type: PermanentProfessional Communities: Strategy & Transformation At Capgemini Inven…

View Details
Posted 2025-10-27

Senior Indirect Tax Manager - Powering the Future of Energy

Capital Tax Recruitment
London

A fantastic opportunity has opened for an Indirect Tax Senior Manager to join a rapidly expanding Energy & Resources team, providing expert advice on all aspects of indirect tax, including VAT, Custo…

View Details
Posted 2025-09-10

Home based Supervising Social Worker

NonStop Consulting
London

Home based Supervising Social Worker Location: London Salary: £39,000 + £3,400 car allowance per annum NonStop Consulting is working with a well established Independent Fostering Agency wi…

View Details
Posted 2025-11-24

Senior QA Automation Engineer

Intec Select Ltd
London

Senior QA Automation Engineer | £500 p/d Inside IR35 | London | Financial Services Overview We are seeking a Senior QA Automation Engineer to join a leading financial services organisation on a …

View Details
Posted 2025-11-24

Live In Carer

Active Care Group
Romford, Greater London

Job Details Would you like to do live in care, where you work an assignment for 2-4 weeks and then have a break? Would you like a rewarding role in which you can support a client one to one in t…

View Details
Posted 2025-11-06

School Business Manager | Wandsworth

Marchant Recruitment
London

A community primary school in Wandsworth is seeking a reliable and experienced School Business Manager to start in January 2026. The School Business Manager will take ownership of financial planning,…

View Details
Posted 2025-11-21

Assistant Restaurant Manager - GSQ

Rosewood Hotel Group
London

Press space or enter keys to toggle section visibility he Chancery Rosewood, a Rosewood Hotel Group property, is housed in the iconic former U.S. Embassy on Grosvenor Square, reimagined …

View Details
Posted 2025-10-27

Recycling Assistant - SC6

Hackney Council
Hackney, Greater London

Job Category: Admin / Clerical Location: Hackney Service Centre, Hackney Council Hours Per Week: 36.00 Start Date: Immediate Start Start Time: 09:00 End Time: 17:10 Salary: £17.2…

View Details
Posted 2025-09-10