Senior Data Engineer
We're seeking a highly skilled and motivated Senior Data Engineer to join our growing data team. In this role, you'll architect and maintain robust, scalable data pipelines and infrastructure that power our analytics, machine learning, and business intelligence initiatives. You'll work with cutting-edge technologies like Python, PySpark, AWS EMR, and Snowflake, and collaborate across teams to ensure data is clean, reliable, and actionable.
Responsibilities
Build and maintain scalable ETL pipelines using Python and PySpark to support data ingestion, transformation, and integration
Develop and optimize distributed data workflows on AWS EMR for high-performance processing of large datasets
Design, implement, and tune Snowflake data warehouses to support analytical workloads and reporting needs
Partner with data scientists, analysts, and product teams to deliver reliable, well-documented datasets
Ensure data integrity, consistency, and accuracy across multiple sources and systems
Automate data workflows and processes to improve efficiency and reduce manual intervention
Monitor pipeline performance, identify bottlenecks, and resolve issues proactively
Apply best practices in CI/CD, version control (e.g., Git), and infrastructure-as-code (e.g., Terraform, CloudFormation)
Enforce data security, compliance, and governance standards in line with industry regulations
Mentor junior engineers, conduct code reviews, and foster a culture of continuous learning and knowledge-sharing
Skills
Must have
5+ years of experience in data engineering or software development
Strong proficiency in Python and PySpark
Hands-on experience with AWS services, especially EMR, S3, Lambda, and Glue
Deep understanding of Snowflake architecture and performance tuning
Solid grasp of data modeling, warehousing concepts, and SQL optimization
Familiarity with CI/CD tools (e.g., Jenkins, GitHub Actions) and infrastructure-as-code
Experience with data governance frameworks and security best practices
Excellent communication and collaboration skills
Nice to have
N/A
Other
Languages
English: C1 Advanced
Seniority
Senior
Recommended Jobs
Hospitality Assistant
Job Details We're looking for a Hospitality Assistant who will be the face of our business, delivering exceptional customer service for Restaurant Associates on a part time basis, contracted to 20…
Steel Erector
Core Group are currently seeking experienced Steel Erectors for an ongoing project in the London Area NW6 Job Title: Steel Erectors Job Type: Temporary Duration: Ongoing Location: London (N…
HR ERP Transformation - Consultant/ Senior Consultant - London
GBS/ERP Transformation and Adoption Specialist - Senior Consultant - GlasgowReference Code: 346550-en_GBContract Type: PermanentProfessional Communities: Strategy & Transformation At Capgemini Inven…
Senior Indirect Tax Manager - Powering the Future of Energy
A fantastic opportunity has opened for an Indirect Tax Senior Manager to join a rapidly expanding Energy & Resources team, providing expert advice on all aspects of indirect tax, including VAT, Custo…
Home based Supervising Social Worker
Home based Supervising Social Worker Location: London Salary: £39,000 + £3,400 car allowance per annum NonStop Consulting is working with a well established Independent Fostering Agency wi…
Senior QA Automation Engineer
Senior QA Automation Engineer | £500 p/d Inside IR35 | London | Financial Services Overview We are seeking a Senior QA Automation Engineer to join a leading financial services organisation on a …
Live In Carer
Job Details Would you like to do live in care, where you work an assignment for 2-4 weeks and then have a break? Would you like a rewarding role in which you can support a client one to one in t…
School Business Manager | Wandsworth
A community primary school in Wandsworth is seeking a reliable and experienced School Business Manager to start in January 2026. The School Business Manager will take ownership of financial planning,…
Assistant Restaurant Manager - GSQ
Press space or enter keys to toggle section visibility he Chancery Rosewood, a Rosewood Hotel Group property, is housed in the iconic former U.S. Embassy on Grosvenor Square, reimagined …
Recycling Assistant - SC6
Job Category: Admin / Clerical Location: Hackney Service Centre, Hackney Council Hours Per Week: 36.00 Start Date: Immediate Start Start Time: 09:00 End Time: 17:10 Salary: £17.2…