Data Engineer
- Data Pipeline Development: Build and maintain scalable data pipelines using PySpark notebooks in Azure Databricks and Microsoft Fabric. Automate ingestion of raw data from various sources into our lakehouse, ensuring reliability and performance.
- Medallion Architecture: Support the development and maturity of our bronze-silver-gold layer architecture, ensuring clean separation of raw, refined, and curated data.
- Dimensional Modelling: Apply star schema and other dimensional modelling techniques to transform data into analytics-ready structures.
- Collaboration: Work closely with analytics, insight, and business teams to understand data requirements and deliver solutions that meet their needs.
- Governance & Lineage: Implement and maintain data governance practices, including data lineage, documentation, and metadata management.
- Quality Assurance: Conduct data validation, testing, and monitoring alongside wider testing teams to ensure pipeline integrity and model accuracy.
- 3-5 years' experience applying relational data modelling and data warehousing techniques, including dimensional modelling (e.g. star schema and slowly changing dimensions) is essential to the role.
- Technical experience building and deploying models using PySpark and Python in a data warehouse or data lakehouse environment.
- Exposure to Delta Lake, lakehouse tables, or similar technologies - familiarity with Azure Databricks or Microsoft Fabric in particular is advantageous.
- Experience with SQL for data transformation and querying.
- Experience with Git and version control in a collaborative environment.
- Familiarity with CI/CD pipelines for data workflows.
- Awareness of data governance principles and tools (e.g., Purview, Unity Catalog, Fabric Data Governance).
- An understanding of the structure and purpose of the Financial Advice and Wealth Management markets within the UK Financial Services sector is highly advantageous.
- Knowledge of the Agile methodology would be beneficial.
- No specific qualifications are required for this role; however, the successful candidate will be expected to complete the Microsoft Certified: Fabric Data Engineer Associate certification within their probation period (6 months).
- Excellent numerical skills are essential.
- Strong problem-solving and analytical thinking.
- Willingness to learn and grow in a fast-paced environment.
- Good communication skills and ability to work collaboratively.
- Attention to detail and commitment to data quality.
Recommended Jobs
Project Manager / Business Analyst (Contractor)
Project Manager / Business Analyst (Contractor) Contract: 3–4 days per week, initial 3-month contract (with potential extension) Location: Dalston Office (at least 1 day per week, travel costs c…
Quality Manager-Pathology
Quality Manager (3 days a week) - Pathology Service s We have an exciting opportunity for a n experienced Quality Manager to join our Pathology service based at The London Clinic in Central Lo…
Nursery Practitioner Level 3 - Part Time
Role Overview: Join Our Team at Busy Bees – Leading Nursery Group in the UK Busy Bees is looking for passionate Level 3 Nursery Practitioners to join our team on a part-time basis , working 2–…
After-school Nanny for a busy family, Job ID J1D38D
A lovely family based in Wimbledon, London, is seeking an After-school Nanny to care for their two school-aged children. The role includes all general nanny duties, with light housekeeping and cookin…
IT Support Specialist
New Job Opportunity - IT Support Specialist - Initial 12 Month PAYE Contract Our Client has a requirement for an IT Support Specialist , who will be required to work on a Contract basis in King…
Hospitality Leader
Are you a hospitality leader looking for a career change? This is your chance to step into a growing sector with one of the UK’s largest charities that can really develop your career. What’s more, it’…
Principal Product Manager
We are looking to hire a Principal Product Manager with deep experience in building and scaling SaaS products for global enterprise customers. The ideal candidate will have a proven track record of ow…
CRM Insight & Data Planning Manager
CRM Insight & Data Planning Manager Currys London Campus at Waterloo Permanent Full Time Grade 4 We’re iD Mobile, one of the UK’s leading mobile virtual network operators. We launc…
Quant Linux Systems Engineer
Quant Linux Systems Engineer £120,000 – £160,000 Quant Capital is urgently looking for a talented Senior Linux Engineer to join our client’s Quant Systems team in London. Our client is a well-…
Corporate Insurance Associate, London, 3-5 PQE
Our US firm client's Financial Institutions Group is seeking a talented mid-level associate with 3-5 years' PQE to join their market-leading Corporate Insurance practice in London. The team advise…