Lead Data Engineer (Data Science Team)

The Role

We are looking for a skilled and experienced Lead Data Engineer to join our Data Science team. The team ingests large amounts of complex sensor data (billions of data points a day), combines it with data from other teams, and produces advanced modelling products that help people park their car or charge their electric vehicle. For example, we predict the availability of parking in cities across the world and provide drivers with routes that reduce the time they will spend searching for a space near their destination. These machine learning models are high-quality production services and are updated regularly using fresh data.

You will lead the design, development, and enhancement of pipelines to ingest and process streaming data for use in our machine learning models. You will be an important member of our team, lead engineering initiatives and work with smart colleagues in a supportive environment.

Responsibilities

You will develop pipelines for scalable big data processing with Spark, and real-time data streaming with Kafka. These pipelines will need to be written using efficient, testable, and reusable Python code using (for example) Numpy, Pandas and Pyspark. We manage our numerous pipelines using Airflow to meet our data serving and modelling requirements. Our services are reliable, robust, and follow industry best practice in data validation, transformation, and logging. We are hands-on with our infrastructure and cloud deployments.

We are also looking for this position to lead initiatives enhancing our processes and infrastructure. The areas for these improvements could be our CI/CD pipelines, our data monitoring capabilities, or our feature stores. We are always looking for new senior engineers to use their experience to promote best practices amongst our data scientists and junior engineers. Although the work is quite autonomous, we value working in a team and like to collaborate and support each other in any way we can.

Requirements

• Proven experience as a software or data engineer in complex production environments.

• High proficiency in Python, including software development standards and knowledge of the Python data science / engineering ecosystem (e.g. Numpy, Pandas).

• Strong command of Linux, containers (Docker), and infrastructure as code for cloud deployments (AWS preferred).

• Comfortable leading initiatives and mentoring others.

• Experience with:

  • Large-scale data processing in the cloud (we use AWS).

  • Distributed processing frameworks, such as Apache Spark.

• Desirable, experience with:

  • Workflow management tools, such as Apache Airflow.

  • Streaming data processing, such as Apache Kafka.

  • Data or ML platforms, such as Snowflake or Databricks.

Benefits

You can expect great employment benefits such as:

• Flexible working - hybrid home and office-based opportunities.

• Paid Leave if you participate in an event for Charity.

• 25 Days holiday entitlement.

• An enhanced Workplace Pension Scheme - 5% by Arrive, 3% by you.

• Private Medical Health Insurance.

• Fantastic wellbeing programmes, including On-site Sports massages, Reiki and Head massages every week.

• Discounted gym membership.

• Access to Blue Call, a mental health support platform.

• Enhanced Maternity and Paternity offering.

About us

We’ve signed up to an ambitious journey. Join us!

As Arrive, we guide customers and communities towards brighter futures and more livable cities, it isn’t a challenge just anyone could take on. Luckily, we have something to help us make it happen. Our people and our values. We Arrive Curious, Focused and Together. Just as our entire brand is inspired by the North Star, the shining light leading travelers to their destinations since time began, our values guide us. They help us be at our best. For our customers. For the cities and communities we serve. For ourselves. As a global team, we are transforming urban mobility. Let’s grow better, together.

One of the key brands within the Arrive is Parkopedia.

Parkopedia is the world’s leading connected car services provider, used by millions of drivers and organisations such as Apple, Here, TomTom, and 20 automotive brands ranging from Audi to Volkswagen. Its mission is to provide the best in-car data and transaction services, to make mobility ecological, efficient and convenient.