Backend Software Engineer - Search, Crawler Team (London, Belgrade, Berlin)

Perplexity
London
We are seeking an experienced Backend Software Engineer to join our Crawler team. In this role, you will design, develop, and operate systems that ingest, process, and manage web-scale data in support of our next generation of advanced search technologies. This is a critical, high-impact engineering position, requiring expertise across both backend and frontend components of our data acquisition stack.

Responsibilities

  • Take ownership of and lead projects focused on developing large-scale web crawlers, ingestion pipelines, and data processing systems.
  • Build, maintain, and optimize core backend and frontend components for crawler services, including storage, retrieval, and UI dashboards for data management.
  • Collaborate closely with Search and Infrastructure teams to ensure the reliable, high-quality ingestion and processing of billions of web pages.
  • Architect and implement fullstack features and scalable distributed systems that handle high-load and real-time data operations.
  • Rapidly iterate, experiment with novel approaches, and continuously enhance system performance, usability, and reliability.

Qualifications

  • Minimum of 5 years of software development experience, with strong knowledge of data structures and algorithms in at least one of the following languages: Python, C++, Rust, or Go.
  • Experience with large-scale web crawlers is highly desirable.
  • Proven experience building, deploying, and optimizing high-load, distributed, and hardware-adjacent services.
  • Deep understanding of cloud infrastructure, with hands-on experience in Kubernetes (K8s) and AWS.
  • Demonstrated passion for writing clean, efficient, and scalable systems.

Posted 2025-12-27

Recommended Jobs

English ECT - Independent School, Islington

Marchant Recruitment
London

A prestigious Independent School in Islington is seeking a permanent, full-time Early Career Teacher (ECT) of English, starting January 2026. Start your career in an intellectually vibrant English de…

View Details
Posted 2025-10-24

Solution Design Associate Director

Novartis
London

Summary The Solution Design Associate Director is a visionary accelerator of standard practice, driven through the combination of creative-thinking and business analytics, and is accountable for d…

View Details
Posted 2025-12-27

HR Analyst

Bruin Financial
London

We are working alongside a growing Lloyd’s Broker who are looking for a highly experienced Human Resources Analyst as they enter an incredibly busy period in the HR lifecycle. The opportunity will …

View Details
Posted 2025-09-12

Team Member London Fenchurch Street (N107444)

Next Careers
London

Description SHIFTS YOU ARE APPLYING FOR: 10.75hrs p/w; Mon 10:45 - 13:45; Thu 10:45 - 14:30; Fri 15:00 - 19:00 Working as part of a fast-paced store team your top priority will be to offer out…

View Details
Posted 2025-11-24

Banking Job - Cantonese speaking Banking Assistant Customer Service Officer - London - rj

People First Recruitment
Central London

Please follow us on WeChat to see all our Cantonese and Mandarin jobs, interview tips and London news: Your New Job Title: Cantonese speaking Banking Assistant Customer Service Officer , Londo…

View Details
Posted 2025-10-09

Confirmed Data Analyst - Revenue Operations

Spendesk
London

We are seeking a data analyst focused on Go-to-Market topics to join our team and revolutionise how we deliver value across our entire customer lifecycle through advanced analytics and AI-powered ins…

View Details
Posted 2025-11-15

Cruise Consultant

Platinum Travel Recruitment Ltd
East London

Are you passionate about cruise holidays and looking to take the next step in your travel career? We’re proud to be partnering with a highly respected, established travel company who are now look…

View Details
Posted 2025-09-30

Geography ECT - Brent Outstanding Academy

Marchant Recruitment
Brent, Greater London

School Status & Location Sector: Outstanding Ofsted-rated Secondary Academy (Global Citizen Centre). Borough: Brent (Outer London). Start Date: Permanent, full-time role commencing January 2…

View Details
Posted 2025-11-22

EYFS SEN Teaching Assistant | Supportive Primary School...

Marchant Recruitment
Brent, Greater London

ASAP Start | Long-Term Role | £90–£110 per day Are you passionate about Early Years education and eager to support children who require additional help? This friendly Brent primary school is looki…

View Details
Posted 2025-12-09

Sales Executive

Enfield, Greater London

Job Location – Enfield,  Salary – £24,000 Gurantee earnings for the first 3 months, OTE will be £50,000 What you will be doing: As a Sales Executive, you will be responsible for every as…

View Details
Posted 2025-09-11