Lead Data Engineer

Job summary

Can you lead secure, production-grade data pipelines on GCP while balancing live operations and innovation?

Do you enjoy mentoring engineers and translating complex data engineering concepts for diverse stakeholders?

If so, we'd love to hear from you!

In recent years DfT’s digital and data teams have implemented a range of advanced data services, making use of the latest cloud technologies to deliver the services and platforms that our users need, with excellent customer satisfaction rates. We are proud of our ability develop and grow as a team, and we look forward to you sharing that sense of pride!

At DfT, we recognise that everyone has different needs and aspirations. We have created an inclusive and welcoming working environment so you can feel comfortable to be yourself at work. We’ll help you to reach your full potential, offering rewarding opportunities alongside access to the latest training and technologies.

Joining our department comes with many benefits, including:

• Employer pension contribution of 28.97% of your salary. Read more about Civil Service Pensions here
• 25 days annual leave, increasing by 1 day each year of service (up to a maximum of 30 days annual leave), plus 8 bank holidays a privilege day for the King’s birthday
• Flexible working options where we encourage a great work-life balance.

Read more in the Benefits section below!

Find out more about what it's like working at DfTc: Department for Transport Central - Department for Transport Careers

Job description

Working as part of a talented and collaborative team, you will:

• Lead the build and operation of DfT’s production‑grade data pipelines and platforms, ensuring reliability and security across our Google Cloud Platform environment.
• Own and manage live data services, triaging and resolving issues at pace to maintain high‑quality data delivery for analysts, policy teams and external commitments.
• Drive innovation within data engineering, identifying opportunities to modernise tooling, adopt emerging GCP capabilities and introduce new approaches that improve efficiency and data quality.
• Plan delivery across legacy migration, operational support and new development, ensuring that resources are allocated effectively and that risks, dependencies and priorities are well managed.
• Work closely with technical and non-technical stakeholders, translating technical concepts, shaping data‑related decisions, and responding to business need.
• Line manage and develop engineers at varying levels, providing technical guidance, coaching and oversight, and fostering a culture of continuous improvement, collaboration and knowledge‑
• Drive adoption of Infrastructure as Code (IaC), establishing repeatable patterns for environments, access, and data services.
• Lead the development of our metadata catalogue, curating business and technical metadata so users can effectively discover and use data.

In return, we can offer you:

• access to new and emerging technologies,
• varied projects developed in a cloud-first environment,
• support and investment to further your training and development,
• flexible and hybrid working supporting a healthy work-life balance,
• industry-leading pension and employee benefits package.

For further information on the role, please read the role profile. Please note that the role profile is for information purposes only - whilst all elements are relevant to the role, they may not all be assessed during the recruitment process. This job advert will detail exactly what will be assessed during the recruitment process.

About Us

At the heart of data innovation and evolution in DfT, you will join a talented, experienced, data engineering team imagining and shaping the delivery of the next wave of data services. The team is embedded within the wider data directorate, and works alongside analysts, data scientists, architects and other engineers to deliver some of the most impactful data projects within DfT. You will support and shape various areas within the business which delivers an innovational transport policy agenda. As DfT is a cloud-only enterprise, you will develop the latest cloud solutions meeting complex digital, identity and data needs.

This role will give you the opportunity to share your experience and further develop your skills every day as you work on new and exciting projects with advanced technologies. We provide a supportive and constructive learning environment where your career growth is important.

Person specification

You will be an experienced data engineer with deep technical foundations and expertise in both Python and SQL. You will also be highly proficient in Google Cloud Platform, or an expert user of AWS or Azure with a willingness to apply your skills to a new cloud platform. You combine hands on engineering excellence with the ability to communicate complex ideas simply, engaging effectively with a wide range of technical and non technical stakeholders. You are comfortable balancing the demands of operating reliable, production grade data services with delivering innovation: shaping new approaches, modernising legacy systems, and driving improvements in data quality and tooling. Alongside this, you bring thoughtful planning and change management skills, helping the organisation evolve its data capabilities while ensuring continuity, stability, and high quality outcomes across DfT.

You will need to demonstrate the following experience:

• Enterprise-scale delivery of robust, maintainable data pipelines in Google Cloud Platform. Including building reusable components, optimising performance and cost on cloud data platforms while meeting security, privacy and governance controls.
• Expert user of data engineering tools including relevant languages (e.g. Python/SQL), IaC tools (e.g. terraform), GCP or equivalent cloud tooling or (e.g. BigQuery, Cloud Functions) CI/CD (e.g. Github Actions), logging and testing.
• Setting and leading engineering standards, ensuring high‑quality coding practices, maintainable solutions, and consistent technical approaches across the team.
• Leading Agile delivery of data engineering work, managing and operating live services at pace to maintain continuity and high‑quality data delivery into DfT, while also driving innovation by developing new data engineering approaches and patterns.
• Stakeholder leadership and technical translation, partnering with architecture and data teams to align designs with strategy and standards, and managing change and innovation within DfT’s data landscape.

Additional Information

The role is part of the Government Digital and Data (or Government Security Profession Career Framework) profession and utilises an enhanced Capability–Based Pay Framework which provides access to a Digital and Data allowance.

The base pay is £57,515. In addition to this the role includes a Digital and Data allowance of up to £22,885.

Working hours, office attendance and travel requirements

Full time roles consist of 37 hours per week.

Whilst we welcome applications from those looking to work with us on a part time basis, there is a business requirement for the successful candidate to be able to work 30 hours per week. We can accommodate part time through job share.

This role is suitable for hybrid working, and therefore a combination of workplace and home-based working can be accommodated subject to business requirements. Hybrid working is a non-contractual arrangement where employees have the flexibility to work remotely combined with a minimum of 60% of their working time a month at either their principal workplace (one of the locations cited in the advert) or, when required for business reasons, visiting stakeholders and colleagues, or carrying out detached duty in another DfT workplace. There may be occasions where you are required to attend above the minimum expectation.

If you have questions regarding how hybrid working is practiced within the business area, or any reasonable adjustments or flexible working arrangements you may currently have or need in place if successful in your application, please contact the Vacancy Holder (see advert for contact details).

Behaviours

We'll assess you against these behaviours during the selection process:

• Leadership

Technical skills

We'll assess you against these technical skills during the selection process:

• Lead Data Engineer - Communicating between the technical and non-technical (Level: practitioner)
• Lead Data Engineer - Data Innovation (Level: practitioner)
• Lead Data Engineer - Problem Management (Level: practitioner)
• Lead Data Engineer - Programming and build (data and analytics engineering)(Level: practitioner)
• Lead Data Engineer - Technical Understanding (Level: practitioner)

Benefits

Alongside your salary of £57,515, Department for Transport contributes £16,662 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.Being part of our brilliant Civil Service means you will have access to a wide range of fantastic benefits:

• Employer pension contribution of 28.97% of your salary. Read more about Civil Service Pensions here
• 25 days annual leave, increasing by 1 day each year of service (up to a maximum of 30 days annual leave).
• 8 Bank Holidays plus an additional Privilege Day to mark the King’s birthday.
• Access to the staff discount portal.
• Excellent career development opportunities and the potential to undertake professional qualifications relevant to your role paid for by the department, such as CIPD, Prince2, apprenticeships, etc.
• Joining a diverse and inclusive workforce with a range of staff communities to support all our colleagues.
• 24-hour Employee Assistance Programme providing free confidential help and advice for staff.
• Flexible working options where we encourage a great work-life balance.
• DfT prioritises investing in our project delivery professionals, because successful projects begin with skilled people. We offer funding for industry recognised qualifications e.g. APM PMQ, PRINCE2, and MSP, alongside leadership programmes. You will have access to targeted technical training, support for project delivery accreditation, and potential funding toward Chartered Project Professional (ChPP). From day one, mentoring, coaching, and the Government Online Skills Tool (GOST) are available to support your learning and development throughout your career in DfT.

Find out more about the benefits of working at DfT and its agencies

Things you need to know

Artificial intelligence

Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use.

Selection process details

This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Experience and Technical skills.

The selection process for this role will be:

Stage 1: Sift of CV and personal statement.

Stage 2: Interview and Presentation

You must be successful at each stage to progress to the next stage.

Stage 1: Sift

At sift, you will be assessed against the following Success Profile elements:

Experience – you will be asked to provide a CV (unlimited wordcount) and personal statement (1250-word count). Further details around what this will entail are listed on the application form.

Please provide evidence of your Experience of the following:

• Enterprise-scale delivery of robust, maintainable data pipelines in Google Cloud Platform. Including building reusable components, optimising performance and cost on cloud data platforms while meeting security, privacy and governance controls.
• Expert user of data engineering tools including relevant languages (e.g. Python/SQL), IaC tools (e.g. terraform), GCP or equivalent cloud tooling or (e.g. BigQuery, Cloud Functions) CI/CD (e.g. Github Actions), logging and testing.
• Setting and leading engineering standards, ensuring high‑quality coding practices, maintainable solutions, and consistent technical approaches across the team.
• Leading Agile delivery of data engineering work, managing and operating live services at pace to maintain continuity and high‑quality data delivery into DfT, while also driving innovation by developing new data engineering approaches and patterns.
• Stakeholder leadership and technical translation, partnering with architecture and data teams to align designs with strategy and standards, and managing change and innovation within DfT’s data landscape.

Further details around what this will entail are listed on the application form.

The sift will take place week from 09th February 2026.

Stage 2: Interview and Presentation

At interview stage, you will be assessed against the following Success Profile elements:

• Behaviours – Leadership.
• Experience
• Technical - DDaT Framework - Lead Data Engineer

As part of your interview, you will be asked to deliver a presentation that assesses your technical capability. Further details will be provided if successful at sift.

The interviews will take place week commencing 23rd February 2026.

This interview will be conducted online via Microsoft Teams. Further details will be provided to you should you be selected for interview.

You can find out more about our hiring process, how to apply, and application and interview guidance on our careers site (opens in a new window).

Please note that we will try to meet the dates set out in the advert. There may be occasions when these dates will change.

Further information on the selection process

We will also hold a 12-month reserve list for this role, which may lead to potential opportunities beyond the role you applied for. You can read more about our reserve lists here.

During your application, you should indicate which location(s) you wish to be considered for and, if successful, you will be placed on an individual list of candidates for each location. Candidates will be held on that list and drawn from it in merit order. We advise you to carefully consider which locations you wish to be considered for. If you decline an offer for a location you have expressed a preference in or have expressed an interest in more than one location and accept an offer, you will be withdrawn from any lists you may be held on. We may also offer candidates a location that they have not expressed a preference for where we have the requirement to do so but this will again be done on the basis of your place in the overall merit order and, in this event, you will not be removed from the list if you decline.

Appointments for this position will be made in order of merit. If you are successful in the selection process but there are no further available posts for the advertised role, you may be contacted to discuss an offer for a lower graded role (with similar experience and responsibility requirements).

If you are unsuccessful in the selection process, your application may be considered for a lower graded position if your demonstrated skills and experience meet the requirements of the alternative position. Candidates will be considered in order of merit.

Reasonable Adjustments

As a Disability Confident Leader employer, we are committed to ensuring that the recruitment process is fair, accessible and allows all candidates to perform at their best. If a person with a visible or non-visible disability is substantially disadvantaged, we have a duty to make reasonable changes to our processes.

Complete the “Assistance required” section in the “Additional requirements” page of your application form to tell us what changes or help you might need during the recruitment process. For instance, you may need wheelchair access at an interview, or if you’re deaf, a Language Service Professional.

If you need a reasonable adjustment so that you can complete your application, you should contact Government Recruitment Service via dftrecruitment.grs@cabinetoffice.gov.uk as soon as possible before the closing date to discuss your needs.

Document Accessibility

If you are experiencing accessibility problems with any attachments on this advert, please contact the email address in the 'Contact point for applicants' section.

This job advert contains links to the DfT Careers website. Our website provides useful guidance and information that can support you during the application process. If you cannot access the information on our website for any reason, please email DRGComms@dft.gov.uk for assistance.

Further Information

For more information about how we hire, and for useful tips on submitting your application for this role, visit theHow We Hire page of our DfT Careers website. You can find detailed information about the recruitment process and what to expect when applying for a role.

Pre-employment Checking

Applicants who are successful at interview will be, as part of pre-employment screening, subject to a check on the Internal Fraud Database (IFD). This check will provide information about employees who have been dismissed for fraud or dishonesty offences. This check also applies to employees who resign or otherwise leave before being dismissed for fraud or dishonesty had their employment continued. Any applicant’s details held on the IFD will be refused employment.

A candidate is not eligible to apply for a role within the Civil Service if the application is made within a 5-year period following a dismissal for carrying out internal fraud against government.

All External applicants and current employees of accredited non-departmental public bodies (NDPBs) will be required to undergo a Social Media Check. A Social Media Check is a review of your publicly available online activity, typically across platforms like LinkedIn, Facebook, X (formerly Twitter), Instagram, and others. The purpose is to identify any public posts or content that could raise concerns for employers, such as:

Hate speech or discriminatory behaviour

Threats or acts of violence

Illegal activity or substance misuse

Sexually explicit material

Extremist views or affiliations

Importantly, this check does not involve hacking into your accounts or accessing private messages. It only considers content you have chosen to make public.

Employers use this kind of screening to help ensure their workplace remains safe, inclusive, and aligned with company values. It’s not about judging your personality or lifestyle—it’s about checking for potential red flags that might affect the role or company culture.

If you have questions or concerns about the social media check, we would be happy to explain in more detail what’s being looked at and how your data is handled securely and fairly.

For further information on National Security Vetting please visit the Demystifying Vetting website.

Feedback

Feedback will only be provided if you attend an interview or assessment.

Security

Successful candidates must undergo a criminal record check.Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).

See our vetting charter (opens in a new window).People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.

Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see theCivil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).

Apply and further information

This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative.The Civil Service welcomes applications from people who have recently left prison or have an unspent conviction. Read more about prison leaver recruitment (opens in new window).Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Contact point for applicants

Job contact :

• Name : Francesca Bryden
• Email : francesca.bryden@dft.gov.uk

Recruitment team

• Email : dftrecruitment.grs@cabinetoffice.gov.uk

Further information

If you feel your application has not been treated in accordance with the Recruitment Principles and you wish to make a complaint, in the first instance, you should contact Government Recruitment Services via email: dftrecruitment.grs@cabinetoffice.gov.uk
If you are not satisfied with the response you receive from the Department, you can contact the Civil Service Commission: Visit the Civil Service Commission website Here

Attachments

Role Profile Opens in new window (pdf, 208kB)