Senior Data Engineer
Job summary
Can you design, build and optimise robust, scalable data pipelines in GCP (or similar cloud platforms), applying engineering best practices to deliver high-quality, cost-effective solutions?
Have you used Python, SQL and cloud-based data engineering tools (such as BigQuery, Cloud Functions, Terraform or CI/CD pipelines) to successfully deliver and support data products in a production environment?
Are you experienced working in Agile teams, collaborating with both technical and non-technical stakeholders, and contributing to engineering standards that improve quality, consistency and long-term maintainability?
If so, we’d love to hear from you!
Join a collaborative and forward-thinking data team focused on delivering high-impact solutions that enable better decision-making across the organisation. Working within a modern cloud-based environment, you'll be part of a department that values innovation, continuous improvement and knowledge sharing. The team brings together a range of technical expertise to solve complex challenges, transform data into actionable insight, and drive meaningful change. You'll have the opportunity to work closely with stakeholders from across the business, contributing to strategic initiatives while helping shape the future of our data capabilities. With a strong emphasis on learning, quality and teamwork, this is an exciting environment for individuals who enjoy solving problems, embracing new technologies and making a tangible impact through data.
Joining our department comes with many benefits, including:
- Employer pension contribution of 28.97% of your salary. Read more about Civil Service Pensions here
- 25 days annual leave, increasing by 1 day each year of service (up to a maximum of 30 days annual leave), plus 8 bank holidays a privilege day for the King’s birthday
- Flexible working options where we encourage a great work-life balance.
Read more in the Benefits section below!
Find out more about what it's like working at DfTc:
DfTc roles: Department for Transport Central - Department for Transport Careers
Job description
Join the Data Engineering team, a collaborative group of specialists working across Leeds, Birmingham, Hastings and London, delivering the department’s critical data services and cloud-based analytical platforms. You'll work closely with data and digital professionals to develop reliable, scalable solutions, support live services, modernise legacy systems, and help drive innovation. This is an exciting opportunity to contribute to high-impact projects that improve data capabilities and support informed decision-making across the organisation.
Your responsibilities will include, but aren’t limited to:
- Collaborate with team members and stakeholders to build and operate DfT’s production‑grade data pipelines and platforms, ensuring reliability and security across our Google Cloud Platform environment.
- Manage live data services; resolving issues at pace to maintain high‑quality data delivery for analysts, policy teams and external commitments.
- Take a leading role in developing and improving legacy systems, ensuring our data infrastructure is modern and meets ongoing needs.
- Support innovation within data engineering, identifying opportunities to modernise tooling, adopt emerging GCP capabilities and introduce new approaches that improve efficiency and data quality.
- Work closely with technical and non-technical stakeholders, translating technical concepts, shaping data‑related decisions, and responding to business need.
- Act as a champion of Infrastructure as Code (IaC), improving and developing repeatable patterns for environments, access, and data services.
- Drive the development of our metadata catalogue, curating business and technical metadata so users can effectively discover and use data.
For further information on the role, please read the role profile.Please note that the role profile is for information purposes only - whilst all elements are relevant to the role, they may not all be assessed during the recruitment process. This job advert will detail exactly what will be assessed during the recruitment process.
Person specification
You will be required to provide evidence that you hold any essential qualifications/licenses at some point during the recruitment process. If you cannot provide evidence, your application will be withdrawn.
Required experience:
To be successful in this role you will need to have the following experience:
- Hands-on delivery of robust, maintainable data pipelines in Google Cloud Platform or alternative cloud providers. Including building reusable components, optimising performance and cost, and applying engineering standards and high‑quality coding practices.
- Proficient user of data engineering tools, including relevant languages (e.g. Python and SQL) and GCP or equivalent cloud tooling (e.g. BigQuery, Cloud Functions). Experience working with or knowledge of CI/CD (e.g. Github Actions), logging and testing, and Infrastructure as Code tools (e.g. Terraform).
- Contribution to engineering standards and best practice, supporting the development of maintainable, consistent technical approaches and improving quality across codebases and services.
- Experience delivering data engineering work in Agile environments, independently managing components of delivery while contributing to wider team objectives.
- Working across technical and non‑technical teams to understand requirements, communicate solutions, and support delivery of change and improvement within the data landscape.
Additional Information
The role is part of the Government Digital and Data (or Government Security Profession Career Framework) profession and utilises an enhanced Capability–Based Pay Framework which provides access to a Digital and Data allowance.
The base pay is £44,241 (National) £48,581 (London). In addition to this the role includes a Digital and Data allowance of up to £14,756.
The value of allowance awarded will be based on an assessment of your skills and experience as demonstrated through the selection process. Here are more details on the pay framework.
This role is suitable for hybrid working, which is a non-contractual arrangement where a combination of workplace and home-based working can be accommodated subject to business requirements.
The expectation at present is a minimum of 60% of your working time a month will be spent at either your designated workplace (one of the locations cited in the advert) or, when required for business reasons, in another office/work location/visiting stakeholders. Your designated workplace will be your contractual place of work. There may be occasions where you are required to attend above the minimum expectation.
Occasional travel to other offices will be required, which may involve overnight stays.
Visa Sponsorship
DfTc does not offer Visa Sponsorship for this role.
Behaviours
We'll assess you against these behaviours during the selection process:
- Communicating and Influencing
- Managing a Quality Service
Technical skills
We'll assess you against these technical skills during the selection process:
- Programming and Build (Data Analytics and Engineering)
- Problem Management
- Data Innovation
- Data Development Process
Benefits
Being part of our brilliant Civil Service means you will have access to a wide range of fantastic benefits:
- Employer pension contribution of 28.97% of your salary. Read more about Civil Service Pensions here
- 25 days annual leave, increasing by 1 day each year of service (up to a maximum of 30 days annual leave).
- 8 Bank Holidays plus an additional Privilege Day to mark the King’s birthday.
- Access to the staff discount portal.
- Excellent career development opportunities and the potential to undertake professional qualifications relevant to your role paid for by the department, such as CIPD, Prince2, apprenticeships, etc.
- Joining a diverse and inclusive workforce with a range of staff communities to support all our colleagues.
- 24-hour Employee Assistance Programme providing free confidential help and advice for staff.
- Flexible working options where we encourage a great work-life balance.
Find out more about the benefits of working at DfT and its agencies (opens in a new window).
Things you need to know
Artificial intelligence
Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use.Selection process details
This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Experience and Technical skills.How to apply:
Our selection process ensures a comprehensive assessment of each applicant's qualifications, skills, and potential fit within our organisation.
The selection process for this role will be:
Stage 1: Sift of CV and personal statement
Stage 2: Interview
You must be successful at each stage to progress to the next stage.
Stage 1: Sift
At sift, you will be assessed against the following Success Profile elements:
Experience – you will be asked to provide a CV (unlimited wordcount) and personal statement (1250-word count). Please provide evidence of your Experience of the following:
- Hands-on delivery of robust, maintainable data pipelines in Google Cloud Platform or alternative cloud providers. Including building reusable components, optimising performance and cost, and applying engineering standards and high‑quality coding practices.
- Proficient user of data engineering tools, including relevant languages (e.g. Python and SQL) and GCP or equivalent cloud tooling (e.g. BigQuery, Cloud Functions). Experience working with or knowledge of CI/CD (e.g. Github Actions), logging and testing, and Infrastructure as Code tools (e.g. Terraform).
- Contribution to engineering standards and best practice, supporting the development of maintainable, consistent technical approaches and improving quality across codebases and services.
- Experience delivering data engineering work in Agile environments, independently managing components of delivery while contributing to wider team objectives.
- Working across technical and non‑technical teams to understand requirements, communicate solutions, and support delivery of change and improvement within the data landscape.
The sift will take place week commencing 10/08/2026
Stage 2: Interview
At interview stage, you will be assessed against the following Success Profile elements:
- Behaviours – Communicating & Influencing, Managing a Quality Service
- Technical – Programming and Build (data analytics and engineering) (Level: Practitioner), Problem Management (Level: Working), Data Innovation (Level: Working), Data Development Process (Level: Practitioner)
Technical skills will be linked against the Government Digital and Data Profession Capability Framework
The interviews will take place week commencing 24/08/2026
This interview will be conducted online via Microsoft Teams. Further details will be provided to you should you be selected for interview.
You can find out more about our hiring process, how to apply, and application and interview guidance on our careers site (opens in a new window).
Please note that we will try to meet the dates set out in the advert. There may be occasions when these dates will change.
Reasonable Adjustments
As a Disability Confident Leader employer, we are committed to ensuring that the recruitment process is fair, accessible and allows all candidates to perform at their best. If a person with a visible or non-visible disability is substantially disadvantaged, we have a duty to make reasonable changes to our processes.
Complete the “Assistance required” section in the “Additional requirements” page of your application form to tell us what changes or help you might need during the recruitment process. For instance, you may need wheelchair access at an interview, or if you’re deaf, a Language Service Professional.
If you need a reasonable adjustment so that you can complete your application, you should contact Government Recruitment Service via dftrecruitment.grs@cabinetoffice.gov.uk as soon as possible before the closing date to discuss your needs.
Document Accessibility
This job advert contains links to the DfT Careers website. Our website provides useful guidance and information that can support you during the application process. If you are experiencing accessibility problems with any attachments on this advert or the information on our website, please contact the email address in the 'Contact point for applicants' section.
Further information on the selection process
For more information about how we hire, and for useful tips on submitting your application for this role, visit theHow We Hire page of our DfT Careers website. You can find detailed information about the recruitment process and what to expect when applying for a role.
We will also hold a 12-month reserve list for this role, which may lead to potential opportunities beyond the role you applied for. You can read more about our reserve lists here.
If your application is successful but you have been dismissed from the Civil Service, your application could be removed at the pre-employment checking stage depending on the nature of the dismissal.
Pre-employment Checking
Applicants who are successful at interview will be, as part of pre-employment screening, subject to a check on the Internal Fraud Database (IFD). This check will provide information about employees who have been dismissed for fraud or dishonesty offences. This check also applies to employees who resign or otherwise leave before being dismissed for fraud or dishonesty had their employment continued. Any applicant’s details held on the IFD will be refused employment. A candidate is not eligible to apply for a role within the Civil Service if the application is made within a 5-year period following a dismissal for carrying out internal fraud against government.
All External applicants and current employees of accredited non-departmental public bodies (NDPBs) will be required to undergo a Social Media Check. A Social Media Check is a review of your publicly available online activity, typically across platforms like LinkedIn, Facebook, X (formerly Twitter), Instagram, and others. The purpose is to identify any public posts or content that could raise concerns for employers, such as:
Hate speech or discriminatory behaviour
Threats or acts of violence
Illegal activity or substance misuse
Sexually explicit material
Extremist views or affiliations
Importantly, this check does not involve hacking into your accounts or accessing private messages. It only considers content you have chosen to make public.
Employers use this kind of screening to help ensure their workplace remains safe, inclusive, and aligned with company values. It’s not about judging your personality or lifestyle—it’s about checking for potential red flags that might affect the role or company culture.
If you have questions or concerns about the social media check, we would be happy to explain in more detail what’s being looked at and how your data is handled securely and fairly.
Feedback will only be provided if you attend an interview or assessment.
Security
Successful candidates must undergo a criminal record check.Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).See our vetting charter (opens in a new window).People working with government assets must complete baseline personnel security standard (opens in new window) checks.
Nationality requirements
This job is broadly open to the following groups:
- UK nationals
- nationals of the Republic of Ireland
- nationals of Commonwealth countries who have the right to work in the UK
- nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
- nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
- individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
- Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service
Working for the Civil Service
The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.
Diversity and Inclusion
The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see theCivil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).Apply and further information
This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative.The Civil Service welcomes applications from people who have recently left prison or have an unspent conviction. Read more about prison leaver recruitment (opens in new window).Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.Contact point for applicants
Job contact :
- Name : Gregory Carter
- Email : gregory.carter@dft.gov.uk
Recruitment team
Further information
If you feel your application has not been treated in accordance with the Recruitment Principles and you wish to make a complaint, in the first instance, you should contact Government Recruitment Services via email: dftrecruitment.grs@cabinetoffice.gov.uk If you are not satisfied with the response you receive from the Department, you can contact the Civil Service Commission: Visit the Civil Service Commission website at https://civilservicecommission.independent.gov.ukAttachments
Role profile V2 Opens in new window (docx, 321kB)Salary range
- £44,241 - £58,997 per year