Data Engineer

Job summary

This role sits within the Data Analytics Service Team. You will be working as our Data Engineer and will be required to review/ design, document, and streamline the ETL processes for the datasets we bring together for analysis and AI projects. We operate in Azure using ABFSS for storage and Databricks Unity catalogue for our data processing. As part of the Senior Management Team, you will ensure new datasets are being added with the correct processes, existing pipelines execute correctly, and proposing changes to these processes going forward in line with best practices.

As part of the senior management of the data analytics team, you will also have leadership responsibilities within the PSFA data team to help us improve our services.

You will work with the platform lead to identify where and how we should continue to build our Azure tenant to best support the ambitions of the PSFA data analytics service going forward. You will develop relationships with key partners across the civil service, including our key data providers and our cloud provider, to identify how we can work together to improve our tenant.

Job description

We are recruiting a Data Engineer to help us build and maintain a data model in Microsoft Azure to aid in the development of our network analysis and AI Service offerings. This role will support our mission by working across government to reduce the impact of fraud, representing the Public Sector Fraud Authority (PSFA) externally, and ensuring that our ML, AI and data services deliver maximum value.

Person specification

Essential criteria

• Degree / Masters in computer science, maths, statistics or related disciplines.
• Coding background in Python/PySpark and strong SQL skills.
• Experience working in the Microsoft Azure cloud.
• Experience in Data Modelling, and understanding best practices when applying data models.
• Experience in building and maintaining data pipelines (ETL and/or analytical pipelines).
• Experience working with Databricks as a data hosting/processing tool, or a similar platform (ideally experience with Databricks unity catalogue).
• Experience in cleansing, managing and transforming high volume data.
• Understanding of No Code/Metadata driven automation and CI/CD best practices including Azure DevOps.
• Senior stakeholder management skills including the ability to manage a diverse set of stakeholders and to prioritise needs.

It will be beneficial for applicants to have

• Experience in fraud software deployment; and configuration of fraud detection tools and/or analytics platforms.
• Experience in fraud, compliance or financial crime.
• Working with Azure DevOps for CI/CD.

Additional information:

A minimum 60% of your working time should be spent at your principal workplace. Although requirements to attend other locations for official business will also count towards this level of attendance.

Behaviours

We'll assess you against these behaviours during the selection process:

• Making Effective Decisions
• Changing and Improving
• Managing a Quality Service

Technical skills

We'll assess you against these technical skills during the selection process:

• Data Engineering - Experience

Benefits

Alongside your salary of £69,308, Cabinet Office contributes £20,078 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.

• Learning and development tailored to your role.
• An environment with flexible working options.
• A culture encouraging inclusion and diversity.
• A Civil Service Pension which provides an attractive pension, benefits for dependants and employer contributions of 28.97%.
• A minimum of 25 days of paid annual leave, increasing by one day per year up to a maximum of 30.

Things you need to know

Artificial intelligence

Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use.

Selection process details

This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Experience and Technical skills.

As part of the application process you will be asked to complete a CV, 500 word Personal Statement (aligned to the essential criteria) and a number of Behaviours.

Further details around what this will entail are listed on the application form.

Note: In the event of high application volumes - Cabinet Office will sift on the following behaviours:

• Making Effective Decisions

Selection process

This vacancy is using Success Profiles, and will assess your Behaviours and Technical skills.

We'll assess you against these behaviours during the selection process:

• Making Effective Decisions
• Changing and Improving
• Managing a Quality Service

Technical

We'll assess your technical ability during the selection process with:

• Data Engineering - Experience

You will be asked at the interview to describe a project you worked on and how it delivered robust data to support analytical work and model development. Technical ability will be assessed against the “Lead Data Engineer” specification on the Government Digital and Data Professional Capability Framework.

Expected timeline (subject to change)

Expected sift date – w/c 19/01/2026

Expected interview date/s – w/c 02/02/2026

Interview location - Your interview will either be conducted face to face or by video. You will be notified of the location if you are selected for interview.

Reasonable Adjustment

If a person with disabilities is put at a substantial disadvantage compared to a non-disabled person, we have a duty to make reasonable changes to our processes.

If you need a change to be made so that you can make your application, you should:

Contact Government Recruitment Service via cabinetofficerecruitment.grs@cabinetoffice.gov.uk as soon as possible before the closing date to discuss your needs.

Complete the ‘Assistance required’ section in the ‘Additional requirements’ page of your application form to tell us what changes or help you might need further on in the recruitment process. For instance, you may need wheelchair access at interview, or if you’re deaf, a Language Service Professional.

Further information

If you are experiencing accessibility problems with any attachments on this advert, please contact the email address in the 'contact point for applicants' section.

Please note that this role requires SC clearance, which would normally need 5-years UK residency in the past 5-years]. This is not an absolute requirement, but supplementary checks may be needed where individuals have not lived in the UK for that period. This may mean your security clearance (and therefore your appointment) will take longer or, in some cases, not be possible.

For further information on National Security Vetting please visit the Demystifying Vetting website.

Please note terms and conditions are attached. Please take time to read the document to determine how these may affect you.

Any move to Cabinet Office from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may however be eligible for other government schemes, including Tax Free Childcare. Determine your eligibility at: https://www.childcarechoices.gov.uk

A reserve list will be held for a period of 12 months, from which further appointments can be made.

If successful and transferring from another Government Department a criminal record check may be carried out.

In order to process applications without delay, we will be sending a Criminal Record Check to Disclosure and Barring Service or Disclosure Scotland on your behalf.

However, we recognise in exceptional circumstances some candidates will want to send their completed forms direct. If you will be doing this, please advise Government Recruitment Service of your intention by emailing Pre-EmploymentChecks.grs@cabinetoffice.gov.uk stating the job reference number in the subject heading.

For further information on the Disclosure Scotland confidential checking service telephone: the Disclosure Scotland Helpline on 0870 609 6006 and ask to speak to the operations manager in confidence, or email Info@disclosurescotland.co.uk

Applicants who are successful at interview will be, as part of pre-employment screening, subject to a check on the Internal Fraud Database (IFD). This check will provide information about employees who have been dismissed for fraud or dishonesty offences. This check also applies to employees who resign or otherwise leave before being dismissed for fraud or dishonesty had their employment continued. Any applicant’s details held on the IFD will be refused employment.

A candidate is not eligible to apply for a role within the Civil Service if the application is made within a 5-year period following a dismissal for carrying out internal fraud against government.

New entrants are expected to join on the minimum of the pay band.

Please note terms and conditions are attached. Please take time to read the document to determine how these may affect you.

Feedback

Feedback will only be provided if you attend an interview or assessment.

Security

Successful candidates must undergo a criminal record check.Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).

See our vetting charter (opens in a new window).People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.

Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see theCivil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).

Apply and further information

This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative.The Civil Service welcomes applications from people who have recently left prison or have an unspent conviction. Read more about prison leaver recruitment (opens in new window).Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Contact point for applicants

Job contact :

• Name : Ricky Reed-Killick
• Email : Ricky.Reed-Killick1@cabinetoffice.gov.uk

Recruitment team

• Email : cabinetofficerecruitment.grs@cabinetoffice.gov.uk

Further information

Appointment to the Civil Service is governed by the Civil Service Commission’s Recruitment Principles. If you feel that your application has not been treated in accordance with the Recruitment Principles, and wish to make a complaint, then in the first instance you should contact Government Recruitment Service by email at : cabinetofficerecruitment.grs@cabinetoffice.gov.uk
If you are not satisfied with the response you receive, then you can contact the Civil Service Commission at info@csc.gov.uk. For further information on the Recruitment Principles and bringing a complaint to the Civil Service Commission, please visit their website at: https://civilservicecommission.independent.gov.uk.

Attachments

Cabinet Office Employee Offer Opens in new window (pdf, 302kB)Cabinet Office Terms and Conditions - External 2025-02-20 Opens in new window (docx, 83kB)