Lead Data Engineer

Job summary

COVID‑19 has reinforced the impact that threats to health can have on society and the economy and driven health security as a critical component of our national security architecture. The United Kingdom Health Security Agency (UKHSA) is a system leader for health security; taking action internationally to strengthen global health security, providing trusted advice to government and the public and reducing inequalities in how different communities experience infectious disease, environmental hazards and other threats to health.

UKHSA will prevent threats by anticipating risks and building readiness and defences; detect and monitor infectious diseases using cutting‑edge environmental and biological surveillance; analyse threats using world‑class science and data analytics; respond with rapid, collaborative and effective actions;and lead strong global, national, regional and local partnerships designed to save lives and protect the nation from public health threats. UKHSA will build onthe legacy of the COVID‑19 response to ensure resilient and scalable infrastructure to protect the public from future threats to health.

UKHSA recognises that diversity of background and life experience brings different insights, creates challenge and encourages change and innovation. We want all our employees to feel that they can be themselves at work and that they are valued.

Working for your organisation

We pride ourselves as being an employer of choice, where Everyone Matters promoting equality of opportunity to actively encourage applications from everyone, including groups currently underrepresented in our workforce.

UKHSA ethos is to be an inclusive organisation for all our staff and stakeholders. To create, nurture and sustain an inclusive culture, where differences drive innovative solutions to meet the needs of our workforce and wider communities. We do this through celebrating and protecting differences by removing barriers and promoting equity and equality of opportunity for all.

Please visit our careers site for more information https://gov.uk/ukhsa/careers

Job description

You will work in the Data Engineering and Enablement Division that reports into the Digital and Data Directorate. It’s an exciting time to join as we seek to create leading Health Protection data capabilities that enable data scientists and data analysts to develop insights that inform public health decisions with resilience platforms and curated data assets.

The UKHSA Chief Data Officer (CDO) provides strategic leadership and oversight for developing and managing a common and bespoke suite of performant, modern, secure, data and analytical tools, services, platforms and applications, ensuring access to high quality data and analytics services, internally and externally.

The role requires the ingestion of a wide range of data assets, building acquisition, orchestration, data pipelines and curated data marts and data egresses. We are seeking to mature our advanced capabilities with standardised practices, machine learning and agility needed to pro-active detect and respond to new public health issues.

You will be involved in the delivery lifecycle from engaging with stakeholders on new initiatives, analysing use cases, developing optimal designs that where possible re-use and extend capabilities, and implement and operate the design. Our teams work collaboratively in an agile multi-disciplinary structure and we are looking for engineers who have the ability to operate in feature teams, data operations and DevSecOps models. We are developing our engineering community of practice to share knowledge, enhance our standards and processes to provide a strong foundation to develop individuals, teams and innovation.

You will draw from an extensive background of experience in data engineering to enable and deliver performant solutions on our data platforms. You will act as a lead technical expert and provide leadership and guidance to small project teams in a collaborative way. You will develop and maintain the technology infrastructure that is integrating new and existing sources of data into our Surveillance processes and workflows so that it can be exploited with maximum efficiency. You can design, write and iterate code from prototype to production‑ready, understand security, accessibility and version control, and use a range of coding tools and languages. You can develop code that self‑generates documentation. You know about the specific technologies that underpin delivery and can apply the required breadth and depth of technical knowledge. You can plan, design, manage, execute and report tests, ensure that risks associated with deployment are understood, and log, analyse and manage problems to identify and implement appropriate solutions.

This is not an exhaustive list of duties and responsibilities.

Person specification

Essential Criteria

• Extensive enterprise‑scale Python experience, including Pandas, pytest, clean code, and PEP8 standards.
• Extensive experience designing, building, and operating robust, automated ETL/ELT data pipelines, including large‑scale processing with Spark.
• Hands‑on experience using AWS data services, including Glue, Lambda, Step Functions, and S3.
• Strong SQL skills with experience in data warehousing and relational databases.
• Proficiency with git‑based development, GitHub workflows, pull requests, and CI/CD pipelines.
• Strong analytical skills to understand data requirements, ensure data quality, support data modelling, and resolve technical or performance issues.
• Ability to collaborate effectively in agile, multidisciplinary teams, with excellent written and verbal communication skills and high attention to detail.

Desirable Criteria

• Experience with agile tooling such as JIRA or Azure DevOps and familiarity with software/data lifecycle management.
• A relevant numerate or technical degree (or equivalent qualification), or relevant experience in a similar senior data role

Benefits

Alongside your salary of £56,185, UK Health Security Agency contributes £16,276 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.

• Learning and development tailored to your role
• An environment with flexible working options
• A culture encouraging inclusion and diversity
• A Civil Service pension with an employer contribution of 28.97%

Things you need to know

Artificial intelligence

Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use.

Selection process details

This vacancy is using Success Profiles and will assess your behaviours and technical skills.

Stage 1: Application & Sift

You will be required to complete an application form. You will be assessed on the listed 7 essential criteria, and this will be in the form of:

• an application form (‘Employer/ Activity history’ section on the application)
• a 1000 word supporting statement

This should outline how your skills, experience, and knowledge, provide evidence of your suitability for the role, with reference to the essential criteria.

You will receive a joint score for your application form and statement. (The application form is the kind of information you would put into your C.V –please be advised you will not be able to upload your CV. Please complete the application form in as much detail as possible). Please do not email us your CV.

Healthjobs UK has a word limit of 1500, but your supporting statement must be no more than 1000 words. We will not consider any words over 1000 words.

Longlisting: In the event of a large number of applications we will longlist into 3 piles of:

• Meets all essential criteria
• Meets some essential criteria
• Meets no essential criteria

Only those meeting all essential criteria will be taken through to shortlisting.

Shortlisting: In the event of a large number of applications we will shortlist on:

• Hands on experience using AWS data services, including Glue, Lambda, Step Functions, and S3.
• Extensive enterprise scale Python experience, including Pandas, pytest, clean code, and PEP8 standards.
• Extensive experience designing, building, and operating robust, automated ETL/ELT data pipelines, including large scale processing with Spark.

Desirable criteria may be used in the event of a large number of applications / large amount of successful candidates.

If you are successful at this stage, you will progress to interview and assessment.

Feedback will not be provided at this stage.

Stage 2: Interview

You will be invited to a remote interview.

Behaviours and technical skills will be tested at interview.

The Behaviours tested during the interview stage will be:

• Delivering at Pace (Lead behaviour)
• Communication and Influencing
• Working Together
• Managing a Quality Service

Interviews dates are to be confirmed.

Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Location

This role is being offered as hybrid working based at any of our Core HQs in Birmingham, Leeds, Liverpool, London (Canary Wharf).

We offer great flexible working opportunities at UKHSA and operate using a hybrid working model where business needs allow. This provides us with greater flexibility about how and where we work, to get the best from our workforce. As a hybrid worker, you will be expected to spend a minimum of 60% of your contractual working hours (approximately 3 days a week pro rata, averaged over a month) on-site.

Our core HQ offices are modern and newly refurbished with excellent city centre transport link and benefit from benefit from co-location with other government departments such as the Department for Health and Social Care (DHSC).

Eligibility criteria

Open to all external applicants (anyone) from outside the Civil Service (including by definition internal applicants).

Security clearance level requirement

Successful candidates must pass a basic Disclosure and Barring Security check before they can be appointed.

Successful candidates must meet the security requirements before they can be appointed. The level of security needed is Security Check clearance.

For meaningful National Security Vetting checks to be carried out individuals need to have lived in the UK for a sufficient period of time. You should normally have been resident in the United Kingdom for the last 5 years as the role requires Security Check (SC) clearance. UK residency less than the outlined periods may not necessarily bar you from gaining national security vetting and applicants should contact the Vacancy Holder / Recruiting Manager listed in the advert for further advice.

Salary information

If you are successful at interview, and are moving from another government department, NHS, or Local Authority, the relevant starting salary principles for level transfers or promotions will apply. Otherwise, roles are offered at the pay scale minimum for the grade, but in exceptional circumstances there may be flexibility if you are able to demonstrate you are already in receipt of an existing, higher salary. Pay increases are through the relevant annual pay award for the role and terms.

Please be aware that the salary is based on the office location:

Civil Service Grade 7

£56,185- £66,581 (National)
£58,340- £68,574 (Outer London)
£60,494- £70,566 (Inner London)

Future location

UKHSA is investing in a new state-of-the-art National Biosecurity Centre in Harlow, Essex, which will eventually bring together teams currently based at Canary Wharf, Colindale and Porton Down. For more details, please see: Huge biosecurity centre investment to boost pandemic protection - GOV.UK.

The new facilities will start becoming operational in the mid-2030s, with full completion by 2038. Staff will move in phases as facilities become available. If you're appointed to a role currently based at Canary Wharf, Colindale or Porton Down, please note that we'll continue investing in these sites for the next decade. As we get closer to the transition, we'll provide full information about relocation support available to staff.

Reasonable adjustments

The Civil Service is committed to making sure that our selection methods are fair to everyone. To help you during the recruitment process, we will consider any reasonable adjustments that could help you. An adjustment is a change to the recruitment process or an adjustment at work. This is separate to the Disability Confident Scheme. If you need an adjustment to be made at any point during the recruitment process you should contact the recruitment team in confidence as soon as possible to discuss your needs.

You can find out more information about reasonable adjustments across the Civil Service here: https://www.civil-service-careers.gov.uk/reasonable-adjustments/

International police check

If you have spent more than 6 months abroad over the last 3 years you may need an International Police Check. This would not necessarily have to be in a single block, and it could be time accrued over that period.

Internal fraud check

If successful for this role as one aspect of pre-employment screening, applicant’s personal details – name, national insurance number and date of birth - will be checked against the Cabinet Office Internal Fraud Hub and anyone included on the database will be refused employment unless they can show exceptional circumstances. Currently this is only for External candidates to the Civil Service.

Careers website

Please visit our careers site for more information https://gov.uk/ukhsa/careers

Feedback will only be provided if you attend an interview or assessment.

Security

Successful candidates must undergo a criminal record check.Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).

See our vetting charter (opens in a new window).People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.

Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see theCivil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).

Apply and further information

Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Contact point for applicants

Job contact :

• Name : Jon Payne
• Email : jonathan.payne@ukhsa.gov.uk

Recruitment team

• Email : recruitment@ukhsa.gov.uk

Further information

The law requires that selection for appointment to the Civil Service is on merit on the basis of fair and open competition as outlined in the Civil Service Commission's Recruitment Principles. If you feel your application has not been treated in accordance with the Recruitment Principles, and you wish to make a complaint, in the first instance, you should contact UKHSA Public Accountability Unit via email: Complaints@ukhsa.gov.uk If you are not satisfied with the response you receive from the Department, you can contact the Civil Service Commission: Visit the Civil Service Commission website: https://civilservicecommission.independent.gov.uk

http://www.healthjobsuk.com/vacancy/7987337