Data Engineer (G7)

Job summary

Please note this role requires you to pass Security Check clearance. For further information, please see 'Selection process details'.

You must have resided continuously in the UK for at least 3 of the last 5 years, 2 of which must have been the immediately preceding years from the point of applying for this job.

Are you a Senior Data Engineer with expert-level experience in Azure data engineering, ETL/ELT pipeline design and technical team leadership?

Join DWP as part of our Data Engineering community, where you will lead the design and delivery of secure, scalable data solutions that power services used by millions of people across the UK. This role plays a key part in ensuring our data estate is reliable, well-governed and continuously improving, enabling analysts, data scientists and operational teams to access trusted data at scale.

Job description

Roles and Responsibilities:

• Lead the team in designing, developing and maintaining scalable ETL/ELT pipelines using a strong knowledge of PySpark and Python to ensure best practice and standards are met in optimizing, validating and standardising all team data workflows as well as batch processes.
• Lead collection of complex data requirements from a wide range of stakeholders, interpret the data and shape data sets to for Data Analysts to fulfil requirements in their creation of analysis and insight dashboards.
• Lead the build and management as well as the supply of data to multiple systems and applications including database management. This to secure data availability and the construction of efficient scripts to extract data for cross business usage.
• Leading immediate team though large-scale data and platform transformation and/or migration programmes, ensuring best practice and building capability within the team as the transformation occurs. Set direction of change and proactively identify key issues.
• Apply a deep understanding and experience of cloud data engineering toolsets in developing complex pipelines, lead development of products and automation processes. Lead development of logging, monitoring and assuring tool performance information.
• Oversee, support and manage team in developing and using a range of operational tracking and cataloguing tools to track data and metadata through its journey such as: Google Cloud Data Catalog, AWS Glue Data Catalog, Azure Data Catalog, Informatica Enterprise Data Catalog, Alation, Apache Atlas etc. Ensuring compliance with business standards.
• Lead the use of Agile practices in the team to leverage data engineering tools like version control, automation, monitoring to streamline data pipelines, automate testing, deployment and track changes.

Skills England is an executive agency of the Department for Work and Pensions (DWP), working with employers, training providers, local partners and government to create better skills for better jobs. We play a leading role in identifying current and future skills needs across the economy, helping to ensure that people can access high-quality education, training and employment opportunities while supporting businesses to develop the skilled workforce they need. As part of the Digital and Data Services team you will play a key role in ensuring Skills England creates, maintains and uses data efficiently and effectively to drive decisions and produce high quality technical education products and labour market insights.

Locations for this team are: London, Coventry, Darlington, Sheffield, Leeds. Roles within other teams will be based in the locations advertised above. This can be discussed further, should you be successful.

Person specification

See selection process for further details.

If you would like to learn more about the role, please contact Aisha.gultasab@dwp.gov.uk.

Technical skills

We'll assess you against these technical skills during the selection process:

• Leading a team in designing Data Engineering Pipelines

Benefits

Alongside your salary of £57,946, Department for Work and Pensions contributes £16,786 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides (opens in a new window).

We also have a broad benefits package built around your work-life balance which includes:

• Working patterns to support work/life balance such as job sharing, term-time working, flexi-time and compressed hours.
• Generous annual leave – at least 25 days on entry, increasing up to 30 days over time (pro–rata for part time employees), plus 9 days public and privilege leave.
• Support for financial wellbeing, including interest-free season ticket loans for travel, a cycle to work scheme and an employee discount scheme.
• Health and wellbeing support including our Employee Assistance Programme for specialist advice and counselling and the opportunity to join HASSRA a first-class programme of competitions, activities and benefits for its members (subscription payable monthly).
• Family friendly policies including enhanced maternity and shared parental leave pay after 1 year’s continuous service.
• Funded learning and development to support progress in your role and career. This includes industry recognised qualifications and accreditations, coaching, mentoring and talent development programmes.
• An inclusive and diverse environment with opportunities to join professional and interpersonal networks including Women’s Network, National Race Network, National Disability Network (THRIVE) and many more.

Hybrid Working

This role may be suitable for hybrid working, which is where an employee works part of the week in their DWP office and part of the week from home. This is a voluntary, non-contractual arrangement and your office will be your contractual place of work.

If a hybrid working arrangement is suitable for the role and for you, you will normally be required to spend a minimum of 60% of your contracted working hours from your DWP office.

If you have a disability, caring responsibilities, or other circumstances that may affect your ability to meet the minimum office attendance requirement, please discuss this with us using the contact details in this advert.

Salary Information

Pay for this role is from £57,946 to £75,664.

The maximum salary for the grade is £68,205, however a Digital Allowance of up to £7,459 per annum is available for exceptional candidates. Digital Allowance is non-salary, and non-pensionable, and is classed as a temporary allowance. It is reviewed annually where it could be retained, amended, or removed.

Pay for this role in London is from £63,075 to £74,005. (Please refer to location details to check eligibility)

Our offer to successful candidates will be based on an assessment of your skills and experience as demonstrated at interview.

Existing Civil Servants who secure a new role on lateral transfer will maintain their current substantive salary.

Civil Servants who gain promotion will move to the bottom of the grade pay scale or receive a 10% increase applied to their current substantive salary, whichever is greater.

Any temporary allowances that you are currently in receipt of will not form part of the calculation to determine your pay. Any allowances that are in payment will cease when you move into your new role.

Things you need to know

Artificial intelligence

Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use.

Selection process details

This vacancy is using Success Profiles (opens in a new window), and will assess your Experience and Technical skills.

Stage 1: Application

Your application will consist of four parts:

1. A Personal Details application form.

2. Employment history - this should contain your work experience and any skills, qualifications and accomplishments relevant to the jobs you have completed.

3. Personal statement - up to 1000 words. This statement should be used to provide examples of how you meet the essential criteria below:

• PRACTITIONER: Ability to lead the team in developing complex ETL pipelines, including effective delegation and prioritisation of work.
• PRACTITIONER: Sound ability in communicating between the technical and non-technical: you can break down complex concepts to communicate them in plain language, adapting communication style to large audiences of none-specialists and senior stakeholders.
• PRACTITIONER: Strong understanding of the principles of data processing in a distributed or cloud systems, and ability to use this understanding to ensure and design robust coding in a distributed or cloud environment using a variety of native toolsets.
• PRACTITIONER: Understand how to collate and review complex technical requirements and specifications, leading the definition of test conditions for the team.
• PRACTITIONER: Understand how to lead and investigate complex problems/issues in data, pipelines and processes, systems and platforms, with an understanding of the level of a problem, for example, strategic, tactical or operational
• PRACTITIONER: Strong knowledge of assessing, mapping, and migrating complex data from a range legacy or on-prem systems to cloud-native formats and platforms across an area.

4. Technical statement (up to 250 words). The following statement is aligned to the required technical skill of: Leading a team in designing Data Engineering Pipelines. This statement should be used to provide example(s) of how you meet the criterion below:

• EXPERT: Strong understanding of leading the design and automation of ETL pipelines using PySpark and Spark SQL, ensuring seamless extraction, transformation, and loading of structured and unstructured data across cloud environments such as AWS, GCP and Azure.

Ensure that all examples provided in your statements are taken directly from your own experience and that you describe the examples in your own words. If you choose to use gen-AI to support your statements, you must follow the guidelines outlined in the Artificial intelligence and recruitment guide.

The sift panel will use the information in your employment history, personal statement, and technical statement to assess your experience, skills and knowledge.

An initial sift will be conducted using the technical statement. Candidates who pass the initial sift will be progressed to a full sift. If you do progress to a full sift, you will be provided with one combined overall assessment score for both your employment history and Personal Statement.

For Hints and Tips on completing your application visit Applying for jobs at DWP Digital.

Important Information

• You will be asked to complete your employment history. Any information that you would customarily share on a CV should therefore be entered onto the application form.
• Personal details that could be used to identify you including your name, contact details and address must be removed for your application to be considered.
• If your employment history, personal statement or technical statement contain any personal details your application will be withdrawn.

Stage 2: Interview 1

If you’re successful at sift stage you will be invited to a video interview via Microsoft Teams.

You will be asked questions to further assess the experience and evidence you provided in your sift application.

Stage 2: Interview 2

If you are successful at interview 1, you will be invited to a video interview via Microsoft Teams. There, you will be assessed against the experience criteria listed under both the essential criteria and technical skill.

Interviews will take place from late November 2026 onwards. Sift and Interview dates to be confirmed.

Other information:

All successful candidates and those placed on reserve will be posted in merit list order by location.

Please note that for these vacancies we recommend that applicants seek specialist advice on their eligibility for Visa Sponsorship (if applicable) when considering whether to apply for any role. We can never guarantee sponsorship, requests will be considered in line with the relevant DWP guidance and Home Office policy in place at the time.

Security Clearance Requirement

You must meet the security requirements before you can be appointed. The level of security needed is security check.

You must have resided continuously in the UK for at least 3 of the last 5 years, 2 of which must have been the immediately preceding years from the point of applying for this job.

For further information on National Security Vetting please visit the Demystifying Vetting website.

Further Information​

At the Department for Work and Pensions (DWP) we value diversity and inclusion and actively encourage and welcome applications from everyone, including those that are underrepresented in our workforce.

Find out more about working for DWP on the DWP Careers Hub.

Reserve List

A reserve list may be held for a period of 6 months from which further appointments can be made.

If you are placed on a reserve list but we cannot immediately offer you a post, please note:

• If you are later offered the role you applied for, in a location you have expressed a preference for, and you decline the offer or are unable to take up the post within a reasonable timeframe you will be withdrawn from the campaign and removed from the reserve list, other than in exceptional circumstances (e.g. a contractual Part Time Working Pattern cannot be accommodated in the initial role offered or in cases of serious ill health).
• If DWP makes an offer of an alternative role or location to that which you originally applied for, and you decline that offer, you will be able to remain on the reserve list.

Disability Confident Scheme

If high application volumes are received, the benchmark for candidates to proceed to the next stage may be raised. In line with our commitment to the Disability Confident Scheme (DCS), we aim to advance all candidates applying under the DCS who meet the minimum standard. However, we may only progress those candidates who best meet the required standards.

Reasonable Adjustments

We consider visible and non-visible disabilities, neurodiversity or learning differences, chronic medical conditions, or mental ill health. Examples include dyslexia, epilepsy, autism, chronic fatigue, or schizophrenia.

If you need a change to be made so that you can make your application, you should contact the Government Recruitment Service via DWPRecruitment.grs@cabinetoffice.gov.uk as soon as possible before the closing date to discuss your needs.

Complete the “Reasonable Adjustments” section in the “Additional requirements” page of your application form to tell us what changes or help you might need further on in the recruitment process. For instance, you may need wheelchair access at interview, or if you’re deaf, a Language Service Professional.

If you are experiencing accessibility problems with any attachments on this advert, please contact the email address in the 'Contact point for applicants' section.

Integrity, plagiarism and Civil Service Principles

The Civil Service values honesty and integrity and expects all candidates to abide by these principles.

Please ensure that all examples provided in your application are taken directly from your own experience and that you describe the examples in your own words.

Examples of plagiarism can include:

- Presenting the work, ideas and experience of others as your own

- Copying content or answers from an online or published source that is not your own

Disclosure and Barring Service and Internal Fraud Database Checks

To process applications without delay, we will be sending a Criminal Record Check to Disclosure and Barring Service on your behalf. However, we recognise in exceptional circumstances some candidates will want to send their completed forms direct. If you will be doing this, please advise Government Recruitment Service of your intention by emailing Pre-EmploymentChecks.grs@cabinetoffice.gov.uk stating the job reference number in the subject heading.

Applicants who are successful at interview will be, as part of pre-employment screening, subject to a check on the Internal Fraud Database (IFD). This check will provide information about employees who have been dismissed for fraud or dishonesty offences. This check also applies to employees who resign or otherwise leave before being dismissed for fraud or dishonesty had their employment continued. Any applicant’s details held on the IFD will be refused employment. A candidate is not eligible to apply for a role within the Civil Service if the application is made within a 5 year period following a dismissal for carrying out internal fraud against government.

If successful and transferring from another Government Department, a criminal record check will be carried out.

Important

New entrants are expected to join on the minimum of the pay band.

Before applying for this vacancy, current employees of DWP should check whether a successful application would result in changes to their terms & conditions of employment, e.g. mobility, pay, allowances. If you are a current employee and are successful you must be able to be released from your current post within four weeks.

Those on protected TUPE/ COSoP terms and conditions applying on promotion or voluntary permanent level move will adopt DWP’s Terms and Conditions and this may have a different impact on pay and allowances. Please review this prior to acceptance of a role.

Civil Servants that would transfer into DWP from other government organisations, following successful application, will assume DWP's terms & conditions of employment current on the day they are posted, unless DWP has stated otherwise in writing. Accepting a post will be taken to mean acceptance of revised terms & conditions.

Civil Servants applying on promotion will usually be appointed to the salary minimum or within 10% of existing salary.

Any move to DWP from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may, however, be eligible for other government schemes, including Tax Free Childcare. Determine your eligibility at the Childcare Choices Website.

Feedback

Feedback will only be provided if you attend an interview or assessment.

Security

Successful candidates must undergo a basic (or equivalent) criminal record check.Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).

See our vetting charter (opens in a new window).People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.

Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see theCivil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).

Apply and further information

This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative.The Civil Service welcomes applications from people who have recently left prison or have an unspent conviction. Read more about prison leaver recruitment (opens in new window).Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Contact point for applicants

Job contact :

• Name : Aisha Gultasab
• Email : Aisha.gultasab@dwp.gov.uk

Recruitment team

• Email : digitalrecruitment.grs@cabinetoffice.gov.uk

Further information

Appointment to the Civil Service is governed by the Civil Service Commission’s Recruitment Principles. If you feel your application has not been treated in accordance with these principles and you wish to make a complaint, you should in the first instance contact DWP by email: HR.BUSINESSASSURANCE@DWP.GOV.UK.
If you are not satisfied with the response you receive from the Department, you can contact the Civil Service Commission. Click here to visit the Civil Service mmission.

Attachments

Success-Profiles-Candidate-Overview Opens in new window (pdf, 635kB)DWP Terms and Conditions January 2024 Opens in new window (docx, 17kB)