Lead Data Engineer, Data Services and Analytics

Job summary

Home Office Digital designs, builds and develops services for the rest of the department and for government. Every year our systems support up to 3 million visa applications, checks on 100 million border crossings, up to 8 million passport applications and deliver 140 million police checks on people, vehicles and property.

Data Services and Analytics (DSA) is the part of DDaT which leads the build of data products to integrate data from multiple sources, and then deliver products for business intelligence, reporting and data science. DSA has the ambition to be one of the leading providers of insight services in UK Government and is currently growing significantly with over 300 colleagues, providing services to variety of functions within the Home Office.

In this role, you will identify and deliver new data sources and develop ETL processes, collaborate with the Data Acquisition, Analytics and Infrastructure teams to deliver the data strategy through the design and implementation of technology systems.

We value diversity and provide an open, inclusive, and supportive environment to help you do your best work.

Where business needs allow, some roles may be suitable for a combination of office and home-based working. Where this is the case, employees will be expected to spend a minimum of 60% of their working time in the office. Applicants can raise any queries to the email address at the bottom of the advert.

Watch this short video to hear from members of Home Office Digital talking about the projects they work on and their experience of working here: Working for Home Office Digital.

Job description

This is an opportunity to lead a team of data engineers working on complex data pipelines, ingesting and transforming data from/to a variety of sources (email, csv, ODBC/JDBC, JSON, XML, Excel, Avro, Parquet). Data pipelines are primarily developed in AWS using a variety of tools and languages including AWS S3, AWS Athena, AWS Redshift, AWS Glue, java (Docker / Spring).

The position requires leading the data engineering development for a team of data engineers working on a legacy Oracle data platform and delivering data integration and data retrieval services using PL-SQL. A complex transformation programme is delivering the change of people, process and technology to redesign and transform the processing of the Oracle data outputs onto the strategic data platform. This role would ideally have experience of Oracle and PL-SQL, with the aptitude to move to cloud native tooling such as AWS Athena, python, S3. Additionally, a key priority is to manage the people change to ensure the team stability and business continuity throughout the transformation.

Person specification

Main Responsibilities

As a lead Data Engineer, you will showcase a true passion for data and exhibit expertise in the following areas:

• Manage data sources effectively to improve data quality and make them suitable for analysis.
• Drive and develop new data models and ETL processes, working collaboratively with the analytics team and mapping data to an appropriate matching data model.
• Utilise essential key stakeholder management skills to effectively manage expectations, actively listen and interpret business needs, communicate progress, and navigate challenging conversations, all while fostering a service orientated culture around data.
• Managing the performance of suppliers and the data team to improve capability, as well as creating the roadmap which balances immediate user needs with long-term investment.
• Developing processes to balance data availability, capacity and data archiving, while continually improving processes and efficiency in the acquisition and ingestion of data.
• Identifying data issues in data pipelines and managing their resolution.
• Proficiency in Oracle and Java for developing and maintaining data pipelines and integration solutions.
• Proven track record of leading engineering teams through transformation programs, specifically migrating from legacy platforms to modern cloud-native solutions (e.g., AWS).
• The successful candidate will be responsible for upskilling staff to transition to a new platform. They must also have leadership experience, including coaching and professional development skills.

Working Pattern

This role is available on a full-time basis with the option of compressed hours working.

Essential Skills

As a Lead Data Engineer you’ll have a demonstrable passion for Data, with the following skills or some experience in:

• Effectively leading and managing teams, helping them deliver technical products and projects in data analysis, solution design and end user reporting.
• Automate IT structures by advising on best practice IT programming and automation techniques and tools such as GIThub, GITLab or other source control software, Python.
• You will lead the implementation and strategic use of Cloud Data technologies, solutions, and future Cloud Data strategies particularly in the context of data warehousing and data lake architectures. This includes expertise in AWS services such as Redshift, Athena, EC2, IAM, AWS Lambda and CloudWatch
• Working with stakeholders to gather requirements and deliver findings.
• Presenting complex ideas and recommendations that inform decisions in a clear way to technical and non-technical audiences and senior customers.
• Using tools such as industry standard ETL tools, network databases and scheduling and orchestration SQL tools.
• Strong analytical and problem-solving skills troubleshooting data issues and optimising data processes.
• Strong experience with transactional databases, including design, optimisation, and troubleshooting of high-volume data systems.

SFIA capability framework

Skills for the Information Age (SFIA) version 8 is the technical framework that sets the standard capability and development of all levels in the Home Office. This is a link to the capability framework: All skills A - Z English (sfia-online.org).

We use set SFIA technical skills to form our interview questions and we will assess you against these technical skills during the selection process.

The essential skills listed above are reflective of the Home Office Government Digital and Data Profession Career Framework (based on the industry standard SFIA framework). Use the SFIA Levels of responsibility to understand what would be expected for each technical skills listed below.

Strategy and Architecture

• Strategy and Planning
  • Innovation (INOV) – Level 3

• Advice and Guidance
  • Methods and Tools (METL) – Level 4

Development and Implementation

• Data and Analytics
  • Data Management (DATM) – Level 4
  • Data Modelling and Design (DTAN) – Level 4
  • Database Design (DBDS) – Level 3
  • Data Engineering (DENG) – Level 4

Behaviours

We'll assess you against these behaviours during the selection process:

• Communicating and Influencing

Technical skills

We'll assess you against these technical skills during the selection process:

• Innovation (INOV) – Level 3
• Methods and Tools (METL) – Level 4
• Data Management (DATM) – Level 4
• Data Modelling and Design (DTAN) – Level 4
• Database Design (DBDS) – Level 3
• Data Engineering (DENG) – Level 4

Benefits

Alongside your salary of £62,109, Home Office contributes £17,992 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.

Why work for us...

Find out more information at: Benefits - Home Office Careers, but some of the primary ones are:

• A Civil Service Pension with employer contribution rates of at least 28.97%.
• In-year reward scheme for one-off or sustained exceptional personal or team achievements.
• 25 days annual leave on appointment, rising with service.
• 8 days of public holidays, plus 1 additional privilege day.
• Where business needs allow, some roles may be suitable for a combination of office and home-based working. This is a non-contractual arrangement where all employees will be expected to spend a minimum of 60% of their working time in an office.

Sign-up on our website to receive emails with information about careers at the Home Office.

Things you need to know

Artificial intelligence

Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use.

Selection process details

This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Experience and Technical skills.

As part of the application process, you will be asked to complete a CV and Personal Statement (up to 1000 words). Please note your CV and personal statement should include all relevant experience that relates to our essential skills criteria listed in the advert and role description. We recommend that you use the STAR format in your examples and ensure that both components are completed thoroughly.

Remove information that identifies you (for example your name, age or place of education) so that you will be judged on merit alone and not your personal background, circumstances, race or gender. Do NOT include e-mail addresses or links to online profiles, resumés, or prior work, either personal or business. Active links or e-mail addresses will result in your application being rejected.

Watch our three short videos on how to apply for our roles: Applying - Home Office Careers.

Sift Stage

The sift will be held on theCV and Personal Statement. Please read the essential skills for this position carefully. We will only consider those who meet the listed requirements.

In the event of a high number of applications received, an initial sift may be held on the Personal Statement.

Interview Stage

Candidates reaching the required standard will then be invited to attend an interview. The interview will assess your Technical Skills (SFIA Framework) and Behaviours using technical and behaviour-based questions.

If you are invited to an interview, you will be required to bring a range of documentation for the purposes of establishing identity and to aid any pre-employment checks. Please see the attached list of Home Office acceptable ID documents.

Sift and Interview dates

The sift will commence from 22/04/2026.

Interviews are expected to take place from 18/05/2026. (Subject to the Panel’s operational requirements/priorities).

Interviews will be conducted remotely via MS Teams.

We will try to meet the dates set out in the advert. There may be occasions when these dates will change. You will be provided with sufficient notice of the confirmed dates.

Further information

If you have previously made an unsuccessful application for a role with the same essential skills and are not able to demonstrate how you have developed these skills since your last application, please reconsider applying as your application is unlikely to be successful.

In order to process applications without delay, we will be sending a Criminal Record Check to Disclosure and Barring Service on your behalf. However, we recognise in exceptional circumstances some candidates will want to send their completed forms directly. If you are doing this, please advise Government Recruitment Service of your intention by emailing Pre-EmploymentChecks.grs@cabinetoffice.gov.uk stating the job reference number in the subject heading.

Reserve List

A reserve list of successful candidates will be kept for 12 months. Should another role become available within that period you may be offered this position.

Reserve lists may be held for each location. Candidates will be appointed in merit order by location where this applies.

Similar Roles

Job offers to this post are made based on merit. We often have similar roles available at different grades. If a candidate is suitable for a similar role or a lower grade than they have applied for, we may offer the candidate that role without the need to go through a further selection process providing the role has the same competencies and essential skills.

UK residency and security requirements

For meaningful security checks to be carried out, individuals need to have lived in the UK for a sufficient period of time. Learn more on our website. Security Checks - Home Office Careers.

Please note that this role requires Security Check (SC) clearance, which would normally need 5 years’ UK residency in the past 5 years.

However, in exceptional circumstances security clearance applications for candidates who have been present in the UK for at least 3 of the last 5 years may be considered. Failure to meet this residency requirement will result in your security clearance application being rejected.

For further information on National Security Vetting please visit the following page Demystifying Vetting - GOV.UK.

Visa Sponsorship

We are unable to sponsor any individuals via Skilled Worker Sponsorship / Tier 2 (General) work visas as we do not hold a UK Visa & Immigration (UKVI) Skilled Worker License.

Capability allowance

The advertised role is part of the Home Office Government Digital and Data Profession. This role has access to a digital capability-based allowance. Applicants who are successful at interview will be invited to complete a Capability and Skills Assessment post-interview. Any allowance awarded will be based on the assessment of your capability against the six skills advertised for this role.​ Please see the Home Office Pay Framework Allowance Careers page for more information. ​

The allowance values are set by the Home Office, subject to remaining in a qualifying role. This allowance is non-contractual, subject to an annual review and could be withdrawn at any time.​

For both new entrants and existing civil servants, the total compensation offer is a combination of base salary and, if applicable, a capability-based allowance. The pay ranges for this role are National: £62,109 - £68,320 and London: £66,229 - £72,852. New entrants to the Civil Service will start on the pay range minimum. For existing civil servants, our policies on level transfer and promotion will apply.

Working at the Home Office

Every day, Home Office civil servants do brilliant work to develop and deliver policies and services that affect the lives of people across the country and beyond. To do this effectively and fairly, the Home Office is committed to representing modern Britain in all its diversity, and creating a welcoming, inclusive workplace where all our people can bring their whole selves to work and perform at their best.

We are flexible, skilled, professional and diverse. We work to recruit and retain disabled staff and area Disability Confident Leader. We are proud to be one of the most ethnically diverse departments in the civil service. We are a Social Mobility Foundation top 75 employer.

New entrants are expected to join on the minimum of the pay band.

Applicants who are successful at interview will be, as part of pre-employment screening, subject to a check on the Internal Fraud Database (IFD). This check will provide information about employees who have been dismissed for fraud or dishonesty offences. This check also applies to employees who resign or otherwise leave before being dismissed for fraud or dishonesty had their employment continued. Any applicant's details held on the IFD will be refused employment.

For further information please see the attached notes for candidates which must be read before making an application.

Existing Civil Servants should note that some of the Home Office terms and conditions of employment have changed. It is the candidate’s responsibility to ensure they are aware of the Terms and Conditions they will adopt should they be successful in application and should refer to the notes for candidates for further details.

Transfer Terms: Voluntary.

You will need to meet the nationality requirements for this role and obtain the necessary security clearance to take it up.

Any move to the Home Office from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may however be eligible for other government schemes, including Tax Free Childcare. Determine your eligibility at https://www.childcarechoices.gov.uk.

Reasonable Adjustments

If a person with disabilities is at a substantial disadvantage compared to a non-disabled person, we have a duty to make reasonable changes to our processes.

If you need a change to be made so that you can make your application, you should:

• Contact Government Recruitment Service via HOrecruitment.grs@cabinetoffice.gov.uk as soon as possible before the closing date to discuss your needs.

• Complete the “Assistance Required” section in the “Additional Requirements” page of your application form to tell us what changes or help you might need further on in the recruitment process. For instance, you may need wheelchair access at interview, or if you are deaf, a language service professional.

If you are experiencing accessibility problems with any attachments on this advert, please contact the email address in the ‘Contact point for applicants’ section.

Feedback

Feedback will only be provided if you attend an interview or assessment.

Security

Successful candidates must undergo a criminal record check.Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).

See our vetting charter (opens in a new window).People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.

Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see theCivil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).

Apply and further information

This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative.Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Contact point for applicants

Job contact :

• Name : AC Recruitment
• Email : ACRecruitment@homeoffice.gov.uk

Recruitment team

• Email : HOrecruitment.grs@cabinetoffice.gov.uk

Further information

If you feel that your application has not been treated in accordance with the Civil Service Recruitment Principles. and you wish to make a complaint, then contact the Government Recruitment Service via HOrecruitment.grs@cabinetoffice.gov.uk. If you are not satisfied with the response that you receive, then you can contact the Civil Service Commission.

Attachments

candidate_information_v16_January_2026 Opens in new window (pdf, 429kB)2025-11-01 Pre-employment Checking - Document Requirements (V17) Opens in new window (pdf, 642kB)HO Government Digital and Data Skills Library V1.2 Opens in new window (pdf, 400kB)