Engineer - Data Test & Assurance

Job summary

Job purpose

As a Data Test & Assurance Engineer, you will operate as a senior technical subject matter expert providing cross-platform data test assurance practice leadership across Homes England’s Azure Databricks Platform.

You will define, embed, and assure automated data and platform testing practices end-to-end across data engineering, analytics, and Data Science / AI‑enabled workloads, ensuring solutions are reliable, secure by design, ethical, and auditable.

The role carries delegated technical leadership responsibility and may require leading and directing the work of contractors and external specialists through technical authority rather than formal line management.

Engineering Community of Practice, mentoring teams and embedding modern ways of working.

This role is crucial to providing the technical leadership and governance required for the Agency to deliver consistent data services, mature its data culture, and respond effectively to strategic and operational priorities.

Job description

Key accountabilities and responsibilities

Cross-Platform Test Strategy and Standards

• Define, own, and continuously evolve the Data Platform Test Strategy, ensuring consistent assurance across infrastructure, CI/CD pipelines, data ingestion, transformation, analytics, and AI/ML workloads. Establish secure‑ by‑design testing standards and patterns adopted across internal teams and suppliers.

Leadership of Contractors and External Resources

• Lead, direct, and quality‑assure the work of contractors and external suppliers delivering data testing, platform assurance, and AI/ML validation capabilities. Set technical direction, review deliverables, and ensure outputs meet Homes England and DDaT standards.

Automated Data, Analytics, and AI Assurance

• Design and implement automated tests for ETL/ELT pipelines, data models, analytical outputs, and machine learning artefacts, including validation of feature engineering, model inputs and outputs, data drift, and reproducibility using Python and SQL‑based frameworks.

CI/CD, Secure-by-Design, and Environment Assurance

• Embed quality and security gates into Azure DevOps CI/CD pipelines. Validate environment parity across Dev, Pre‑Prod, and Prod, and apply secure‑by‑design principles including identity and access management, secrets handling, data masking, and least‑privilege access.

Quality, Observability, and Compliance

• Implement data profiling, anomaly detection, and assurance evidence to support governance, audit, GDPR, responsible AI, and regulatory compliance.

Delivery Tracking and definition

• Maintaining and processing the data test engineering and assurance backlog in collaboration with the product and delivery managers as part of the data service offered to our customers.

Data Community of Practice

• Actively participate and contribute to the Data Analytics Community of Practices from the Data Engineering perspective.

Person specification

Key skills, knowledge and experience

• Hold a relevant degree or professional qualification in computer science, data engineering, mathematics, or a related discipline, or demonstrate minimum equivalent minimum practical experience gained though foundation level Data Engineering Level 5 apprenticeship or higher.
• Strong background in data testing and assurance across cloud data platforms
• Experience with Azure Databricks, Azure Data Factory, Synapse Analytics, and Data Lake Gen2
• Proficiency in Python (PySpark, Pandas) and SQL for automated testing and profiling
• Working knowledge of data science and machine learning lifecycles, including feature engineering and model deployment
• Understanding of AI/ML assurance topics such as bias, explainability, reproducibility, and drift
• Strong understanding of secure‑by‑design principles including IAM, data protection, and least‑privilege access
• Experience embedding testing and security controls into Azure DevOps CI/CD pipelines
• Ability to communicate assurance outcomes clearly to technical and non‑technical stakeholders
• Experienced in managing large, complex structured and unstructured datasets, including collection, validation, analysis, and transformation.
• Able to define solutions for ambiguous or complex problems independently, apply innovative approaches and adopt modern technologies.
• Understanding of project management principles, Agile methodologies, and software delivery practices, capable of leading cross-functional data initiatives.
• Awareness of data governance, security, and regulatory standards, with professional competence equivalent to formal qualifications in data engineering or analytics.
• Experience working within the Homes England sector or similar public sector organisation’s, with an understanding of sector-specific data requirements, compliance, and governance frameworks.

Benefits

Homes England is an executive non-departmental public body with its own terms and conditions, benefits and pension scheme. We are a government agency working in partnership with the private sector and want to employ and retain great people. In return we offer a range of excellent benefits, some of the key benefits include:

Holiday - Homes England offers all staff 33 days annual leave per year in addition to 8 statutory bank holidays.

Working Flexibly - At Homes England we have a strong culture of flexible working and encourage all employees to have a healthy work/life balance. We offer a wide range of potential working patterns, including part time, job share and compressed hours.

Health and Wellbeing - We offer health screening with Nuffield Proactive Health Clinics, a free Employee Assistance Programme which offers confidential support 24 hours a day, 7 days a week as well as access to Occupational Health.

Professional Development - At Homes England we are committed to your continued professional and personal development with learning as one of our core values. We pay for one job-related professional membership subscription per year for our employees.

We also offer an excellent range of family friendly benefits, for example, maternity, paternity, shared parental leave.

Things you need to know

Artificial intelligence

Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use.

Selection process details

To apply please follow the links to our careers page.

Security

Successful candidates must pass a disclosure and barring security check.

For more information on the role, please see the attached Job Description.

Successful applicants will be advised of the outcome of their application and interviews held as soon as practical after the closing date. Feedback will only be provided if you attend an interview or assessment. We would like to take this opportunity to thank you for your interest in this role and Homes England.

Please note – Homes England has a responsibility to ensure compliance with HMG Baseline Personnel Security Standard to address the potential problems of identity fraud, illegal working and deception generally. To ensure compliance we will carry out a pre-employment Personal Identity and Vetting Report through Experian to meet the standard and you will therefore be asked to provide certain information in order that the verification check can be carried out if you are successful in your application.

Homes England are an equal opportunities employer and positively encourages applications from suitably qualified and eligible candidates. We are proud to be an equal opportunities employer that values and respects the people who work for us. Our aim is to be an employer of choice, enabling everyone to reach their full potential.

Homes England is committed to protecting the privacy and security of your personal data. For further information please refer to our Privacy Notice which can be viewed here:

https://www.gov.uk/government/organisations/homes-england/about/personal-information-charter

Nationality requirements

Open to UK, Commonwealth and European Economic Area (EEA) and certain non EEA nationals. Further information on whether you are able to apply is available here.

Working for the Civil Service

Please note this Post is NOT regulated by the Civil Service Commission.

The Civil Service embraces diversity and promotes equality of opportunity. There is a guaranteed disability confident interview scheme for candidates with disabilities who meet the minimum selection criteria.

Further information

Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Contact point for applicants

Name: Recruitment Team

Email: recruitmentinfo@homesengland.gov.uk

Feedback will only be provided if you attend an interview or assessment.

Security

Successful candidates must undergo a basic (or equivalent) criminal record check.People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

Working for the Civil Service

Please note this Post is NOT regulated by the Civil Service Commission.The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.

Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see theCivil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).

Apply and further information

The Civil Service welcomes applications from people who have recently left prison or have an unspent conviction. Read more about prison leaver recruitment (opens in new window).Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Contact point for applicants

Job contact :

• Name : RecruitmentInfo@homesengland.gov.uk
• Email : RecruitmentInfo@homesengland.gov.uk

Recruitment team

• Email : Recruitmentinfo@homesengland.gov.uk