Lead Data Engineer

Job summary

If you want to work at the forefront of technology, regulating digital markets and protecting consumers, this role offers the chance to develop cutting-edge data engineering skills within a supportive team and make a real difference in society.

The CMA is a world-leading competition authority at the forefront of tackling pressing public policy questions, including the regulation of the biggest digital markets. Our mission is to make markets work well in the interests of consumers, businesses, and the economy.

The Data, Technology and Insights (DTI) team is an established multidisciplinary team that brings together data engineers, data scientists, technologists, behavioural scientists and digital forensics and eDiscovery experts.

We are fully committed to being a flexible, respectful and inclusive employer. It is a priority to make our workforce as diverse as possible, and we welcome applications from under-represented groups.

You can read more about working at the CMA in our Candidate Information Pack

Job description

As a Lead Data Engineer in DTI, you will provide technical leadership for data engineering at the CMA. You will act as technical lead on specific projects and programmes, taking responsibility for architectural decisions, engineering standards, and delivery quality within those initiatives. You will ensure solutions are scalable, maintainable, and aligned with agreed organisational principles.

You will own the technical implementation of workflow orchestration, scheduling and monitoring to ensure data pipelines are resilient and maintainable. You will oversee containerised data and analytics applications running on our AWS-based infrastructure and ensure Infrastructure as Code, security and operational standards are applied consistently.

You will line-manage and technically supervise Data Engineers, setting expectations for code quality and design, conducting detailed reviews, and ensuring consistent adherence to agreed patterns and best practice.

You will work closely with analysts, data scientists, and engineers to translate organisational priorities into robust and scalable technical solutions, identifying risks early and proposing pragmatic improvements within the agreed strategic framework.

Person specification

It is essential that you can provide evidence and examples for each of the following selection criteria in your application. For tips on how to make the most your application, please have a look at our guidance document.

• Extensive experience delivering and leading production data pipelines end-to-end in a cloud environment (AWS Preferred), including architectural design and operational ownership. (Lead Criteria)

• Demonstrable experience acting as technical lead on significant initiatives or projects, including setting standards, reviewing code, and guiding other engineers. (Lead Criteria)

• A degree in a quantitative subject or equivalent professional experience in data engineering, data science, or cloud-based software development. (Lead Criteria)

• Strong experience writing and reviewing production-quality Python code to work with complex or messy datasets.

• Strong understanding of data engineering best practice, including data modelling, validation, testing, observability, security and reproducibility.

• Experience designing and operating data systems using Infrastructure as Code in a cloud environment.
• Strong communication skills, with the ability to explain complex technical decisions clearly and provide constructive challenge.

Behaviours

We'll assess you against these behaviours during the selection process:

• Developing Self and Others
• Working Together
• Making Effective Decisions
• Leadership

Technical skills

We'll assess you against these technical skills during the selection process:

• Data Engineering knowledge
• Coding skills

Benefits

Alongside your salary of £63,800, Competition & Markets Authority contributes £18,482 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.

• 25 days leave (increasing to 30 days over five years), plus 8 public holidays and an additional day off for the King’s birthday. In addition, you’ll be able to access a wide range of other types of leave as and when you need it, including generous maternity, paternity, shared parental leave and adoption options, as well as paid special leave for volunteering
• Season ticket loans, cycle to work scheme, flu vaccinations and eye tests
• Access to the Civil Service Sports & Leisure, giving discounted gym membership, high street discounts, free access to UK wide attractions and a free Tastecard
• A range of wellbeing benefits, including an employee assistance programme, flexible working options and family friendly policies, regular networking events and professional learning opportunities at work

You can read more about our benefits in our Candidate Information Pack

Any move to the CMA from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may however be eligible for other government schemes, including Tax-Free Childcare. Determine your eligibility at https://www.childcarechoices.gov.uk/.

Like many other organisations, we operate a hybrid working model with a mix of office and home working. We believe this supports us to deliver our best work while allowing the flexibility to maintain balanced and healthy lives.

We are fully committed to being an inclusive employer and ensuring equal opportunities. We want a diverse workforce that reflects the consumers we serve. We welcome applications from under-represented groups, including ethnically diverse people, people with a disability, people with diverse gender identities and from people of different ages.

We are an accredited Disability Confident Employer and disabled applicants meeting the minimum criteria for the job will be given the opportunity to demonstrate their abilities at an interview.

Our Values

The CMA delivers impactful outcomes for people, business and the UK economy.​ We are Ambitious and Evidence-based, and always strive for Excellence.​ We treat everyone with Respect and are Collaborative and Inclusive.​

Everything we do is underpinned by the Civil Service values: Honesty, Integrity, Impartiality and Objectivity.

You can read more about our values in our Candidate Information Pack

Things you need to know

Artificial intelligence

Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use.

Selection process details

This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Experience and Technical skills.

Application and sift stage

• As part of the application process, you will be asked to complete a CV, personal statement, and an online application form. Further details about application requirements are listed on the application form.
• The selection process will comprise of the sift (CV & application review), and an interview. Arrangements for the technical test and interview will be enclosed in the invitation.
• Please note that it is essential that your personal statement is aligned to provide evidence that you have the knowledge and experience for each of the essential criteria (as included on the role profile attached and listed above under 'Person Specification') in your application. Your personal statement should be no more than 1250 words. You will first be sifted on the first three criteria points (Lead Criteria). If your application progresses to a full sift, all elements of the essential criteria listed under Person Specification will then be considered.
• We may raise the score required at sift stage to progress to interview if we receive a high number of applications and take through the highest performing candidates to the interview stage.

Interview stage

• At the CMA we apply a blended interview technique, allowing us to find out more about you and we use the Success Profiles framework assessing the Behaviours, experience and technical skills listed in the advert and attached role profile.
• During the interview you will be asked questions based on the Behaviours, Experience and Technical skills outlined in the role profile and will be asked questions related to the technical test.
• A reserve list may be held for up to 12 months from which further appointments may be made for the same or similar roles.

Timeline (the dates outlined below are indicative and may be subject to change)

• The advert closing date is 11 March 2026 (23:55). We are not able to accept late applications.
• The sift is scheduled to take place between 12 and 18 March 2026.
• Interviews will be held from the week commencing 07 April 2026,

Reasonable adjustments

We want to make sure no one is at a disadvantage during our recruitment process because of a disability, condition or impairment. To assist you with this, we will reduce or remove any barriers where possible and provide additional support where appropriate.

If you need a change to be made so that you can make your application, you should complete the “Assistance required” section in the “Additional requirements” page of your application form to tell us what changes or help you might need during the recruitment process.

For example, you may need wheelchair access at interview, or if you are deaf, a Language Service Professional. You might also require additional time to complete a timed assessment or a sign language interpreter to support with the relaying of information.

Feedback will only be provided if you attend an interview or assessment.

Security

Successful candidates must undergo a criminal record check.People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.

Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see theCivil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).

Apply and further information

This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative.Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.

Contact point for applicants

Job contact :

• Name : Serena Connor
• Email : recruitment@cma.gov.uk

Recruitment team

• Email : recruitment@cma.gov.uk

Further information

If your application has not been treated in accordance with the Recruitment Principles and you wish to make a complaint, you should contact Recruitment@cma.gov.uk in the first instance. If you are not satisfied with the response you receive you can contact the Civil Service Commission http://civilservicecommission.independent.gov.uk.

Attachments

Lead Data Engineer - Role Profile Feb 2026 Opens in new window (pdf, 191kB)