Azure Data Engineer - Leeds

More details

At UK Government Investments (UKGI), we create value for society from government’s most complex commercial interests. What we do is unique, and so are we. Wholly owned by HM Treasury (HMT), our team comprises c.150 experts who work together with Civil Servants to provide high quality corporate finance and corporate governance expertise to departments, advising in support of their policy objectives.

Working for UKGI provides a unique opportunity for talented individuals from the public and private sector to contribute to work of national importance. The work we do is challenging and fast-paced and is always significant in terms of social, economic, and political impact. Continuously interesting and often novel, our work spans sectors including: AI, defence, transport, energy security and space travel.

Within UKGI, we have undertaken a key advisory role in the most recent high-profile transactions involving government, including the return of NatWest group to the private sector, the acquisition of Royal Mail by IDS, the acquisition of OneWeb and subsequent merger with Eutelsat, the equity raise to fund the construction of Sizewell C nuclear plant and the re-acquisition of the Annington Homes estate. Our advice ranges from supporting companies in distress, responding to major private sector mergers and acquisitions, moving companies into public ownership and commercial negotiations with the private sector.


About the team:

UKGI’s Data and Analytics team plays a central role in enabling data-driven decision making, strengthening organisational insight, and supporting high-profile cross-government programmes. As part of the next phase of maturing UKGI’s data capabilities, we are recruiting a Data Engineer to lead the hands-on engineering required to build, scale, and maintain our data pipelines, platforms, and development practices.

The work of the team includes designing and building new tools to support operational priorities as well as cross-government data initiatives relating to monitoring, reporting and informing government’s management of its financial assets and liabilities.

This role is designed to take on technical responsibilities that include ensuring robust engineering foundations, high-quality data flows, and improved efficiency across analytical and operational workflows alongside UKGI’s Head of Data.


Job description

You will be working in a highly collaborative, open-minded and growth-oriented team that is developing improvements to how UKGI collects data across government, streamlines data structures, and utilises this data to generate products that support the delivery of government’s fiscal and financial priorities.

You will be line managed by the Head of Data, but also work in close collaboration with the Head of Analysis, Chief Data and Analytics Officer as well as UKGI teams spanning all of UKGI's disciplines.


About the Role

You will take ownership of hands-on engineering tasks critical to UKGI’s data architecture and workflows. This includes:

1) Data Pipeline Engineering
- Build, maintain and optimise data ingestion, transformation, and enrichment pipelines in Microsoft Fabric, Azure Data Factory, SQL environments and related Azure services.
- Ensure reliable scheduled execution, data validation, lineage tracking, and version control across all pipelines.
- Implement schema management practices to ensure consistency across datasets used for analytical, governance and reporting purposes.

2) Development Operations
- Own the Azure DevOps workflows for data engineering, including branch design, merge strategies, pull request standards, and validation rules.
- Maintain and troubleshoot build pipelines relevant to data workflows.
- Automate deployment processes for data assets, ensuring consistent movement across development, testing and production environments.

3) Platform Reliability and Performance
- Monitor pipeline failures, environmental constraints, and system bottlenecks, taking proactive action to resolve and prevent issues.
- Lead performance tuning across SQL workloads, Fabric transformations, and ADF orchestration.
- Implement best practices in alerting, logging, and recovery.

4) Engineering Standards, Quality and Governance
- Enforce engineering standards for version control, testing, documentation, and data validation, ensuring alignment with UKGI’s DevOps guidance.
- Ensure appropriate handling of sensitive or production-grade datasets.
- Support adoption of quality assurance and peer-review processes across the function.

About You

This role would be suitable for someone with 5+ years of professional experience in a data engineering role, with a strong foundation and appetite for further growth, though we are happy to consider all level of experience.


Essential Skills and Experience

- Strong hands-on experience with SQL, including writing complex queries, performance tuning, and working with relational data stores.
- Practical experience building and maintaining ETL/ELT pipelines using Azure Data Factory, Fabric Dataflows, or equivalent tools.
- Working knowledge of Azure DevOps for CI/CD processes, including branch management, pull requests, and automated validations.
- Experience with data modelling, schema design, and structuring datasets for analytical and operational workloads.
- Ability to troubleshoot and resolve pipeline failures, permission issues, and environment-level blockers.
- Familiarity with Python or similar scripting languages for data tasks.
- Strong communication skills and ability to work with non-technical colleagues.
- Experience working within Agile or iterative delivery teams.


Desirable

- Experience with Microsoft Fabric engineering components (e.g. Data Factory in Fabric, Lakehouse, SQL Endpoint).
- Exposure to CI/CD for Fabric-native data workflows.
- Familiarity with principles of data governance and quality assurance.
- Understanding of containerisation (e.g. Docker) concepts as they relate to engineering workflows.


Diversity statement

UKGI has a strong commitment to equality and diversity. Our aim is to be an open and inclusive organisation, recruiting and retaining diverse, talented and high-performing people who support and develop one another.

UKGI is a Disability Confident Employer. This means we’ve been recognised as an employer which is confident and leading the way in recruiting and retaining staff with disabilities. We will offer an interview to any applicants with a disability who have indicated they wish to take part in the disability confident scheme, provided they meet the essential criteria for the post set out in the person specification. If you need any reasonable adjustments to take part in the selection process, please tell us about this in your application.

UKGI is also part of the ‘Great Place to Work for Veterans’ initiative. This means that as part of our commitment to diversity and equality in our recruitment and retention policy, we will offer an interview to any eligible applicants from His Majesty’s Armed Forces provided they meet the essential criteria for the post set out in the person specification. Please indicate on your application that you wish for your application to be considered as part of this initiative.

In line with company policy, we will be employing a hybrid working model based, with 40% of your time being home-based. At UKGI we place a high degree of value on work life balance and as such applications from individuals seeking flexible working, reduced hours contracts and job shares are also actively encouraged.

If you wish to learn more about how your data is used, please click on the document attached to this role.