Senior Data Engineer
This is the SFIA description for the Senior Data Engineer position.
Responsibilities
Summary as follows:
| Responsibility | Level |
|---|---|
| Autonomy | 4 |
| Influence | 4 |
| Complexity | 4 |
| Business skills | 4 |
| Knowledge | 4 |
Autonomy
- Works under general direction within a clear framework of accountability. Exercises substantial personal responsibility and autonomy. Uses substantial discretion in identifying and responding to complex issues and assignments as they relate to the deliverable/scope of work. Escalates when issues fall outside their framework of accountability. Plans, schedules and monitors work to meet given objectives and processes to time and quality targets.
Influence
- Influences customers, suppliers and partners at account level. Makes decisions which influence the success of projects and team objectives. May have some responsibility for the work of others and for the allocation of resources. Engages with and contributes to the work of cross-functional teams to ensure that customers and user needs are being met throughout the deliverable/scope of work. Facilitates collaboration between stakeholders who share common objectives. Participates in external activities related to own specialism.
Complexity
- Work includes a broad range of complex technical or professional activities, in a variety of contexts. Investigates, defines and resolves complex issues. Applies, facilitates and develops creative thinking concepts or finds innovative ways to approach a deliverable.
Business skills
- Communicates fluently, orally and in writing, and can present complex information to both technical and non-technical audiences when engaging with colleagues, users/customers, suppliers and partners.
- Selects appropriately from, and assesses the impact of change to applicable standards, methods, tools, applications and processes relevant to own specialism.
- Demonstrates an awareness of risk and takes an analytical approach to work.
- Maximises the capabilities of applications for their role and evaluates and supports the use of new technologies and digital tools.
- Contributes specialist expertise to requirements definition in support of proposals.
- Shares knowledge and experience in own specialism to help others.
- Learning and professional development — maintains an awareness of developing practices and their application and takes responsibility for driving own development. Takes the initiative in identifying and negotiating their own and supporting team members' appropriate development opportunities. Contributes to the development of others.
- Security, privacy and ethics — fully understands the importance and application to own work and the operation of the organisation. Engages or works with specialists as necessary.
Knowledge
- Has a thorough understanding of recognised generic industry bodies of knowledge and specialist bodies of knowledge as necessary. Has gained a thorough knowledge of the domain of the organisation. Is able to apply the knowledge effectively in unfamiliar situations and actively maintains own knowledge and shares with others. Rapidly absorbs and critically assesses new information and applies it effectively.
Skills
Core
Summary as follows:
| Skill | Code | Level |
|---|---|---|
| Data engineering | DENG | 4 |
| Programming/software development | PROG | 4 |
| Testing | TEST | 4 |
| Release and deployment | RELM | 4 |
| IT infrastructure | ITOP | 4 |
Data engineering
- Designs, implements, and maintains complex data engineering solutions to acquire and prepare data.
- Creates and maintains data pipelines to connect data within and between data stores, applications and organisations.
- Carries out complex data quality checking and remediation.
Programming/software development
- Designs, codes, verifies, tests, documents, amends and refactors complex programs/scripts and integration software services.
- Contributes to the selection of the software development methods, tools and techniques.
- Applies agreed standards and tools to achieve well-engineered outcomes.
- Participates in reviews of own work and leads reviews of colleagues' work.
Testing
- Selects appropriate testing approach, including manual and automated testing.
- Develops and executes test plans and test cases. Implements scalable and reliable automated tests and frameworks.
- Collaborates across parties involved in product, systems or service design and development to enable comprehensive test coverage. Identifies improvements in requirements, design or specification processes to increase the effectiveness and efficiency of testing.
- Analyses and reports on test activities, results, issues and risks, including the work of others.
Release and deployment
- Assesses and analyses release components for input to release scheduling.
- Maintains and administers tools and methods for software delivery, deployment and configuration.
- Maintains release processes and procedures.
IT infrastructure
- Provides technical expertise to enable the correct application of operational procedures.
- Contributes to the planning and implementation of infrastructure maintenance and updates. Implements agreed infrastructure changes and maintenance routines.
- Uses infrastructure management tools to determine load and performance statistics. Configures tools and/or creates scripts to automate the provisioning, testing and deployment of new and changed infrastructure. Maintains operational procedures and checks that they are executed following agreed standards.
- Investigates and enables the resolution of operational issues. Provides reports and proposals for improvement, to specialists, users and managers.
Data Maturity
Summary as follows:
| Skill | Code | Level |
|---|---|---|
| Data science | DATS | 4 |
| Business intelligence | BINT | 4 |
| Machine learning | MLNG | 4 |
Data science
- Investigates the described problem and dataset to assess the usefulness of data science and analytics solutions.
- Applies a range of data science techniques and uses specialised programming languages. Understands and applies rules and guidelines specific to the industry, and anticipates risks and other implications of modelling.
- Selects, acquires and integrates data for analysis. Develops data hypotheses and methods and evaluates analytics models. Advises on the effectiveness of specific techniques based on project findings and comprehensive research.
- Contributes to the development, evaluation, monitoring and deployment of data science solutions.
Business intelligence
- Supports business intelligence needs of specific management or governance processes or operational areas.
- Investigates the need for business intelligence reporting and analysis where there is some complexity and ambiguity.
- Selects and applies non-standard business intelligence tools and techniques to provide insights and aid decision-making. Selects, acquires and integrates data for analysis.
- Identifies opportunities to digitise and streamline operational data handling and optimise business intelligence capabilities.
Machine learning
- Given a well-described problem and dataset, assesses whether machine learning is likely to provide an effective solution.
- Implements algorithms developed by others. Advises on the effectiveness of specific techniques, based on project findings and wider research.
- Contributes to the development, evaluation, monitoring and deployment of machine learning systems.
- Understands and applies rules and guidelines specific to the industry, and anticipates risks and other implications of modelling.
Engineering Maturity
Summary as follows:
| Skill | Code | Level |
|---|---|---|
| Systems and software life cycle engineering | SLEN | 4 |
| Systems integration and build | SINT | 4 |
| Database design | DBDS | 4 |
| Acceptance testing | BPTS | 4 |
Systems and software life cycle engineering
- Elicits requirements for systems and software life cycle working practices and automation.
- Prepares design options for the working environment of methods, procedures, techniques, tools, and people.
- Selects systems and software life cycle working practices for software components and micro-services.
- Deploys automation to achieve well-engineered and secure outcomes.
Systems integration and build
- Provides technical expertise to enable the configuration of system components and equipment for systems testing.
- Collaborates with technical teams to develop and agree system integration plans and report on progress. Defines complex/new integration builds. Ensures that integration test environments are correctly configured.
- Designs, performs and reports results of tests of the integration build. Identifies and documents system integration components for recording in the configuration management system.
- Recommends and implements improvements to processes and tools.
Database design
- Implements physical database designs to support transactional data requirements for performance and availability.
- Develops and maintains specialist knowledge of database and data warehouse concepts, design principles, architectures, software and facilities.
- Assesses proposed changes to object/data structures and evaluates alternative options.
- Implements data warehouse designs that support demands for business intelligence and data analytics.
Acceptance testing
- Develops acceptance criteria related to functional and non-functional requirements, business processes, features, user stories and business rules.
- Designs, specifies and executes test cases and scenarios to test that systems, products and services fulfil the acceptance criteria and deliver the predicted business benefits.
- Collaborates with project colleagues and stakeholders involved in the analysis, development and operation of products, systems or services to ensure accuracy and comprehensive test coverage.
- Analyses and reports on test activities, results, issues and risks including the work of others.