Junior Data Engineer

Accountabilities

• Work as part of the Data Engineering team to uphold and evolve common standards and best practices, collaborate to ensure that our data solutions are complementary and not duplicative.
• Build and maintain high-performance, fault-tolerant, secure, and scalable data platform to support multiple data solutions use cases.
• Interface with other technology teams to design and implement robust products, services and capabilities for the data platform making use of infrastructure as code and automation.
• Build and support platforms to enable our engineers to build our cloud based big data platform.
• Design, build, and maintain scalable data pipelines on Azure using technologies such as Azure Data Factory and Azure Synapse Analytics
• Help to engineer our platform ingestion, data warehouse/data lake and API strategies for our data management ecosystem
• Work with large, complex datasets and integrate data from multiple sources
• Work with our DevOps team to provide high availability data solutions that scale geographically

Experience

• Experience working with Cloud architecture/administration in production environments.
• Experience with object oriented and functional design, coding, and testing patterns as well as experience in engineering software platforms and large-scale data infrastructures.
• Experience writing production quality code in C#/Python/Bash/Powershell/Go, etc
• Experience of building and maintaining distributed platforms to handle high volume of data.
• Good platform-level design, architecture, implementation and troubleshooting skills.
• Experience with MS SQL
• Good understanding of Enterprise patterns and best practices applied to data engineering and data science use cases at scale.
• Good understanding of cloud storage and orchestration, and computing platforms (especially document/blob stores, GraphDB Kafka, Airflow, Elastic, Spark ).
• Good understanding of DevOps/DataOps in an Agile Environment, familiarity with Jira and Confluence.
• Experience of Docker/Kubernetes would be beneficial.
• Expertise in Databases (Postgres, MySQL, etc)
• Understanding network and security concepts on cloud-based environments, specifically on cloud services such as VPCs, Security Groups, NACLs and IAM roles.
• Good understanding of CI/CD using tools like Jenkins/CircleCI/Azure Data Pipelines, along with deep experience in source control like Git.

Competencies

• Great problem-solving skills, and the ability and confidence to hack their way out of tight corners.
• Ability to prioritise and meet deadlines.
• Conscientious, self-motivated, and goal orientated.
• Excellent attention to detail and solid written and verbal English communication skills
• Willingness and an enthusiastic attitude to work within existing processes / methodologies