Description:
The Role:We are looking for a Senior Data Engineer for a 12 month contract.
Job Overview:
This person is responsible for creating data pipelines to support downstream data delivery.
A data pipeline is a series of automated, programmatic steps. These data pipelines gather data from numerous, potentially disparate data sources and bring everything together into a central place (data lake warehouse) where it is made readily available for downstream consumption.
The incumbent works with a variety of technologies on a day to day basis. The nature of the job requires constant upskilling and personal development.
Skills and Experience:
Qualifications Skills required:
Relevant IT degree, Diploma or certification
Good programming skills in both Python and SQL.
Good database design skills and an understanding of data warehousing techniques and approaches.
An understanding of and ability to access data from sources such as:
o Microsoft SQL Server
o Oracle
o MongoDB
o Amazon S3
o Windows and Linux file systems (Excel, delimited files, etc.)
o HTTP APIs
o SFTP sites
An ability to engineer cloud-first solutions in the Amazon Web Services cloud.
An interest in machine learning and advanced analytics, and the development of data pipelines in support thereof.
An understanding of the software development process, with proficiency in Git for version control and build deployment pipelines in GitLab.
An ability to create reports and BI dashboards to communicate data analysis results, using a variety of technologies, which might include:
o Microsoft SQL Server Reporting Services
o Microsoft PowerBI
o Microsoft Excel
17 Apr 2022;
from:
gumtree.co.za