Description:
An expert in integrated corporate benefits well known for outstanding usability and integration within the banking space is looking for a specialist capable of working within the capacity of a BI Big Data Engineer who will be responsible for building and maintaining data pipelines for new data in a variety of formats using an optimal orchestration method into a modern data platform. The candidate will have deep data engineering skills with experience using different programming languages and working comfortably on Linux or Windows environments. Responsibilities: Perform data analysis and normalisation to create features for machine learning applications.Ingest data into the data environment in batch or real-timeBuild Meta-data driven data integration solutionsDevelop and maintain APIs for seamless integration of modern data platform with other systems.Productise Machine Learning models on a Linux containerized environment to ensure scalability and reliability.Administer Linux operating system and exhibit a strong programming experience.Utilise distributed programming techniques on a cluster of servers for optimised data processing.Collaborate with teams to advise on optimal data system architecture for data warehousing, data lakes, and lakehouses.Conduct root cause analysis on production issues surrounding data pipelining and executionTechnical leadership of entire information management process of both structured and unstructured dataOptimisation of the solutions for loading and transforming multi structure dataCreate comprehensive technical documentation for data systems, APIs, and processes.Qualifications and Experience: National Diploma /BTech/Mtech IT or BSC/MSC Computer Science5 years plus experiencePython, Java, JavaScript, C# (Advanced)Spark SQL and R programming (Advanced)Building and maintaining API(s) and Webhooks (Advanced)Google Analytics data (Advanced)Real-time streaming and batch orchestration (Advanced)Object Oriented Programming (Advanced)SQL programming (Advanced)Data Warehouse, Data Lake and Lakehouse (Advanced)Git versioning of code (Advanced)The Reference Number for this position is NG58552 which is a Permanent position in Sandton, Johannesburg offering a salary negotiable between R750k to R950k CTC based on experience and skills. E-mail Nokuthula on
22 Apr 2024;
from:
gumtree.co.za