Description:
RequirementsAn MS or BS degree or the equivalent practical experience3+ years’ experience working as a DataOps Engineer or in a similar role, with a strong focus on data engineering and automation3+ years of experience using Python to develop enterprise-grade software using appropriate design patterns and principlesStrong experience with cloud platforms such as AWS, Azure, or GCP, and expertise in utilizing cloud-based data services (e.g., AWS Glue, Azure Data Factory, Google Cloud Dataflow)Solid understanding of modern data architecture, including data modelling, ETL/ELT processes, and data warehousing conceptsExperience with version control systems (e.g., Git), containerization technologies (e.g., Docker/Kubernetes), infrastructure as code (e.g., terraform), and CI/CD pipelinesFamiliarity with data governance, data security, and data privacy practicesResponsibilitiesDesigned, developed, and maintained scalable data pipelines and workflows to support data acquisition, processing, and transformation activities in an agile and iterative mannerCollaborated with data scientists, software engineers, and other stakeholders to understand data requirements and implemented efficient data solutions to foster team self-service and improve collaboration and communication between departmentsImplemented and optimized data integration processes, ensuring data consistency, accuracy, and reliability across different data sources and systemsAutomated data operations tasks, including data ingestion, data validation, and data quality monitoring, using appropriate tools and technologiesIdentified and resolved data-related issues, troubleshoot performance bottlenecks, and ensured optimal data processing and storage
19 Apr 2024;
from:
gumtree.co.za