Lead Data Engineer

About the role

We’re looking for an experienced Lead Data Engineer to lead the design and evolution of enterprise data platforms supporting operational, analytical, AI/ML, and Generative AI workloads. The role involves overseeing data architecture, building robust pipelines, and mentoring a team of data engineers.

Key responsibilities

• Own and evolve enterprise data architecture for operational, analytical, and AI workloads.
• Design scalable data platforms across transactional databases, data warehouses/lakehouses, and AI/ML pipelines.
• Define data models, integration standards, metadata management, and data lifecycle strategies.
• Build and optimise ETL/ELT pipelines for structured and unstructured data, including batch and real‑time integrations from EHRs, practice management systems, APIs, and third‑party healthcare applications.
• Develop data pipelines for AI/ML model training, feature engineering, vector databases, RAG, and LLM/SLM applications.
• Evaluate emerging technologies and recommend architectural improvements.
• Lead, mentor, and develop a team of Data Engineers, conducting code and architecture reviews.

Required profile

• Bachelor’s degree in Computer Science, Information Technology, Engineering or a related discipline.
• 6+ years of experience in Data Engineering, Data Platform Engineering, or Data Architecture.
• Hands‑on experience with modern data engineering technologies such as Apache NiFi, Airflow, dbt, Spark, and Kafka.
• Familiarity with healthcare interoperability standards like HL7 and FHIR.

Required skills

• Apache NiFi
• Airflow
• dbt
• Apache Spark
• Kafka
• Power BI
• HL7 and FHIR standards
• CI/CD and Infrastructure as Code
• Docker
• Kubernetes