Data Engineer – Build Scalable Data Pipelines

About the role

iLabs is seeking a Data Engineer to design, build, and maintain scalable data pipelines that support both batch and real‑time workloads. You will work closely with software engineers, DevOps, ML engineers, and data scientists to deliver reliable data solutions that drive business insights.

Key responsibilities

• Design, develop, and operate scalable data pipelines for batch and streaming workloads.
• Implement data integration, storage, and processing solutions aligned with analytics needs.
• Ensure data quality, reliability, and performance across the platform.
• Build and maintain APIs and services that expose data to internal teams and applications.
• Write SQL, DDL, and Python code for pipeline development, validation, troubleshooting, and tuning.
• Define best practices for data modeling, documentation, backup, and recovery.
• Mentor junior engineers and contribute to code reviews and engineering standards.

Required profile

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• 3+ years of experience in a data engineering role.
• Strong experience with lakehouse architectures and Delta Lake.
• Hands‑on experience with streaming and CDC technologies such as Kafka and Debezium.
• Proficiency with workflow orchestration tools like Apache Airflow.
• Experience with distributed query engines (Trino/Presto, Hive) and advanced SQL.

Required skills

• Python
• SQL
• Delta Lake
• Lakehouse architectures
• Kafka
• Debezium
• Apache Airflow
• Trino / Presto
• Hive
• FastAPI / Django / Flask
• AWS
• Linux
• Databricks
• GCP
• Azure
• Spark
• Docker
• Kubernetes
• CI/CD
• Monitoring tools
• Selenium
• Beautiful Soup
• NoSQL databases
• Data warehousing