Data Engineer – Build Scalable Data Pipelines

About the role

We are looking for a hands‑on Data Engineer to own end‑to‑end data pipelines that move data from operational source systems onto our data platform. You will design, develop, and operate both batch and streaming pipelines while ensuring their reliability and performance.

Key responsibilities

• Own the extraction, ingestion, transformation, and orchestration of data pipelines.
• Build foundational data products (raw → record‑of‑truth) and derivative products from upstream assets.
• Design, develop, deploy, monitor, and remediate batch and streaming pipelines.
• Handle backfills, replays, late‑arriving data, schema drift, and SLA breaches.
• Implement ingestion patterns such as full‑load, incremental, CDC (Debezium, Fivetran, GoldenGate), event‑driven, API and file‑based intake.
• Package data products with schema contracts, versioning, SLAs/SLOs, and output‑port designs.

Required profile

• Strong experience building and operating large‑scale data pipelines.
• Ability to work with both batch and real‑time streaming technologies.
• Familiarity with open‑table formats and data mesh concepts.
• Hands‑on mindset for troubleshooting and performance tuning.

Required skills

• SQL & Python (core); Scala or Java for high‑throughput streaming.
• Pipeline frameworks: dbt, Apache Spark, Apache Beam, Airflow, Dagster, Prefect.
• Streaming platforms: Kafka, Kinesis, Flink, Spark Structured Streaming.
• Storage formats: Iceberg, Delta Lake, Hudi.
• Data quality & observability: Great Expectations, Soda, Monte Carlo, OpenLineage.
• CI/CD & GitOps pipelines for data.
• AI‑adjacent tools: pgvector, Pinecone, Feast, Tecton.
• Schema contracts: Avro, Protobuf, JSON Schema.