Jobiglo

No results.

Senior Data Engineer

NO Comp · Colombo

New
Senior 🇬🇧 English
Python FastAPI Spring Boot Hadoop Apache Spark Kafka NiFi Apache Airflow dbt SQL DB2 PostgreSQL IBM AS400 Docker Kubernetes OAuth2

Job description

About the role

This Senior Data Engineer position leads the design, build and production of scalable ETL/ELT pipelines and big data environments, while also mentoring junior and mid-level data engineers. The role covers backend data-serving APIs, orchestration, infrastructure and MLOps.

Key responsibilities

  • Architect and build highly scalable, fault-tolerant ETL/ELT pipelines from scratch and oversee them to production.
  • Design and manage open-source big data environments using Hadoop, Apache Spark, Kafka and NiFi for high-throughput, real-time and batch processing.
  • Lead backend development for data serving with Python FastAPI and Spring Boot, securing APIs with OAuth2.
  • Use Apache Airflow for complex workflow orchestration and dbt for data transformation.
  • Deploy and manage containerized applications with Docker and Kubernetes, using Linux command-line skills and network configuration.
  • Design infrastructures supporting MLOps practices for model deployment and monitoring, and handle complex time-series data storage and analysis.
  • Mentor junior and mid-level data engineers through technical guidance and code reviews.

Required profile

  • Minimum 5+ years of hands-on data engineering experience.
  • Ability to lead a small team and guide junior engineers.
  • Follows best practices for production-ready setups and uses efficient code design patterns.
  • Knowledge of multithreading and Git version control.

Required skills

  • Strong Python; FastAPI and Spring Boot.
  • Hadoop, Apache Spark, Kafka, Airflow and dbt (mandatory).
  • Advanced SQL, including DB2 SQL and PostgreSQL; experience with IBM AS400 is advantageous.
  • Docker, Kubernetes, Linux CLI, network configuration and OAuth2.

Questions fréquentes

Le salaire n'est pas communiqué publiquement par le recruteur. Vous pouvez postuler et négocier directement avec NO Comp.
Cliquez sur "Postuler maintenant" en haut de la page. Vous pouvez importer votre CV en 1 clic — Jobiglo extrait automatiquement vos informations et postule pour vous.

Why are you reporting this job?

Thank you for your report. We will review this job.

Apply in 30 seconds

Enter your email to apply. An account will be created automatically.

Apply now →

By continuing, you accept our terms of use.

Already have an account? Login

A question about this job?

Ask it here: you will get the full job summary by e-mail, right away.

Published 6 days ago

Expires 1 month from now

12 views · 0 interested

Boost your chances

Upload your CV — we will match you with relevant openings.

Analyzing your CV...

NO Comp

Colombo