Obligations

• Design, build, and operate Spark Structured Streaming pipelines (event-driven, CDC, API-based)
• Implement and maintain Slowly Changing Dimensions (Type 1/2) at scale
• Own data modeling and table design using Apache Iceberg (partitioning, compaction, schema evolution)
• Ensure pipeline reliability, SLAs, and data correctness across batch and streaming workloads
• Develop and operate Airflow-orchestrated data workflows
• Collaborate closely with Senior Data Engineers and DataOps on platform stability and performance
• Enforce data access controls, governance, and secure data sharing

Benefits

Opportunities for professional growth and development

Competitive salary and bonuses

Comprehensive insurance coverage

Supportive work environment

Visa Premium salary card

Corporate discounts and events

Additional vacation days

Discounted education and employee loans

New cozy studio office in Port Baku Tower 2

Strong teamwork with Agile principles, regular team buildings and themed events

Multicultural environment with foreign colleagues sharing their best experiences

Requirements

5+ years experience in data engineering

2+ years experience in production Spark Streaming

Deep understanding of Iceberg internals, lakehouse patterns, and large-scale analytics

Proven experience implementing SCDs in distributed systems

Strong Python & SQL, comfortable reading JVM-level Spark behavior

Experience running Spark on Kubernetes

Operational mindset: monitoring, alerting, incident response

Zero tolerance for fragile pipelines or undocumented logic

Nice to have:

• CDC systems (Debezium, Kafka-based ingestion)
• Performance tuning under cost and latency constraints
• On-prem or hybrid data platforms