Senior Data Engineer

Description: Join a team modernizing OAG’s core Flight Status data platform. Build high-throughput, near real-time ingestion and processing pipelines using Azure Databricks and Delta Lake. Requirements: Strong experience in building and scaling production-grade streaming and batch ETL/ELT data pipelines. Proficiency with Azure Databricks, Apache Spark (PySpark), Structured Streaming, Delta Lake, and Unity Catalog. Deep understanding of modern data patterns and pipeline design. Experience with Databricks Workflows and event streaming platforms. Strong background in automated testing, CI/CD tools, and data observability practices. Experience using AI coding assistants to enhance development. Excellent analytical skills and technical communication abilities. Benefits: Flexible working arrangements with options for remote work. Attractive compensation package including private health insurance, bonus scheme, and pension. 22 days holiday plus additional benefits for service and personal milestones. Annual learning and development budget. Opportunity to work remotely anywhere in the world for up to 1 month per year. Free lunch and snacks provided in the office.