DE&A - Core - Lead Data Engineer – Databricks
This role focuses on designing, building, and optimizing large-scale data engineering solutions using Azure Databricks. The professional will develop robust ETL/ELT pipelines with PySpark and Spark SQL, integrate data from diverse sources (SAP, APIs, Azure Data Lake, SQL databases, and streaming platforms), and support analytics, BI, and data science use cases. Responsibilities include performance tuning of Databricks clusters and Delta Lake, implementing scalable lakehouse architectures, ensuring data quality, governance, and security, and collaborating closely with onshore teams. The role also involves CI/CD automation, code reviews, troubleshooting, and maintaining reliable, production-grade data pipelines.
