The Azure Databricks - Data Engineer – FinOps & Cloud Ops is responsible for building, operating, and optimizing cloud cost and operations analytics platforms using Azure Databricks, ADLS Gen2, and FinOps frameworks. The role combines data engineering, cloud operations, and financial governance to support Cloud Center of Excellence (CCoE) and Operational Excellence programs across multi-cloud environments (Azure, AWS, GCP).
Azure Databricks & Data Engineering
- Design and implement Bronze / Silver / Gold data lake architecture using Azure Databricks + ADLS Gen2.
- Build scalable ingestion pipelines using Databricks Autoloader, Azure Data Factory, and APIs.
- Develop and optimize Spark (PySpark / SQL) jobs for large cloud billing, usage, and telemetry datasets.
- Implement Unity Catalog for fine-grained access control, auditing, and governance
-
inOps (Cloud Cost Management)
- Ingest and normalize cost & usage data from Azure Cost Management, AWS CUR, Cloudability, Excel-based finance sources, and ServiceNow CMDB.
- Build cost allocation, chargeback/showback, and optimization datasets.
- Develop Power BI semantic models and dashboards on Databricks Gold layer.
- Partner with Finance and Cloud teams to deliver cost transparency and optimization insights.