Assistant Manager

Role: Data Bricks Developer 

Experience: 5+ Years 

Location: Gurgaon OR Bangalore 

Work Mode: Work From Office [5 Days Office] 

POSITION SUMMARY 

The Databricks Data Engineer will be responsible for designing, building, and optimizing scalable data pipelines and lakehouse solutions using Databricks. The role requires strong hands-on experience in data engineering, distributed data processing. 

ROLES AND RESPONSIBILITIES: 

• Design, build, and maintain ETL/ELT pipelines on Databricks using PySpark, Spark SQL, and Delta Lake. 

• Develop and optimize data ingestion frameworks, data transformations, and end to end workflows for batch and streaming use cases. 

• Implement Delta Lake based architectures, including versioning, schema evolution, and ACID compliant pipelines. 

• Work with stakeholders to understand data requirements and translate them into scalable data engineering solutions. 

• Manage and optimize Databricks clusters, jobs, and notebooks for performance and cost efficiency. 

• Ensure data quality, reliability, and observability through validation frameworks and monitoring. 

• Contribute to data modeling, metadata management, and best practices within the data platform. REQUIRED QUALIFICATIONS 

• 3+ years of experience in data engineering with 2+ years of hands on expertise in Databricks. 

• Hands on experience with Spark (PySpark/Spark SQL) and distributed data processing. 

• Solid SQL knowledge and experience working with large-scale datasets 

• Strong understanding of Delta Lake, medallion architecture, and scalable lakehouse patterns. 

• Good understanding of CI/CD, Git, and modern DevOps practices for data pipelines. 

• Familiarity with structured/unstructured data, data quality frameworks, and performance tuning. 

EDUCATION: Bachelor’s degree in computer science, Software Engineering, MIS or equivalent combination of education and experience 

KEY SKILLS: Data Engineering, Python, Pyspark, Azure Cloud, Azure Data Bricks

• Design, build, and maintain ETL/ELT pipelines on Databricks using PySpark, Spark SQL, and Delta Lake.  

• Develop and optimize data ingestion frameworks, data transformations, and end to end workflows for batch and streaming use cases. 

• Implement Delta Lake based architectures, including versioning, schema evolution, and ACID compliant pipelines. 

• Work with stakeholders to understand data requirements and translate them into scalable data engineering solutions. 

• Manage and optimize Databricks clusters, jobs, and notebooks for performance and cost efficiency. 

• Ensure data quality, reliability, and observability through validation frameworks and monitoring. 

• Contribute to data modeling, metadata management, and best practices within the data platform. REQUIRED QUALIFICATIONS 

• 3+ years of experience in data engineering with 2+ years of hands on expertise in Databricks. 

• Hands on experience with Spark (PySpark/Spark SQL) and distributed data processing. 

• Solid SQL knowledge and experience working with large-scale datasets 

• Strong understanding of Delta Lake, medallion architecture, and scalable lakehouse patterns. 

• Good understanding of CI/CD, Git, and modern DevOps practices for data pipelines. 

• Familiarity with structured/unstructured data, data quality frameworks, and performance tuning. 

 Bachelor’s degree in computer science, Software Engineering, MIS or equivalent combination of education and experience