SeniorAdministrator - Extract Transform Load (ETL), Data Lakes

Job Summary

Job Summary : Job Description –Data Engineer Job Title: Data Engineer Job Family: Engineering / Data & Analytics Experience: 4+ years Location: Bangalore Role Overview Looking for a hands-on Data Engineer to design, develop and maintain scalable data pipelines, data-processing services and integration solutions supporting enterprise applications, analytics and AI-enabled solutions. The candidate should possess strong Python and SQL programming skills, along with practical experience in data ingestion, ETL/ELT, API integration, relational and NoSQL databases, data modeling and data-quality engineering. The role requires someone who can independently develop production-quality data components while collaborating with application developers, architects, AI/ML engineers, business analysts and other engineering teams. Job Responsibilities : Key Responsibilities • Design, develop and maintain scalable data ingestion, transformation and processing pipelines. • Develop production-quality data engineering components using Python. • Write complex and optimized SQL queries for data extraction, transformation, validation and analysis. • Develop and maintain ETL/ELT pipelines for structured and semi-structured data. • Integrate data from databases, enterprise applications, REST APIs, files and other data sources. • Develop reusable Python modules for ingestion, transformation, validation, logging and exception handling. • Design and implement relational database schemas and data models. • Work with SQL and NoSQL databases based on application requirements. • Implement data-quality controls including validation, reconciliation, deduplication and completeness checks. • Develop and consume REST APIs for data integration. • Work with enterprise search technologies such as Elasticsearch/OpenSearch or Apache Solr where required. • Optimize database queries and pipelines for performance and scalability. • Implement appropriate error handling, retry, logging and monitoring mechanisms. • Develop automated unit, integration and data-quality tests. • Troubleshoot data, database, API and pipeline-related issues across development and production environments. • Follow Git-based development, code-review and CI/CD practices. • Work collaboratively with architects and engineering teams to translate requirements into technical implementations. • Create and maintain appropriate technical documentation.

Key Responsibilities

Key Sourcing Information : NA

Skill Requirements

Skill Requirement : Required Technical Skills Skill Area Expected Proficiency Python Strong – Mandatory SQL Strong/Advanced – Mandatory Data Engineering ETL/ELT, ingestion, transformation, validation Python Libraries Pandas, NumPy, PyArrow or equivalent Relational Databases SQL Server / PostgreSQL / MySQL NoSQL MongoDB / document databases or equivalent API Integration REST APIs, JSON, authentication Data Modeling Relational modeling, schema design, dimensional concepts Data Formats JSON, CSV, XML, Parquet Search Technologies Elasticsearch/OpenSearch, Apache Solr Version Control Git / GitHub / GitLab Testing PyTest, unit testing, integration and data-quality testing Linux Shell commands, scripting and troubleshooting Containerization Docker CI/CD Basic understanding of automated build/test/deployment pipelines SQL Skills The candidate should have practical experience with: Complex Joins • CTEs • Subqueries • Window Functions • Aggregations • Views • Stored Procedures • Transactions • Indexes • Query Execution Plans • Query Optimization The engineer should understand how database design and indexing decisions affect application and pipeline performance. Data Engineering Skills The candidate should understand an end-to-end data flow such as: Source Systems → Ingestion → Validation → Transformation → Data Quality → Storage → Search/Analytics/Application Consumption Experience should include: • Batch data processing • Incremental data loading • ETL/ELT • Schema validation • Data cleansing • Deduplication • Reconciliation • Error handling • Retry/recovery • Data lineage fundamentals • Metadata handling AI/GenAI exposure – desirable Educational Qualification Bachelor\'s or Master\'s degree in Computer Science, Information Technology, Engineering, Data Science or a related technical discipline, or equivalent practical experience.

Other Requirements

Other Requirement : Professional Competencies The candidate should demonstrate: • Strong analytical and problem-solving skills • Good programming discipline • Ability to independently troubleshoot technical issues • Understanding of software engineering best practices • Ability to work within Agile development teams • Effective communication and collaboration • Ability to understand technical requirements and translate them into working solutions • Willingness to learn emerging data and AI technologies.