Apache Spark Technical Specialist - Scala, Python

Job Summary

Quantexa

Hands on development work on Scala/Spark, Hadoop, platform operations with Quantexa Certified Experience.Data integration – ETL tools like IBM Datastage and Informatica. Ingestion mechanism like Flume & Kafka.
Data modelling – Dimensional modelling using RDBMS, NO-SQL and Big Data technologies.
Big data – Hadoop eco-system, Distributions like Cloudera / Hortonworks, Pig and HIVE.
Data processing frameworks – Scala/python/Spark.
Hands-on experience with databases like Oracle, MS SQL Server, NOSQL (HBase / Cassandra, MongoDB)
Data Platform administration for Hadoop, Elastic, Oracle.
Exposure to containers, kubernetes, application setup (load balancer), logging, Prometheus

Key Responsibilities

1. Develop and implement data processing applications using apache spark, scala, and python.
2. Design and maintain scalable data pipelines to process large volumes of data efficiently.
3. Perform data analysis and optimization to enhance the performance of data processing tasks.
4. Collaborate with cross functional teams to understand business requirements and translate them into technical solutions.
5. Troubleshoot and resolve issues related to data processing and data quality.
6. Stay updated with the latest advancements in apache spark, scala, and python to continuously improve data processing capabilities.

Skill Requirements

1. Proficiency in apache spark for big data processing and analytics.
2. Strong programming skills in scala for developing scalable applications.
3. Advanced knowledge of python programming for data manipulation and analysis.
4. Experience with data processing frameworks and tools in the apache spark ecosystem.
5. Understanding of distributed computing principles and parallel data processing.
6. Excellent problem-solving skills and ability to work in a fast paced environment.
7. Strong communication skills and ability to collaborate effectively with team members.

Other Requirements

null