Job Location: Chennai
- Create and maintain optimal data pipeline architecture
- Build the infrastructure required for optimal data extraction, transformation, and loading of data from a wide variety of sources using big data technologies
- Assemble large, complex data sets that meet functional/non-functional business requirements
- Identify/design/implement internal process improvements, automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
- Build analytics tools that utilize the data pipeline to provide actionable insights to business
- Maintain security and data privacy in an environment secured using Kerberos and LDAP
- Build test prototypes for the functional/technical problems identified
- Propose ETL best practices/standards
- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs
- Work with cross-functional teams in a dynamic environment
- 4+ years of experience in big data tools: Hadoop, hive, Spark, Kafka, HBase, Flume, Pig, KUDU etc.
- 6+ years of experience in programming languages: Python/Java/Scala, etc. Flueny in writing shell scripts [bash, korn]
- Experience with relational SQL and any NoSQL databases
- Basic understanding in AWS cloud services: EC2, EMR, RDS
- Experience with stream-processing systems: Storm, Spark-Streaming, etc.
- Strong ability in building complex analytical queries on various relational databases
- Experience in building and optimizing data pipelines, architectures and data sets
- Analytical and problem solving skills, applied to Big Data domain
- Experience in performing root cause analysis on internal/external data and processes to answer specific business questions and identify opportunities for improvement
- Strong analytic skills related to semi/unstructured datasets
- Experience in manipulating, processing and extracting metrics from large disconnected datasets
- Strong project management and organizational skills
- Advertisement -