Job Location: Jaipur
As Big Data Engineer, you will develop, maintain, evaluate and test big data solutions. You will be involved in data engineering activities like Creating pipelines / workflows for Source to Target etc..
- You will be involved in the design of data solutions using Hadoop based technologies along with Hadoop, Azure, HDInsights for Cloudera based Data Late using Scala Programming.
- Responsible to Ingest data from files, streams and databases. Process the data with Hadoop, Scala, SQL Database, Spark, ML, IoT
- Develop programs in Scala and Python as part of data cleaning and processing
- Responsible to design and develop distributed, high volume, high velocity multi-threaded event processing systems Develop efficient software code for multiple use cases leveraging Python and Big Data technologies for various use cases built on the platform
- Provide high operational excellence guaranteeing high availability and platform stability
- Implement scalable solutions to meet the ever-increasing data volumes, using big data/cloud technologies Pyspark, Kafka, any Cloud computing etc.
- Bachelor’s degree in Economics, Computer Science, or a related field. MBA preferred but not required
- Strong analytical skills
- Excellent verbal and written communication skills
- Experience with writing requirement/specifications of technology related products
- Basic knowledge of product design trends and good practices
- Working skills of basic computer programs (Word, Excel, Visio, etc.)
- Knowledge of writing database/SQL queries
Required Technical and Professional Expertise
- Minimum 2+ years of experience in Big Data technologies
- Proficient in any of the programming languages – Python, Scala or Java
- Mandatory experience in Mid to Expert Level programming capabilities in a large-scale enterprise In-depth experience in modern data platform components such as the Hadoop, Hive, Pig, Spark, Python, Scala, etc
- Experience with Distributed Versioning Control environments such as GIT
- Familiarity with development tools – experience on either IntelliJ / Eclipse / VSCode IDE, Build Tool Maven
- Demonstrated experience in modern API platform design including how modern UI’s are built consuming services / APIs.
- Experience on Azure cloud including Data Factory, Databricks, Data Lake Storage is highly preferred Solid experience in all phases of Software Development Lifecycle – plan, design, develop, test, release, maintain and support, decommission