Yulu Bikes | Data Engineer | Bengaluru | Bharat | BigDataKB.com | 2023-02-18

The role entails implementing methods to improve data reliability, and quality and combining raw information from different sources to create consistent and machine-readable formats. You lead the team in a complete lifecycle of using machine learning to solve real business problems – including data understanding, feature engineering, model selection, model training, model optimization, and deployment. You would also develop and test architectures that enable data extraction and transformation for predictive or prescriptive modeling.

Key Responsibilities

• Working with Big Data tools and frameworks to provide requested capabilities
• Identify development needs in order to improve and streamline operations
• Develop and manage BI solutions implementing ETL process and Data Warehousing concepts
• Dimensional modeling and maintaining data quality
• Monitoring performance and managing infrastructure
• Interacts with product and service teams to identify questions and issues for data analysis and experiments
• Working to visualize data to gain an understanding of it, then identifying differences in data distribution that could affect performance when deploying the model in the real world

Key Requirements

• 4+ years of experience in relevant field
• Strong programming skills, familiarity with data structures and algorithms, and object-oriented programming concepts.
• Proficient understanding of distributed computing principles
• Some experience with Hadoop and Spark with building stream-processing systems, using solutions such as Kafka and Spark-Streaming is preferred
• Good knowledge of Data querying tools SQL and Hive
• Experience with integration of data from multiple data sources
• Knowledge of various ETL techniques and frameworks
• Experience with Python/Java/Scala (at least one)
• Experience with cloud services such as AWS or GCP
• Experience with NoSQL databases, such as DynamoDB, MongoDB will be an advantage