Before u proceed below to check the jobs/CVs, please select your favorite job categories, whose top job alerts you want in your email & Subscribe to our Email Job Alert Service For FREE
Job Location: Bangalore/Bengaluru
Roles and Responsibilities
Job Description
- Build High level technical design both for Streaming and batch processing systems
- Design and build reusable components, frameworks and libraries at scale to support analytics data products
- Perform POCs on new technology, architecture patterns
- Design and implement product features in collaboration with business and Technology stakeholders
- Anticipate, identify and solve issues concerning data management to improve data quality
- Clean, prepare and optimize data at scale for ingestion and consumption
- Drive the implementation of new data management projects and re-structure of the current data architecture
- Implement complex automated workflows and routines using workflow scheduling tools
- Build continuous integration, test-driven development and production deployment frameworks
- Drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards
- Analyze and profile data for the purpose of designing scalable solutions
- Troubleshoot complex data issues and perform root cause analysis to proactively resolve product and operational issues
- Lead, Mentor and develop other Sr Data Engineers and Data engineers in adopting best practices and deliver data products.
- Partner closely with product management to understand business requirements, breakdown Epics,
- Partner with Engineering Managers to define technology roadmaps, align on design, architecture and enterprise strategy
Capabilities Required
- Expert level expertise in building big data solutions
- Hands-on experience building cloud scalable, real time and high-performance data lake solutions using AWS, EMR, S3, Hive & Spark, Athena
- Hands-on experience in delivering batch and streaming jobs
- Expertise in an agile and iterative model
- Expert level expertise relational SQL
- Experience with scripting languages such as Shell, Python
- Experience with source control tools such as GitHub and related dev process
- Experience with workflow sc
- Scheduling tools like Airflow
- In-depth understanding of micro services architecture
- Strong understanding of developing complex data solutions
- Experience working on end-to-end solution design
- Able to lead others in solving complex Data and Analytics problems
- Strong understanding of data structures and algorithms
- Strong hands-on experience in solution and technical design
- Has a strong problem solving and analytical mindset
- Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders
- Able to quickly pick up new programming languages, technologies, and frameworks
Submit CV To All Data Science Job Consultants Across India For Free