Vivid Infotech Software Solutions (P) Ltd. | is Hiring | US Based MNC | Looking For Data Engineer | Immediate joiner | BigDataKB.com | 2022-04-05

Before u proceed below to check the jobs/CVs, please select your favorite job categories, whose top job alerts you want in your email & Subscribe to our Email Job Alert Service For FREE

 

Job Location: Chennai( Ashok Nagar )

Job Description

Data Engineer will work with the Analytics team to build, maintain, and optimize data pipelines for key data and analytics consumers including business and data analysts and data scientists. Data engineers also need to guarantee compliance with data governance and data security requirements while creating, improving, and operationalizing these integrated and reusable data pipelines. This would enable faster data access, integrated data reuse and vastly improved time-to-solution for analytics initiatives. The data engineer will be measured on their ability to integrate analytics and (or) data science results with business processes.

This role will require both creative and collaborative working with IT and the wider business. It will involve evangelizing effective data management practices and promoting better understanding of data and analytics. The data engineer will also be tasked with working with key business stakeholders, IT experts and subject-matter experts to plan and deliver optimal enterprise data assets.

ROLE AND RESPONSIBILITIES

BigDataKB.com Jyotish
BigDataKB.com Jyotish - Career & Life Prediction
  • Build data pipelines: The primary responsibility of data engineers is to architect, build, and maintain data pipelines that will provision high quality data ready for analysis. This includes ingestion, exploration, and curation of high value data.
  • Drive Automation through effective metadata management: The data engineer will be responsible for using innovative and modern tools, techniques, and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks in order to minimize manual and error-prone processes and improve productivity.
  • Learning and using modern data preparation, and integration
  • Tracking data consumption patterns.
  • Performing intelligent sampling and caching.
  • Monitoring schema changes.
  • Recommending or sometimes even automating existing and future integration flows.

QUALIFICATIONS AND EDUCATION REQUIREMENTS

  • A bachelor’s or master’s degree in computer science, data management, information systems, information science or a related quantitative field is required.
  • The ideal candidate will have a combination of IT skills, data engineering skills, and analytics skills.
  • At least 3 years or more of work experience in data management disciplines including data integration, modeling, optimization, and data quality, and/or other areas directly relevant to data engineering responsibilities and tasks.

PREFERRED SKILLS

  • Strong experience with data engineering tools / frameworks on cloud for Object-oriented/object function scripting using languages such as Python, PySpark, Scala, or similar
  • Strong ability to design, build and manage data pipelines in PySpark, Apache Airflow and related technologies for data structures encompassing data transformation, data models, schemas, metadata and workload management.
  • Strong experience with popular database programming in relational and nonrelational environments like Snowflake, AWS Redshift, Google Big Query, Azure SQL DB, and similar platforms
  • Experience in working with large, heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using traditional data integration technologies. These should include ETL/ELT, CDC, message-oriented data movement and upcoming data ingestion and integration technologies such as stream data integration.
  • Strong experience in working with and optimizing existing data pipeline processes and data integration and data preparation flows and helping to move them in production
  • Experience in working with both open-source and commercial message queuing technologies such as Kafka, Amazon Simple queuing Service, stream data integration technologies such as Apache Nifi, Apache Kafka Streams, Databricks and stream analytics technologies such as Apache Kafka KSQL
  • Demonstrated ability to work across multiple deployment environments including cloud, on-premises and hybrid, multiple operating systems and through containerization techniques such as Docker, Kubernetes, and others
  • Experienced in agile methodologies and capable of applying DevOps and increasingly DataOps principles to data pipelines to improve the communication, integration, reuse and automation of data flows between data managers and consumers across an organization

ADDITIONAL NOTES

  • Strong experience supporting and working with cross-functional teams in a dynamic business environment.
  • Required to be highly creative and collaborative.

Apply Here

Submit CV To All Data Science Job Consultants Across India For Free

LEAVE A REPLY

Please enter your comment!
Please enter your name here