PlanSource | Jobs | Data Engineer | BigDataKB.com | 09-02-22

โ€”

by

Job Location: Bangalore/Bengaluru

 

About the Data Engineer Position

The Data Engineer will be responsible for growing and optimizing our data and data pipeline architecture. The ideal candidate is experienced with building data pipelines and data storage infrastructure from the ground up, and is obsessed with developing systems that are scalable, highly performant, and easy to understand, maintain and grow. The data engineer will support the software development and data science teams on data initiatives, and will ensure data is accurate, reliable, and up to date throughout ongoing projects. They must be self-motivated and comfortable supporting the needs of multiple teams, systems, and products.

Some of your responsibilities will include:

  • Create and maintain optimized data pipeline architecture.
  • Assemble large, complex data sets that meet business requirements.
  • Build infrastructure for ingestion, transformation, and loading of data from a variety of sources using SQL and AWS โ€œbig dataโ€ technologies.
  • Work with stakeholders in Engineering, Product, and Data Science teams to assist with data-related technical issues and support their infrastructure needs.
  • Ensure highest level of data security and disaster recovery across multiple AWS regions.
  • Create data tools for analytics and data science team members that assist them in building and optimizing our product into an innovative industry leader.

About You

As a Data Engineer, you will possess the following essential strengths, skills, and competencies:

  • Bachelorโ€™s degree in a quantitative discipline such as statistics, mathematics, engineering, computer science, or physics.
  • Two (2) years of experience in data acquisition, data mining, data transformation, and cleaning structure/unstructured data.
  • Strong analytic mindset and problem-solving skills.
  • Advanced SQL knowledge and experience working with relational databases.
  • Experience building and optimizing data pipelines, architectures, and data sets.
  • Experience building processes supporting data transformation, data structures, metadata, dependency, and workload management.
  • Working knowledge of scalable data stores.
  • Excellent written and oral communication skills and strong interpersonal skills.
  • Must thrive in a fast-paced environment and be able to work independently.
  • Familiarity with Python and Python data processing tools (e.g., NumPy, pandas).

We would love to have a candidate that also has the following skills:

  • Proficiency with big data technologies (e.g., Spark, Kafka).
  • Experience with data warehouse tools (Snowflake preferred).
  • Experience with data pipeline and workflow management tools (Airflow preferred).
  • Experience with AWS cloud services.
  • Experience building scalable APIs in Python (using FastAPI is a plus!).
  • Working knowledge of stream-processing systems.
  • Proficiency with Scala (preferred), Java, or C/C .
  • Experience productionalizing and deploying machine learning models.

Apply Here

Submit CV To All Data Science Job Consultants Across India For Free

๐Ÿ” Explore All Related ITSM Jobs Below! ๐Ÿš€ โœ… Select your preferred “Job Category” in the Job Category Filter ๐ŸŽฏ ๐Ÿ”Ž Hit “Search” to find matching jobs ๐Ÿ”ฅ โž• Click the “+” icon that appears just before the company name to see the Job Detail & Apply Link ๐Ÿ“๐Ÿ’ผ

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *