Reputation | is Hiring | Data Engineer | BigDataKB.com | 2022-04-05

We are looking for a savvy Data Engineer to join our growing team of analytics
experts. The Data Engineer will be responsible for expanding and optimizing our
data and data pipeline architecture, as well as optimizing data flow and collection
for cross-functional teams. The ideal candidate is an experienced data pipeline
builder and data wrangler who enjoys optimizing data systems and building them
from the ground up.

The Data Engineer will support the Data Management/BI team  on data initiatives and will ensure optimal data delivery architecture is consistent  throughout ongoing projects. They must be self-directed and comfortable  supporting the data needs of multiple teams, systems, and products. The right  candidate will be excited by the prospect of optimizing or even re-designing our  company s data architecture to support our next generation of products and data  initiatives.

Responsibilities

• Work cross-functionally to help shape the long-term data strategy; prioritize and execute initiatives to enable us to realize this strategy
  Architect, improve and manage high-quality and reliable data pipelines across multiple sources by leveraging the latest techniques and tools to automate and optimize our existing processes, provide visibility into and monitor pipeline status, and identify opportunities to improve performance
• Optimize our data infrastructure and processes to ensure scalability, reliability, quality, and security; implement and automate performant and large-scale data storage and processing systems
• Build warehouses and data pipelines that efficiently deliver data sets at scale and enable teams to build, maintain, and manage machine learning models
• Communicate and collaborate with technical and non-technical partner teams to align on business needs, document requirements, evaluate tradeoffs, design solutions, and execute technical data requirements
• Provide technical guidance, training, and mentorship to your team and technical partner teams (e.g., Engineering, Analytics) around data best practices
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources (CRM, ERP, SQL NoSQL databases, etc.)

Qualifications

• 4+ years of industry experience in data engineering, data operations, data science, or data analytics
• Experience with large-scale data techniques including data warehousing and modeling, building data pipelines, batch or streaming data processing leveraging Spark, Flink, Kafka, Airflow, or similar big data frameworks
• Experience with programming languages including Java or Python
• Experience with NoSQL or relational databases including MySql or Postgres; Familiar with at least one OLAP data-warehouses such as Snowflake; Strong in SQL for data querying and processing
• Strong experience with ETL/ELT/Pipeline tools
• Experience with Business Intelligence tools such as Mode, Looker, Tableau, DOMO, etc.
• Strong collaboration and communication skills with both technical and non- technical audiences
• Meticulous attention to detail and strong organizational skills; must have the ability to prioritize projects based on business needs, manage multiple projects simultaneously across stakeholders, and learn quickly
• Bachelor in Computer Science, Engineering, or related quantitative field