Job Location: Pune( Kharadi )
Your Position
As a Data Engineer, you will work in the newly created Warehouse Process optimization team, collaborating with data scientists, an architect, and a product owner. The WS Process Optimization tool is a new product under development, and upon hitting milestones, the team will be further expanded with more people to support the successful scaling and productization.
In your role, you’ll flexibly work on creating and scaling a prototype, by creating a backend to embed the models developed by the data scientists and the data ingestion connectors/pipelines to run the model. You will need to communicate with multiple technical stakeholders to agree on the requirements and interfaces. The team architect will be your sparring partner and you will collaborate on creating an end-to-end tool.
Roles and Responsibilities
- You participate in end-to-end development through prototyping, validation, implementation, launch, and further iterations. You participate in architecture and roadmap discussions.
- You develop new features for our products and services which provide our customers with actionable information through multiple channels.
- You participate in the development of our back-end applications, (relational and graph) databases, data pipelines, and APIs.
- You ensure that applications are running in a cloud-based environment.
- Collaborate together with front-end developers and data scientists in the team to reach the shared goal.
- You will develop scalable data pipelines to transform and aggregate data for business use, following software engineering best practices. For these data pipelines, you will make use of the best frameworks available for data processing like Spark.
- You develop our data services for customer sites towards a product, using (test & deployment) automation, componentization, templates, and standardization in order to reduce the delivery time of our projects for customers.
- You design and build a CI/CD pipeline, including (integration) test automation for data pipelines. In this process, you strive for an ever-increasing degree of automation.
- You will work with an infrastructure engineer to extend storage capabilities and types of datacollection (e.g. streaming)
- You enjoy rapid prototyping and getting your hands dirty.
- You are a team player and do not shy away from opportunities to help the team and product to grow.
- You are proactive and a strong communicator.
Desired Candidate Profile
- You have a Bachelor’s or Masters degree in Computer Science or equivalent.
- You can deal with the uncertainty of a new product introduction: you can take ownership of unclear requirements and support refining them, and you are flexible if the requirements change on the way.
- You have experience developing online applications, especially those containing a lot of data. Experience with Event Driven Architecture is a strong plus.
- You have a strong knowledge of web and cloud technologies such as web services, REST APIs, messaging protocols (MQTT, AMQP, MQ), and security (OAuth, OpenID, Keycloak).
- You have experience with Azure, including serverless concepts and CI/CD pipelines. Experience working with Kubernetes and Docker is a plus.
- You have experience creating and deploying SQL and NoSQL databases. You have strong experience with Graph databases (e.g. deployment, querying, ETLs, ).
- You have experience building and optimizing data pipelines, architectures, and data sets. Knowledge of Scalable data processing frameworks (e.g.Spark) is desired. Python fluency is desired.
- You are fluent in English.
Perks and Benefits
Competitive Salary & Other Benefits
Job Location: Pune (Hybrid Work Culture)
Submit CV To All Data Science Job Consultants Across Bharat For Free