Job Location: Bangalore/Bengaluru
As a Data Engineering Architect, you will be responsible for developing best practices andmaking architectural decisions to rapidly improve critical data processing analytics pipelines.You will tackle hard problems to improve the platform s reliability, resiliency, and scalability.
What You ll Be Doing
- Building scalable data pipelines for both real-time and batch using best practices in datamodeling, ETL/ELT processes utilizing various technologies such as Spark, Kafka,Python, Airflow, Glue, Elastic Search, Lambda, Presto and EMR
- Setup reliable data ingestion pipelines for new data sources and integrate them with otherdata sets
- Provide engineering support to investigate, identify and set up new tools and processesfor data warehousing, data quality, reporting, business intelligence, data governance, anddata cataloging
- Build and scale data infrastructure that powers batch and real-time data processing ofbillions of records
- Interface with data engineers, data scientists, product managers, and all data stakeholdersto understand their needs and promote best practices
- Participate in user story creation process through dissection of high-level requirementsduring team review sessions
- Actively participate in development along with team members for as much as 75% oftheir time, creating modules; systems that can then be treated as a working reflection ofthe best practices
- Drive the data architecture, data modeling, design, and implementation of dataapplications using standard open source big data tech stack, Data Warehouse, anddistributed systems.
- Gather business and functional requirements from external and/orinternal users, and translate requirements into technical specifications to build robust,scalable, supportable solutions.
- Mentor data engineers at various levels of seniority by doing their design and codereviews, providing constructive and timely feedback on code quality, design issues,technology choices with performance and scalability being critical drivers. Manageresources on multiple technical projects and ensure schedules, milestones, and prioritiesare compatible with technology and business goals
- Challenge the status quo and propose innovative ways to process, model, consume datawhen it comes to tech stack choices or design principles.
- Implementation of long-term technology vision for your team.
- Active participant in technology forums; represent MFine in external forums.
What We Need To See
- Bachelor s degree in Computer Science, MIS, Computer Engineering, or equivalenttechnical degree
- 5 years of relevant work experience or DBA experience in Relational DatabaseManagement Systems[RDBMS] (MySql, SQLServer, etc.)
- Strong software development and programming skills using Python (PySpark)
- Building and designing RESTful APIs
- Experience with AWS big data technologies: S3, Glue, EMR, Kinesis, RDS, Redshift,Athena ( AWS experience preferred)
- Designing and developing Data Pipelines for Data Ingestion or Transformation usingJava or Scala or Python Strong Python programming skills with the ability to implement OOPs and functionalprogramming
- Strong knowledge of RDBMS and NoSQL databases with the ability to implement themfrom scratch
- Expertise in building optimizing data pipelines, architectures, and data sets
- Advanced skills in big data technologies like Hadoop, Hive, Sqoop, Python, Spark,Presto, Kafka, ORC, Parquet, Avro, Apache Hudi
- Experience designing database environments, analyzing production deployments, andmaking recommendations to optimize performance
- Quick; Excellent Problem-Solving skills for complex large scale data applicationsproblems
- Solid hands-on experience with Enterprise Data Warehouse and Big data storage andcomputation frameworks like OLAP Systems, MPP (SQL DW, Redshift), HadoopCompute (MR, Spark, Flink, Hive). Awareness of pitfalls use cases for a large varietyof solutions.
- Ability to drive capacity planning, performance optimization, andlarge-scale system integrations
- Strong proficiency in building resilient and cloud-ready solutions based on MicroServices, Multi-tenancy architecture
- Cloud: AWS or Azure
Nice To Have:
- Automate cloud infrastructure, services, and observability
- Develop CI/CD pipelines and testing automation
- Experience with CI/CD pipelines and unit test
- Worked in a small setup (either a start-up or a small reasonably independentteam/development center) Comfortable working in fuzzy environments – where boundaries aren t clearly definedand Proven experience as a leader capable of motivating and energizing the very bestsoftware engineers, as well as gaining their respect. – Be the technical mentor to yourteam
- Knowledge of Scala/Java would be plus
Submit CV To All Data Science Job Consultants Across India For Free
๐ Explore All Related ITSM Jobs Below! ๐
โ
Select your preferred “Job Category” in the Job Category Filter ๐ฏ
๐ Hit “Search” to find matching jobs ๐ฅ
โ Click the “+” icon that appears just before the company name to see the Job Detail & Apply Link ๐๐ผ

Leave a Reply