Job Location: Eden Prairie, MN
JOB DESCRIPTION
An employer is seeking a Senior Cloud Data Engineer who will be working in an enterprise level environment in the healthcare industry. You will be part of a team that is migrating an application suite from the current on-prem Big Data Tenant to a Cloud Technology stack. The application primarily does ETL processing and data extracts of healthcare data. The team will be comprised of other contractors, vendor partners and employees working in a global setting. You will be a part of a Scrum team of about 4-5 engineers.
You will be responsible for:
You will be responsible for:
- Build and deploy performant, scalable solutions for complex data processing pipelines by applying modern data engineering concepts and development best practices
- Build, test, deploy and monitor scalable, end to end ETL pipelines for large-scale healthcare data warehousing using Cloud technologies
- Adopt modern technologies and practices like Git code management, open source tools, test automation, CI-CD, application telemetry, monitoring and visualization
- Optimize data pipelines to handle growing data volumes to meet performance and quality standards
- Collaborate with other team members in the design. Document architecture, design and best practices.
- Ability to prioritize tasks and work concurrently on multiple tasks
- Participate in business analysis and data discovery to gather data requirements
- Ability to build complex but compact well performing SQL queries.
- Ability to analyze on-prem ETL components that are written using a combination of Unix scripts, Spark jobs, Python and Scala code and Apache Airflow jobs
- You will develop and test components of a configurable ETL pipeline framework
- You will develop and test components to integrate with Airflow orchestration and other services like Data Quality and Snowflake Cloud platform
MINIMUM REQUIREMENTS
- Experience building ETL applications using Cloud technology stack
- Ability to write complex SQL
- 8+ years of experience in distributed systems, data engineering, software engineering, and working with big data systems
- 5+ years of solid coding skills in a language such as Scala or Java or Python and ETL
- 3+ years of Apache Spark and CI-CD development environment
- 2+ years of experience with Azure Cloud with Azure Databricks technologies including Autoloader, Delta Lake and Snowflake Cloud Platform
- 2+ years of experience with job scheduling tools like Apache Airflow and automation pipelines like Jenkins
DESIRED SKILLS
- Healthcare experience with knowledge of concepts like claims and eligibility
- Experience with DevOps, Continuous Integration, and Continuous Delivery in the cloud environment
- Experience with cloud infrastructure, including infrastructure as code, and cloud security set up
- Strong design skills, knowledge of design patterns and experience building scalable, extendable architectures
- Ability to effectively document and communicate the design to other engineers
- Experience working in an Agile environment
- Experience working with modern databases, including Snowflake Cloud Platform
Submit CV To All Data Science Job Consultants Across United States For Free
🔍 Explore All Related ITSM Jobs Below! 🚀
✅ Select your preferred "Job Category" in the Job Category Filter 🎯
🔎 Hit "Search" to find matching jobs 🔥
➕ Click the "+" icon that appears just before the company name to see the Job Detail & Apply Link 📝💼

