Data Engineer jobs in India

The purpose of the Data Engineering specialist is to build and unit test code for Projects and Programmes on Azure Cloud Data and Analytics Platform

Job Description: looking for a talented and experienced Senior Data Engineer with a strong expertise in Snowflake to join our team. As a Senior Data Engineer, you will play a critical role in the development and optimisation of data systems, data pipelines, and ensuring seamless data transformations to support business intelligence and data-driven decisions.

As a Software Engineer, you'll be a key part of a team improving a discrete set of tools and processes into a repeatable, automated, and audited pipeline to better onboard and process data for clinical trials. You will be working with clinical counterparts to improve their work pipeline as well as automating system validation and documentation efforts for a low-friction rigorous product.

Seeking a highly skilled and motivated Data Engineer The ideal candidate will have strong expertise in Python, Apache Spark, Apache Flink, BigQuery, Google Cloud Dataproc/ Databricks, and Confluent Kafka. Will be responsible for designing, developing, and optimizing large-scale data processing systems, enabling real-time and batch data workflows to support business intelligence and advanced analytics.

As a Data Engineer you will be developing and maintaining data infrastructure and writing, deploying, and maintaining software to build, integrate, manage, maintain, and quality-assure data

The Data Integration Engineer will lead initiatives to integrate, validate, and optimize data pipelines for clinical research using real-world data, ensuring high data quality and compliance. The role involves working with internal teams and external partners, leveraging technologies like AWS, SQL, and Python, to support healthcare research and improve medical outcomes.

 Collaborate with solution architect, data engineers, business stakeholders, business analysts, and DQ testers to ensure data management and data governance framework is defined as critical components.
 Design and develop data models using industry-standard modeling techniques and tools.

As a Product Champion, you'll leverage your creativity, analytical abilities, and leadership from day one. Collaborate with skilled product managers, designers, and data engineers to create world-class analytics products that tackle major customer challenges and transform teamwork. This is an exciting chance to lead the analytics domain as a PM, shaping how companies engage with and connect their employees across the platform.

Total Exp- 6+yearsKey Responsibilities:

• Develop and optimize ETL pipelines using PySpark and Databricks.
• Work with large-scale datasets and build distributed computing solutions.
• Design and implement data ingestion, transformation, and processing workflows.
• Write efficient and scalable Python code for data processing.
• Collaborate with data engineers, data scientists, and business teams to deliver insights.
• Optimize performance and cost efficiency for big data solutions.
• Implement best practices for CI/CD, testing, and automation in a cloud environment.
• Monitor job performance, troubleshooting failures, and tuning queries.

Required Skills & Qualifications:

• 6years of experience in Python programming for data engineering.
• Strong hands-on experience with Apache Spark (PySpark) for data transformation.
• Experience working with Databricks for data processing and analytics.
• Good knowledge of SQL and NoSQL databases.
• Experience with Cloud platforms (Azure, AWS, or GCP) and storage solutions.
• Familiarity with CI/CD pipelines, DevOps practices, and automation tools.
• Understanding of data lake, data warehouse, and data modeling concepts.
• Knowledge of job orchestration tools like Airflow, Azure Data Factory, or AWS Glue.
• Strong problem-solving and debugging skills in distributed environments.

Good to Have:

• Experience with Delta Lake and Lakehouse architecture.
• Familiarity with Kafka or real-time data processing.
• Knowledge of MLflow and integrating ML models in Databricks.
• Experience with Scala for Spark-based applications.

You will build and mentor a team of data scientists and machine learning engineers, collaborating closely with cross-functional teams to integrate AI models and analytics tools into our ecosystem. This role involves driving innovation in how we use AI and machine learning to automate and optimize the organisation's strategies.