Big Data jobs in India

• Bachelor's or master's degree in computer science, or a comparable field.
• More than 4 years of experience working in the field of software testing.

• Lead and manage the delivery of data projects, ensuring timely execution and quality standards.
• Collaborate with business and technical teams to define project scope, timelines, and milestones.

This role requires a highly organized and experienced individual to lead and manage complex data-centric programs, incorporating strong change management principles and process analysis capabilities. The Data Program Owner will oversee all aspects of data initiatives, from inception to completion, ensuring projects are delivered on time, within budget, and meet business objectives.

The Data and AI Governance Leader, will play an integral role in orchestrating the governance council and ensuring compliance with governance policies in data, analytics, and AI. Their participation will be instrumental in defining a robust Data and AI Governance Framework, codifying organizational and industry policies into standards, procedures and technical controls ensuring effective, ethical, and legal use of Data & AI technologies

As a Data Engineer you will be developing and maintaining data infrastructure and writing, deploying, and maintaining software to build, integrate, manage, maintain, and quality-assure data

The Data Architect will be responsible for designing enterprise data management frameworks, ensuring data security and compliance, implementing data management processes, build data models and strategies to support various business needs and initiatives.

Qualifications
*
Bachelor's or master's degree in computer science, or a comparable field.
*
More than 4 years of experience working in the field of DevOps engineering.
*
Interest in developing and automating CI/CD and data pipelines for Big Data and analytical / AI solutions.
o
Three to five years of experience with DevOps tools and methods (e.g. GitLab, Container services as AWS ECR, CICD)
o
Cloud development skills in AWS services as Lambda, S3, DynamoDB, ECR and Stepfunctions including experience with infrastructure as code with Terraform (at least two years of experience)
*
Experience in working with other cloud services (Azure/Google) advantageous.
o
Programming skills in Python and knowledge about pipeline configuration in scripting languages like bash required (at least one year).
*
Experience with data engineering tools and patterns incl. solid knowledge of SQL.
o
Knowledge about deployment of frontend applications and development skills with JavaScript libraries or frameworks like React beneficial.
*
Analytical skills, team spirit, holistic thinking, and independent work (especially self-learning) required.
*
Customer-orientation and quality-consciousness
*
Good English skills required due to standard communication language in the team.
Job Description:
*
Work in a highly innovative development team dedicated to implementing a new generation of Big-Data Cloud and AI applications supporting the innovation and the digitalization of the company.
*
Evaluate and implement modern techniques and services to support the Big Data, AI, and business automation journey of the Festo companies worldwide.
*
Design, implementation and operation of CICD pipelines based on the requirements of an international team of cross functional developers.
*
Consultant the team and enable the deployment of backend and frontend applications and support their application lifecycle management.
*
Development and operation of data pipelines and platform services including the integration and automation under the strict consideration of data security and governance aspects.

Total Exp- 6+yearsKey Responsibilities:

• Develop and optimize ETL pipelines using PySpark and Databricks.
• Work with large-scale datasets and build distributed computing solutions.
• Design and implement data ingestion, transformation, and processing workflows.
• Write efficient and scalable Python code for data processing.
• Collaborate with data engineers, data scientists, and business teams to deliver insights.
• Optimize performance and cost efficiency for big data solutions.
• Implement best practices for CI/CD, testing, and automation in a cloud environment.
• Monitor job performance, troubleshooting failures, and tuning queries.

Required Skills & Qualifications:

• 6years of experience in Python programming for data engineering.
• Strong hands-on experience with Apache Spark (PySpark) for data transformation.
• Experience working with Databricks for data processing and analytics.
• Good knowledge of SQL and NoSQL databases.
• Experience with Cloud platforms (Azure, AWS, or GCP) and storage solutions.
• Familiarity with CI/CD pipelines, DevOps practices, and automation tools.
• Understanding of data lake, data warehouse, and data modeling concepts.
• Knowledge of job orchestration tools like Airflow, Azure Data Factory, or AWS Glue.
• Strong problem-solving and debugging skills in distributed environments.

Good to Have:

• Experience with Delta Lake and Lakehouse architecture.
• Familiarity with Kafka or real-time data processing.
• Knowledge of MLflow and integrating ML models in Databricks.
• Experience with Scala for Spark-based applications.