Data.Devops Engineer
Data.Devops Engineer
We are currently looking for Data.Devops Engineers in India for our offices in multiple locations. We are looking for experienced Data.Devops Engineer to work on critical application & product development. You'll be a part of a modern, lean, self-governing engineering team.
Responsibilities
- Build the infrastructure required for optimal extraction, transformation, and loading of data from various data sources using AWS ‘big data’ technologies.
- Build analytics tools that utilize the data pipeline to provide actionable insights into customer use cases, operational efficiency, and other key business performance metrics.
- Contribute to frequent deployments using CI/CD pipeline and improve the Team's DevOps measures
- Work with data and analytics experts to strive for greater functionality in our data systems.
- Implement data applications using big data technologies (Hadoop, AWS) to ingest, process, and analyze large disparate datasets.
- Explore and learn the latest AWS technologies to provide new capabilities and increase efficiency
- Can-do attitude when working with ambiguity and demonstrate learning agility
- Good analytical skills with excellent knowledge of SQL.
Requirements
- Bachelor’s Degree in computer science, Information Technology or area of study related to data sciences and data mining or relevant experience
- 3 years of solid development in big data stack and programming experience in Python/Pyspark, Spark and Scala
- 3+ years of hands-on experience on the following data and analytics tools/services
- I. Ingestion Tools such as DMS, Kinesis, DataSync, AWS SFTP and Snowball/Snowmobile
- II. Storage such as S3, Glacier, RDS, DynamoDb
- III. Transformation Tools such as EMR, Glue, Athena. Lambda
- IV. Analytics Tools such as RedShift and RedShift Spectrum
- V. Visualization/BI Tools Quick Sight, Tableau, Qlikview/QlikSense
- VI. Security/Monitoring: IAM, KMS, CloudWatch/DataDog/Splunk or 3rd party
- VII. Orchestration Tools : Step Functions and Airflow
- Strong experience with writing complex programs, implementing architectures, and enabling automation in environments
- Good hands-on experience with IaC (Terraform/Cloudformation) for deploying the data and analytics stack and version tool such as Gitlab/Github.
- Experience with Agile Development
- Experience building or administering reporting/analytics platforms
- Knows how to optimize the distribution, partitioning, and MPP of high-level data structures.
- Knowledge on Databricks, Snowflake and AI/ML will be an added advantage.
- Strong communication/interpersonal and organizational skills
Technologies
- Big Data
- Spark
- Hadoop
- Hive
- Python
- Scala
- Devops
- CI/CD
- Terraform
Big Data
Apache Spark
Python
Scala
Apache Hadoop
Apache Hive
DevOps
CI/CD
Terraform
Amazon Web Services
Benefits
For you
- Insurance Coverage
- Paid Leaves – including maternity, bereavement, paternity, and special COVID-19 leaves.
- Financial assistance for medical crisis
- Retiral Benefits – VPF and NPS
- Customized Mindfulness and Wellness programs
- EPAM Hobby Clubs
For your comfortable work
- Hybrid Work Model
- Soft loans to set up workspace at home
- Stable workload
- Relocation opportunities with ‘EPAM without Borders’ program
For your growth
- Certification trainings for technical and soft skills
- Access to unlimited LinkedIn Learning platform
- Access to internal learning programs set up by world class trainers
- Community networking and idea creation platforms
- Mentorship programs
- Self-driven career progression tool
Bangalore, Chennai, Gurgaon, Hyderabad, Pune
Cities eligible for the position
