Skip to content

Data Engineer III

PDI TechnologiesChennai, HyderabadOn-Site

Estimated salary, not published by the employer: Est. ₹17–26 LPA

Needs a free Superjobs account. You'll finish on PDI Technologies's Lever page.

Skill score

Summary

The Data Engineer III builds and maintains cloud-based big data and analytics infrastructure, including data lakes, warehouses, ETL pipelines and data modeling, and also mentors team members and supports off-hours operations. It suits an experienced engineer who is strong in Python, SQL and AWS and can lead technical design and work with business stakeholders.

Function
Data Science
Estimated salary
Est. ₹17–26 LPA
Estimated for a senior role, ~7 yrs experience, India. Not published by the employer.
Work mode and location
On-Site, Chennai, Hyderabad
Experience
7+ yrs
Employment type
Full-time

Job description

Overview

As a Data Engineer III, you will be part of an agile data services team responsible for developing and maintaining our industry-leading cloud-based big data and data analytics infrastructure serving major global fortune 500 companies. The candidate will help with design, development, unit testing, performance testing, deployment, troubleshooting, defect resolution, and support of our

Roles and responsibilities

  • Utilize architectural and design skills for a variety of data-oriented tasks.
  • Establish and maintain strong relationships with business stakeholders.
  • Build highly reliable & scalable data systems with good data governance that provide complete, secure, accurate and timely data for analytics.
  • Expertise in data & dimensional modeling with good business acumen in supporting new product features and services - including financial reporting.
  • Assist development and analytics organization to meet its delivery goals with regards to data design, flow and infrastructure.
  • Perform Data Modeling, Big data architecture design, support and implementation of OLTP, real time OLAP/data warehouse systems and ETL pipelines.
  • Track/Build & monitor data/system KPI’s like storage, performance & usage
  • Hands on administration of big data databases/compute engines (Redshift, EMR, Spark or similar) and optimization/tuning techniques
  • Prepare required documentation as outlined by departmental standards.
  • Identify, analyze and back track data problems as needed.
  • Provide recommendations for application and system improvements.
  • Mentor other team members
  • Work with business users to understand business requirements, issues and business and/or client processes.
  • Develop, test, and maintain high-performance of our data systems to meet the requirements of the business and/clients while adhering to departmental standards.
  • Perform quality assurance testing for all work performed.
  • Meet with agile teams as required to define and document application requirements.
  • Follow project development, deployment process & maintain industry standards and best coding practices.
  • Maintain security and organization of the company’s data.
  • Provide off-hour support as assigned.
  • Gather business requirements/features, estimate the tasks, design & develop scalable solutions.
  • Work with manager to ascertain the company’s data requirements.
  • Plan work to meet project deadlines, accommodate demands by development teams, set priorities and escalate issues appropriately.
  •  Provide recommendations to development teams for data restructuring and performing complex maintenance.
  • Deploy and manage changes in development, staging and production.
  • Assist development and dev ops teams on SQL queries and tuning.

Skills required

Top skills

  • Data modeling
  • SQL
  • ETL pipelines
  • Data warehousing

All skills (8)

  • Python
  • Java
  • Spark
  • Hadoop
  • AWS
  • Airflow
  • Data lake architecture
  • Data governance

Qualifications

  • At least 7 years’ experience in data engineering, Data modeling, SQL, ETL, Big database, Data warehousing & basic-level administration of data infrastructure
  • At least 7 years coding experience in Java, Python, R or other equivalent programming language
  • At least 4 years big data experience
  • At least 2 years’ experience as technical lead architecting data lake/data warehouse/lake house systems & mentoring team of Data Engineers / DBAs
  • Proficient in a variety of big-data tools and technologies, including Hadoop, Spark, etc., handling different storage formats like Parquet, Delta/Apache Hudi/Apache Iceberg, & managing/working with different OLTP systems & data ingestion mechanisms.
  • Experience building large scale data lakes & warehouse systems leveraging cloud environments including AWS technologies –S3, Redshift, Aurora/RDS, DMS, EC2, Glue spark, Cloud watch, & EMR
  • Experience with data orchestration tools like Airflow
  • Experience in building pipelines, ingesting data from heterogeneous systems onto data lake environments in real time
  • Proficient in Python, SQL and PL-SQL, query tuning, optimization, ETL, ELT, and Data Warehousing
  • Proficient in Business Intelligence (BI), analytic database and products, able to take business requirements and translate into database tables, pipeline design and tasks.
  • General data-specific concepts, frameworks and processes
  • Agile development practices
  • Working within an SDLC
  • Designing data warehouses including definition, structure, documentation, maintenance, long-range requirements, operational guidelines, and protection
  • Data governance, meta data management, data security measures, retention & archival across data lake layers

Nice to have

  • Cloud experience, such as AWS, GCP or Azure
  • Building/Managing data lake & lake house architectures.
  • Experience working on different data lake storage formats.
  • Excellent oral and written communication
  • Multi-tasking and managing multiple priorities.
  • Working well in a team-oriented, collaborative environment with people from different disciplines and varying degrees of technical experience
  • Working in an Agile team environment.

Other details

  • Cultivates Innovation
  • Decision Quality
  • Manages Complexity
  • Drives Results
  • Business Insight

Benefits

PDI is committed to offering a well-rounded benefits program, designed to support and care for you, and your family throughout your life and career. This includes a competitive salary, market-competitive benefits, and a quarterly perks program. We encourage a good work-life balance with ample time off [time away] and, where appropriate, hybrid working arrangements. Employees have access to continuous learning, professional certifications, and leadership development opportunities. Our global culture fosters diversity, inclusion, and values authenticity, trust, curiosity, and diversity of thought, ensuring a supportive environment for all.

Company info

About PDI Technologies

At PDI Technologies, we empower some of the world's leading convenience retail and petroleum brands with cutting-edge technology solutions that drive growth and operational efficiency. By “Connecting Convenience” across the globe, we empower businesses to increase productivity, make more informed decisions, and engage faster with customers through loyalty programs, shopper

More jobs at PDI Technologies