Skip to content

Senior Data Engineer

InFoCusp1101, 11th Floor, M-Ramanujan, Balewadi Phata, Baner, Pune, Maharashtra, IndiaOn-Site

Estimated salary, not published by the employer: Est. ₹14–21 LPA

Needs a free Superjobs account. You'll finish on InFoCusp's site.

Skill score

Function
Data Science
Estimated salary
Est. ₹14–21 LPA
Estimated for a senior role, ~5 yrs experience, India. Not published by the employer.
Work mode and location
On-Site, 1101, 11th Floor, M-Ramanujan, Balewadi Phata, Baner, Pune, Maharashtra, India
Experience
5+ yrs
Employment type
Full-time

Job description

About the Role We are seeking a highly skilled Senior Data Engineer with strong hands-on experience in Google Cloud Platform (GCP) or AWS, cloud infrastructure, data ingestion, and Python. The ideal candidate is also strong in dbt, Databricks, and AWS, and has experience implementing Data pipelines. The ideal candidate will have a proven track record of designing and implementing scalable data platforms, organizing GCP or AWS environments to meet data sovereignty requirements, and building secure and reliable data ingestion and transfer solutions. They should be comfortable working with existing, imperfect codebases and improving them. Key Responsibilities 1. GCP and AWS Architecture & Infrastructure Design, develop, and maintain scalable GCP infrastructure for enterprise data workloads. Architect and organize GCP environments to support data sovereignty and data residency requirements. Design appropriate GCP projects, environments, networking, storage, IAM, and security configurations for data platforms. Establish scalable and reliable cloud infrastructure patterns for data processing and storage. Work across GCP and AWS environments, applying consistent infrastructure and security patterns. 2. Data Ingestion & Pipeline Engineering Design and implement robust data ingestion pipelines for structured and unstructured data sources. Build production-grade ingestion frameworks to reliably process and move data into GCP or AWS, or similar storage. Develop scalable and fault-tolerant data pipelines with appropriate validation, monitoring, and error handling. Optimize data ingestion workflows for performance, reliability, and cost. Ensure data pipelines maintain data quality, integrity, and traceability throughout the ingestion process. Design and implement Medallion architecture (Bronze, Silver, Gold layers) to deliver clean, reliable, analytics-ready data. Build and maintain transformation workflows using dbt, including modeling, testing, and documentation. Develop and optimize data processing workloads on Databricks. 3. Pipeline Architecture Review & Refinement Assess existing data pipelines, including messy or poorly documented legacy code, and refactor them into clean, maintainable, well-tested solutions. Refine and improve pipeline architecture to increase scalability, reliability, performance, and cost efficiency. Identify technical debt and drive improvements without disrupting production workloads. 4. Python Development & Automation Develop production-grade data engineering applications and automation using Python. Build reusable libraries, services, and utilities to support data ingestion and processing workflows. Automate infrastructure and data engineering processes to improve operational efficiency. Implement appropriate logging, monitoring, error handling, and testing for Python-based services and pipelines. 5. Collaboration & Ownership Work closely with Data Engineers, Software Engineers, Cloud/Infrastructure Engineers, Security teams, and business stakeholders. Take ownership of technical initiatives from architecture and design through production deployment and ongoing support. Requirements 5+ years of experience in Data Engineering, Cloud Engineering, or a related field. Strong proficiency in Python, with experience building production-grade applications and data pipelines. Good to have experience with dbt and Databricks. Proven experience designing and implementing data pipelines. Proven experience designing and implementing GCP architecture and infrastructure for data workloads. Good understanding of GCP services such as Google Cloud Storage (GCS), BigQuery, Pub/Sub, Dataflow, Cloud Run/GKE, VPC, IAM, and Cloud Monitoring. Working knowledge of AWS equivalents (S3, Redshift, Glue, Lambda, CloudWatch) is an added advantage. Experience with cloud networking, security, authentication, authorization, and access control. Experience with infrastructure automation and Infrastructure as Code (IaC), preferably Terraform. Strong understanding of data quality, monitoring, observability, and production operational best practices. Strong experience building data ingestion pipelines and data processing workflows. Ability to work with messy or legacy code and refine pipeline architecture. Excellent communication skills and ability to work independently with minimal supervision. Work Location: Ahmedabad/Pune Contact us to apply If you would like to apply for this role, send your resume to [email protected]. Benefits

Skills required

Top skills

  • dbt
  • Databricks
  • AWS
  • Python

All skills (8)

  • SQL
  • Medallion architecture
  • Data quality monitoring
  • Data pipeline refactoring
  • AWS Glue
  • AWS Lambda
  • Amazon Redshift
  • Amazon S3

Company info

Infocusp is a technology company that builds AI, data engineering, cloud, automation and software development solutions for businesses across industries. It serves organizations looking to simplify operational complexity and grow through practical, scalable technology.

More jobs at InFoCusp