Skip to content

Data Engineer (SDE 2)

FamPayBengaluruOn-Site

Estimated salary, not published by the employer: Est. ₹9.5–14 LPA

Needs a free Superjobs account. You'll finish on FamPay's Lever page.

Skill score

Summary

The Data Engineer II will build and scale a high-performance lakehouse with CDC ingestion, distributed compute, and orchestration, while designing OLAP domain models and partnering with product and business stakeholders. The role suits an engineer with 3 to 5 years in distributed data systems who is hands-on with PySpark, SQL, AWS, and AI-assisted development.

Function
Data Science
Estimated salary
Est. ₹9.5–14 LPA
Estimated for a mid-level role, ~3 yrs experience, India. Not published by the employer.
Work mode and location
On-Site, Bengaluru
Experience
3–5 yrs
Employment type
Full-time

Job description

Overview

About the role

We are looking for a Data Engineer II (SDE-2) to join our data team. The ideal candidate will be a play a key role to develop of high performant and scalable Data Lake-house, moving us toward a world of sub-minute data latency and unified batch/streaming compute. This is an engineering-heavy role where you will manage complex CDC flows, optimize distributed query engines and leverage AI to accelerate our development lifecycle.

Skills required

Top skills

  • Python
  • PySpark
  • SQL
  • AWS

All skills (8)

  • Infrastructure as Code
  • Airflow
  • Temporal
  • Dimensional modelling
  • Go
  • Java
  • Scala
  • Debezium

Qualifications

  • Experience: 3–5 years in Data Engineering, specifically with distributed systems and cloud-native architectures.
  • Coding: Expert-level Python/PySpark and SQL. Familiarity with Go/Java/Scala is a plus
  • Infrastructure: Hands-on experience with AWS (S3, EKS, MSK) and Infrastructure-as-Code.
  • Orchestration: Experience with Airflow or Temporal for complex workflow management.
  • AI-Native: Proficiency in using AI tools (Claude, Codex, Copilot) to write, test, and document code efficiently.
  • Systems Thinking: Ability to explain the trade-offs between different storage formats and processing frameworks.
  • Tech Execution : Drive key tech initiatives by preparing TRD and actively involve in design reviews.
  • Domain Modelling - Should be hands on in designing Domain models for OLAP like Fact, Dimension, Cumulative , types of SCD’s and OBT pattern tables.
  • Self Starter - Lead the team technically and bring in new ideas to contribute to the growth of the charter.
  • Stakeholder Interaction - Interact with the Product & Key Stakeholders & help them by adding value to the business workflow with data & analytics.

Nice to have

  • Real-time CDC: Ownership of high-throughput ingestion from RDBMS to Lakehouse using Debezium, PeerDB.
  • Lakehouse Architecture: Designing and optimizing table formats (Iceberg, Delta, Hudi) for both performance and storage efficiency.
  • Unified Compute: Developing robust ETL/ELT frameworks in PySpark and Flink (handling both batch and streaming workloads).
  • Infrastructure & Ops: Managing data workloads on AWS (EMR, EKS, MSK, S3) and automating everything via Gitlab/Github Actions.
  • Query & BI: Tuning Trino or Clickhouse to power real-time dashboards in Metabase, Superset, and PowerBI.

Other details

  • Ingestion & CDC: OLake and PeerDB for near real-time sync from production systems into Alchemy; Debezium/Kafka for CDC-heavy use cases, with support for Kafka and S3-based sources.
  • Lakehouse / Storage: Alchemy on Apache Iceberg with S3 as the data lake storage layer and AWS Glue Catalog for metadata; exposure to Delta/Hudi is a plus.
  • Processing & Compute: PySpark on EMR/EKS for batch and streaming workloads; Flink and Spark Structured Streaming fundamentals for low-latency pipelines.
  • Streaming Platform: MSK / Kafka for event-driven ingestion, CDC propagation, replay, backfills, and operational monitoring through Kafka UI.
  • Query & Serving Layer: Trino over Alchemy/Iceberg for lakehouse analytics, ClickHouse for high-throughput operational and real-time dashboards, and BigQuery exposure where applicable.
  • Workflow Orchestration: Airflow for scheduled data pipelines, DQ/reconciliation DAGs, backfills, and SLA-driven jobs; Temporal for durable workflow execution in ingestion services.
  • Data Quality & Governance: DQ checks, freshness/SLA monitoring, source-to-lake reconciliation, deduplication, schema evolution handling, and cataloging/lineage through OpenMetadata.
  • Infrastructure & DevOps: AWS (S3, EKS, EMR, MSK), Kubernetes, Terraform/IaC, GitLab/GitHub Actions, observability via Grafana/CloudWatch, and production runbook discipline.
  • BI & Analytics: Metabase, Superset, Tableau, and PowerBI for business dashboards; strong ability to model curated marts, fact/dimension tables, SCDs, and OBT patterns for stakeholder-facing analytics.

Benefits

  • Relocation assistance to make your move seamless.
  • Free office meals (lunch & dinner).
  • Generous leave policy, including birthday leave, period leave, paternity and maternity support, and more.
  • Salary advance and loan policies for any financial help.
  • Quarterly rewards and recognition programs, and a referral program with great incentives.
  • Access the latest gadgets and tools.
  • Comprehensive health insurance for you and your family, mental health support.
  • Tax benefits with options like food coupons, phone allowances, car/device leasing.
  • Retirement perks like PF contribution, leave encashment and gratuity.

PS - We’re looking for someone with an AI-native mindset, who actively leverages AI in their day-to-day work, as we encourage and embed it across every level at Fam.

Company info

Fam is a payments app in India that lets users make online and offline payments through UPI and its FamCard, with a flagship Spending Account, FamX, for managing and saving money. It is used by people above 11 years old in India who want to make UPI and card payments.

More jobs at FamPay