Data Engineer
SynechronBengaluru - Thanissandra (BCIT)On-Site
Needs a free Superjobs account. You'll finish on Synechron's site.
Skill score
Summary
The Senior Data Engineer analyzes existing Omnia-based Spark pipelines, derives their business and transformation logic, and rebuilds it as scalable pipelines on AWS, Snowflake and dbt. This suits an experienced engineer strong in PySpark, Python, advanced SQL and Airflow who can work across legacy and cloud data platforms.
- Function
- Data Science
- Estimated salary
- Est. ₹9.5–14 LPA
- Estimated for a mid-level role, ~3 yrs experience (typical for the level), India. Not published by the employer.
- Work mode and location
- On-Site, Bengaluru - Thanissandra (BCIT)
- Employment type
- Full-time
Job description
Overview
Role: Senior Data Engineer
Role Overview
Experienced Data Engineer to analyze, design and develop enterprise-scale data pipelines across legacy and strategic cloud data platforms.
A key component of the position is analyzing existing data pipelines, particularly Omnia-based Spark pipelines, understanding the underlying business and transformation logic, and translating that logic into scalable data solutions on modern cloud platforms. The role requires strong hands-on experience with AWS, Snowflake, dbt, Airflow, PySpark, Python and advanced SQL.
Roles and responsibilities
- Conduct detailed analysis of existing end-to-end data pipelines across multiple platforms, with a particular focus on Omnia - PySpark-based processing.
- Review existing source code, SQL, transformation rules, mappings and data flows to understand the current business and technical logic.
- Derive, document, and implement new transformation logic based on the analysis of existing pipelines and business requirements.
- Design, develop and maintain reliable, scalable, and reusable data pipelines using AWS, Snowflake and dbt.
- Develop batch and, where required, event-driven data processing solutions using cloud-native and distributed data technologies.
- Translate existing Omnia data processing logic into strategic cloud-based data engineering solutions.
- Build ingestion, transformation, enrichment and data integration pipelines across data lakes, data warehouses and cloud lakehouse platforms.
- Develop complex and performance-optimized SQL for data transformation, reconciliation, validation and analytical processing.
- Analyze end-to-end data flows from source capture through transformation, enrichment and downstream consumption.
- Work with data designers, product owners, and subject matter experts to translate business requirements into technical data rules and durable engineering solutions.
- Implement data quality, reconciliation and balancing controls across data pipelines.
- Troubleshoot data pipeline failures, transformation issues, data discrepancies and performance bottlenecks across multiple environments.
Skills required
Top skills
- AWS
- Snowflake
- dbt
- Apache Airflow
All skills (8)
- PySpark
- Python
- Advanced SQL
- Spark ETL
- Git
- CI/CD
- Jenkins
- Omnia platform
Qualifications
- Strong hands-on experience designing, developing and supporting scalable enterprise data pipelines.
- Strong Spark-etl experience, including the ability to analyze existing Omnia pipelines, understand their processing logic and derive new transformation and business rules.
- Strong working knowledge of AWS services, including S3, Glue, EMR, Lambda, Athena, Step Functions, SQS, IAM, Secrets Manager and CloudWatch, combined with hands-on experience in orchestrating data pipelines using Apache Airflow.
- Practical experience with Snowflake, covering data ingestion, data modelling, transformation and performance optimization.
- Experience using dbt to develop data models, incremental processing, reusable macros, automated tests, documentation and dependency management.
- Excellent SQL skills, with experience in complex data transformations, query optimization, reconciliation and understanding existing business logic.
- Familiarity with Git, automated testing, CI/CD, Jenkins and DevSecOps practices within a data engineering environment.
- Good Python programming skills for data processing, automation, and reusable framework components.
Nice to have
- Previous CBA experience, particularly with Omnia or CBA data platforms and engineering standards, is highly desirable.
Company info
About Synechron
Synechron provides agentic AI, data, cloud and software engineering services that help banks, asset managers and insurers modernize their operations. Its clients are financial services firms in banking, asset management and insurance.