Data Engineer, Data Platforms
S&P GlobalIN - CHENNAI PRESTIGE POLYGON BUILDING, 2 Locations, IN - HYDERABAD ORIONOn-Site
Needs a free Superjobs account. You'll finish on S&P Global's site.
Skill score
Summary
The data engineer builds, supports, and stabilizes data pipelines as they move onto a Databricks and AWS lakehouse platform, and helps offshore teams onboard datasets using shared frameworks. Suited to a senior engineer with 5 to 7 years of data platform experience who can give technical guidance across time zones and uses AI tools responsibly.
- Function
- Data Science
- Estimated salary
- Est. ₹13–19 LPA
- Estimated for a mid-level role, ~5 yrs experience, India. Not published by the employer.
- Work mode and location
- On-Site, IN - CHENNAI PRESTIGE POLYGON BUILDING, 2 Locations, IN - HYDERABAD ORION
- Experience
- 5–7 yrs
- Employment type
- Full-time
Job description
Roles and responsibilities
Grade Level (for internal use)
10Key Responsibilities
Data Pipeline Transition and Platform Delivery
- Partner with Databricks and platform teams to transition existing pipelines into the target enterprise data platform.
- Provide hands-on engineering support to stabilize and optimize pipelines post-transition, focusing on performance, scalability, maintainability, and operational reliability.
- Implement and follow repeatable engineering patterns for ingestion, transformation, testing, deployment, and monitoring.
- Support secure AWS deployment patterns and governed data access using AWS IAM, encryption, and platform guardrails.
- Work with lakehouse technologies such as Delta Lake, Apache Iceberg, and Databricks Unity Catalog, and support metadata-driven pipeline design.
- Assist with establishing and maintaining observability practices (logging, telemetry, alerts, dashboards) for production pipelines.
Data Onboarding and Asset-Agnostic Enablement
- Support asset-agnostic onboarding by applying standard ingestion and transformation frameworks to new datasets.
- Help offshore teams adopt common onboarding standards, delivery practices, and technical templates.
- Contribute to reusable technical assets such as reference implementations, onboarding templates, documentation, and operational runbooks.
- Support both batch and streaming onboarding patterns using AWS services (e.g., AWS Glue, AWS Lambda, Amazon Kinesis) or event-driven integrations as appropriate.
Data Mastering Platform Integration
- Support integration of platform datasets with the enterprise data mastering platform.
- Help establish and/or maintain data quality controls, reconciliation, and metadata alignment for trusted mastered data.
- Support golden record / golden copy generation and maintenance activities for key financial datasets (as applicable to the program).
Technical Leadership and Engineering Excellence
- Provide technical guidance to engineering peers through code reviews, design reviews, and implementation planning.
- Contribute to engineering standards and delivery rigor across CI/CD, infrastructure automation, testing practices, and operational readiness.
- Help ensure documentation quality and operational discipline (runbooks, support procedures, incident learnings).
- Act as a technical bridge between offshore execution teams and global platform stakeholders to resolve delivery blockers and keep alignment.
AI-Assisted Engineering and Context Engineering
- Use AI tools (Claude, GitHub CoPilot, LLMs) to accelerate engineering tasks such as drafting code, refactoring, writing documentation, generating unit tests, and troubleshooting.
- Apply basic prompt engineering and context engineering practices by providing the right technical background, constraints, schemas, examples, and acceptance criteria to improve output quality.
- Validate and verify AI-generated outputs before adoption, especially for correctness, security, and maintainability.
- Follow responsible AI usage expectations, including protection of sensitive data, credentials, and regulated information.
Skills required
Top skills
- Databricks
- AWS Glue
- AWS Lambda
- Amazon S3
All skills (8)
- AWS Lake Formation
- Amazon Kinesis
- AWS IAM
- Delta Lake
- Apache Iceberg
- Unity Catalog
- Data pipelines
- Data mastering MDM
Qualifications
Technical Experience
- 5–7+ years of experience in data engineering, data platforms, or related technical domains.
- Hands-on experience building and operating batch and/or streaming data pipelines.
- Experience working with Databricks-based platforms and lakehouse architectures.
- Strong familiarity with AWS data engineering services such as Amazon S3, AWS Glue, AWS Lambda, AWS Lake Formation, Amazon Kinesis, and AWS IAM.
- Working knowledge of Delta Lake, Apache Iceberg, and Databricks Unity Catalog (or equivalent governed lakehouse concepts).
- Solid understanding of data quality, schema management, lineage/metadata basics, access control, encryption, and operational support.
Leadership and Collaboration
- Ability to provide technical guidance without people management responsibility.
- Experience collaborating with global teams across time zones and working through ambiguous delivery problems.
- Strong communication skills and ability to translate technical requirements into implementable tasks.
AI Tooling (Required)
- Practical experience using AI-assisted engineering tools such as Claude, GitHub CoPilot, or LLM-based development features.
- Ability to use AI effectively with basic context engineering practices (inputs/constraints/examples/acceptance criteria).
- Demonstrated ability to validate AI-generated outputs and follow responsible, secure usage practices.
Nice to have
- Experience with enterprise onboarding frameworks, reusable ingestion patterns, and pipeline templates.
- Familiarity with data cataloging/metadata management and governed data product concepts.
- Knowledge of semantic modeling (e.g., semantic layers, business definitions, canonical models) is a plus.
- Exposure to enterprise data mastering/MDM workflows, golden record concepts, and trusted data distribution.
- Experience in regulated environments with strong governance, auditability, and security expectations.
- Databricks and/or AWS certifications (or equivalent practical experience) are a plus.
Other details
Benefits
We take care of you, so you can take care of business. We care about our people. That’s why we provide everything you—and your career—need to thrive at S&P Global.
Our benefits include
- Health & Wellness: Health care coverage designed for the mind and body.
- Flexible Downtime: Generous time off helps keep you energized for your time on.
- Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.
- Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.
- Family Friendly Perks: It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families.
- Beyond the Basics: From retail discounts to referral incentive awards—small perks can make a big difference.
For more information on benefits by country visit: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspgbenefits.com%2Fbenefit-summaries&data=05%7C02%7Clauren.washington%40spglobal.com%7C2110bb7975e647763da908dc57fae79d%7C8f3e36ea80394b4081a77dc0599e8645%7C1%7C0%7C638481982335774379%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=MLyuUlul44KSiQWXG3isvYhaT7f6JLL5jLpgtgOcHyE%3D&reserved=0
Hiring process
Recruitment Fraud Alert
If you receive an email from a spglobalind.com domain or any other regionally based domains, it is a scam and should be reported to [email protected]. S&P Global never requires any candidate to pay money for job applications, interviews, offer letters, “pre-employment training” or for equipment/delivery of equipment. Stay informed and protect yourself from recruitment fraud by reviewing our guidelines, fraudulent domains, and how to report suspicious activity here.
Company info
About S&P Global
S&P Global provides governments, businesses, and individuals with market data, expertise, and technology solutions for confident decision-making. Our services span from global energy solutions to sustainable finance solutions. From helping our customers perform investment analysis to guiding them through sustainability and energy transition across supply chains, our solutions help unlock
- Founded
- -
- null
- Industry
- Finance
- Funding stage
- Post IPO debt
- Employees
- 5001+
- Company type
- Public Company