Skip to content

Sr. Data Engineer-Hadoop,Hive,Spark,Trino, Star Rocks, Clickhouse,Data engineering-3+yrs AI exposure

VisaIN - Bengaluru, IndiaOn-Site

Estimated salary, not published by the employer: Est. ₹11–16 LPA

Needs a free Superjobs account. You'll finish on Visa's site.

Skill score

Summary

The Sr. Data Engineer builds and owns data pipelines and products, translating stakeholder requirements into technical architecture and coding, testing, automating, and monitoring data processes in batch and real time. The role suits an experienced engineer who works with distributed data systems and can also bring AI/ML and secure, regulated-environment practices to the team.

Function
Data Science
Estimated salary
Est. ₹11–16 LPA
Estimated for a senior role, ~3 yrs experience, India. Not published by the employer.
Work mode and location
On-Site, IN - Bengaluru, India
Experience
3+ yrs
Employment type
Full-time

Job description

Overview

Sr. Data Engineers are responsible for collaborating with stakeholders, including product teams, business partners, and cybersecurity, to identify and clarify requirements for data-driven features and products. They independently understand the data ecosystem, security, privacy, and retention requirements necessary to support business and product features. Sr. Data Engineers create and document technical designs for components, identify systems, architectures, tools, and storage needs, and present technical solutions to peers. They stay current with market trends and industry dynamics, applying this knowledge to drive innovation and ensure the safety and security of product components.

Roles and responsibilities

  • Collaborate proactively with stakeholders to clarify requirements and identify key product stakeholders.
  • Translate functional requirements into architecture designs for one or more components, leveraging existing architecture design patterns, and communicate these to Data Engineers.
  • Implement extensible, maintainable, and reusable code using appropriate coding patterns, guidelines, and best practices; participate in code reviews to ensure standards are followed.
  • Provide end-to-end ownership of data products or data pipelines, including mitigation of technical debt and documentation of errors or unexpected bugs.
  • Identify and conduct required testing for moderately complex use cases, including unit testing, coding standards, and security scans.
  • Build tools and standard automation processes to assist in transforming, managing, accessing, deploying, and monitoring data processes in batches and in real time.
  • Write queries to extract and compile raw data across end-to-end pipelines, and implement orchestration techniques to automate data extraction logic.
  • Ensure adherence to data management principles, governance, and tools to maintain data quality across multiple features.
  • Develop and update troubleshooting guidance and procedures for reviewing, addressing, and fixing advanced problems flagged by testing.
  • Manage software upgrades and server patches for security remediation where applicable.

Skills required

Top skills

  • Hadoop
  • Hive
  • HBase
  • Spark

All skills (8)

  • Trino
  • ETL/ELT
  • Java or Python
  • SQL
  • CI/CD pipelines
  • Containerization and orchestration
  • AI agents
  • Real-time systems

Qualifications

All roles require digital fluency, including the ability to work with emerging technologies such as Generative AI tools (e.g. ChatGPT, Microsoft Copilot) to support everyday work.

Basic Qualifications

  • 3+ years of relevant work experience and a Bachelor's degree, OR 5+ years of relevant work experience.
  • Experience in solving data problems using data technologies
  • Experience with Hadoop, Hive, HBase, Spark, Trino, or comparable distributed data systems.
  • Experience in designing and implementing data pipelines and large-scale data processing jobs.
  • Experience in writing, modifying, and reviewing high-quality, testable, and efficient code.
  • Data pipelines, ETL/ELT, data-quality frameworks, Java or Python, SQL, Git, and production troubleshooting.
  • Experience in developing unit, integration, and end-to-end tests with a focus on automation.
  • Experience in building and maintaining CI/CD pipelines and deploying applications using containerization and orchestration tools.
  • Experience in monitoring applications in production and troubleshooting issues using observability tools.
  • Experience in integrating AI or ML services into applications, where applicable.
  • Experience in collecting and analyzing metrics to guide optimizations and improvements.
  • Experience in partnering with stakeholders to clarify requirements and ensure deliverables meet business needs.
  • Experience in applying secure coding practices and complying with regulatory standards.

Nice to have

  • 3 or more years of work experience with a bachelor’s degree or more than 2 years of work experience with an Advanced Degree (e.g. Masters, MBA, JD, MD).
  • Experience in solving data problems using data technologies
  • Experience with Hadoop, Hive, HBase, Spark, Trino, or comparable distributed data systems.
  • Experience in contributing in Open-source components.
  • Use/Build AI agents, and agentic patterns to improve development, testing, operations, troubleshooting, automation, onboarding, and documentation.
  • Data pipelines, ETL/ELT, data-quality frameworks, Java or Python, SQL, Git, and production troubleshooting.
  • Experience in enterprise-class systems engineered for high availability, low latency, scalability, and supportability. Architecture depth across storage/compute separation, query and table formats, streaming, metadata, governance, and interoperability.
  • Experience in building and pushing code into production.
  • Experience in implementing and supporting real-time systems.

Other details

Visa requires at least 3 days in office, expectations of these days will be confirmed by your Hiring Manager.

Company info

Visa is a multinational financial services corporation that facilitates electronic funds transfers worldwide, primarily through Visa-branded credit cards, debit cards, and prepaid cards.

Founded
68 years ago
1958

Recent news

  • TechCrunch

    Visa Expands Digital Payment Solutions with New Feature Launch

  • Finance Weekly

    Visa Announces CFO Leadership Change

  • Retail Dive

    Visa Strengthens Partnership with Major Retail Chain

More jobs at Visa