Databricks Engineer - Senior/ Lead
Blend360Hyderabad, TS, IndiaRemote
Needs a free Superjobs account. You'll finish on Blend360's SmartRecruiters page.
Skill score
Summary
This Azure Databricks Engineer builds and tunes scalable data pipelines and analytics solutions on Azure, using Python, SQL, and Spark to process large datasets and maintain Delta Lake tables. It suits a senior or lead data engineer with six or more years of experience who can work with analytics and ML teams and enforce data quality and governance.
- Function
- Development
- Estimated salary
- Est. ₹14–20 LPA
- Estimated for a manager role, ~6 yrs experience, India. Not published by the employer.
- Work mode and location
- Remote, Hyderabad, TS, India
- Experience
- 6+ yrs
- Employment type
- Full-time
Job description
Overview
We are looking for an experienced Azure Databricks Engineer with strong hands-on expertise in Python, SQL, and Apache Spark to design, build, and optimize scalable data pipelines and analytics solutions on the Azure cloud platform. The ideal candidate should have experience working with large datasets, distributed data processing, and modern data engineering practices.
Roles and responsibilities
- Design, develop, and maintain scalable data pipelines using Azure Databricks
- Implement ETL/ELT workflows using PySpark, Spark SQL, and Python
- Optimize Spark jobs for performance, cost, and scalability
- Work with structured and semi-structured data (Parquet, Delta, JSON, CSV)
- Build and manage Delta Lake tables (ACID, time travel, schema evolution)
- Integrate Databricks with Azure Data Lake Storage (ADLS Gen2)
- Develop complex queries and transformations using SQL
- Collaborate with data scientists, analysts, and stakeholders to support analytics and ML use cases
- Ensure data quality, validation, and monitoring
- Follow best practices for security, access control, and governance in Azure
Skills required
Top skills
- Azure Databricks
- Python
- SQL
- Apache Spark
All skills (8)
- PySpark
- Delta Lake
- Azure Data Lake Storage Gen2
- Git
- Distributed computing
- Azure Data Factory
- CI/CD pipelines
- Streaming data
Qualifications
- 6+ years of experience in Data Engineering
- Strong hands-on experience with Azure Databricks
- Proficiency in Python for data processing
- Strong knowledge of SQL (joins, window functions, performance tuning)
- Hands-on experience with Apache Spark / PySpark
- Experience working with Delta Lake
- Knowledge of Azure Data Lake Storage (ADLS Gen2)
- Understanding of distributed computing concepts
- Experience with Git version control
Nice to have
- Experience with Azure Data Factory
- Exposure to CI/CD pipelines (Azure DevOps, GitHub Actions)
- Basic understanding of data modeling
- Familiarity with cloud security and RBAC in Azure
- Exposure to streaming data (Spark Structured Streaming, Event Hub, Kafka)
Company info
About Blend360
Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. With a mission to fuel bold visions, Blend tackles significant challenges by seamlessly aligning human expertise with artificial intelligence. The company is dedicated to unlocking