Azure Databricks Developer (SJ-IQ)
SE Mentor SolutionsBangalore, Karnataka; Kochi, KeralaOn-Site
Needs a free Superjobs account. You'll finish on SE Mentor Solutions's site.
Skill score
- Function
- Development
- Estimated salary
- Est. ₹12–18 LPA
- Estimated for a mid-level role, ~6 yrs experience, India. Not published by the employer.
- Work mode and location
- On-Site, Bangalore, Karnataka; Kochi, Kerala
- Experience
- 6+ yrs
- Employment type
- Full-time
Job description
Job Description: Senior Developer – Azure Databricks (Unity Catalog & Genie AI)
Job Title: Senior Developer – Azure Databricks, Unity Catalog & Genie AI
Experience: 6-10 Years
Location: Bengaluru / Gurgaon/ Kochi
Employment Type: Full-Time
Role Overview
We are seeking an experienced Senior Developer – Azure Databricks to design, develop, and optimize enterprise-scale data and AI platforms on Azure. The ideal candidate will possess strong expertise in Azure Databricks, Unity Catalog, Delta Lake, Medallion Architecture, and hands-on experience with Databricks Genie AI for conversational analytics and AI-driven data consumption.
The role involves building modern data platforms, implementing governance frameworks, enabling self-service analytics, and supporting AI/GenAI initiatives across business functions.
Key Responsibilities
Data Engineering & Platform Development
• Design and develop scalable data pipelines using Azure Databricks and Apache Spark.
• Implement ETL/ELT solutions for structured, semi-structured, and unstructured data.
• Develop and optimize Delta Lake-based data platforms following Medallion architecture (Bronze, Silver, Gold).
• Create reusable frameworks for ingestion, transformation, data quality, and monitoring.
• Optimize Spark workloads for performance, scalability, and cost efficiency.
Unity Catalog & Data Governance
• Design and implement enterprise data governance using Unity Catalog.
• Configure catalogs, schemas, tables, views, and external locations.
• Implement fine-grained access controls including row-level and column-level security.
• Establish data lineage, metadata management, and governance policies.
• Collaborate with data stewards and security teams to ensure compliance and auditability.
Databricks Genie AI & Self-Service Analytics
• Develop and configure Genie Spaces for conversational analytics.
• Enable natural language querying for business users across governed data domains.
• Create semantic layers, business definitions, and trusted datasets for Genie.
• Partner with business stakeholders to improve data discoverability and self-service adoption.
• Integrate Genie with enterprise AI and analytics use cases.
• Collaborate with Data Scientists and ML Engineers for model operationalization.
Required Qualifications
Technical Skills
• 6+ years of experience in Data Engineering.
• 4+ years of hands-on experience with Azure Databricks.
• Strong expertise in:
- Apache Spark (PySpark)
- Delta Lake
- Unity Catalog
- Databricks Workflows
- Databricks SQL
• Experience implementing Medallion Architecture and Lakehouse solutions.
• Strong understanding of data modelling principles and dimensional modelling.
• Experience with Azure ecosystem:
- Azure Data Factory
- Azure Storage (ADLS Gen2)
- Azure Key Vault
- Azure Entra ID (Azure AD)
• Experience with Git, CI/CD, and DevOps practices.
Unity Catalog Expertise
• Catalog and schema management.
• Data lineage and metadata governance.
• Security and access management.
• External locations and storage credentials.
• Regulatory compliance and audit support.
Genie AI Expertise
• Designing Genie Spaces and conversational experiences.
• Creating trusted semantic datasets.
• Natural language query optimization.
• Business glossary and metadata integration.
• User enablement and self-service analytics adoption.
Company info
About SE Mentor Solutions
SE-Mentor Solutions provides IT services and AI solutions, including data engineering, automated quality assurance, and full software development lifecycle management. Its offerings are aimed at enterprises looking to modernize their business.