Responsibility
Job Title: Databricks Architect
Experience: 7–10 Years
Employment Type: Contract (6–12 Months + Extendable)
Location: Bengaluru / Noida / Gurugram / Mumbai / Pune
Work Mode: Hybrid
Role Summary
We are looking for an experienced Databricks Architect with a strong background in Data Engineering and Lakehouse Architecture. The ideal candidate will be responsible for designing scalable, secure, and high-performance data platforms using Databricks, Apache Spark, and Delta Lake. This role involves defining enterprise architecture, optimizing data pipelines, implementing governance, and collaborating with business and technical stakeholders to deliver modern data solutions.
Key Responsibilities
- Design and implement modern Lakehouse architectures using Bronze, Silver, and Gold (Medallion) data models, supporting both batch and streaming workloads.
- Build scalable data pipelines using Databricks, Apache Spark, Delta Lake, and Databricks Workflows, while ensuring high performance, reliability, and operational excellence.
- Establish governance and security using Unity Catalog, access controls, data lineage, and data quality standards.
- Optimize platform performance through cluster management, autoscaling, partitioning strategies, Spark tuning, and workload orchestration.
- Develop CI/CD processes and infrastructure-as-code for notebooks, repositories, jobs, and deployments.
- Integrate Databricks with enterprise ecosystems, including cloud storage, event streaming platforms, data warehouses, and BI tools.
- Conduct solution workshops, provide architectural guidance, prepare implementation roadmaps, and mentor engineering teams on best practices.
Required Skills & Experience
- 7–10 years of overall experience in Data Engineering, including strong expertise in ETL/ELT, distributed computing, and enterprise data platforms.
- Minimum 5+ years of hands-on experience in Databricks Architecture or technical leadership roles.
- Strong expertise in:
- Databricks
- Apache Spark (PySpark/Scala)
- Delta Lake
- Python
- Data Pipeline Design
- Data Architecture
- Performance Tuning
- Experience with data orchestration, Git, CI/CD pipelines, and DevOps practices.
- Strong understanding of secure data platform design, including RBAC, secrets management, networking, and compliance.
- Excellent customer-facing, solution design, and stakeholder management skills.
- At least one Databricks Certification is mandatory.
Preferred Skills
- Experience with Kafka, Azure Event Hubs, Structured Streaming, or Change Data Capture (CDC).
- Exposure to MLflow, feature engineering, and machine learning platform architecture.
- Cloud platform certifications (Azure, AWS, or GCP).
- Experience designing enterprise-scale Lakehouse solutions.
Qualifications
- Bachelor’s or Master’s degree in Computer Science, Engineering, Information Technology, or a related discipline.