Job Position: Databricks Forward Deployed Engineer (FDE)
Experience: 5–8 Years
Location: PAN India
Function: Data & AI Engineering
Level: Senior G7
We are looking for an experienced Databricks Forward Deployed Engineer (FDE) to design, develop, and optimize production-grade data platforms for enterprise clients. The ideal candidate should have strong expertise in Python, PySpark, Apache Spark, Databricks, Microsoft Azure, Generative AI, and CI/CD, along with excellent communication and client-facing skills.
The role involves working directly with clients to understand business requirements, design scalable Lakehouse architectures, develop data pipelines, troubleshoot performance issues, and deliver reliable, cost-effective data engineering solutions.
Data Engineering & Lakehouse Architecture
Lead client discussions, technical discovery sessions, and solution design activities.
Design and implement enterprise Lakehouse solutions using Medallion Architecture (Bronze, Silver, and Gold).
Develop and optimize production-grade data pipelines using Python, PySpark, Spark SQL, and Delta Lake.
Implement batch and streaming data ingestion using Auto Loader, Kafka, and related frameworks.
Configure Databricks clusters, Unity Catalog, access controls, and data governance.
Performance Optimization & Deployment
Optimize Spark jobs and Delta Lake tables for performance, scalability, and cost efficiency.
Implement CI/CD pipelines using Databricks Asset Bundles (DABs), GitHub Actions, or Azure DevOps.
Monitor production workloads, troubleshoot technical issues, and implement data quality checks and alerting.
Manage production deployments and ensure reliable, maintainable, and scalable data solutions.
Generative AI & Client Engagement
Support GenAI solutions involving RAG pipelines, vector stores, and LLM evaluation.
Present technical recommendations and architecture decisions to clients and business stakeholders.
Translate business requirements into technical solutions and take ownership of end-to-end delivery.
Explain complex technical concepts clearly to technical and non-technical stakeholders.
5+ years of experience in data engineering within production environments.
At least 2 years of hands-on experience with Databricks or Apache Spark.
Strong programming skills in Python.
Hands-on experience with Databricks, Apache Spark, PySpark, and Spark SQL.
Experience with Microsoft Azure cloud services.
Experience with React.
Strong SQL and data pipeline development skills.
Experience designing Medallion Architecture and enterprise Lakehouse platforms.
Knowledge of Delta Lake, data ingestion, performance tuning, and data governance.
Hands-on experience with CI/CD pipelines and deployment automation.
Exposure to Generative AI (GenAI) and related solutions.
Excellent communication, client-facing, and stakeholder management skills.
Experience with Structured Streaming, Delta Live Tables, or Kafka.
Knowledge of Unity Catalog, Auto Loader, Change Data Capture (CDC), and Slowly Changing Dimensions (SCD).
Exposure to GenAI, Retrieval-Augmented Generation (RAG), vector databases, and LLM evaluation.
Databricks Data Engineer Associate or Professional certification.
Experience with AWS or GCP in addition to Azure.
Previous experience in consulting, Global System Integrators (GSIs), or client-embedded engineering roles.
Excellent verbal and written communication skills.
Strong client-facing and stakeholder management capabilities.
Ability to communicate complex technical concepts to diverse audiences.
Strong problem-solving, analytical thinking, and decision-making skills.
Proactive mindset with end-to-end ownership of technical delivery.
Ability to work directly with enterprise clients and deliver high-quality technical solutions.
Databricks | Python | PySpark | Apache Spark | Spark SQL | Microsoft Azure | Delta Lake | Lakehouse Architecture | Medallion Architecture | SQL | React | CI/CD | Databricks Asset Bundles (DABs) | GitHub Actions | Azure DevOps | Unity Catalog | Auto Loader | Kafka | GenAI | RAG | Vector Databases | LLM Evaluation | Client Engagement | Data Engineering