Databricks Data Engineer with 4 years of experience building scalable Lakehouse solutions on Azure. Experienced in PySpark, Spark SQL, Delta Live Tables, Unity Catalog, Auto Loader, Azure Data Factory, ADLS Gen2, and Azure DevOps. Skilled in performance tuning, ETL automation, and delivering enterprise-grade analytics platforms.
Expertise & skills
Cloud
Microsoft Azure
Databricks
DatabricksDelta LakeDelta Live TablesUnity CatalogPhotonDatabricks SQL
Programming
PythonPySparkSQL
Data Engineering
Apache SparkETLData LakehouseData Modeling
Azure Services
ADLS Gen2Azure Data FactoryAzure Key VaultAzure Monitor
Databases
SQL ServerAzure Synapse
DevOps
GitAzure DevOpsCI/CD
Visualization
Power BI
Selected projects
Insurance Data Modernization
- Designed end-to-end ETL pipelines for policy, claims, and customer datasets.
- Built Bronze, Silver, and Gold layers using Delta Lake architecture.
- Implemented Delta Live Tables to automate data quality validation.
- Optimized Spark transformations using Photon and partitioning.
- Integrated curated datasets with Power BI for business reporting.
Telecom Customer Usage Analytics
- Developed incremental ingestion pipelines using Auto Loader.
- Created PySpark notebooks to process large-scale CDR datasets.
- Implemented Unity Catalog for secure data governance.
- Reduced ETL runtime through Spark optimization techniques.
- Supported deployment using Azure DevOps CI/CD pipelines.
Supply Chain Data Platform
- Developed reusable ETL framework using parameterized notebooks.
- Integrated ERP and warehouse datasets into Azure Data Lake.
- Created business-ready Gold tables for KPI reporting.
- Performed production support and root cause analysis.
- Collaborated with business analysts in Agile sprints.
Certifications
- Databricks Fundamentals
- Microsoft Azure Fundamentals (AZ-900)
- Microsoft Azure Data Engineer Associate (DP-203)
Education
Bachelor of Engineering (Information Technology)
Charotar University of Science and Technology
Achievements
- 4 years of hands-on Azure Databricks experience.
- Built enterprise Lakehouse solutions using Delta Lake.
- Experience with Delta Live Tables, Unity Catalog, and Photon.
- Strong Spark performance tuning and production support skills.
- Experienced working in Agile/Scrum delivery teams.
Similar engineers
Other Data Engineering engineers with comparable experience.
