Databricks Developer with 4 years of experience designing scalable data engineering solutions on Azure Databricks. Skilled in PySpark, Spark SQL, Delta Lake, Unity Catalog, Azure Data Factory, Azure Data Lake Storage Gen2, CI/CD, performance optimization, and ETL pipeline development. Experienced in building medallion architecture, implementing data quality frameworks, and supporting analytics workloads.
Expertise & skills
Core Skills
DatabricksApache SparkPySparkSpark SQLDelta LakeDelta Live TablesUnity CatalogAzure Data FactoryAzure Data Lake Storage Gen2Azure DevOpsGitPythonSQLPower BIREST APIsCI/CDAgile / Scrum
Selected projects
1. Enterprise Sales Data Lakehouse
Built Bronze, Silver, and Gold layers using Delta Lake.
- Developed PySpark ETL pipelines for 120+ source tables.
- Optimized Spark jobs reducing execution time by 40%.
- Automated orchestration using Azure Data Factory.
- 2. Customer 360 Analytics Platform
- Developed incremental ingestion pipelines with Auto Loader.
- Implemented Unity Catalog for centralized governance.
- Created reusable notebooks and parameterized workflows.
- Improved data quality through validation frameworks.
Certifications
- Databricks Fundamentals
- Microsoft Azure Fundamentals (AZ-900)
Education
Bachelor of Engineering in Computer Engineering
Gujarat Technological University
Achievements
- 4 years of hands-on Databricks development.
- Experience with Agile/Scrum.
- Strong troubleshooting and production support.
- Worked with batch and near real-time data pipelines.
Similar engineers
Other Data Engineering engineers with comparable experience.
