Databricks Developer with 4 years of experience in designing modern data platforms on Azure. Skilled in Azure Databricks, PySpark, Spark SQL, Delta Lake, Unity Catalog, Delta Live Tables, Structured Streaming, Azure Data Factory, and Azure Data Lake Storage Gen2. Experienced in building scalable ETL frameworks, optimizing Spark applications, and delivering analytics-ready data.
Expertise & skills
Cloud
Microsoft Azure
Databricks
DatabricksDelta LakeDelta Live TablesUnity CatalogStructured StreamingAuto LoaderDatabricks SQL
Programming
PythonPySparkSQL
Big Data
Apache SparkSpark SQL
Azure Services
Azure Data FactoryADLS Gen2Event HubsAzure Key VaultAzure Monitor
Databases
SQL ServerCosmos DB
DevOps
GitAzure DevOpsCI/CD
Reporting
Power BI
Selected projects
Pharmaceutical Sales Analytics Platform
- Built scalable ETL pipelines to consolidate sales and distributor data.
- Implemented Medallion Architecture using Delta Lake.
- Developed incremental ingestion with Auto Loader.
- Optimized Spark jobs using adaptive query execution and caching.
- Delivered Gold datasets for executive dashboards.
Logistics Shipment Tracking Platform
- Processed shipment events using Structured Streaming.
- Integrated Event Hubs with Azure Databricks for near real-time processing.
- Implemented Unity Catalog for centralized governance.
- Created reusable PySpark notebooks for transformation.
- Reduced processing latency through Spark optimization.
HR Workforce Analytics Platform
- Migrated legacy ETL workloads to Azure Databricks.
- Developed Delta Live Tables pipelines with data quality rules.
- Created parameterized workflows using Databricks Workflows.
- Supported production deployments and incident resolution.
- Collaborated with QA and business stakeholders in Agile sprints.
Certifications
- Databricks Fundamentals
- Microsoft Azure Fundamentals (AZ-900)
- Microsoft Azure Data Fundamentals (DP-900)
Education
Bachelor of Technology (Computer Engineering)
Indus University
Achievements
- 4 years of enterprise Azure Databricks experience.
- Expertise in Delta Live Tables, Structured Streaming, and Unity Catalog.
- Hands-on experience building Lakehouse architecture.
- Strong Spark performance tuning and troubleshooting skills.
- Experienced with Agile, code reviews, and CI/CD deployments.
Similar engineers
Other Data Engineering engineers with comparable experience.
