Dedicated Databricks Data Engineer with 2 years of experience developing scalable data engineering solutions on the Azure platform. Proficient in Databricks, Apache Spark, PySpark, Delta Lake, Azure Data Factory, and SQL. Experienced in building robust ETL pipelines, optimizing distributed data processing, and delivering trusted datasets for business intelligence and analytics.

Expertise & skills

Big Data

DatabricksApache SparkPySparkDelta LakeSpark SQL

Programming

PythonSQL

Cloud

Microsoft Azure

Azure Services

Azure Data FactoryAzure Data Lake Storage Gen2Azure Synapse AnalyticsAzure Key VaultAzure Monitor

Databases

Azure SQL DatabaseSQL Server

Data Engineering

ETLELTData ModelingData Validation

Version Control

GitAzure DevOps

Reporting

Power BI

Methodology

Agile / Scrum

Core Skills

Databricks DevelopmentPySparkApache SparkDelta LakeAzure Data FactoryData Pipeline DevelopmentSpark Performance OptimizationSQLProblem SolvingTeam Collaboration

Selected projects

Energy Consumption Analytics Platform

DatabricksPySparkDelta LakeAzure Data FactoryADLS Gen2

Designed and implemented scalable cloud data engineering solutions using Databricks and Azure services.

Developed PySpark-based ETL pipelines for smart meter data.
Implemented Bronze, Silver, and Gold Delta architecture.
Automated ingestion using Azure Data Factory.
Performed data quality validation and reconciliation.
Optimized Spark jobs for improved execution time.
Created reusable Databricks notebooks.
Supported production releases and monitoring.
Collaborated with business stakeholders.

Customer 360 Data Integration

DatabricksSpark SQLPythonAzure SQL Database

Designed and implemented scalable cloud data engineering solutions using Databricks and Azure services.

Integrated customer data from CRM and ERP systems.
Developed Spark SQL transformations.
Built incremental loading frameworks.
Implemented logging and exception handling.
Optimized Delta tables using OPTIMIZE and VACUUM.
Prepared analytics-ready datasets for reporting.

Marketing Campaign Analytics

DatabricksPySparkAzure Data Lake StorageGit

Designed and implemented scalable cloud data engineering solutions using Databricks and Azure services.

Built ETL workflows for campaign performance analysis.
Processed large datasets using PySpark.
Managed source code with Git and Azure DevOps.
Supported UAT and production deployments.
Prepared technical documentation.
Participated in Agile ceremonies.

Core competencies

Databricks DevelopmentPySparkApache SparkDelta LakeAzure Data FactoryData Pipeline DevelopmentSpark Performance OptimizationSQLProblem SolvingTeam Collaboration