Results-oriented Databricks Data Engineer with 2 years of experience building modern data platforms on Azure. Skilled in developing PySpark applications, Delta Lake implementations, ETL automation, and cloud data engineering solutions. Experienced in optimizing Spark jobs, integrating enterprise data sources, and delivering high-quality datasets for analytics and reporting.

Expertise & skills

Big Data

DatabricksApache SparkPySparkDelta Lake

Programming

PythonSQL

Cloud

Microsoft Azure

Azure Services

Azure Data FactoryAzure Data Lake Storage Gen2Azure Synapse AnalyticsAzure Key Vault

Databases

Azure SQL DatabaseSQL Server

Workflow

ETLELTData Integration

Version Control

GitAzure DevOps

Reporting

Power BI

Methodology

Agile / Scrum

Core Skills

Databricks DevelopmentApache SparkPySparkDelta LakeAzure Data FactorySQL DevelopmentETL Pipeline DevelopmentPerformance TuningAnalytical Problem SolvingTeam Collaboration

Selected projects

Pharmaceutical Data Processing Platform

DatabricksPySparkDelta LakeAzure Data FactoryADLS Gen2

Developed cloud-based data engineering solutions using Databricks and Azure technologies.

Responsibilities

Developed ETL pipelines for pharmaceutical sales and inventory data.
Built scalable PySpark transformations for large datasets.
Created Delta Lake Bronze, Silver, and Gold layers.
Implemented incremental loading strategies.
Optimized Spark performance using partitioning and caching.
Validated source and target data quality.
Supported deployment and production monitoring.
Prepared technical documentation.

Logistics Shipment Analytics

DatabricksSpark SQLPythonAzure SQL Database

Developed cloud-based data engineering solutions using Databricks and Azure technologies.

Responsibilities

Designed Spark SQL transformations for shipment analytics.
Developed reusable Databricks notebooks.
Automated daily batch processing jobs.
Created curated datasets for reporting.
Implemented logging and exception handling.
Worked with business users to validate data outputs.

HR Workforce Reporting Solution

DatabricksPySparkAzure Data LakeGit

Developed cloud-based data engineering solutions using Databricks and Azure technologies.

Responsibilities

Integrated employee data from multiple HR systems.
Performed data cleansing and standardization.
Maintained ETL workflows in Databricks.
Managed source code using Git.
Resolved production issues and defects.
Participated in sprint planning and code reviews.

Core competencies

Databricks DevelopmentApache SparkPySparkDelta LakeAzure Data FactorySQL DevelopmentETL Pipeline DevelopmentPerformance TuningAnalytical Problem SolvingTeam Collaboration