Databricks Data Engineer with 2 years of professional experience designing and implementing cloud-based ETL solutions using Databricks, Apache Spark, Delta Lake, and Azure services. Experienced in data ingestion, transformation, workflow automation, performance tuning, and delivering analytics-ready datasets for business intelligence.
Expertise & skills
Big Data
DatabricksApache SparkPySparkDelta LakeSpark SQL
Programming
PythonSQL
Cloud
Microsoft Azure
Azure Services
Azure Data FactoryADLS Gen2Azure Synapse AnalyticsAzure Key VaultAzure Monitor
Databases
Azure SQL DatabaseSQL Server
Version Control
GitAzure DevOps
BI
Power BI
Methodology
Agile / Scrum
Core Skills
Databricks DevelopmentPySparkApache SparkDelta LakeETL DevelopmentAzure Data FactoryPerformance OptimizationSQL DevelopmentAnalytical ThinkingTeam Collaboration
Selected projects
Banking Data Warehouse Migration
DatabricksPySparkDelta LakeAzure Data FactoryAzure SQL
Supported migration of banking data pipelines from legacy ETL to Databricks.
Responsibilities
- Developed scalable PySpark ETL pipelines.
- Created Delta Lake tables using Bronze, Silver, and Gold layers.
- Implemented incremental data loading.
- Optimized Spark jobs with partitioning and caching.
- Performed data validation and reconciliation.
- Supported production deployments and issue resolution.
- Prepared technical documentation.
- Worked closely with QA and business teams.
Telecom Usage Analytics
DatabricksSpark SQLPythonADLS Gen2
Built analytical datasets for telecom customer usage reporting.
Responsibilities
- Designed Spark SQL transformations.
- Developed reusable Databricks notebooks.
- Automated batch processing workflows.
- Implemented exception handling and logging.
- Improved query performance using Delta optimization.
- Created curated datasets for Power BI dashboards.
Customer Master Data Integration
DatabricksAzure Data FactoryPySparkGit
Integrated customer information from multiple enterprise systems.
Responsibilities
- Developed ETL workflows in Databricks.
- Performed data cleansing and standardization.
- Managed source code using Git.
- Monitored scheduled pipeline executions.
- Resolved production support issues.
- Participated in Agile sprint activities.
Core competencies
Databricks DevelopmentPySparkApache SparkDelta LakeETL DevelopmentAzure Data FactoryPerformance OptimizationSQL DevelopmentAnalytical ThinkingTeam Collaboration
Certifications
- Databricks Lakehouse Fundamentals
- Microsoft Certified: Azure Data Fundamentals (DP-900)
- Microsoft Azure Fundamentals (AZ-900)
Education
Bachelor of Engineering (Information Technology)
L.D. College of Engineering
Similar engineers
Other Data Engineering engineers with comparable experience.
