Results-oriented Databricks Data Engineer with 8 years of experience delivering enterprise data platforms on Azure and AWS. Expertise in building scalable Lakehouse solutions, data ingestion frameworks, Delta Lake optimization, Spark performance tuning, and analytics engineering.
Expertise & skills
Databricks
DatabricksDatabricks SQLDelta LakeUnity CatalogDelta Live TablesMLflow
Data Engineering
Apache SparkPySparkSpark SQLStructured StreamingAuto Loader
Cloud
Azure Data FactoryADLS Gen2Azure SynapseAWS S3
Programming
PythonSQLScala
DevOps
GitGitHubAzure DevOpsJenkins
Visualization
Power BITableau
Selected projects
Retail Demand Forecasting Platform
DatabricksPySparkDelta LakePower BI
Responsibilities
- Built scalable ETL pipelines for sales and inventory data.
- Designed Bronze, Silver and Gold data layers.
- Optimized Spark jobs using partitioning and caching.
- Developed reusable transformation notebooks.
- Reduced reporting time by 40%.
Digital Banking Data Hub
DatabricksSpark SQLUnity Catalog
Responsibilities
- Implemented secure data governance using Unity Catalog.
- Created curated datasets for business reporting.
- Automated daily data ingestion workflows.
- Improved data quality with validation rules.
- Optimized Delta tables.
Telecom Customer Insights
DatabricksStructured StreamingKafka
Responsibilities
- Built real-time streaming pipelines.
- Processed event data from multiple sources.
- Created analytics-ready datasets.
- Implemented monitoring and alerting.
- Improved streaming throughput.
Healthcare Data Integration
DatabricksPythonDelta Live Tables
Responsibilities
- Integrated data from EMR systems.
- Implemented automated data quality checks.
- Built reusable ETL framework.
- Optimized batch processing.
- Delivered standardized reporting datasets.
Enterprise Data Warehouse Migration
DatabricksADLS Gen2PySpark
Responsibilities
- Migrated legacy warehouse workloads.
- Converted SQL procedures into PySpark.
- Automated deployment pipelines.
- Validated migrated datasets.
- Improved overall processing efficiency.
Certifications
- Databricks Certified Data Engineer Professional
- Microsoft Certified: Azure Data Engineer Associate (DP-203)
Education
Bachelor of Engineering (Information Technology)
Achievements
- Built 250+ production Databricks notebooks.
- Delivered 45+ enterprise data engineering solutions.
- Reduced ETL execution time by 40% through Spark optimization.
- Processed more than 90 TB of enterprise data.
- Established reusable Lakehouse development standards across projects.
Similar engineers
Other Data Engineering engineers with comparable experience.
