RAKESH SHARMA
Jaipur
Summary
A data engineer having rich experience in development of data processing modules and reporting using variety of tools and technologies within Insurance and retail domain. Skilled in pre-sales activities, promoting and demonstrating advanced analytical products.
Overview
3
3
years of professional experience
1
1
Certification
Work History
DATA ENGINEER
TATA AIG
11.2022 - 07.2024
- Migrated large-scale insurance data from on-premises to Azure Data Lake using ADF, automating real-time and batch ingestion pipelines for seamless data flow
- Processed and transformed data in Databricks using PySpark, applying business rules and analytics to optimize workflows and enhance efficiency
- Developed parameterized PySpark scripts for dynamic dataset retrieval and implemented year-over-year business growth analysis with flexible date parameters
- Built scalable pipelines leveraging Databricks features like Unity Catalog for data governance, Delta Lake for ACID storage, Auto-Scaling Clusters, Job Scheduling, and Notebooks for collaboration
- Optimized big data processing with PySpark, utilizing RDDs, Data Frames, UDFs, Broadcast Variables, Window Functions, and Adaptive Query Execution (AQE) for performance tuning
- Enhanced PySpark & SQL query execution by applying partitioning, caching, and broadcast joins, fine-tuning executor cores, memory configurations, and parallelism settings for efficient resource utilization
- Developed dashboards using Mcube for data reporting and visualization, providing actionable insights to business teams
- Ensured data accuracy and compliance through output validation and root cause analysis, collaborating with cross-functional teams to maintain data consistency
- Engaged with clients and stakeholders to handle ad-hoc data requests and optimize reporting solutions
- Insurance Domain
PYSPARK DEVELOPER
REDTAG (TCG)
06.2022 - 10.2022
- Migrated IBM DataStage jobs to PySpark, optimizing scripts for performance and efficiency
- Implemented data quality checks to ensure integrity and supported production jobs, resolving issues within SLAs
- Retail Domain
PRESALES ENGINEER
TCG
12.2021 - 06.2022
- Demonstrated TCG Mcube, highlighting its capabilities in data storage, ingestion, reporting, and analytics
- Engaged with clients to address business challenges using data-driven solutions, advanced analytics, and low-code APIs
- Demonstrated pre-built solutions across industries, such as the OLPIS (Online Plant Information System) solution for the petrochemical sector, enabling real-time plant monitoring, data-driven decision-making, and parameter optimization recommendations to maximize yield
Education
Master's - Data science
CHRIST UNIVERSIT
01.2022
Skills
- Apache Spark (PySpark)
- Azure Data Engineering Tools
- Hadoop
- HDFS
- Hive
- SQL
- PL/SQL
- No-SQL
- Python
- Databricks
- Elasticsearch
- Greenplum
- PostgressSQL
- Unix Shell Scripting
Certification
- DP-203 - Data Engineering on Microsoft Azure
- Azure Databricks & Spark For Data Engineers: Hands-on Project
- Data Engineering: Learn SQL, Python & Spark
Awards
- On the Spot Award, 2023
- Best client feedback, 2023
Timeline
DATA ENGINEER
TATA AIG
11.2022 - 07.2024
PYSPARK DEVELOPER
REDTAG (TCG)
06.2022 - 10.2022
PRESALES ENGINEER
TCG
12.2021 - 06.2022
Master's - Data science
CHRIST UNIVERSIT
Similar Profiles
HEMAL H. TOPIWALAHEMAL H. TOPIWALA
Sales Manager at Tata AigSales Manager at Tata Aig
Dr. Shweta Todankar SalunkeDr. Shweta Todankar Salunke
Senior Manager at Tata AIGSenior Manager at Tata AIG
ARPITA RAIARPITA RAI
Underwriter at Tata AIGUnderwriter at Tata AIG
Rijo RajanRijo Rajan
Channel Sales Manager at TATA AIGChannel Sales Manager at TATA AIG
DIKSHA SHARMADIKSHA SHARMA
HR Executive at TTL Engineering Pvt LtdHR Executive at TTL Engineering Pvt Ltd