Led Azure resource provisioning and designed the architecture. Also architected & developed the Generic Data Engineering framework for ingestion pipelines with Databricks, Azure Data Factory, SQL DB, Blob Storage and Snowflake. Set up the standards and guidelines.
Designed data solutions which process terabytes of data on monthly scale
Designed CICD process for Azure services being used and worked with infra team for end to end implementation
Worked on front end Streamlit Apps and developed back end Fastapi API's
Worked closely with business stakeholders and helped them with solutions from Data Engineering side to achieve their goals
Senior Data Engineer
Novartis
Dublin, Ireland
11.2020 - 11.2022
Built ingestion framework for Data Science/AI team using Databricks, ADF, Azure SQL, Azure Keyvault and Synapse.
Performed 80% hands-on development and 20% team leading and reviewing of work.
Worked in python dash front end Apps to help business come up with a repository for business documents.