CODING & ANALYTICS by Gbenga Oridupa

In today’s post, I will like to talk about considerations for choosing to use Azure Blob Storage or Azure Data Lake Store when processing data to be loaded into a data warehouse. Here is a Data Warehouse Architecture published by Microsoft, where it suggests loading data from your source into Azure Blob Storage. Pause for a second!!! Ok, Let's continue!!! So, here are my thoughts on why you may choose one over the other based on my experience in some projects. It really "depends".In most cases you can’t go wrong either way because they are both powerful storage systems.Let's go: Firstly, Text files. ADLS is better with text files compared to ABS.When you talk about non-text data like media files,database backup files etc, are better off with ABS.There are trade-offs with both. Secondly, Geographic redundancy. ABS gives you that out of the box. For ADLS,i...

Microsoft recently introduced the use of templates in Azure Data Factory(ADF). I believe this is a very solid implementation. By having this addition, data engineers can easily get started quickly with building data factory pipelines and improve developer productivity along with reducing development time for same workflows repeatedly. Ready to explore?. Let's jump in for a cool ride: 1. Just click Create pipeline from template on the Overview page or click +-> Pipeline from template on the Author page in your data factory UX to get started. 2. Select any template from the gallery and provide the necessary inputs to use the template. You can also read detailed description about the template or visualize the end to end data factory pipeline. 3. You can also create new connections to your data store or compute while providing the template inputs. 4. Once you cli...