Posts

Showing posts from May, 2020

ADF using Parameterized Stored Procedure

Image
Introduction Here in this article, we are trying to demonstrate the Stored Procedure Activity of ADF and how we pass the parameters in it. Hope it will be interesting. Case Scenario We have a table named [ ProductInfo ]. It contains all the product name.   CREATE TABLE ProductInto (       ProductID    INT ,       ProductName VARCHAR ( 50 ) ) GO INSERT INTO ProductInto VALUES ( 1 , 'Tooth Past' ),( 2 , 'Tooth Brush' ) Now, we have another Table named [ OrderInfo ], contains all the order information. CREATE TABLE OrderInfo (    OrderID               INT ,    ProductID             INT ,    Qty                   INT ,    Rate                  DECIMAL ( 18 , 2 ) ) GO INSERT INTO OrderInfo VALUES ( 1 , 1 , 2 , 50 ), ( 1 , 2 , 1 , 30 ), ( 2 , 1 , 1 , 50 ) We are going to read one by one Product from our [ ProductInfo ] table, calculate the Order amount and insert records in [ ProductWiseOrderAmount ] Tabl

Copying Multiple File from Blob Storage to Single SQL Table – Part-2

Image
Introduction This is the extension of my proves blog post. If you not go with previous, I strongly recommended you to read my previous blog post named “Copying Multiple File from Blob Storage to Single SQL Table”. Case Scenario Here in this project, we are going to retrieve the CSV file from Azure Blob Storage, but we must retrieve the current date file only and store it into a Azure SQL DB Table. For SSIS developer it is work of using Expression, here I am also using the expression but in different ways that supports by Azure ADF. Source Files in Azure Blob Storage We have 3 file in our Azure Blob Storage. But only one file have Modified Date 30/5/2020. So, we need to retrieve that file only which matches the current system date.   ADF Solution   Step-1 We are using Get MetaData   Activity which retrieve all file information from Blob Storage and send it to the ForEach Look container Activity. Step-2 Within the ForEach Loop Activity we used another Get MetaData

Copying Multiple File from Blob Storage to Single SQL Table

Image
Introduction This blog is for SSIS people, who want to move to ADF v2. Here we are using a simple case scenario like SSIS and continuing to extend the solution in my coming blog post. Here we assume that the readers has basic understanding with ADF. Hope it will be interesting. Case Scenario The case scenario is most common for SSIS developer. I have a Blob Storage and there are two txt file with same metadata.   I need to load it an Azure SQL Server Table (Both the file).   Source Files in Azure Blob Storage ADF Solution Lookup Activity Configuration Here I am using a lookup to extract data from a table that have table name that need to be load from Azure Blob storage to Azure SQL Table object. SELECT FileNames FROM    LookupTab WHERE   NeedLoad = 'Y'   There is a property in Lookup named First Row Only We need to unchecked it as we need all the records that is return by our SQL statement. ForEach Activity Configuration The output tha