Azure DataBricks Accessing Data Lake (Using Access Key)
Introduction: In this Blog post we are going to discuss about accessing Azure Data Lake Gen2. How can we Access the Azure Data Lake Gen2: Access can be done by Ø Using Storage Access key Ø Using Shared access signature (SAS token) Ø Using Service Principal Ø Using Azure Active directory authentication pass-through Ø Using unity catalog In this post we are going to discuss about accessing Azure Data Lake Gent 2 by using Access key. Authenticate Data Lake with Access Key: · Each storage account comes with 2 access keys · Each access key is 512 bits · Access key gives full access of storage account · Conceder it as super user · Key can be rotated (re-generated) Access Key Spark configuration: Here we take myschool as Azure Data Lake gen2 and there is a container within this data lake named bronze . The bronze containers have a csv file named school.csv spark.conf.set( "