How to create and use Databricks backed secret scope?

In this article, we will learn how to create a Databricks-backed secret scope. Here is the step-by-step explanation for the same:

1. Create a Databricks-backed scope. Here I am creating a scope named mynewscope .

# Syntax
#databricks secrets create-scope --scope <scope-name>

databricks secrets create-scope --scope mynewscope

Please note that If your account has the Azure Databricks Premium Plan, you can change permissions at any time after you create the scope.

2. Set the permission for the scope

databricks secrets create-scope --scope mynewscope --initial-manage-principal users

#You can list the ACLs by using this comand

databricks secrets list-acls --scope <scope Name>
# this will diosplay the results in the following format:
Principal             Permission
--------------------  ------------
User Name(emailID)  MANAGE

3. Once the scope is created you can add the secret

databricks secrets put --scope <scope-name> --key <key-name>

#For example
databricks secrets put --scope mynewscope--key mykey

Here is the screenshot after you enter the above command

This will open a notepad where you need to enter the key.

Once you enter the secret please save the notepad and close it.

4. Now note down the Application client ID and Directory ID from the service principal created to access the data lake so you can use the same in the Powershell.

Now Access the secret and scope by using dbutils utility. We are trying to access the data lake from the python code below.

configs = {"dfs.adls.oauth2.access.token.provider.type": "ClientCredential",
           "dfs.adls.oauth2.client.id": "Client ID is the application id taken from above screenshot",
           "dfs.adls.oauth2.credential": dbutils.secrets.get(scope = "mynewscope", key = "mykey"),
           "dfs.adls.oauth2.refresh.url": "https://login.microsoftonline.com/Azure AD Directory ID/oauth2/token"
 }