Access Azure Databricks data with Entra ID identities from Alteryx Designer using multi-tenant application
- Alteryx Designer
- Azure Databricks
- Entra ID (Azure AD) admin access
OAuth is a widely accepted industry standard for authorization. This flow is suitable for applications that need to access Databricks on behalf of users. In this specific scenario, organization can manage access to a group of services via EntraID.
EntraID Databricks OAuth Diagram (U2M Flow)
Pros
This is the industry standard for the User to Machine (U2M) authentication. It relies on the user to explicitly grant their consent to third-party applications to access Databricks data on their behalf. The flow results in a short-lived access token, used to establish connection to Databricks, which mitigates the risk of token misuse. Additionally, it allows organizations to manage all identities from within one interface assigning users to groups of services at once. Note, Alteryx also requires the EntraID OAuth client to issue a refresh token that is used to obtain a new access token.
Cons
Management - this flow requires additional configuration for Databricks environments running on AWS to establish trust between your EntraID tenant and Databricks instance.
EntraID tenant configuration - this flow might require additional configuration of your EntraID tenant to permit the use of Alteryx-managed multi-tenant application, addition details can be found in the configuration guide - link provided below.
Refresh token lifetime - to run automated workflows, organizations are required to configure a reasonable lifetime for refresh tokens.
Internal policies - organizations might have internal mandate to only rely on the user identities when a workflow execution is triggered by an individual, making this authentication not suitable for automated runs. The M2M, machine-to-machine, OAuth flow can be used for automated runs, instead.
Detailed instructions covering how to setup this flow for your Databricks users and create a connection from Alteryx can be found below.
Alteryx Designer enables users leverage two types of Azure AD application configurations to access Databricks data: single-tenant and multi-tenant application. This configuration covers multi-tenant application configuration.
To access Databricks data with Azure AD accounts, users are required to have the following configuration in place:
-
Have the latest version of Databricks ODBC driver installed on users' machines. The latest version can be downloaded from Alteryx Data sources page:
Data Sources
-
Have Azure Databricks account created within their tenant;
-
Obtain authentication details required to setup a new connection between Alteryx Designer and Databricks.
Multi-tenant application setup
In this post we will cover how to:
-
Obtain required details of your Azure Databricks instance;
-
Connect to Databricks using Azure AD account from Alteryx Designer.
Please note, the following example is intended for demonstration purposes only. We recommend engaging your systems team to help you with the configuration. This example covers multi-tenant OAuth implementation.
Obtain details of your Azure Databricks instance
To allow users access Azure Databricks data using Azure Active Directory identities organisations need to make sure that users exist and are identified by the same user principal names (UPNs) in both Databricks and Azure AD. Databricks recommends using user email address as UPN.
In order to access your Azure Datbaricks data, you need to obtain the following inputs from your Azure Databricks account administrator:
-
Azure Databricks server URI;
-
Port;
-
HTTP path of your Azure Databricks cluster;
Access Databricks data with Azure AD Account
You can access your data in Databricks from Alteryx Designer using your Azure AD account. Add input or output tool, check “Use Data Connection Manager (DCM)” box and select Databricks from the list of available data sources in Alteryx Designer. Select Quick Connect, provide your Databricks instance details.
Next, create new credential and select Azure AD authentication method:
Do not change the value of the
scopeparameter. It represents the programmatic ID for Azure Databricks (2ff814a6-3304-4ab8-85cb-cd0e6f879c1d) along with the default scope (/.default, URL-encoded as%2f.default).
After filling out above details and clicking connect, you will be redirected to the Azure AD login page. You will be prompted to login with your Azure AD account and grant this application required permissions. Once done, you will be able to explore your Azure Databricks data.
Related Documentation
Please refer to the following documentation to learn more about Azure Active Directory and Azure Databricks integration.
Get Microsoft Entra ID tokens for service principals - Azure Databricks
Common problems
Depending on your Azure AD configuration, individual users might not be able to grant required consent to Azure AD application. In this case you might be presented with one of the following cases:
-
Approval request submitted to account admin. You will be presented with the following prompt asking you to submit approval request to your account admin who would need to approve and grant required access to this application;
Upon submitting request, your account admin will receive a notification and will need to approve your request. Once approval is granted, you can reset the connection and access your Azure Databricks data.
- Need admin approval. This screen appears when your Azure AD settings don’t allow individual users grant consent to applications. In this specific case, accounts setting don’t allow users to submit approval requests to your directory admin. To resolve this problem, ask your account admin to use his/her credentials when first setting up connection from Alteryx designer and provide consent to this app on behalf of the organisation. Upon completing these steps, all further Azure AD tenant users shall be able to grant consent without further approvals. Alternatively, you might want to ask your Azure AD admin to update your tenant consent policy to allow users submit approval requests to account admin.