This post is for extracting data from Data Cloud. For Ingesting/uploading data in Data Cloud there is a separate post that I had created in the past. Streaming Data with Salesforce Data Cloud Ingestion API and MuleSoft.
Accessing Data Cloud Data: What’s the Difference Between Direct API and Connect API
Salesforce Data Cloud (fka Salesforce CDP) provides two different kinds of APIs to access its data: Connect API and Direct API.
Salesforce Data Cloud Direct APIs
Direct APIs are faster and are documented in the Data Cloud Developer Guide. If we are building use cases such that Data Cloud data needs to be surfaced in real-time (e.g., UI displaying data fetched from Data Cloud in real-time) then direct API may be a good option.
The Direct APIs need to be authenticated with in several steps as explained in the Data Cloud Developer documentation.
Salesforce Data Cloud Connect APIs
The Connect APIs are documented in the Salesforce Connect API guide.
Connect APIs have a simpler authentication process – we simply authenticate with the Data Cloud home org. However the Connect APIs are not as expected to be as fast as Direct APIs so it may not be the best choice for real-time UI based integration (e.g., display data fetched directly from Data Cloud in the UI). Connect APIs however provide the convenience, example they can be easily called from Salesforce Flow – though conceptually we should be able to call any api from flow using Apex invocable action.
The above summary is based on the information contained in the Data Cloud Postman collection. According to the documentation in this collection: “Direct API is considered the preferred approach to API connections.”
Using Data Cloud Direct API
Following are the steps that I had used to access Data Cloud data in the DMOs using Direct API.
(1) The first step is creating the private key and self signed digital certificate. We shall need them for the next steps.
(2) The next step is creating a Connected App in the Data Cloud home org, enable OAuth, and enter the callback URL (Callback URL – For Postman, this should be: https://oauth.pstmn.io/v1/callback). The scopes related to Data Cloud (e.g., cdp_query_api) are required plus a couple of other scopes, e.g., Manage user data via APIs (api) and Perform requests at any time (refresh_token, offline_access). We also upload the server.crt file created in step (1) by selecting the option “Use Digital Signatures” setting.
Please note that you verify that the digital signature file did indeed get uploaded (e.g., the options you had given for digital signature show up in connected app). The connected app takes a good amount of time (from several minutes to much more) to get activated after creation, so if you get error during subsequent steps wait for some time before trying the next steps.
Once the Connected App is created, find it in Setup – App Manager and click “Manage”. We then complete the following steps.
* Permitted Users – Change to “Admin approved users are pre-authorized”
* IP Relaxation – Change to “Relax IP Restrictions”
* Refresh Token Policy – Change to “Refresh Token is Valid until revoked.” If you do not see any options in the Refresh Token Policy section, you need to update the OAuth Scopes to include “Perform requests at any time (refresh_token, offline access)”
Save these settings.
Click “Manage Profiles” to add your Profile (and the other users whom we want to authorize) to the Connected App. We need to add our own Profile to the Connected App.
(3)We now need a JWT token as described in the following two guides:
Authenticate with Data Cloud Direct APIs
I used jwt.io for getting the token.
On the right side of the panel, put in the following inputs.
Algorithm – “RS256”
Header values will automatically get populated:
* “alg”: “RS256”,
* “typ”: “JWT”
* “iss”: “Connected App Client ID”,
* “aud”: “https://login.salesforce.com”,
* “sub”: “username”,
* “exp”: “1735743600”
Public Key – empty (delete all the placeholder content)
Private Key – Contents of the “server.key” file that we obtained in step 1.
Copy and save the JWT from the left side panel.
A few words about this process. The jwt.io will show “Invalid Signature” after you generate the token – which is “ok.” I spent hours trying to figure this out as when my next steps did not work I thought that there was something wrong in this step.
For production implementations we would like use our own program to generate the jwt token, instead of a public site like jwt.io.
The official Salesforce guide for creating JWT token explains the significance of “exp” in the json claim used to generate the jwt token. It says that exp is the date and time at which the token expires, expressed as the number of seconds from 1970-01-01T0:0:0Z measured in UTC. For example, the expiration time set to 1735743600 seconds means that our token expiry is set to January 1, 2025 at 15:00:00 UTC. In fact this token will actually expire at 15:03:00 UTC on January 1, 2025 since Salesforce allows a 3 minute buffer.
(4)In this step we use Postman to make an API call to Salesforce Data Cloud Home Org. Postman is a useful tool to very quickly check the various API calls. There are some good primers on the basics of using Postman for Salesforce like this: How to connect to Salesforce with Postman?
Now we need to use our JWT from the previous step and acquire the core token (i.e., from the CDP Home org). The below screenshot shows the options in the Postman API call for this.
(5) In this step we exchange the core (CDP Home Org) access token for Data Cloud token. We also use the instance url returned in the previous call.
In this step –
– We send back in addition to the various parameters the subject_token, which is the access token obtained for the core (CDP Home) org in the previous step.
– We get back (1) the Data Cloud token and (2) the instance URL of Data Cloud.
(6) Get data from Data Cloud DMO.
In this step we send back the Data Cloud access token obtained in the previous step and retrieve the data from a DMO in Data Cloud.
The following screenshot shows the body of the post call.
The following screenshot shows the options used in the headers in the same post call. Note the Authorization header in which we have input the Data Cloud Access token obtained in the previous call.
Voila we can retrieve data using Direct API. If we want to simplify our SQL queries we can appropriately customize our Data Cloud data model.
While these seem like a lot of steps to get data in real life all these various steps would be merely 6 simple lines in a program. How to mint Salesforce CDP access token shows how to do all this a Java program so you can extract Data Cloud data in an external system.
Extract Data Cloud Data with Connect APIs
This is a relatively simpler process than making a Direct API call. In this we use the connected app we had created previously in the Data Cloud Home Org.
From Postman we pass the values of the client_id etc as noted in the following screenshot. I have used the Postman body to pass these values to Data Cloud home org.
In the following screenshot we sent a GET request to the Connect API endpoint and got back the data from one of the Data Cloud DMOs.
If we are making a call to Connect API from another Salesforce Org then we can utilize the Named Credential method of authentication instead of the manually done OAuth dance shown in this step.
Above are some examples of making calls to Data Cloud APIs.
Please use the official Salesforce guides for your project. Ref: Data Cloud resources