Saturday, August 31, 2019

Methods to Ingest Data Into Adobe Audience Manager

I've written a few articles about Adobe Audience Manager and gave an introduction to the concept of a data management platform last year. To recap, A data management platform (DMP) allows marketers to integrate multiple data sources such as 1st party (online, email, media, offline/CRM), 2nd party (partner) and 3rd party (psychographic) data into a centralized system typically leveraged for digital advertising media activation and optimization.

In this post, I'll give a general overview of the various ways to bring data into Audience Manager and cross reference a few blog posts I've written as well as Adobe documentation on some of these methods. 

Before we get started, let's understand some common identifiers that are ingested or identified by Audience Manager:

  • UUID: This is the most common identifier in AAM. The UUID or demdex is basically a 3rd party website cookie that is 38 characters in length and is generated by the Adobe Experience Cloud ID Service.
  • IDFA: Identifier for Advertisers on iPhones represent devices that run the iOS operating system. I wrote about mobile marketing and covered mobile  marketing with this in detail.
  • GAID: Google Advertising ID represent devices that run the Android operating system. 
  • CRM IDs: These are your user identifiers such as email address or user id used to identify your own customers in your backend CRM system. Given that we cannot ingest PII in AAM, the CRM IDs should ideally be hashed using SHA-256 if you plan to send it to AAM for cross device and data onboarding requirements.

This is just a small list but there are a lot more IDs that are described in greater detail here. Let's now dive into the various ways by which we can bring data into Audience Manager.

1. Onsite/Mobile App Data Collection

The most common way to bring in onsite data is to implement the Audience Manager deployment code on your website ideally using a Tag Management solution such as Launch by Adobe. There are two main ways to bring in onsite site but let's first look at the use case. 

Typical Use Cases: Capture real time user behavioral data from the website/mobile app for personalization based on what products a user has browsed or retargeting based on cart abandonment

Client Side DIL 

This is the original methodology to bring in web/onsite data into Audience Manager where we deploy the Data Integration Library (DIL) code on the page to collect page and behavioral attributes client side. The recommended way to deploy it is to leverage Launch by Adobe where you can install the AAM client side extension but it can be deployed using a different Tag Manager or deployed directly on the page. The latest version of the DIL library can be accessed here. DIL leverages the UUID as the unique identifier.

I won't go into details but will cover what the network call in a web browser looks like when DIL code is implemented. I've taken an example from one of my customers who have DIL deployed and are pulling custom attributes from Google Analytics. Please note that the only way to capture data from non-Adobe analytics solutions is DIL code which is what we did here as my client didn't have Adobe Analytics deployed.

In this example, I filtered for "event" where the host of is parsed and we can see the various custom dimensions captured which are also sent to Google Analytics. Given that client side DIL generates its own network request, it differentiates it from server side forwarding (covered below) .

This is what the DIL code snippet looks like after it's loaded on the page. The first block has some mandatory params such as partner code etc. encapsulated inside the DIL.create function. The second block is the integration hook which captures data for Google and a similar module is available for Adobe Analytics to send custom variables to AAM.

For mobile apps, you can install the AAM extension from the AEP SDK in Launch as explained here

Server Side Forwarding (SSF)

This is the recommended method to send onsite data to AAM and it piggy backs off the existing Adobe Analytics call without firing its own call like DIL. In this case, the data is forwarded to AAM server side where SSF is enabled at the report suite level in Adobe Analytics.

To troubleshoot SSF online, we can filter simply for "/b/ss" for the Analytics call and if you look at the response,  you'll see "/10/" (unique to SSF) in the Analytics beacon, various the various IDs syncs as well as the dcs_region populated. 

dcs_region is a the region ID (populated by the Visitor ID Service) that's tied to a regional server name, which you need to send as part of data collection server (DCS) call. 

If you have a client side DIL implementation of AAM, then I'll highly recommend that you migrate to SSF but make sure you either have SSF or DIL deployed to avoid being double billed. T
his Adobe article walks through the steps to migrate from client side DIL to server side forwarding. It has some videos and I highly recommend following the steps laid out here. The awesome Doug Moore recorded these so kudos to him for doing a great job here.

One other point I want to call out is that you can enable or disable SSF based on user content tied to GDPR requirements. This article explains how you can use a context variable to disable SSF by setting cm.ssf=1 when you DON'T want to forward data to Audience Manager.

For mobile apps, this article covers the various steps you can take to enable SSF. 

2. Media or Email Pixels

Media pixels is the most common way for you to bring in display banner clicks and impressions in AAM. Similarly, we can deploy email pixels to capture impression and click level data in AAM for emails sent. 

Typical Use Cases: AAM media pixels can be deployed on display banners and this data can be leveraged in AAM segments for frequency capping and retargeting to help improve return on Ad spend (ROAS). 

Email pixels populated either on impression or click can capture user IDs to sync them with AAM for cross device targeting or integration with onsite data. This can also help fill any gaps in case you don't have the capability to capture authenticated user IDs upon registration or sign in.

Media Pixels

There's already a lot of documentation on how to capture media impressions and clicks in AAM and these pixels are deployed in an Ad Server. I'm not going to cover it but the general pixel looks something like this (both HTTP and HTTPS are supported): where d_event is replaced with "click" for clicks instead of "imp". clientname or partner code is unique to your org and can be provided to you by your Adobe consultant.

Please note that media pixels are the only way to capture data in European Markets as Actionable Log Files can no longer being used due to Google's announcement to stop supporting log file transfers. It's likely that a similar announcement might take place for Non-EU markets.

Email Pixels 

I already wrote about email pixels so will encourage you to read that post for more information on how to send capture email data in Audience Manager. This pixel is deployed in the Email Marketing Software. The general pixel for email is exactly the same as the media pixel except for campaign specific parameters such as site, placement and campaign among others.

3. Data Onboarding

Data onboarding allows us to ingest offline data into Audience Manager. This is possible only when an ID sync occurs upon authentication where the backend hashed CRM ID  is captured online and sent to AAM. The most common data onboarding scenario deals with ingesting 1st party data but 2nd or 3rd party data can also be ingested in AAM.

Typical Use Cases: The most common use case for data onboarding is to tie offline data to online data to get a unified customer view across multiple touch points. The other use case is personalizing the web/app experience when a user signs in based on their offline attributes such as loyalty status, demographic information etc.

An example of 2nd party data use case is a ski resort getting data about customers shopping for skis or snowboards from a retail company to serve them relevant ads.
An example of 3rd party data use case is a financial company purchasing data about C level execs to advertise about their upcoming donation campaign for cancer patients.

1st Party CRM File Onboarding

1st party data is your own customer data. I wrote a two part series on how to ingest data in AAM which goes into the various steps to onboarded CRM data. I also wrote about how we can integrate LiveRamp with Audience Manager which you should find helpful.

2nd Party Data Share

2nd Party data is collected by "partners" who are also AAM customers and have a different business model than yours. The reason why it's possible to share data from one AAM customer to the other is the demdex 3rd party cookie which is the same across all AAM customers. The recommended way of bringing in 2nd party data is to ask your partner to share their data as a private feed that will be accessible in the Audience Marketplace in AAM.

3rd Party Marketplace

3rd Party data is collected by data providers such as Acxiom who collect data across a plethora of websites and aggregate it to sell it to enterprises. This data is available at a cost within the Audience Marketplace. This video provides you an overview of the Marketplace and how to subscribe to feeds.

4. Data Collection Server (DCS) API

DCS APIs can be leveraged when you want to send data to AAM in real-time either client side or server side along with User/Device ID syncs. The official Adobe documentation covers how we send data to DCS and I've also covered it at a high level below

Typical Use Cases: The most common use case for leveraging DCS APIs is to send data to AAM server side from smart devices such as Roku or Alexa for which we don't have SDKs built out yet. The final outcome can be to share user segments across multiple smart devices. 

The other use case is to send AAM segment data back to your offline CRM system for activation across multiple platforms such as a chat application or call center.


Here's an example of a DCS API call (browser based): In this case, I've sent a call from my browser (client side) to DCS.

  • Domain alias and destination domain
  • d_dst parameter which sends back qualified cookies tied to a URL destination (optional field)
  • d_rtbd parameter which I highly recommend if you want a JSON response back and based on this, we get back the cookie qualifications within the "stuff" object among other fields like the UUID, dcs_region etc.
Please note that the dcs_region is automatically returned based on the user IP in this case client side but in case of server side, where if AAM is unable to deduce it (absence of ID Service), it might make sense to hard code the dcs_region to the nearest region as it's a required field in the API call. 

Here's a full list of all supported parameters in the DCS API call which can be passed as part of the API call. I've only scratched the surface here but the official Adobe covers it in a lot more detail.

5. Experience Cloud Audiences

There are two primarily two Adobe solutions which share segments with Audience Manager:

Adobe Analytics

This is the most common way for clients to ingest segments created within Adobe Analytics when historical data or complex segments (not possible to create in AAM) are involved. I wrote about this in detail last year where I covered the specific use cases where these should be used. Here's some more information on it.

Please note that there is a limit on how many of these Analytics segments can be shared with AAM so it's better to create segments directly in AAM wherever possible.

Adobe Campaign

The commonly used integration between AAM and Adobe Campaign allows us to send AAM segments to Campaign but there is also a way to ingest audiences via batch from Campaign to AAM which are covered in step #11 in this Adobe article.

I hope this post was helpful to provide you with an overview of the various ways by which you can bring data in Audience Manager.


Q3 technologies said...

HI Rohan
Very nice post . I have a quick question on PROS and CON of SSF and DLL approach .

I see the major problem as users enterprise are now moving away with the full stack setup as the bussiness value ( ROI ) is not thatr much .

moving away from SSF to DILL can you have share some content around it


Rohan Kapoor said...

Hi Santosh,
The cost breakup between SSF and SIL is the same as they both incur server calls similarly but please make sure you don't have both DIL and SSF deployed on the page to avoid being charged twice.
As far as advantages are concerned, I like SSF simply because it saves you an extra server call which DIL captures that adds to the hits going out of the page so I'll suggest SSF if you have Analytics.
If you don't have Analytics, then I'd go with DIL as you can send data for non-Analytics solutions with SSF.
Hope that helps,

Anonymous said...

Hi Rohan,

Is there a way we can stop media pixels to fire from the backend? As one of the team had done this before (right now difficult to identify which it was), still Pixels are firing and showing in AAM billing report.

Please advise if there is any alternative to stop this.


Rohan Kapoor said...

Hello, Unfortunately there's no way to disable pixels from our end as it's hard to pinpoint where exactly is the pixel coming from proactively. The only way to remove it is from the DSP.

Anonymous said...

Hi Rohan,

Thanks for the article, I am new to Adobe products and am looking for some approaches.

Is there a way to update the Audience segment using a custom in-house algorithm and then integrate it back to Audience Manager for Re-targeting.

Anonymous said...

Hi Rohan,

Thanks for the article, I am new to Adobe products and am looking for some approaches.

Is there a way to update the Audience segment using a custom in-house algorithm and then integrate it back to Audience Manager for Re-targeting.

Rohan Kapoor said...

Hi Rohit,

Thanks for your comment. As far as I know, AAM has its own algorithms and models (lookalike and predictive) which can be used with segments within AAM. However, you can still export AAM data or run your models externally. One other option would be Adobe Experience Platform which allows you to run your own models within Data Science Workspace. Hope that helps.

Anonymous said...

Hi Rohan,

Thanks for taking the time and providing the approach.
It was really helpful

Durga Nuvvula said...

Hi Rohan,

Thank you for the article, its very informative.

If the customer do not have the Adobe Audience Manager, is there a way to import the google analytics audience into Target or Experience Cloud?

Best Regards,

Rohan Kapoor said...

Hi Durga,

There's no easy way to do so but the only viable option is to capture the same attributes in a data layer and pass it over to Target via page parameters.