Saturday, August 31, 2019

Methods to Ingest Data Into Adobe Audience Manager

I've written a few articles about Adobe Audience Manager and gave an introduction to the concept of a data management platform last year. To recap, A data management platform (DMP) allows marketers to integrate multiple data sources such as 1st party (online, email, media, offline/CRM), 2nd party (partner) and 3rd party (psychographic) data into a centralized system typically leveraged for digital advertising media activation and optimization.

In this post, I'll give a general overview of the various ways to bring data into Audience Manager and cross reference a few blog posts I've written as well as Adobe documentation on some of these methods. 


Before we get started, let's understand some common identifiers that are ingested or identified by Audience Manager:



  • UUID: This is the most common identifier in AAM. The UUID or demdex is basically a 3rd party website cookie that is 38 characters in length and is generated by the Adobe Experience Cloud ID Service.
  • IDFA: Identifier for Advertisers on iPhones represent devices that run the iOS operating system. I wrote about mobile marketing and covered mobile  marketing with this in detail.
  • GAID: Google Advertising ID represent devices that run the Android operating system. 
  • CRM IDs: These are your user identifiers such as email address or user id used to identify your own customers in your backend CRM system. Given that we cannot ingest PII in AAM, the CRM IDs should ideally be hashed using SHA-256 if you plan to send it to AAM for cross device and data onboarding requirements.

This is just a small list but there are a lot more IDs that are described in greater detail here. Let's now dive into the various ways by which we can bring data into Audience Manager.



1. Onsite/Mobile App Data Collection

The most common way to bring in onsite data is to implement the Audience Manager deployment code on your website ideally using a Tag Management solution such as Launch by Adobe. There are two main ways to bring in onsite site but let's first look at the use case. 


Typical Use Cases: Capture real time user behavioral data from the website/mobile app for personalization based on what products a user has browsed or retargeting based on cart abandonment. 


Client Side DIL 

This is the original methodology to bring in web/onsite data into Audience Manager where we deploy the Data Integration Library (DIL) code on the page to collect page and behavioral attributes client side. The recommended way to deploy it is to leverage Launch by Adobe where you can install the AAM client side extension but it can be deployed using a different Tag Manager or deployed directly on the page. The latest version of the DIL library can be accessed here. DIL leverages the UUID as the unique identifier.


I won't go into details but will cover what the network call in a web browser looks like when DIL code is implemented. I've taken an example from one of my customers who have DIL deployed and are pulling custom attributes from Google Analytics. Please note that the only way to capture data from non-Adobe analytics solutions is DIL code which is what we did here as my client didn't have Adobe Analytics deployed.


In this example, I filtered for "event" where the host of clientcode.demdex.net/event is parsed and we can see the various custom dimensions captured which are also sent to Google Analytics. Given that client side DIL generates its own network request, it differentiates it from server side forwarding (covered below) .


This is what the DIL code snippet looks like after it's loaded on the page. The first block has some mandatory params such as partner code etc. encapsulated inside the DIL.create function. The second block is the integration hook which captures data for Google and a similar module is available for Adobe Analytics to send custom variables to AAM.

For mobile apps, you can install the AAM extension from the AEP SDK in Launch as explained here. 


Server Side Forwarding (SSF)

This is the recommended method to send onsite data to AAM and it piggy backs off the existing Adobe Analytics call without firing its own call like DIL. In this case, the data is forwarded to AAM server side where SSF is enabled at the report suite level in Adobe Analytics.



To troubleshoot SSF online, we can filter simply for "/b/ss" for the Analytics call and if you look at the response,  you'll see "/10/" (unique to SSF) in the Analytics beacon, various the various IDs syncs as well as the dcs_region populated. 

dcs_region is a the region ID (populated by the Visitor ID Service) that's tied to a regional server name, which you need to send as part of data collection server (DCS) call. 


If you have a client side DIL implementation of AAM, then I'll highly recommend that you migrate to SSF but make sure you either have SSF or DIL deployed to avoid being double billed. T
his Adobe article walks through the steps to migrate from client side DIL to server side forwarding. It has some videos and I highly recommend following the steps laid out here. The awesome Doug Moore recorded these so kudos to him for doing a great job here.

One other point I want to call out is that you can enable or disable SSF based on user content tied to GDPR requirements. This article explains how you can use a context variable to disable SSF by setting cm.ssf=1 when you DON'T want to forward data to Audience Manager.

For mobile apps, this article covers the various steps you can take to enable SSF. 



2. Media or Email Pixels

Media pixels is the most common way for you to bring in display banner clicks and impressions in AAM. Similarly, we can deploy email pixels to capture impression and click level data in AAM for emails sent. 

Typical Use Cases: AAM media pixels can be deployed on display banners and this data can be leveraged in AAM segments for frequency capping and retargeting to help improve return on Ad spend (ROAS). 

Email pixels populated either on impression or click can capture user IDs to sync them with AAM for cross device targeting or integration with onsite data. This can also help fill any gaps in case you don't have the capability to capture authenticated user IDs upon registration or sign in.


Media Pixels

There's already a lot of documentation on how to capture media impressions and clicks in AAM and these pixels are deployed in an Ad Server. I'm not going to cover it but the general pixel looks something like this (both HTTP and HTTPS are supported): 

https://clientname.demdex.net/event?d_event=imp&d_src=datasource_id&d_site=siteID&d_creative=creative_id&d_adgroup=adgroup_id&d_placement=placement_id&d_campaign=campaign_id where d_event is replaced with "click" for clicks instead of "imp". clientname or partner code is unique to your org and can be provided to you by your Adobe consultant.

Please note that media pixels are the only way to capture data in European Markets as Actionable Log Files can no longer being used due to Google's announcement to stop supporting log file transfers. It's likely that a similar announcement might take place for Non-EU markets.


Email Pixels 

I already wrote about email pixels so will encourage you to read that post for more information on how to send capture email data in Audience Manager. This pixel is deployed in the Email Marketing Software. The general pixel for email is exactly the same as the media pixel except for campaign specific parameters such as site, placement and campaign among others.



3. Data Onboarding

Data onboarding allows us to ingest offline data into Audience Manager. This is possible only when an ID sync occurs upon authentication where the backend hashed CRM ID  is captured online and sent to AAM. The most common data onboarding scenario deals with ingesting 1st party data but 2nd or 3rd party data can also be ingested in AAM.


Typical Use Cases: The most common use case for data onboarding is to tie offline data to online data to get a unified customer view across multiple touch points. The other use case is personalizing the web/app experience when a user signs in based on their offline attributes such as loyalty status, demographic information etc.

An example of 2nd party data use case is a ski resort getting data about customers shopping for skis or snowboards from a retail company to serve them relevant ads.
An example of 3rd party data use case is a financial company purchasing data about C level execs to advertise about their upcoming donation campaign for cancer patients.

1st Party CRM File Onboarding
 

1st party data is your own customer data. I wrote a two part series on how to ingest data in AAM which goes into the various steps to onboarded CRM data. I also wrote about how we can integrate LiveRamp with Audience Manager which you should find helpful.

2nd Party Data Share


2nd Party data is collected by "partners" who are also AAM customers and have a different business model than yours. The reason why it's possible to share data from one AAM customer to the other is the demdex 3rd party cookie which is the same across all AAM customers. The recommended way of bringing in 2nd party data is to ask your partner to share their data as a private feed that will be accessible in the Audience Marketplace in AAM.


3rd Party Marketplace

3rd Party data is collected by data providers such as Acxiom who collect data across a plethora of websites and aggregate it to sell it to enterprises. This data is available at a cost within the Audience Marketplace. This video provides you an overview of the Marketplace and how to subscribe to feeds.


4. Data Collection Server (DCS) API

DCS APIs can be leveraged when you want to send data to AAM in real-time either client side or server side along with User/Device ID syncs. The official Adobe documentation covers how we send data to DCS and I've also covered it at a high level below. 


Typical Use Cases: The most common use case for leveraging DCS APIs is to send data to AAM server side from smart devices such as Roku or Alexa for which we don't have SDKs built out yet. The final outcome can be to share user segments across multiple smart devices. 

The other use case is to send AAM segment data back to your offline CRM system for activation across multiple platforms such as a chat application or call center.


DCS API Call

Here's an example of a DCS API call (browser based):
https://ags542.demdex.net/event?c_prod=skis&d_dst=1&d_rtbd=json. In this case, I've sent a call from my browser (client side) to DCS.


  • ags542.demdex.net: Domain alias and destination domain
  • d_dst parameter which sends back qualified cookies tied to a URL destination (optional field)
  • d_rtbd parameter which I highly recommend if you want a JSON response back and based on this, we get back the cookie qualifications within the "stuff" object among other fields like the UUID, dcs_region etc.
Please note that the dcs_region is automatically returned based on the user IP in this case client side but in case of server side, where if AAM is unable to deduce it (absence of ID Service), it might make sense to hard code the dcs_region to the nearest region as it's a required field in the API call. 

Here's a full list of all supported parameters in the DCS API call which can be passed as part of the API call. I've only scratched the surface here but the official Adobe covers it in a lot more detail.


5. Experience Cloud Audiences

There are two primarily two Adobe solutions which share segments with Audience Manager:

Adobe Analytics

This is the most common way for clients to ingest segments created within Adobe Analytics when historical data or complex segments (not possible to create in AAM) are involved. I wrote about this in detail last year where I covered the specific use cases where these should be used. Here's some more information on it.

Please note that there is a limit on how many of these Analytics segments can be shared with AAM so it's better to create segments directly in AAM wherever possible.

Adobe Campaign

The commonly used integration between AAM and Adobe Campaign allows us to send AAM segments to Campaign but there is also a way to ingest audiences via batch from Campaign to AAM which are covered in step #11 in this Adobe article.

I hope this post was helpful to provide you with an overview of the various ways by which you can bring data in Audience Manager.

Sunday, July 28, 2019

Miscellaneous Facts About Adobe Analytics

At the start of this year, I wrote an article about Audience Manager on which I'm going to structure this article. The only difference is that this article will cover some random facts about Adobe Analytics which are not commonly known or are rarely discussed. One of my co-workers Adam Waddington has also shared his experience on an issue he faced with a client which is covered in the first bullet point followed up by some other facts which I've shared.

1. Segments Are Tied To SAINT Classified Dimensions


This is based on a recent issue which Adam faced while deleting a classified column as part of a new SAINT upload where segments mapped to one of the deleted columns stopped working. Below is a verbatim writeup of what Adam sent me and I hope you will find what he shared useful. Thanks Adam!



Segments built using classified variables follow a unique reference method that is highly sensitive to changes in SAINT tables and order. Unlike eVars, props, and standard OOTB dimensions, classified variables are referenced by segments using a back-end numeric ID (aka ‘Div’) prior to use in reporting. These Div#’s are set at the time AA Classifications are added, incremented by 1 from top-to-bottom.


AA Classification Order
Backend Numeric ID
Segment Reference
Classification 1
Div1
Div1
Classification 2
Div2
Div2
Classification 3
Div3
Div3

Similarly, the segment reference Div is set at the time each segment is created and - depending on the type of change being made to Classifications in AA - there can be several (potentially severe) business and reporting impacts: 
  • DELETING a classification in AA removes that variable and its associated Div data from the backend. Any segment which leveraged the classified variable linked to that Div will then become ‘Undefined’ and segments will not function as intended.
  • EDITING or renaming a classification in AA has no effect on the Div itself. However, any segment which uses that modified classification will now show the updated name provided.
  • ADDING a classification will always default to incrementing by +1 on the highest Div# in the system, even if a lower Div# was deleted. For example, deleting Classification 2 from the above table and subsequently adding a new Classification will create Div4; Div2 (Classification 2) will still not exist. In order to add Div2 (in this example), the user must delete all Classifications below 1, add Classification 2 again, and re-upload the desired SAINT tables.

2.  s.transactionID Can Be Used To Reconcile Missing Orders

Late last year I wrote an article about data variance between Web Analytics tools and mentioned that there is a way to reconcile missing eCommerce orders on the purchase page with the help of Adobe engineering.  This can be accomplished by passing s.transactionID on a page prior to a confirmation page and data can be reconciled by engineering. The trick is to pass the s.transactionID right before the purchase page on the page (client side) as well as in the backend (server side). 


This is not straightforward as it will require additional support from your developers but our engineering team can take the two transactionIDs, reconcile them in the backend then backfill any missing orders. Please note that there will be an additional cost to work with engineering as well so ONLY do this if it's absolutely necessary.

3. Sending Historical Data To Adobe Analytics Is Possible BUT..


This is accomplished by passing over historical attributes (eVars, props, events etc.) along with the timestamp variable (s.timestamp) but the main issue is out of order nature of events being sent in as it can affect the sequence of events and attribution of eVars. The only use case I've seen for this is backfilling historical or missing data and should ONLY be done when you have no other option. I've included a sample below using a GET request via the Data Insertion API method. 


Notice that the timestamp is for a previous time frame. I'd also suggest changing the timestamp setting in your report suite to optional for best results especially if you're using a global setup capturing both mobile app and desktop data.

http://(tracking server)/b/ss/(rsid)/0?AQB=1&pccr=true&g=none&ts=1559335477&products=;testvarn;1;5.95&events=event1&pageName=testPN&mid=5670954657508492467610283427418697012&AQE=1


4. Possible To Send Negative Revenue In The Products String

One other rare scenario is where your client side code accidentally captures a really large order value which messes up your overall Revenue metric. This happened recently for a client where they accidentally sent over a Million dollars for a single SKU. There is a way to fix that by sending an equivalent negative revenue amount for the same SKU for which you accidentally sent the wrong amount. I've demonstrated it using a small amount of $400 but the concept is the same.



The first screenshot shows that we "accidentally" sent a revenue of $400.

This shows what the products string will look like when we send negative revenue amount. Note that we sent a different purchase ID as the same purchase ID cannot be used to send a different order as duplicate order IDs are ignored by Adobe.

Finally, we can see that the overall revenue for SKU123ABC is now $0 as we sent a negative revenue amount. The other thing to consider is that you'll have an additional Orders event captured as we used a new purchase ID but in the larger scheme of things, you can live with that and ignore it via segmentation.

5. tntA Variable Can Be Sent To Tie Analytics With Target (A4T)

This is from a recent test I did with a client while testing A4T where Analytics is deployed server side but Target is deployed client side. The data is sent to Adobe Analytics via Data Insertion API and the challenge was that there was no concept of a Supplemental Data ID matching Analytics and Target which is used to tie Analytics with Target client side. 

In this case, the fix is to send the tntA variable (captures A/B test variant information) which can be passed as a query string parameter both on page view and click events. This ensures that client side Target matches up with Adobe Analytics deployed server side and populates the A4T reports. Below is a sample (event) snippet you can send to test this which will populate the Analytics for Target reports in Adobe Analytics. Note that we've added pe=tnt to denote that it's a click event other for page views, you'll simple pass the tnta param without pe=.

http://(tracking server)/b/ss/(rsid)/0?ts=1559335477&products=;testpev;1;5.95&events=event1&pageName=testpev&mid=5670954657508492467610283427418697012&pe=tnt&tnta=291105:0:0|1

6. Submit Expected Traffic Spikes Well In Advance

A lot of businesses have expected traffic spikes each year during seasonal events such as Thanksgiving and it's quite common for them to schedule traffic spikes for their Adobe Analytics report suites. However, I've seen a lot of cases where these spikes are scheduled quite late which leads to delayed data processing of their data. 

To avoid that, it's better to schedule your expected traffic spikes well in advance to avoid any data processing delays or issues. As an example, expected spikes above 25,000,000 server calls should be submitted one or more months in advance which gives Adobe enough time to allocate any necessary hardware. Here's a summary of how much lead time is needed by traffic volume.

So 
that's it! Hopefully you found some of these uncommon scenarios pertaining to Adobe Analytics helpful.

Saturday, June 22, 2019

Send Real-Time Adobe Analytics Events to Adobe Campaign via Triggers

Triggers are part of the Experience Cloud Activation Core Service and have been in existence for a few years now. There's a lot of documentation already out there but with this post, I'll fill in some gaps that exist in the public facing documentation. I will also provide a quick overview of Triggers from my sandbox account, cover common use cases, share a logical architecture diagram and discuss the provisioning process along with some common access issues. Let's get into it!


Overview of Triggers


Triggers allow marketers to activate Adobe Analytics events in Adobe Campaign to re-market and engage customers via emails and other channels. Adobe Campaign is the main solution that leverages Triggers to send emails or SMS based on an action performed on the site in near real-time. I will be covering the integration between Triggers and Adobe Campaign Standard (ACS) at a high level.

We can access Triggers by going to the Activation site and clicking "Manage Triggers" (provisioning covered in the next section). There are 4 types of Triggers we can create which are:

  • Abandonment (Send an email or SMS after a gap of between 10 minutes to 2 hours)
  • Actions (Send an email or SMS if a particular action is performed on the site)
  • Session Start (Send an email or SMS if an action occurred at the start of a session)
  • Session End (Send an email or SMS if an action occurred at the end of a session)

Below is the UI we see when we create a new Trigger based on an abandonment.


Once created, Triggers will start collecting data immediately based on the conditional logic. In my actual example, I've created a simple Trigger based on an action tied to a particular Page Name.




Few Considerations While Using Triggers

  • We can create up to 100 Triggers.
  • We cannot bring in any calculated metrics or segments created in Adobe Analytics.
  • There's no way to select a Visitor container like we can in the Analytics Segment UI.
  • Values from Adobe Analytics reporting don't automatically populate in the dropdown menu when we use the "Equal" operator.
  • For Products or other dimensional data, there's no way to know which particular product has been added to the cart etc. so all the data is sent as it comes in. Prior manipulation of data will have to done to specify exactly which product was sent.
  • The conditional logic to create Triggers can get really cumbersome and there's a lot of flexibility provided around boolean logic and Containers provided in the Triggers UI.


Common Use Cases For Triggers 


There are numerous use cases that can be met using Triggers and below are some examples. Please note that for these to work properly, the user would have to be signed in.

  • Abandonment (Time Bound)

    • A user abandons the cart or form and needs to be contacted within 10 minutes
    • A user signs up for an offer and you need to send a confirmation email
    • Notify of a shipment or password reset
    • A user purchases a product
  • Actions (Event Driven)
    • A user views a promotional video
    • A user completes a survey 
    • Personalize email to show last product viewed
    • Confirm when a prospect opts-in for a newsletter

  • Session Start and End-I haven't used these yet

    • A user visits a campaign landing page after clicking on an ad
    • A user did not sign up for a form and exited the site

Aside from these, we can leverage the propensity score attribute to NOT trigger an email if a user is likely to visit the website in the next 30 days.

Triggers Provisioning


Email the Digital Request Alias

Anyone can can email the Digital-Request@adobe.com alias to get Triggers enabled for their Experience Cloud Org. This team will enable Triggers in the Admin Console as well as enable Triggers in Adobe Campaign Standard (They create an additional ticket). Below is what needs to be communicated in the email:

  • Marketing Cloud Company Name 
  • IMS ORG ID
  • Analytics Login Company (could be same as marketing cloud company name)

Map Report Suites to the Experience Cloud Org

All Adobe Analytics Report Suites need to be mapped to the correct Experience Cloud Org as explained here.

Enable Triggers in the Admin Console

In the Adobe Admin console, we need to turn on the toggle to enable Triggers as shown below.

Create Data Sources

Create a data source in Customer Attributes to capture authenticated users which includes an integration code. This is needed to tie users logging in to the website with profile data in Adobe Campaign Standard. Please note that the user ID is hashed and needs to be reconciled in Adobe Campaign Standard (covered below).


Sync the Hashed User IDs with the Experience Cloud


This is a very important piece of this integration and is a required step to sync Customer or Declared IDs captured on the site with AAM or Customer Attributes and eventually with Campaign. This can be done easily in a Tag Manager like Launch.


Reconcile Triggers with the ID Service in Campaign Standard

The final step is to reconcile the an encrypted ID with a Declared ID data source in Adobe Campaign Standard. Below is an example of a Declared ID data source in ACS which is a new data source that needs to be created tied to the authenticated Customer ID by going to Administration -> Application settings -> Shared Data Sources. The link below the image covers this step in more detail.
Here's some more information on how to configure Triggers to send emails in Adobe Campaign Standard: 


Triggers are also Accessible via Adobe I/O

I recently wrote about Adobe APIs last time and covered how to create an integration in Adobe I/O but this link covers how to connect with the Triggers API.


Common Access Issues


I recently ran into some issues while trying to access Triggers and had to log tickets internally at Adobe so if you run into these issues, you'll have to ask your Campaign consultant to log these on your behalf.

The first issues I ran into was around not being able to create a trigger. The fix for this was to map the report suites to the Experience Cloud (covered above).

This is an issue which I'm currently facing for a client where I'm unable to create an authenticated Customer ID Data Source. I had to log an internal ticket to get this resolved.


Simplified Integration Architecture


Below is a simplified architecture diagram I put together to outline the integration between Analytics and Adobe Campaign Standard using Triggers. I've leveraged some internal documentation to provide more details primarily around steps 4-6. Below is a screenshot of that and a summary of the various steps.

1 - AEM Services the page
2a - ECID Service is called and relevant IDs are returned back to the page 

2b - If the visitor is logged in, an Authenticated "Declared ID" will be paired with the ECID
3 - Analytics image request fires and page/event level data gets sent to Adobe Analytics
4 - Real-Time Events sent via Analytics are detected using the Triggers State Machine
5 - Events are "enriched" with the Customer ID sent from the ID Service
6 - The enriched trigger events are sent through Pipeline Services and made available to Campaign. ACS "listens" for Trigger data that comes in as IDs starting with analyticsHitSummary
7 - Campaign queries data sources to build personalized multi-channel messages
8 - Campaign ingests/shares data from/to other data sources
9 - ACS delivers marketing messages across channels (Email, SMS) based on Triggers
10 - Customer clicks on the email and is routed back to the client website

So that was it! I hope you found this post helpful and hope that it will pique your interest in Triggers if you aren't already using it. Here's a link to another post where I've written about how to use webhooks to listen for Triggers.