Digital Analytics Insight
Product management, microservices, data management, tag management, Adobe solution integrations and data analysis will be some of the themes of this blog but I may write about other relevant topics occasionally.
Tuesday, February 1, 2022
Medium Blog: Building Enhanced Security into Our Container Platform
Friday, November 19, 2021
Medium Blog: How Ethos Powers a Cloud-Native Transformation at Adobe
I recently moved into a product management role at Adobe. This is my first post as a PM published on Adobe's Medium Tech blog about Ethos (the team I work in). Ethos is a cloud-native and cloud-agnostic platform (and principles) that streamlines the development, operation, and consumption of cloud services inside Adobe. Here's a link to it and below is a high-level architecture of Ethos.
Friday, July 16, 2021
Adobe Experience Platform Query Service Video Demo
In this post, I want to share a demo I created working for the Adobe Experience Platform Query Service product team. The target audience are primarily pre-sales and account teams but anyone can use it get a general understanding of how Query Service connects with external tools. Below is what I worked on to create this demo:
- Identified a common telecom sector customer churn use case
- Ingested online/offline data into Adobe Experience Platform using AWS S3
- Mapped the incoming data files to relevant schemas and datasets
- Used SQL to combine those datasets into one common dataset using a CTAS query
- Created a simple churn analysis dashboard using Tableau and connected it to Query Service using PostgreSQL.
- Here's a link to it. Please note that the dashboard will not populate (due to AEP access requirements) but you can look at the general structure and fields being used.
Adobe Experience Platform Query Service is a powerful serverless tool that allows brands to analyze online-to-offline customer journeys and understand omnichannel attribution. Query service also allows you to join any dataset in the AEP Data Lake and capture the query results as a new dataset for use in reporting, machine learning, or for ingestion into Real-time Customer Profile. It provides customers with a rich set of SQL-based analytic capabilities to provide rich insights on customer data collected from all channels and devices.
In addition, here are some other practical applications of Query Service aside from combining AEP datasets and connecting to external data visualization tools:
- Leverage Query Service to take a back up of AEP datasets.
- Run pathing analysis, attribution analysis and sessionize raw data using Adobe defined functions.
- Create new calculated attributes using existing data like creating a new calculation for Customer Lifetime Value (CLV).
Below is the video which is uploaded on YouTube and a direct link to it.
Sunday, June 6, 2021
Multi-Org Adobe Target and AEM Experience Fragment Integration and Governance Ideas
Adobe Experience Manager and Adobe Target integration is one of the most powerful and robust in the Adobe Experience Cloud. I wrote about some of the benefits of this integration back in 2019. In this post, I want to cover two topics within the context of these two solutions:
- Cross-IMS Org integration between AEM and Target
- General Target workspace governance and XF nomenclature ideas
AEM & Target Cross-IMS Org Integration
This is one topic I never got a clear answer to unless I tried and tested it myself. A well known fact about IMS orgs is that we cannot integrate solutions across which is absolutely true when it comes to the Adobe Experience Cloud Visitor ID among other solution but in this case, it is possible to integrate AEM with different Adobe Target instances via Adobe I/O. Having said this, I must also mention that it's not very common for us to see given most companies have a single Target instance but there are some Adobe customers that have multiple Target instances and orgs.
I've included the steps I took to set this up where I tested this using a single AEM author and two Target instances across two IMS orgs. Let's a closer look:
In AEM (6.5.x), go to Security - > Adobe IMS Configurations and either click on an existing Target integration or create a new one. In this case, I already have two integrations set up with two Target instances (2 IMS orgs).
The first few steps are outlined in the next two screenshots which is key where we first need to generate the public certificate/key in AEM.
Adobe Target Workspace and AEM XF Governance
Before we can set governance in any solution, it's important for us to understand the concepts of user access in any solution. I'm not going to explain each of the user roles or the core access functionality in Target (covered in great depth in the documentation) but I have taken a stab to visualize WHAT the users are given access to in Target.
As you can see, the "Default Workspace" in Target has access to all Target properties and any additional workspaces do not have access to Target properties by default so you will have to manually add them to the workspace. This is an important concept as in AEM 6.5, you can share Experience Fragments directly to Target workspaces.
Target Workspace Governance
- Workspaces by Brand: This works well in companies that have multiple brands and each business group operates differently.
- Workspaces by Geography/Locale: This works well in companies that have multiple country sites/implementations/teams.
- Workspaces by Environment: Workspaces are tied to each unique site environment like Dev, QA, Stage etc. This is getting more and more common as more companies start sending XFs to these environments for testing before a launching to production.
- Hybrid Workspaces: This typically works well for a large companies that are organized by various functions but eventually, it always makes sense to formalize it based on some fundamental construct.
Experience Fragment Nomenclature Ideas
I've seen examples where XFs are named as "XF1" to something which is well structured like "Site:Location:Offer".
In my experience, a good XF nomenclature has a delimited set of values that have a predefined logic to it and I'm only referring to cases where they're meant to be shared with a Target workspace. Some examples are:
- XFs based on Brand: <brand>:home:summercampaign
- XFs based on Geography: en-us:home:springcampaign
- XFs based on Environment: <dev/stage>:home:wintercampaign
- Hybrid XFs: <brand>:en-us:home:fallcampaign:20%off
These examples are probably oversimplified and generic but the main point I'm trying to make is that it's important to add some sort of structure to how XFs are named as we want to minimize any confusion and redundancy when these land up as offers in Adobe Target. The other important point is that both the developers in AEM and Target users should align on these before applying these widely to save time.
Even though Target has robust filtering capabilities for offers (and others), this is how XFs will look in Target if the two groups won't align which in this example, shows inconsistency.
So, that was it! Hope you found this topic helpful. Please feel free to share with me your ideas on workspace organization or XF nomenclature best practices that you've seen work well.
Sunday, February 21, 2021
Overview of Adobe Experience Platform Event Forwarding
Happy (belated) New Year 2021! There is no denying the fact that the introduction of the Adobe Experience Platform Web SDK has revolutionized data collection. There are a lot of advantages of this approach but the main ones are reduced page latency, fewer cookies, a consolidated tracking library and the ability to directly stream data to Adobe Experience Cloud solutions to name a few. It's no surprise that Adobe is openly propagating this approach as the recommended path to implement all client-side solutions moving forward. I coauthored a post about the high level migration approach to the Web SDK on Adobe's official Medium blog site.
Assuming that we know the basics of the Adobe Experience Platform Web SDK, XDM and setting up the AEP schemas & datasets which will all apply in this case, this article will solely focus on Event forwarding which is a recently launched service by the Adobe data collection team. I'm working with one of my clients to evaluate the feasibility of this feature and have already set this up on my sandbox which I'll cover in this article.
Overview and Use Case
Architecture
Steps to Implement
The next step is to create a new property (Rohan's Test in my case) in this section and add the extension called "Adobe Cloud Connector". Please note that there's another extension which allows you to federate data to Google Analytics as well.
What Else Would I Like to See
Below are a few additional things I'd like to see included in Launch Server Side:
- The ability to do the same thing on a mobile app client-side implementation. Adobe is working on a Mobile app equivalent of the Web SDK so hopefully we'll see it soon.
- It'll be great if the Launch Server Side team can collaborate with the Data/Bulk Ingestion API team and see if they can come up with shared learnings to standardize server side data collection within Adobe's own solutions as well.
- Being able to replicate the same data elements created in the client-side Launch property to avoid redundancy.
- Include more extensions in addition to Google Analytics to federate data server side which is collected by the Edge Server . This will hopefully allow clients to get rid of additional 3rd party scripts from the page such as Facebook and Google conversion pixels.
Saturday, December 19, 2020
Overview of Adobe Privacy Service and Data Repair API
Imagine a world where user data and information flows freely without any regulations and guidelines. Information ranging from your date of birth, health records to credit card number is accessible by anyone on the Internet. Well, given that we're still in the middle of a pandemic, let's not make things worse and get back to reality.
We know that web analytics and data management platforms (barring CDPs) are not supposed to store PII or PHI information and are designed to capture anonymous behavioral user information with the ability to store demographic data using indirectly identifiable encrypted IDs. However, it's virtually impossible to control what data gets sent into these tools and in my personal experience working with a lot of different clients and tools, I've seen PII data such as email address, phone number and even SSN still being passed accidentally. Now, I don't think there ever can be a situation where we can completely guarantee that this won't ever happen again but we can certainly put in the right guardrails in place before data is captured.
GDPR and CCPA (and other regulations) are some of the ways which put more power in the hands of individuals to know, opt-out and delete any data collected on them and understand how it's used. This article explains the impact of these two regulations on marketers but before that it is also important to understand the difference between data subjects, data processor and data controller as explained here.
In this post, I will cover how Adobe Analytics customers can obfuscate or delete data which is done using either the Adobe Privacy Service (UI) and Data Repair API respectively. Given that my article will primarily cover how these two tools can be used to execute the deletion requests, please review the official documentation on these two tools for the finer details which I won't cover. So, let's dive in!
Adobe Privacy Service
Use Case
The primary use case for leveraging the Privacy Service is to either access or delete data for users who explicitly reach out to a brand to request a copy of all their personal data or ask for their personal data to be deleted. A good example of how users can do so is shown here.
Overview of Privacy Labels
Privacy Job Request
I'll now process a delete request using the JSON upload method which allows you to upload up to 1000 IDs at a time. In my case, I only have 1 ID to delete called "def456" but you can upload up to 1000 per request but limit the requests to up to 10,000 IDs per day. Below is what my JSON looks like. Note that you need to include your IMS org, specify the namespace and add more ids in the "users" array among other attributes.
I also had my network tab open while doing this so you can see the request actually makes an API call to the Privacy Service and sends over the necessary fields to process a delete request.
Impact on Adobe Analytics Data
Data Repair API
Use Case
The primary use case for leveraging the Data Repair API is to completely delete data from Adobe Analytics variables. The typical scenario is when a customer may have inadvertently captured PII data in an Analytics variable.
Data Repair API Request
The official documentation covers all information around the prerequisites (admin console, console.io access token, global company id etc.) and caveats so I highly recommend that you read that. In this section, I'll cover how I went about sending the API requests using Postman.
Populate the access token, client id (from I/O Console) and global company ID (from Analytics) and as headers in the /serverCallEstimate GET call.
Pass the start and end dates as query string parameters in the same request. You will see that the API response generates a validation token along with specifying the total number of server calls.
The request is the actual delete POST request where we specify the variable that needs to be deleted. You can add multiple variables as part of the same request but I only sent a request for eVar50 in this example. Also, take a look at the response which provides us with a job ID and status.
So, that was the extent of what I will cover but a lot of other useful information is covered in the official Adobe documentation.
Impact on Adobe Analytics Data
What Else Would I Like to See Added
- A user interface to make these requests in addition to APIs
- Ability to add regex or conditional logic built into the UI to selectively delete data instead of deleting everything
- Make this API available for other solutions including AAM and others
- This is not a big deal but when I tried to send multiple requests (one for each variable), I ran into the following error. It will be nice if users are able to send multiple requests without waiting. Regardless, you can get around this by sending a delete requests for multiple variables as part of the same call.
Saturday, October 24, 2020
Marketo Integration with Adobe Experience Cloud Solutions
The Experience Cloud ID service consists of many solution integrations which I've written about in the past. The true value of the Experience Cloud ID service come to the fore with the various integrations that exist helping client really realize the true value of their investment in Adobe technology. A relatively newer addition to the Experience Cloud ecosystem is Marketo which was acquired by Adobe a few years ago for its rich cross-channel activation and marketing automation capabilities primarily in the B2B space but Marketo can be leveraged for B2C use cases as well.
I know that doesn't do justice to how powerful Marketo really is so here's a high-level overview of some of its capabilities especially around marketing automation:
To start off, let's start by understanding what marketing automation is. It is a technology that automates the measurement and orchestration of omnichannel marketing initiatives. It simplifies lead management & nurturing, event marketing, personalization, regular & triggered emails or SMS. I have worked extensively in the automotive vertical in the past and know how crucial it is for automotive companies or any other company to manage their leads throughout the customer journey starting from awareness to purchase. Marketo is the perfect system to do that and if you combine it with the power of the Adobe Experience Cloud, you are truly able to orchestrate and measure the customer journey from start to finish.
In this post, I will walk through the integration of Marketo with other Adobe Experience Cloud solutions focussing more on Audience Manager but the general process is the same for Adobe Analytics and Target. Please note I'm specifically referring to Marketo Engage but will call it Marketo in this post.
Use Cases
Let's start with some common use cases that can be executed with this integration:
- Cross-device media activation of leads (B2B or B2C data from Marketo activated in Audience Manager leveraging the device graph)
- Event-driven (signups, abandonments etc.) user messaging (in Marketo based on user behavioral data from Analytics)
- Cross-channel and cross-device personalization (via Target and Marketo based on onboarded data, user behavioral data from Analytics leveraging the device graph in Audience Manager)
Prerequisites
- An Adobe org admin can enable this integration by mapping the IMS org in Marketo (Admin section).
- It is recommended to setup the cookie sync as early as possible between the Adobe Experience Cloud ID Service and Marketo's munchkin.js to ensure a higher match rate. The "cookie sync" happens automatically as long as long as both these scripts are present on the page.
- The other important piece to keep in mind is that the munchkin cookie needs to match to a known lead with an email address via a hashed id on the website when users authenticate or submit a form or click on an email given that hashed emails IDs are sent to AAM.
Architecture
- One thing to note in this architecture is that the Marketo to AAM integration is currently manual but the AAM->Marketo integration is automated where AAM audiences are refreshed in real-time with a backfill done every 24 hours. Step #4 is explained in more detailed in Marketo's official documentation.
- Another thing to note is that in Marketo, there will be an option to either "Send to Experience Cloud" (hashed email ids) or "Sync from Experience Cloud Audience" (ECIDs) to send and receive segments respectively. For Target, it's currently a one-way sync from Marketo.
Other Facts
- Any email lead data from Marketo to Audience Manager must be hashed (handled automatically from Marketo if these are captured).
- There is already an integration between AEM and Marketo where Marketo can receive assets from AEM to embed in emails. More information can be found here.
- An integration between Ad Cloud, Real-Time CDP and Marketo is also possible. I have not worked with it directly but will share once I learn more.
- This integration is not PHI compliant but can be used for any other customer not mandating PHI compliance.