One of life’s great joys is the oh-so satisfying feeling at the end of a project that went well:

• Thorough discovery focused on aligning requirements to the organizational goals for a project
• Reviewed solution design with sponsor and stakeholder rep who were thrilled
• Trouble-free implementation
• A few minor fixes in testing; validation came back clean
• Users are trained and excited to use the new tools

A few weeks later you connect with the users for a “now that you’ve had a chance to get used to it” Q&A.

Nothing is really wrong, but there are very few questions, and none of them are about the features they or the sponsor were excited about at launch. You ask if they are using them, only to learn that the users’ stakeholders are asking for the same reports and metrics they always have.

I have been trying to crack this nut for years, with some success. Then I took a Change Management course that provided very valuable new perspectives. I’ll describe what didn’t work, and some more effective approaches I will use going forward.

Communicating with Strategic Intent: The limits of organic adoption

When users would report back that they were getting the same old questions despite having fancy new tools, I responded based on what I would do. I’d encourage users to:

• Deliver the requested data, but when a stakeholder has an ask that could be improved upon based on new functionality, also offer related information and analysis now available.
• Offer that if the first suggestion isn’t helpful, they might be able to suggest more pertinent information if the stakeholder wants to share what questions they are trying to answer.
• Over time, users should work to be seen as SME advisors.

I maintain that this is entirely valid advice to people reporting data. But it doesn’t get the organization to use the solution to its fullest as quickly as possible. It is slow and sporadic, relying on countless repetitive (at least from the analyst point of view) conversations, with the right questions and data to provide the opportunity, all against a backdrop of a reasonably open-minded culture.

Change Management is about how to make that kind of change adoption happen at scale. To get stakeholders asking questions that take advantage of new tools quickly, we need to have an intentional strategy to communicate the change with them.

Ignoring Preferred Sender of Key Messages research

A crucial part of a change communication strategy is the concept that people tend to receive information about a change more readily if it comes from the right sources.

There are two preferred types of source, each being better for a certain kind of message:

• Messages about the big picture, like why we need to change and how the change supports our organizations’ goals, are best delivered by senior management or executives.
• Messages about what will change in people’s day to day work, like what the change will actually require work-wise and what the benefits to the individual and team will be, are best delivered by their own manager.

This raises another weakness with our original approach: not only are we relying on the sporadic instead of the systemic, we are sending these messages through the wrong people.

Maybe we’ll get lucky and already have Change Management

We’ve established the need for a strategy to help move stakeholders through analytics change, and that the communication in that strategy is best delivered by people in specific roles. But do we need to take this on ourselves?

Reinventing the wheel is a waste of time, and trying to do someone else’s job can start a turf war, so step one is to take advantage of the Change Management investments you already have access to.

If you are in-house, maybe there are Change Practitioners somewhere in or around HR just wishing that the project teams in the company would integrate with them more closely, and you are off and running. Or you might be with an org that has never heard of Change Management and doesn’t like the sound of it, in which case you might need to be more hands on.

On the agency side, this can be a bit trickier since you are working through a point of contact and sponsors that may or may not have the appetite and relationships to secure the Change Management resources you want to integrate with.

I’m handling it by making it clear that I’m enthusiastic to integrate with a prospective client’s Change Practitioners, and that my Change Practitioner certification means I’m not just winging it in that regard.

If a client answers that they don’t have a Change Office or Practitioner, I’ll offer to supply some elements they would usually provide.

Up Next: Crafting an analytics stakeholder communication strategy without the benefit of experienced Change Practitioners

So what do we do if our or the client’s organisation doesn’t have Change Management professionals to allocate to us? Ok, now we have to figure out at least some minimal plan if we want to see people get the full value our solution can deliver.

The next post will look at exactly that. If you want me to hurry up and post it, or you have any thoughts or questions, please let me know in the comments below.

The post Upgraded analytics tools, trained analytics users: same old questions? Change Management explains why appeared first on Informed Iteration.

One of the main ways people expend effort on analytics without having an impact, or at least, having less of an impact than they could, is failing to understand the business goals of what they are tracking and reporting on. Or, losing sight of them along the way.

And this isn’t just a personal observation of mine. I asked around among some leading voices in the analytics industry about what problems most hold back analytics teams from having an impact. Not aligning what we are doing to the underlying business goals was a recurring theme.

In this post, we’ll look at why it can happen, and what we can do to better align our analytics work to business goals.

It seems obvious that we’d need to understand exactly what someone is trying to accomplish if we are going to give that person the right data and recommendations to help them succeed. Unfortunately, with all of the moving pieces that go into everything from gathering data through to presenting it, it’s very easy to lose the forest for the trees. This can happen for several reasons, but major ones are overreliance on tools, and overreliance on processes.

Don’t Let the Tool Do the Work

One major pitfall that can cause analytics work to become misaligned from business goals is an over-reliance on the metrics and reports automatically provided by analytics tools.

Loren Hadley, who has a wonderful understanding of how to help organizations succeed with data, had the following to say about why the right metrics aren’t always included:

It’s often what does my tool give me? CRM systems and analytics tools (I’m looking at you GA) tend to provide what seems meaningful and is easy to calculate without much context. Not that they are bad metrics in any way. Just that they may over promise. I don’t fault marketers for reaching for this or what an agency did 4 years ago. I’d just like to help them make sure they are focusing on what really matters.

– Loren Hadley

It’s very easy to let automatically gathered and calculated metrics, that are presented prominently in the pre-generated reports offered by analytics tools, get the lion’s share of the attention. Most businesses don’t really live or die based on the total number of people who come to their website or how many pages they look at, but tools put a lot of emphasis on users, visits, and views, so it often gets more prominence than it needs in reporting.

Engagement rate is another metric that people like to use, even though remembering what it actually represents off the top of your head can be tricky, let alone educating stakeholders on it. Some might argue that the point is to give an idea of what portion of traffic is engaged. I’d argue back that the threshold to be considered an engaged visitor is too low for a lot of use cases, and that a more tailored approach to measuring engagement would be useful.

Tools will inherently push us towards defaults instead of the specificity that will enhance our ability to really support decision making, and we need to guard against that. Processes, on the other hand, can either hurt or help us align our work to business goals.

Process: Finding Your Groove vs Getting Stuck in a Rut

We all know that we can’t afford to reinvent the wheel for every ask that comes in. We use repeatable processes for efficiency and reliability, and rightly so. The right processes serve as crucial guidelines to keep things understandable and on track. Overly restrictive processes, or an overreliance on processes, however, can lead people to just repeat what’s worked before.

If your website/app has a lot of funnels, you may get into the swing of things and have a process you follow when setting up tracking and reporting for a new funnel. That process can help you be efficient when planning, tracking, and creating reporting for each new funnel that comes your way. You need to know where the funnel is, what the steps to track are, and establish technical elements like is the funnel a single page app. And you’d certainly be wasting a lot of time to start the reports or dashboards from scratch every time.

The danger is deciding that the system will work as is for every funnel we are asked to track. If we assume that completion rate is the primary KPI for anything we do involving a funnel, our reporting for some funnels won’t be telling a very useful story. Maybe one project is to modify a funnel to introduce a “You may also want to consider” element to the checkout.

Completion rate is still very important, but trying to figure out if this change is working or not means considering at least average order value. And you’ll want information on interactions with the new element, and changes to items in the cart, so that you can segment metrics based on different behaviors.

Another funnel is to open a support ticket, and success for the business looks very different here. Ideally, a content recommender will show the user some help content that resolves their problem without having to open a ticket.

Analysis, reporting, and by extension, tracking, on this funnel are going to be different than how we handle purchase checkouts. Or at least they should be. Treating a funnel drop out as success doesn’t account for people who just get fed up and leave.

We can ask that the funnel be set up with distinct steps so we can clearly track people who drop out while being shown help content, as opposed to those who drop out after indicating that they still have an issue and want to contact support. Now we can start to tell the difference between someone dropping out because they solved their problem, and someone dropping out because our contact form has issues.

Without going further into the example rabbit hole, we see how getting too comfortable with treating similar asks as identical can lead to reporting and analysis that isn’t aligned to business goals.

Making Analytics Discovery Processes That Align to Business Goals

An overreliance on an inflexible process can lead us right past crucial information we need to make sure that the analysis we eventually provide will be valuable. But, we can also adapt our processes to make sure that we gather that information before we make any other decisions.

I’ve written about this a bit before, when discussing requirement gathering for analytics projects using the 5 Ws. The difference is that in that post I tried to look at all the information you’d want to bring together when figuring out what will go into fulfilling a request. And yes, “Why” was presented first. But with five more years of experience under my belt, I feel like the Why questions deserve a deep dive.

You Need to Know Why They Are Doing It If You Want to Help

If you’ll allow an anecdote, I lived in a bit of a rough neighborhood while in university. One day I was out jogging, and a very large friend of mine happened to spot me from a nearby balcony.

He looked concerned, and shouted “Are you running on purpose?”

It took me a second to grasp his meaning, and I answered “Yup – I’m just working out. No one is chasing me!”

My friend, understanding that I was running to improve my health, and not fleeing danger, encouraged me to keep it up. Had someone been chasing me, this friend was the type who would have raced downstairs to help.

Point being, knowing what someone is doing isn’t good enough to render the right assistance. You have to know why they are doing it.

Make “Why” Questions a Discovery Conversation Priority

To fully understand the goals of a project, especially a larger one, you can’t just ask people what they are and write down the answers. You need to ask the right questions to get people talking about their goals, and have a conversation. A conversation both about their project, and about how analytics can support it in succeeding. And conversations are arguably the most important part of your job.

An analytics leader whose LinkedIn posts are laden with insight, Tris J Burns, recently shared the following:

The most powerful analysis tool we can ever hope to master and possess is:

→ CONVERSATION (use your words)

Every analysis must begin and end with conversation.

We use conversation to understand the problem we are hoping to solve.

We use conversation to gather the initial data points, whether it be to gather context on the problem at hand, or to gather highly valuable qualitative data

And finally, we use conversation to deliver the insight, recommended action and estimated business impact of the analysis we’ve performed.

– Tris J Burns

If you don’t like having conversations about analytics with people that don’t know as much about analytics as you, you are going to have a hard time figuring out how to best help them with data.

We’ll tackle the first two use cases of conversation here, and talk more about recommendations towards the end.

Why Are You Doing It?

I start every discovery with “Why are you doing it?”

If I already have a pretty good idea of why based on what I know going in, I’ll confirm that understanding and ask stakeholders to expand if I missed any points or nuances. Note that we are talking about why they are doing the underlying work, not why they want analytics for it. If someone asks for analytics help with a funnel, for now, we ask why the funnel exists, not why they want help analyzing it.

With that broad understanding in place, I move on to defining what the desired outcomes are, in two contexts.

What does success with this feature/asset look like for the end user?

What did the user come here to do? Get information? Buy something? Return something? Apply for a job? Post an ad or message? What does a successful outcome for the individual end-user involve?

What does success with this project look like for you?

This is where we get more specific about why we are doing this project. If the point of updating the checkout funnel is to reduce the exit rate on the shipping step, this is where we get into “what is the rate currently?” and “is there a target improvement we are trying to achieve or threshold we want to reach?”

Analytics Discovery Continued: Questions About Questions

By now, we have a good high level understanding of what people are trying to achieve, and depending on our knowledge of the business and stakeholders, we could start planning a solution. But we’d not have as much detail as we could benefit from. And, we’d be missing a chance to understand and manage expectations.

Liz Oke, a stellar marketing strategist, wisely suggests the question:

“What do you want your analytics to answer?”

– Liz Oke

In some cases, particularly with stakeholders that are reasonably data-savvy, this will yield some great ideas you can include in your solution with little modification. In other situations, you’ll be able to have important conversations about what questions will be possible and useful to answer.

What Inexperienced Stakeholders Want Answered

People who don’t really know what’s possible with analytics will generally give one of two types of answer:

“I don’t know, I was hoping you could tell us what would be most helpful.”

This can be a very good situation to be in, in that you are being trusted to recommend what analytics can do for the stakeholders. As long as you have the right domain expertise around what the business does, you can lead them in the right direction. If you aren’t sure if your domain knowledge is sufficient, ask the stakeholders questions until it is.

“I have no idea, but it would be cool if we could know this thing YOU’VE NEVER DREAMED OF INCLUDING BEFORE.”

These can be frustrating, but they can also be incredibly fun. A lot of the time, it’s just a matter of interpreting the desires of someone who isn’t used to analytics jargon and doing a bit of translation. Sometimes, though, the ask will be for something you’ve never considered reporting on before.

The temptation here is to dismiss it as an impossible ask from someone that doesn’t understand the limitations of the technology. And that might be the right response, in the end. But before we throw out every crazy-sounding   idea a stakeholder has, we do them, and ourselves, a disservice, by not at least giving the idea some thought.

People who don’t know what to expect from analytics, but know the business well, don’t have the same habits and biases baked into their ideas that we do. And that can be a fantastic source of outside the box thinking on what analytics can do for the organization.

Is the idea actually impossible because it’s asking for something that is more magic than technology?  Or, does it seem impossible because it’s asking for something our tools don’t already do, or maybe more personally, something that you’ve never done before?

Depending on how valuable the information would be, and the resources available, maybe you need to think about extending the abilities of yourself or your tools, or adding a new tool to your stack.

What Semi-Experienced Stakeholders Want Answered

This is the stakeholder that has been exposed to analytics enough to understand what is possible in a technical sense, and get comfortable with some common measurements. But, they have trouble focussing on what they actually need in a business sense. Instead of focusing on their own business priorities, they put on their analytics hat and try to think of everything that might be useful.

If they are used to seeing it in the tools, they’ll ask for it. If they are used to seeing it in dashboards and reports, they’ll ask for it. If it was useful to them in a completely different context, they may very well, still, ask for it.

This is, in many ways, the “overreliance on tools and what we did in the past” problem, but this time it is coming from the stakeholders instead of our team. Which is entirely forgivable, as it’s not their job to avoid these issues, it’s ours.

Let’s look at some techniques to help us eliminate, or at least deprioritize, the questions that are less relevant.

Prioritizing Requirements In Analytics Discovery

Veteran voice of the analytics community, Jim Sterne, has a great method to help discern idle curiosity and old habits from the burning questions that will have an impact:

Dive into their goals and find out what they will change based on the results being above or below expectations.

– Jim Sterne

If someone has a laundry list of things they want to know, you can get a feel for how important each item is by asking stakeholders what they will do with that information. If someone insists that engagement rate (or anything else) be reported prominently, ask them what they will do if that metric were to increase or decrease by 10% – Nothing?

Ok, what about 20% – Still nothing? What about 30%, 40%, 50%?

If something can change drastically and it wouldn’t elicit a response from the stakeholders, it stands to reason that reporting on it won’t have an impact on the business. No action is being taken based on the information.

You can use this to help the stakeholders prioritize their questions in terms of value to them. Maybe you’ve only got enough resources to get to the most important ones. Or maybe you have plenty of resources, and they’ve asked for really basic stuff that’s built into the tools and won’t even use that many resources.

Either way, you need to help them focus on what’s important, both right now through prioritization, and down the road by making them analysis and dashboards that focus on those priorities.

Moving Forward

It may seem like we are skipping a lot in the middle, but solution design and implementation depend too much on the specifics of the project to get into here. And, to be fair, we have laid the foundation we need to be successful in doing those things in a way that will lead to having an impact on business decisions.

As long as you design a solution that delivers on what stakeholders want to know, as we established in the questions above, you can follow that map. Especially with a larger project, though, you can get lost in technical, documentation, or taxonomy details along the way, so it’s good to revisit the summaries you made of your discovery notes periodically.

I make it a point to do this when I move to the visualization, and again when I move to the analysis, phases of a project. Especially if it was a longer timeline, I want to make sure I remember what the point of all the work to get here was. It would certainly be unfortunate to have done everything right up until that point, then drop the ball when actually sharing the output.

Wrapping It Up

Before we can call it a day, we need to finish strongly. We need to make sure that what we get back into the hands of our users is actually going to help them achieve the goals we asked about on day 1.

Jason Thompson is not just a renowned analytics leader, but someone who often reminds us that we are human beings first, and analytics professionals second. He also had a fantastic response to my LinkedIn post asking about what prevents analytics teams from making an impact:

The biggest issue I observe in analytics teams that are struggling to make an impact comes down to struggling to move beyond data collection and reporting.

To make a true impact, analytics teams must be about more than capturing data, sharing numbers, and observing trends. To make a true impact, analytics teams must not only do meaningful analysis but must make informed business recommendations based on that analysis.

Here is some data we collected

Here is a chart attached to some data we collected

The chart line for the data we collected goes up and down

When we increase paid search budgets, the line goes up

Our analysis shows a strong correlation at a 28-day lag which highlights a monthly cycle in user buying behavior. This means for some high-value purchases, customers may take up to a month from their first visit to finalize their decision and place an order. Recognizing that customers may take several weeks to decide on a purchase we should test nurturing campaigns that present return website visitors, specifically entering into our web store, valuable information and potential incentives through a 28-day decision window…

Like most things, this is a people issue.

– Jason Thomson, 33 Sticks

The point isn’t that analysis needs to be complicated, the point is that it needs to lead to recommendations that will help the business achieve its goals. Especially in predominantly self-serve analytics environments, it’s easy to overlook that we need to make the output focused on the business questions at hand.

Go back to those discovery summaries and remember why you are collecting this data in the first place. Even in self-serve environments, we need to provide guidance and help people find the deeper insights they won’t get from looking at an automated report or dashboard.

Don’t just be a conduit for data. Where can you use what you know about the reasons for individual projects, and the business overall, to turn the data into actionable recommendations?

How do you keep your analytics work aligned to stakeholder goals and your recommendations relevant? Leave a comment and let me know!

The post Making Analytics Useful: Align to Business Goals appeared first on Informed Iteration.

Mix in a couple Radical Analytics principals from Stéphane Hamel, and you end up with a pretty good way of looking at how to best help our users with their requests.

Why – Understand It Like It Was Your Own

As the requirement recipient, you are the bridge between the person that understands the problem, and the org’s analytics resources. That means you need to understand the problem, and what your org can do with analytics. The good news is that you’ll build an ever greater understanding of your team, resources, and tools by virtue of working with them.

The even better news is that the user request side of things will be a never ending series of often totally unforeseen and fascinating topics that will keep your work interesting. It’s crucial to your performance that you understand what users are after – specifically what people want to do and why. Unforeseen questions can often be handled by someone that gets the “why” of a request without having to make that round trip to the original user.

Asking why is a very powerful  business tool, even formalized in the 5 Why method, which you might want to consider using. Getting people to talk about why they are doing something gets them away from talking about what they want you to do, and focuses the conversation on their activities and goals. That leaves room for you and your team to add value by considering solutions the user wouldn’t know about. More on this in the next section.

What – Never Ask, Alway Propose

People might try to tell you what analytics solutions they think they need, or even how to do what they think they need, instead of telling you what they are trying to accomplish. Remember that you and the analytics team can figure out how to track, report, and analyze just about anything, but you can only do a good job of that if you know what the analytics user is trying to do, and why.

Depending on the ask, goal, and need, the solution might be obvious to both of you. If it’s not, present some options you can discuss. The point is to get people talking about solutions without inviting them, let alone forcing them, to think of a solution without your help. You are the one that knows about using analytics to help people succeed at marketing – don’t rely on your users to take the lead.

This principle comes from the Never Ask, Always Propose, part of the Radical Analytics series by Stéphane Hamel. The series is a fantastic look at things that are wrong with our industry, and what we can do about it. I enjoy doing analytics more now that I’m applying it.

Note that you don’t have to come up with great options on the spot. You can let someone know you’ll get back to them and give yourself a chance to do some research or talk to your team. If you are just spitballing ideas on the spot, make sure you tell people that, and avoid promising something you aren’t sure you can deliver.

Who – The Good, The Bad, and The Ugly

We aren’t going to categorize user requests with those three terms, and certainly not the users making them! They are just to break down reasons it’s important to consider and pass along who is asking for the analytics work.

The Good

Understanding who is asking is often the first step to understanding why they want to do something, and a crucial part of scoping a question. The quality of communication you have with a person is greatly influenced by how well you know them. Sometimes, someone on your team will have a relationship with the asker, or at least experience working with them, and can leverage to find a better solution. Of course, in a larger org, you’ll need to keep track of this just so you remember who to tell when something is done.

The Bad

Who do you reach out to if there’s something to clear up, or a deadline that will be missed? Perhaps more importantly, who does whoever is working on that ticket reach out to when they get your auto-repsonder about recovering from surgery for the next couple weeks?

The Ugly

Different people in your org have different levels of ability and inclination to make you and the rest of the analytics team happy or miserable. As much as it can hurt people like us, who value transparency and accuracy, it is in your interest to take this into account when prioritizing who’s stuff gets done first, and how to frame your response. Faking report results to humour a CxO is unethical. Doing something you don’t think is the best technical solution, use of resources, methodology, etc, once your politely but clearly disclosed objections are noted, overruled, or ignored, is part of working with others.

Before you run off to get the “c-suite buy-in” often presented as a silver bullet to every analytics problem, Stéphane’s Radical Analytics lets some of the air out of that myth, and presents a great, sustainable strategy for analytics pros to grow influence in an organization.

Where – Online Is a Big Place

Make sure you clearly understand the scope and targeting that are called for. Very often, people (including me) with decades of experience working in places that have dozens of websites, a bunch of social media accounts, several agency partners, and huge numbers of campaigns, will refer to whichever piece of the puzzle they happen to be thinking of as “the site/account/agency/campaign.”

Are the new content and forms we need to track going up on all the properties? Or just on the English part of the US site for now? Or it’s actually going on a new subdomain?

It’s incredibly easy for someone working hard on something to leave out a detail that is obvious to them, but unknown outside the people really tucked in to that project. Help avoid misunderstandings by double checking on context.

A standard request form or ticket format can help make sure this is clear in every case. Don’t use it as an excuse to avoid a conversation though: it’s there to make sure all the important stuff is covered, not to allow the important stuff to be glossed over and assumed.

When – Preferably Before the Campaign Ends

People are usually pretty good about letting you know when they need something. Unfortunately, depending on resources and culture, not every ask will get done, and it can take a lot of diplomacy to help users understand when their needs can’t be the priority. Try not to leave people empty handed: if new interactive dashboards aren’t in the cards for this quarter, can you at least get the data into a custom report to tide them over?

A lot less thought and discussion goes into how long something will be needed, which is almost as important. Tags that only needs to be around for a short campaign or promotional event don’t require the same degree of thinking about maintenance consequences. It might get added in a faster, simpler way than if the team has to worry about how the tags will have to behave as the site and analytics solution evolve.

Something needed now that doesn’t need to last a long time is usually ok, and something that needs to last a long time but isn’t needed right now is usually ok. Trying to get something that requires long term planning done in a hurry is where you run into problems.

Consider a phased approach if the priority and timelines justify it, particularly if your org or team use Agile. Get a temporary and easily reversible solution in place quickly, then make a new request out of making a long term solution.

Sometimes, it’s a lot easier to wait and do something as part of a larger update or new build. Sometimes, when you go back and ask the stakeholder how things are working out and if there’s anything they want tweaked while you set up the long term solution, you find out that the campaign was a bust and they don’t need the tracking anymore.

Either way, the user got what they needed while your analytics team judiciously conserved resources, and kept the tag container from turning into martech’s answer to the Mutter Museum of Medical Oddities.

How – That’s 6 Questions and You Don’t Start with W

You’re right, we’re done! How is a different thing entirely, in that it’s not up to business to figure out how you are going to implement their requirements for you. That doesn’t mean you should be dismissive if they mention something technical: the last thing you want is a culture where people don’t tell you stuff because they are worried you’ll be frustrated with their lack of understanding.

With less analytics knowledge, they are even less able to determine what is obvious to everyone and what has to be mentioned. And it’s not like we’re 100% accurate when we are deciding what to assume and what to ask. You’ll usually fair better in an environment where people freely mention their concerns and don’t get a bunch of eye rolling in response.

Be reassuring about stuff that is obvious, and be thankful when people give you info that turns on a light bulb. Outright tell them how that info is useful and changes things, without getting technical. Over time, you’ll build a shared concept of what is par for the course and doesn’t need discussion, vs what might be out of left field and require some extra notes.

Let me know in the comments if you have any questions about how this could work in your organization, or if you have some tips to share about how you better understand your users’ requirements.

The post The Five Ws of Analytics Business Requirements appeared first on Informed Iteration.