Your Answers Are Only As Good As Your Questions
Rob Clark
In Douglas Adams’ hilarious Hitchiker’s Guide to the Galaxy, there is the story of Deep Thought; a computer the size of a city and awesome in its capabilities. The computer is tasked with providing the answer to the ultimate question – of life, the universe and everything:
“There is an answer, but I’m going to have to think about it,” the computer advises before going into seven and a half million years of churning over data and calculations.
One day, however, Deep Thought finally has the answer. The people urge the computer to tell them.
“I don’t think you’re going to like it,” it advises. They demand it to tell them anyways.
“You’re really not going to like it,” observed Deep Thought.
“Tell us!”
“All right,” said Deep Thought. “The Answer to the Great Question…”
“Yes..!”
“Of Life, the Universe and Everything…” said Deep Thought.
“Yes…!”
“Is…” said Deep Thought and paused.
“Yes…!”
“Is…”
“Yes…!!!…?”
“Forty-two,” said Deep Thought, with infinite majesty and calm.”
The computer explains that this answer is quite correct; it even double-checked its math. But the problem is, they didn’t quite know what it was they were asking. Until they know how to form the question correctly, they’ll never quite understand what to make of ‘42’.
Whether you’re conducting research, auditing discussion around a brand, or compiling a report on campaign performance, if you don’t have a clear question in mind you will have a hard time putting any resulting information to use. Here are three considerations to ensure your seven-and-a-half -million years of pouring over spreadsheets and data tables isn’t for naught.
Question 1: Why do you need this information?
More specifically, what’s the change you’re hoping to make or the decision to be made? From a business perspective, information’s value comes from its ability to provide clarity for decision making. If information helps mitigate risk by pointing you in the right direction, then it provides a clear financial value. If information isn’t feeding into a decision, then it is simply trivia. Interesting perhaps, but of no true relevance.
Around every decision made, there are things known and a great many things unknown. If you’re not careful, you could spend forever digging through data to find that very few pieces of information will point you in the right direction.
One of the best-in-class examples of big data in action is how Target pinpointed which customers may be expecting a child. This wasn’t a random discovery made after combing through mountains of transactions. It was all to address a very specific need: the arrival of a new child is a time where people change their shopping behavior. So how could Target get their marketing in front of new parents before anyone else?
By understanding this specific need, they framed their question: “Can our customer’s purchases indicate that they are expecting a child”? This ensured they could recognize which transactions mattered. Out of all the items in the store, a purchase of a dozen specific items within a specific timeframe was enough for Target to pinpoint not only that the customer is with-child, but the date the baby will be born (with high accuracy).
Question 2: Are you talking the right talk?
Too often we frame our questions from the perspective of the business rather than the customer. Customers are not going to discuss or describe products with the same words and phrases we use in the brand guidelines. A customer is going to approach an experience, site design or brand touchpoint in the way that works best for them and chart their own path based on their needs.
Putting the focus on yourself and not the people you’re trying to reach provides the answers you want to hear, shutting you off from the answers you need.
For example, if you’re looking at how people discuss your brand online and you are only looking for Official Brand Name™ and not the various folksonomies, nicknames, and even slurs there may be for your brand, you are missing a good part – maybe the most important part – of the conversation.
Many of the answers we’re seeking in communications are led by a desire to understand why consumers do what they do and how our efforts impacted their actions. If you’re not setting up research that takes the consumer perspective into account, you will always have an answer disconnected from the reality you’re operating in.
Question 3: Are you factoring for human failings when gathering data?
The problem with many models of behavior is that they anticipate people will act logically and answer honestly. Have you met the human race? We’re a pack of irrational liars!
People tend to share what they feel puts them in the best light or frames them as the hero of the narrative. They will try to give the ‘right’ answer as opposed to the ‘actual’ answer. For example, as much as someone may acknowledge the importance of good diet and exercise, that won’t change the fact that they spent the evening hoovering down a bag of chips and bottle of soda in front of the TV.
To get better answers, your questions need to take human failings into account and be designed to triangulate the truth. Look to actual past behaviors as opposed to claims of what will be done. Look for other signs or indicators that can be used to circumnavigate past the truths people prefer to tell about themselves and reach the reality of what they desire. Reframe the question so the truth sets the respondent as the hero.
In addition, be careful that this inherent human failing isn’t in the design of your research and that you aren’t leading the respondent to the answers you want to have. Likewise, be very careful that you aren’t reading into data that isn’t there, placing an over-emphasis on the information that fits a preferred narrative while ignoring proof that counters it.
Asking better questions requires a clarity of purpose, an honesty of the self and recognition of the failing in others. It’s a tall order, but it can be done. And doing so will serve you, your brands, your company, and your customers better.