Neal discusses the shift from point-in-time research to continuous intelligence operating systems and explores how generative AI is enabling advanced pattern recognition across various data signals. He also details CMB’s Intelligence Signals service, which combines focused digital signals with primary research data to provide actionable recommendations. The conversation also covers the critical importance of governance, data transparency, and the human-in-the-loop model.
An Evolving Insights Mindset
TMRE just took place in early October, and All Things Insights witnessed a lot of trends happening in market research, including the development of always-on intelligence services and tools. CMB also recently introduced some new tools and services in this regard. Your CEO referenced that research is turning from a point in time deliverable into a living decision capability. Can you talk about this changing mindset?
“That was certainly one of the themes at TMRE,” agrees Neal. “Most corporate insights functions are trying to go from one-off point in time projects to a continuous stream of intelligence, basically like an intelligence operating system with their own data lake, they have their own LLM on top of it, they have different agents talking to each other. Insights functions are basically managing all this in addition to doing net new primary research because you have to feed the models or you’ll get model collapse. There’s a lot of talk about that as well. In our world, every project we do, whether it’s a tracking study or not, we’re trying to help clients get more value out of that over time.”
“They might be going back and leveraging the system for other purposes,” adds Neal, “and finally breaking down the data silos. I’m actually seeing this happen. We’ve been talking about it for years, but I think it’s actually happening in a lot of corporate insights functions now. You can go basically query and get insights from different studies that might not have been directly commissioned to answer a given question that’s coming at them now, one, two, three, six, nine months after those original studies were done. That’s happening across the board with everyone I work with.”
Neal continues, “What’s happening more specifically to signals is trying to get different types of data, to find patterns across different signals. Generative AI has really opened this up for us as it is good at pattern recognition. Of course, most large clients we work with have been doing social media listening for years. They’re tracking the news. They might be looking at industry analyst reports. But now we are helping them bring in all the signals together to see what’s impacting what. Once you put them all in one place and you have a powerful tool to look at their interactions, you start to see patterns happening. Once we get all that data in one place, we can start doing some pretty cool analytics with it to see what’s actually going on.”
Thinking Beyond the Black Box
Trends at the TMRE conference also included the development of more of these insights hubs or repositories that the corporate insights professional can tap into. The community is leveraging AI, but of course humans have to be in the loop as the key part of the judgment process. Where do you see insights teams leaning, making a push for automation or being cautious to adopt some of these practices?
“There’s a wide variety or responses, and there’s tension, of course. A lot of the pressure is coming from their stakeholders, especially executive management who want everything fast and cheap, understandably,” says Neal.
“Insights professionals are very good at being thorough and certain when they give recommendations. There’s attention there. A lot of what we’re helping our clients with really falls under the realm of governance,” says Neal. “Not just compliance, of course, there’s privacy and security threats with AI. But making sure that answers that especially generative AI produces are putting a confidence interval next to them, even if it’s not a statistic. Do they have high confidence in this, medium confidence, low confidence.”
He adds, “Where do you basically say we’re not going to put the tool which could generate an answer in the hands of people who don’t really understand what they’re getting out of it? That was one of the biggest themes at TMRE. It’s exactly where insights is right now, basically making sure that people use these tools in a way that isn’t giving straight-up incorrect information or making a mountain out of a molehill, or when they don’t really have enough valid data to adequately support that.”
“That’s really where a lot of the AI persona discussion is centered,” says Neal. “What are they good for? What are they not? You have to make sure they stay true to the training data, or how far out from that training data do you let them give estimates? It’s the same thing with synthetic data. It’s very much where our profession is right now, and it’s a stressful time. There is a lot of pressure on fast and continuous research. But hang on a second: Where is this data coming from? We need transparency. It can’t be a black box. Insights professionals are very good at that, but things are moving really quickly and they’re trying to stay ahead of that.”
Always-On Intelligent Signals
Pivoting slightly, can you just tell us a little bit about CMB’s Intelligent Signals suite of tools, and how that’s really taking these external signals and leveraging them?
“It is basically taking very focused digital signals and most typically pairing them with custom primary research data for a common purpose,” says Neal. “It is very focused. It is customized. It is a service, not an off-the-shelf ResTech tool. A typical example would be, say, a brand tracking program and we have really solid every six to twelve months brand tracking, KPIs, the brand funnel, and brand perceptions of you and your competitors. That’s great. But then our marketing teams have a lot of needs in between those waves and most companies don’t have the budget to continually be getting real human feedback in the loop.”
Neal adds, “We’ll take ongoing digital signals and do a lot of work on data curation upfront and getting those semantic queries exactly right for what we need to look at—social, of course, paid news, financial signals, in some cases, technical health. We’ll look at AEO, GEO, we’ll look at search, and then bring them all together. We have an agent for each one of those. Then we have an agent that goes across those to look for patterns across the different signals to then basically figure out what’s going on.”
“Once we get all that data formatted, we can do some cool stuff to basically separate the signals from the noise,” he says. “We can start doing some predictive modeling to look at if we have a proxy for purchase intent or more often in social people talking about how they just purchased something, what is driving that, what are the signals or what are specific messages within those signals that are predictive? Then we can move forward with some very specific recommendations to our clients of how to tweak their campaign strategy or their branding, to move the needle where they needed to go.”
Modernizing the Brand Tracker
Brand tracking is kind of evolving as AI grows. What about this expectation of “always on?” Is that new or different than what research teams have been set up to deliver in the past? How can people adjust to that?
“Yes, it’s very different for most organizations,” says Neal. “The digital campaign world has always been using social and other kind of continuous feeds and is used to that. The primary insights world is not. Most people I talk with are at different points in terms of getting together a common data lake internally with all the different data and then basically getting AI functionality on top of that to make it a living pool of resources that’s constantly updated and not siloed.”
“That’s one of the big challenges, breaking down those data silos,” adds Neal. “Some of those biggest challenges aren’t always technological. It could be people and processes within large organizations. There’s been a lot of progress on that with a lot of large companies over the last six, twelve, twenty-four months. We are definitely in the middle of a sea change on that.”
You mentioned that it’s beyond just the tools, but also the people, the processes, the culture of an organization. There are themes of trust, security, and transparency. How do you assure your clients in that regard?
“We tackle it the way we’ve always tackled it. It’s just with an increasing array of data sources,” says Neal. “It all starts with knowing exactly where your data comes from, being very transparent and is it high quality? Survey primary research data is never going to be perfect. There’s always going to be some fraud, but we have spent a lot of time at the bleeding edge of the forever wars against bad data to make sure that what we get is representative of the markets. There might be a few bad actors, but not enough to impact the results.”
He adds, “In the digital world, social data can be really messy. There’s also a lot of actors out there trying to influence what the bots are saying about a topic, different companies, different brands. There’s somebody who spun up a fake deodorant brand and pretty soon got ChatGPT to start recommending it. A lot of our work isn’t the technology itself. It’s just the messy, not sexy work of making sure that the data ingest we get, the raw oxygen for whatever tool we’re doing is really good, and we’re very transparent about that as well.”
“What then do we do with it?” Neal continues. “That’s where the LLMs come on board. LLMs are amazing in what they can do, but they do have flaws. Sometimes a probabilistic model might grab and make up the wrong data point from an underlying data file. With our intelligent agents, we’ve basically put in Python code to make sure that never happens. There are certain things that should be deterministic. Once we make sure we get that right 100% of the time, then we let the LLM do what it’s really good at. Being transparent about that along the way and sharing with our clients what we’re seeing, coming up with recommendations about how to act on it and really being collaborative with them as opposed to here’s a big fancy black box, here’s your answer. When it’s those types of situations, that’s when it relies on a lot of trust. We’re still very much in a kind of wild west in terms of all the different tools and capabilities out there.”
Developing the Right Skill Sets
We talked about the power of LLMs and all of these new tools like the intelligence agents and the intelligent personas. Where do you see the skill set of the market researcher five years from now down the road?
“Not everyone has to have a degree in using AI platforms, but everyone needs to get at least literate in it,” says Neal. “With the corporate insights function, certainly getting savvy at using it effectively, but almost more importantly, it’s the governance. How do you validate what’s coming out of the AI tools? Because that seems to be falling on insights departments a lot now.”
“There’s a lot of different ways to do that,” Neal adds. “People are spending the energy on it and then figuring out some things if we put in these guardrails, it’s going to be pretty good a lot of the time and learning to let go and not keep revalidating it all the time and losing those efficiencies. In our case, getting better at testing and managing tools and increasing to the agents to go do things autonomously on your behalf or your organization’s behalf. Those are skills that I don’t think the typical insights pros had to do before and we’re very much going to need to get good at now.”
Where do you see the industry headed? There is this AI adoption curve happening right now. What do we need to be paying attention to?
“From what I’ve observed, everyone in our neck of the woods is using it now on a pretty regular basis. It’s not early adoption,” he says. “A year or two ago, it was should we use it? Should we not? A lot of legal departments, especially in certain industries, are shutting it down. A lot of that is past us and it’s being used outside the insights department. A lot of times insights people are getting stuff that’s clearly gone through an AI tool. By the time it reaches them, it’s not necessarily bad, but then they have to parse out, what did you actually mean? Because sometimes people are using the tool and don’t really understand the output.”
“I think it’s more about getting good at using it for specific tasks and also realizing what certain tools are not good at,” says Neal. “You can’t throw an LLM at everything and people are figuring that out. We’re really in the learning phase and getting better experienced with the tools that I see.”
