Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Always Be Testing

Always Be Testing artwork

#135 83% of AI Citations Aren't Yours — Here's What To Do | Katerine Tamargo & Tye DeGrange

Sep 22, 202622 minAudio

Katerine Tamargo, Head of AI at CJ (Commission Junction), joins Tye DeGrange to unpack one of the most surprising findings in generative engine optimization: more than 83% of AI citations point to third-party sources, not brand-owned content. They dig into the KPIs that actually define AI visibility - mention rate, sentiment, position, and citations - and why a rigorous test-and-learn methodology beats chasing tactics. Katerine also busts a few of the biggest myths circulating about GEO and agentic AI, and explains how publisher compensation models are evolving as AI reshapes the path to purchase. What you'll learn: • Why third-party content - not brand-owned pages - wins the majority of AI citations • The KPIs that define AI visibility: mentions, sentiment, position, and citations • How to build a real test-and-learn framework for GEO instead of guessing • Why the old "owned media" playbook doesn't translate to AI search • How publisher compensation is shifting as AI answers replace clicks • Common myths about GEO and agentic AI, debunked

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

Hello, hello, hello, welcome to another episode of the Always Be Testing Podcast. I'm your host, Ty DeGrange, and I'm here in beautiful Santa Barbara at CJU. We're talking to Catherine Tomargo right now, the AI Zar. How you doing? That's a new one. I love it. Doing great. Can't complain really. Just got out of a fun agency CJ conversation, diving deep into all the things and CJU is near and dear to my heart because I was a CJU employee and it's great to be back in Santa Barbara.

We've got gauchos on our on our team. I'm a former gaucho, so it's good to be back. And thanks for joining us. Yeah, thanks for having me. Yeah. know a little bit about what's happening all this AI stuff so hopefully you can help us and you also know a lot about CJ because you've kind of been with CJ a while. How did you Yeah. how did you how'd you do that? How did I do that? It's a great question. I mean for many I'd say about

a decade now, we've been talking about how affiliate is the channel of channels, how it we can pretty much do almost everything on a performance basis. And I think I just got really lucky early on with clients that were, you know ~ so I was able to to test into a lot of new emerging models and that eventually just sort of, you know, set me up for for being at the forefront of what's happening now within

Yeah, it's come up so much on the pod, it's obviously the namesake, but you you lived it and made sure that it was happening for your clients. It's something that Tom and other colleagues and I have talked about, which is beyond important. Coming up through CJ, was there you know was there something that sort of stood out for you or or taught you with those experiments that kind of like how did you get to be like Obviously it's become massively important, but I'm excited to hear how you went from CJ Client Services to leading

the AI charge procedure, which is really really amazing. Yeah, so I mean again it it comes back to this test and learn framework. That's something I really encouraged a lot of my clients to ensure they were always doing and with testing and learning they're you know, you gotta look at the data. ~ so just having that Data focus or data focus just persists throughout everything that that I do. Earlier, I would say like late twenties four,

early twenty. that I helped develop from CJ around understanding where referral traffic was coming. ~ where for both, you know, publishers and where advertisers were referring traffic, et cetera, et cetera, ~ was able to pick up on the fact that early days, maybe not so much right now, but early days, ~ some of these emerging platforms were actually driving lots of traffic to publisher sites.

And so quickly understood that as both an opportunity ~ to improve that for publishers, right? So if if you're not yet receiving kind of traffic, how can you do that? And because there wasn't a ton out there, it's really required just for the digging in and we are. Yeah, absolutely. And when clients are coming to you and coming to CJ to say, I need help figuring out AI visibility, what are some of the things that they're getting and how what does that

look like? Sure. So we back to data, right? ~ we've spent spent a lot of last year and beginning of this year just trying to lay the foundation, right? Or or understand and unpack what it is that we the tech stack should be ~ since the existing tech stack doesn't quite tackle

a lot of the pieces and so ~ with that research we've developed this sort of we'll call it our own tech stack or or suite of tools ~ that allow brands to understand that starting with visibility right so like what visibility far as we're concerned and I think most of the industry is whether or not a brand appears across these environments and and how they appear. So there are tools in place that allow brands to track that. That's sort of where we start ~ is establishing those baselines

or conducting an audit in order to establish those baselines. And then from there looking at the sources that are driving that visibility on behalf of that brand to that then you know we can then devise a plan. But it's all grounded in data. I love that. And what you shared earlier today was really cool and like citation, sentiment, mention, like you talked a little bit about like what that maybe looks like without giving up the whole story. It's a really I loved what some of that visibility you shared.

Yeah, sure. So with this this new touch point we'll call it, ~ because it hasn't yet received the full ~ channel crown just yet or designation. but was just information Okay. here on the pod. Not quite. I just think the industry hasn't fully aligned. Like you, GEO, are a channel now. but the It is new and it is different. ~ it is similar to

Other things that we've done in the past, but it is nonetheless different. And where it sits is really like in this upper funnel space. Yeah. And as such the the KPIs are different. Right. So we talked about visibility. Yeah. That's one of the main KPIs. How is your brand appearing? That's what that is. On a per kind Problem. of basis. Yeah. A prompt, a platform, etcetera. ~ so that's a visibility score, which is also known as pension rate. Those two are somewhat interchangeable. ~

And then there is ~ sentiment, right? So like you're appearing, but is it positive or negative? What what's the deal? ~ and then position, right? So you're being listed, that's fantastic. Are you first or are you last? and those are the sort of four KPIs that that we look at when we think about just Yeah, that's amazing. A lot of talk happens around path to purchase, attribution, MTA, all that stuff. There's like kind of the brand led conversations,

AI led, social. You know, where where does C A CJ see it when with AI growing? What's like your is there pieces that you think are kind of more overhyped? And what where do you kind of stand on that those paths to purchase? Yeah, I mean look data exists that shows us that AI referred traffic is converting at higher rates than non-AI referred traffic. So that's not that's not made up or overhyped,

that's that's the truth. The the rate at which it's higher does vary depending on the time frame or platform, etc. But that's true. Tack that call them answers, right? We used to rely on search engines to conduct our own research and Mm. ~ that's another thing that that's that's not overhyped,

that's that's just the truth, right? Even if you're not an AI adopter, you use Google and Google has AI overviews. So ~ it's pretty much ~ reached everyone. So some of the things that might be overhyped or around some of the guess measurement that can be delivered at this point in time. But as far as whether or not users are using it, if that's not overhyped. Absolutely. as far as whether or not brands should be testing,

I I don't think that that's overhyped because that's happening now and it's better to be in a position of being proactive versus reactive. And there's a lot to learn right now. ~ a lot of the conversations I have with with brands are around though can't quite forecast just yet, even though we can't quite project the returns, whether it be, you know, anything further down the funnel. Sales or sales revenue. A learning is a win.

and that's something that you would want to know. Like how how does being visible translate to those downstream ~ APIs? So I'm not I think that the framing of some things are overhyped, but like the fact that you should be doing it and that it is important is not. Yeah, and that's a perfect segue into, you know, namesake of the pod, the testing importance. We've clearly established that like, hey, you kinda have to be doing these types of tests. What's a way you like to see it structured in a GEO,

answer engine environment? How would you like to see a test structure appropriately and set up for success? So ~ it all goes back to the data. Yep. ~ we the data's not absolutely perfect and what I mean by that is in an ideal world we're getting exact query or or prompt information from the PLMs themselves alongside the exact responses and the sources cited. In the absence of that we have synthetic data that we can look at,

we have CDN call signals that we can incorporate. a number of things that we can we can look at to then help help inform what that test should look like. Can you say some credit data, can you share what that means exactly? Sure, so ~ in the absence of receiving the actual prompt information from platforms, there have been a number of emerging tools that have emerged over the past couple of months, eighteen months or so, that run synthetic prompts across those top LLMs,

capture those responses, and that's what they then use in order to put together scores tell you which sources are cited and how frequently across which topics, etc etc. Amazing, amazing. Are there publisher types that are starting to really nail it from a GEO citation perspective? What are are there are there types of reviews or types of formats that are really winning in this Answer Engine environment? Sure, I mean honestly it's the ones that type closest to the types

of prompts that consumers ask that are very structured in a very clear and succinct way. ~ things like the best X of you know for whatever gifts to buy mom, dad, brother, son, etc etc. ~ these are the kinds of this is the kind of language that's easily digested.

Yeah. As far as types of publishers, it really comes down industry Large percentage of affiliate publishers were being cited or were being cited. Each individual publisher has taken a different approach in terms of how they want to perceive those citations. Some really appreciate that and I

Optimizing their schema and their structures and the type of content that they focus on. Yeah. And others are So it's not so much about the type of publisher, although I'll tell you can't you know, publishers that write things on the internet, which is everyone, ~ are the types that could be cited. It's those that are are really leaning into it. I love it. I love it. And we talk a lot about zero click. We talk a lot about last click being flawed. We talk a lot about the need for more visibility and exposure.

And we're working on that ourselves. I mean we we have methodologies to showcase clients visibility. We ha we talk and often talk about how it's a hey, it's a gift with purchase. You're getting core partner marketing, but a massive percentage of this is showing up in AI citations and GEO. I've I've talked about this a little bit and I've written some things about payment and how do you tie payment and compensation to this world? How does CJ I mean how does CJ think about it? How do you personally think about it?

~ the compensation models, how do they evolve in more of a visibility environment when the click's being lost? Sure, so I mean my personal POV is they absolutely need to evolve, right? as search, I'll just say search, refer traffic declines for publishers, they they need different ways to monetize their influence, right? ~ and so there are things that we can use in

the absence of those things that we need, right? So we talked about those things. talked about ~ CDN call signals there there are a lot of things that we can pull in. our univers well This we might need to cut. CD and call signals, other technologies you can share all digital data so that we can analyze trends,

you know, direct conversion trends and apply ~ or create methodologies that then allow us to to make assumptions around publisher influence ~ to conversions. And that's something that absolutely needs and we need to begin testing. I think in the intern we're focusing on flat feed engagements as we develop those, paper influence or just ~ dynamic commissioning structures based on that engagement.

No, it's good. I mean I think that what I'm hearing is you're thinking about a lot of ways to compensate. You've got a number of levers, you know, kind of within the existing ecosystem to compensate. And then there's all these new ways to measure that we can co tie to those payment methods that allow you to have some flexibility around it. So I think it makes sense too. But it is an industry priority ~ that we all need to start thinking about. ~ particularly I would say on the pub publisher front,

right? ~ we I work really closely with with publishers, wanting to hear from them, wanting to hear that feedback to better understand what it is that they need. Yeah. Because at the end of the day, they're the ones providing the content, they're the ones driving that influence. Yeah. ~ and so we wanna make sure that whatever whatever is developed is Yeah, absolutely. I mean we often talk about it like they're the RD of the ecosystem, they're the currency of the ecosystem, they're they're really who is driving a lot of it in so many ways.

So I love that framing. That's great. It's mythbuster time. We ready to bust some myths? Yeah, they're what's some Let's do it. good what's some good like Geo myths that we want to just kind of say ~ Not really not really your jam. meh. Taking performance budgets and defining and leveraging those for upper funnel.

If you do that, you're gonna kill performance. Like if you remove performance budgets, then your pr you won't meet your performance goals. those th that maybe it's not a myth, but we'll call it Is it a mistake or trap that you see brands saying we're going to take away Let's reallocate performance budgets for the purposes of GEO, which then impacts the performance.

Yeah, yeah, you need you need the fuel of what's working at the very least. And that should that should be coming from another experimentation budget or yeah, you're gonna lose something. One of many silly claims I Yeah. The the Yeah. Another big myth. GEO is primarily owned. Or rent it. Yeah, yeah. Yeah. ~ the data shows that the vast majority of sources cited are third party sources.

Go through Yeah. If you want GEO results, you heard it here. Kind of important. Kind little bit. ~ big big myth, there are clients we talk to that I'm like, all right, let me know. ~ would love to hear more about what you're doing. They're like, we got this down. We are re like ~ rewriting our entire website. We are writing so much content. We are doing all of these things. I mean you kinda need both, right?

But you don't wanna neglect definitely don't wanna neglect th third party trusted content that talks about you in a way that doesn't feel as biased as you talking about. Yeah, but it's not even about the bias, really, it's about Let's look at the Like it just is let's go back to the data. And that's really what ~ I encourage every company to do. let's start with the data and who is being cited when it comes to your brand.

That's that's then the work that needs to be done. And I'm telling you, ~ their analys I There are analysis that I have read about on the internet, ~ including analysis that CJ has done and published, that shows that over 83% of sources cited across verticals are Yeah, that percentage sounds like it keeps going up. I kinda like that. good myth busting. The czar of AI, bespoke. Agentic.

What the heck is it and what's going on and what's coming? A gente. So you know, I sometimes I worry about hot takes in AI because they're always like famous last words. Yeah. a gen tick in particular, that one feels it feels a ways away. Yeah. I sure I'll tell you something anecdotally. Yep. Last year.

Last year my friend and I were were talking about back to school. She was doing back to school shopping for her daughters and we built some agents. We're just like, ~ my god, AI is just going to simplify all of our lives. Let's get some agents to get that done and boom. About the wrong stuff. I listened to a keynote today ~ that talked about how either half or over half something like that of the internet is ~ it's actually bots. It's bot traffic. So you know what a lot of websites have done?

They've implemented ways to block bot traffic. And so work agents. Until thank you. ~ so until we really like solve for that that piece and of course there's like authentication and and a lot of but there's a lot of work to be done Turns out we're still employed. it what a concept but even beyond that I mean there's there are two different maybe three but the two main ~ challenges that we need to solve for is one consumer trust right

like who I I don't really know that many people that are handing over are ready anyway to hand over their credit card to agents and have them ~ fully make those decisions. And then there's the the website or internet trust. The websites don't trust the agents to come in and and make purchase decisions. So until those two things are solved for I'm just I th I think it's I think it's always right. Yep, Katarine, head of AI at CJ. Miss 305, some people say. It's

That one says. a it's a it's it's a it's a starting to get traction here in Santa Barbara. What's one big takeaway you want to give the audience in the AI realm that you recommend they they take action on? Yeah. I would say start collaborating cross channel, cross cross almost everything really. ~ as we dig into as we dig into

how these LLMs work and and how they formulate answers, we're really finding that every single channel plays role. And when you are having one channel focus on all of it, you're really missing a lot of context and a lot of the opportunity. So bring it all together, create a center of excellence, create a tiger team, create a strike team, whatever you want to call it. bring in all the right stakeholders and focus on the data.

For the love of God, please look at the data and make decisions based on on what it I love it. Sounds obvious, but it's clearly helped you navigate all all the way up through your career with one great company and you've been able to lead it now with AI and it's been a pleasure to get to know you here and thanks for joining us in a pretty nice location to to figure it all out. So until next time. Thank you so much for having me. Yeah. Thanks everybody. I

Open the transcript file

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. #142 SaaS Class EP3: (Revenge of Affiliate) Tye DeGrange, Nancy Harnett & Guy Yalif

    Nov 10, 202642 minAudio

    In this episode of SaaS Class, Nancy Harnett, Guy Yalif, and Tye DeGrange dig into whether affiliate is finally getting its moment in the age of AI search. Nancy is an agency owner who previously helped build HubSpot’s...

    Credit the people on this episode

  2. #141 Why Campaign-Based Marketing Is Failing | Tye DeGrange and Sandeep Menon

    Nov 2, 202640 minAudio

    Sandeep Menon, co-founder and CEO of Auxia and a former Google Pay marketing leader, joins Tye DeGrange, host of Always Be Testing and CEO of RoundBarnLabs, to discuss how AI and agentic marketing are reshaping...

    Credit the people on this episode

  3. #140 Why Chasing Clicks Is Killing Your Pipeline | Tye DeGrange & Rob Pinkerton

    Oct 26, 202640 minAudio

    Rob Pinkerton is the CMO of Seismic, an AI-powered sales enablement platform, and previously served as CMO of Morningstar and SVP at Oracle, with earlier time at Adobe. He holds a JD and an MBA, and he began his career...

    Credit the people on this episode

  4. #139 Brands Without Creator Partnerships Are Invisible to AI | Tye DeGrange & Dave Yovanno

    Oct 20, 202645 minAudio

    In this episode of Always the Testing Podcast, host Tye DeGrange talks shop with Dave Yovanno, CEO of impact.com, a partner marketing platform serving about 6,000 brands, including Walmart, Target and 60 banks. The...

    Credit the people on this episode

  5. #138 Market Research CEO: Your SaaS Ads Are Burning Cash on a Message Nobody Wants | Peep Laja & Tye DeGrange

    Oct 13, 202638 minAudio

    Tye DeGrange talks with Peep Laja, CEO of Wynter (a B2B market research platform with 12 to 48 hour turnaround on real-human surveys) and a leading voice in SaaS messaging and testing. They unpack why 65% of SaaS...

    Credit the people on this episode