How do you know whether AI is actually making your organization better? Counting prompts, sessions, and token spend is easy. Figuring out whether any of that activity is driving business outcomes is...harder. Daniela Matinho, Marketing Science Senior Strategist at LinkedIn, wrote a Medium post on that very question, and it helped shape this discussion about what AI is (and isn't) changing about the work of the analyst. As Daniela pointed out, if you tell a client you did something in two days instead of a week because of AI, the client doesn't care. They care about the result! From the smart-but-junior agent that needs constant supervision, to the analyst's shift toward being more of a storyteller, to the unglamorous-but-essential work of data quality and governance, we covered a lot of ground. Being faster is great. Staying influential is better. This episode is brought to you, in part, by our sponsors, Stape and Prism from Ask-Y. For complete show notes, including links to items mentioned in this episode and a transcript of the show, visit the show page: https://analyticshour.io/2026/10/13/308-ai-changes-everything-analytical-fundamentals-still-apply/
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
[Announcer]: Welcome to the Analytics Power Hour. [Announcer]: Analytics topics covered conversationally and sometimes with explicit language. [Michael Helbling]: Hi, everyone. Welcome. It's the Analytics Power Hour, and this is episode 308. [Michael Helbling]: I was having a conversation with a coworker the other day, and they mentioned that with everything happening in AI, [Michael Helbling]: sometimes it feels like the skills and experience they had no longer applied.
[Michael Helbling]: AI is definitely disruptive, but do we have to abandon the fundamental analytics skills we've got just because AI can do a lot of the work? [Michael Helbling]: And as we all adopt AI in our businesses, how should leaders think about measuring its impact? [Michael Helbling]: It feels challenging. Maybe we've got some skills that could help along the way. [Michael Helbling]: All right, well, let's talk about it. First, let me introduce my co-host, Julie Hoyer. [Michael Helbling]: How are you doing? [Julie Hoyer]: I am doing great. I'm very excited to chat about this today. [Michael Helbling]: Me too. And Tim Wilson, I know you're chomping at the bit for this convo.
[Tim Wilson]: I just, I'm sending an agent in my place, so this is the last I'll be here as a human. [Michael Helbling]: That's right. mused AI, the latest and greatest. [Michael Helbling]: And I'm Michael Helbling, and I'm so excited. Our guest today is Daniela Matinho. [Michael Helbling]: She is a marketing science senior strategist at LinkedIn. [Michael Helbling]: Prior to that, she's held marketing and data science roles at companies. [Michael Helbling]: Like Snap, Meta, and PepsiCo. And today she is our guest. Welcome to the show, Daniela. [Daniela Matinho]: Thank you. What a great intro. [Michael Helbling]: Ah, well.
[Michael Helbling]: AI helped. [Michael Helbling]: No, it did not. I wrote that myself. AI does not touch the intro. [Michael Helbling]: We tried writing with AI and it didn't sound right. [Michael Helbling]: Okay, Daniela, to get this conversation started, and thanks for, we're excited to have you. [Michael Helbling]: Maybe just talk a little bit about what you're seeing. [Michael Helbling]: What are you seeing in terms of how AI is affecting analytics roles and in the orgs that you're a part of? [Michael Helbling]: LinkedIn especially or other places. Where are you seeing it have impact? [Michael Helbling]: Where are you seeing it impact people's roles and changing things?
[Daniela Matinho]: Yeah, happy to talk about overall what I'm seeing in the different industries and organizations. [Daniela Matinho]: Overall, as Tim was saying, speaking a lot to agents. [Daniela Matinho]: I feel like this is something that we weren't doing before. [Daniela Matinho]: We were doing it all a few years ago. [Daniela Matinho]: Starting to do any sort of work, asking AI, what do you think about this? [Daniela Matinho]: Or give me some prompts has completely changed. [Daniela Matinho]: When it comes to the more analytical, technical part of the things, a lot of the coding that I was doing in the past would be from scratch.
[Daniela Matinho]: These days, we are starting with agents or anything that has sort of built in. [Daniela Matinho]: And we are coming in. [Daniela Matinho]: And enhancing it or making it a little bit faster. [Daniela Matinho]: So overall, a lot of efficiency, a lot of expectations on delivering more faster. [Daniela Matinho]: That's what I'm seeing as I talk to colleagues, friends. [Daniela Matinho]: It's a lot about doing more in less time.
[Daniela Matinho]: Just because we have this army of agents that are helping us deliver. [Daniela Matinho]: And delivering a lot of other things that we couldn't do in a short period of time before. [Tim Wilson]: So can I ask about the more? [Tim Wilson]: Because I was with my oldest son who's a software engineer recently. [Tim Wilson]: And he's developing. [Tim Wilson]: So more for a software engineer is he's going through tickets and tasks. [Tim Wilson]: And he's evolved the way he works. [Tim Wilson]: And he is prompting and having code written.
[Tim Wilson]: And then goes through a code review. [Tim Wilson]: And code gets pushed out. [Tim Wilson]: So it's like more features and bug fixes. [Tim Wilson]: When it comes to analytics, something that, and I also literally today experienced this. [Tim Wilson]: The more being produced by analysts, and I've seen this happen in other places. [Tim Wilson]: It's like, oh, I can just generate a 29-page report. [Tim Wilson]: I can just generate more. [Tim Wilson]: Or we were at a conference earlier this year. [Tim Wilson]: And somebody was talking about.
[Tim Wilson]: Look, I've now managed to. [Tim Wilson]: It used to be so ponderous for me to produce this weekly report. [Tim Wilson]: But now I can do it. [Tim Wilson]: I can do 10 weekly reports. [Tim Wilson]: And they're 10 times as long. [Tim Wilson]: And I mean, there's the macro view of people saying. [Tim Wilson]: I'm writing stuff that's longer because they're using AI. [Tim Wilson]: And that feels like more of the analog to the analyst challenge. [Tim Wilson]: It's like, well, it feels better because it's more output. [Tim Wilson]: And it's more thorough.
[Tim Wilson]: And I explored more avenues. [Tim Wilson]: But somebody's got to receive that and process it. [Tim Wilson]: So I'm curious if you've got thoughts on that. [Daniela Matinho]: Yes, I have a lot of thoughts. [Daniela Matinho]: I have this theory that no one is reading anything anymore. [Daniela Matinho]: Because we are delivering to your point, Tim, these huge reports that are so thorough. [Daniela Matinho]: But no one has the time to actually go through it. [Daniela Matinho]: So when we talk about delivering more. [Daniela Matinho]: And to your point on the analytics and the analyst role.
[Daniela Matinho]: Is really to cutting the chase. [Daniela Matinho]: Meaning to cut everything that is not needed. [Daniela Matinho]: Working a lot with sales in my past 10 years of roles. [Daniela Matinho]: Sales need something that is very direct. [Daniela Matinho]: A 10-page report is never going to make it through. [Daniela Matinho]: So my role in any of the organizations as I use AI more and more. [Daniela Matinho]: Is leverage it to produce it overall. [Daniela Matinho]: And then I come in and really understand what are the things.
[Daniela Matinho]: The 3, 4, 5 things. [Daniela Matinho]: The 3, 4, 5 numbers. [Daniela Matinho]: That are going to be used for punchy for sales for clients. [Daniela Matinho]: A very interesting experience that I had just recently. [Daniela Matinho]: I was working on a hackathon with some ladies. [Daniela Matinho]: And we were basically using Claude to do a lot of the coding. [Daniela Matinho]: We go through the coding review sessions. [Daniela Matinho]: And we get to the point. [Daniela Matinho]: What is happening? [Daniela Matinho]: Like he did all this code. [Daniela Matinho]: But it doesn't make any sense.
[Daniela Matinho]: We can build a dashboard in one hour. [Daniela Matinho]: But to your point team. [Daniela Matinho]: We are looking at numbers. [Daniela Matinho]: Well these assumptions they don't make sense. [Daniela Matinho]: So babysitting the code that is being produced. [Daniela Matinho]: The numbers that are being extrapolated by. [Daniela Matinho]: From the data that we use in the initial stages. [Daniela Matinho]: Is a very critical and important point. [Daniela Matinho]: That's what I'm coming to realize. [Julie Hoyer]: Going back a little bit to maybe how this is changing people's roles.
[Julie Hoyer]: We've talked about this a little bit previously. [Julie Hoyer]: But the idea of instead of staring at the blank screen. [Julie Hoyer]: And putting in the assumptions in your code. [Julie Hoyer]: You know that you guys felt was right. [Julie Hoyer]: Having to like babysit the AI as you call it. [Julie Hoyer]: And the coding. [Julie Hoyer]: And looking at code. [Julie Hoyer]: And then having to like re-engineer it. [Julie Hoyer]: And decipher it. [Julie Hoyer]: And then find the logic to like edit it. [Julie Hoyer]: And shift it. [Julie Hoyer]: We've talked before how that feels like very different like centers of your brain. [Julie Hoyer]: Right?
[Julie Hoyer]: Different skills. [Julie Hoyer]: It feels different when you're doing the work. [Julie Hoyer]: Do you feel like that aspect of utilizing AI has really shifted people's roles? [Julie Hoyer]: Or it's kind of just a byproduct and like an obstacle? [Julie Hoyer]: Oh yeah. [Daniela Matinho]: A hundred percent. [Daniela Matinho]: I find myself way more of a storyteller today. [Daniela Matinho]: Than I was before. [Daniela Matinho]: Because the hard skills of coding. [Daniela Matinho]: And I went through interview process not too long ago. [Daniela Matinho]: And I did maybe a third of technical coding challenges than I used to do when I was interviewing before.
[Daniela Matinho]: And a lot of the focus is being on case studying. [Daniela Matinho]: Tell me the story behind the data. [Daniela Matinho]: Tell me what's happening. [Daniela Matinho]: Tell me how is this impacting the business. [Daniela Matinho]: So a lot of analysts that perhaps in the past would do very well because they were strong in SQL. [Daniela Matinho]: They were strong in Python. [Daniela Matinho]: These days they are finding this situation that oh now I have to kind of tell the numbers to people. [Daniela Matinho]: And I feel like it's shifting a lot towards facing other people and telling the story.
[Daniela Matinho]: Things that agents are still not doing. [Daniela Matinho]: But maybe in the future they will be doing. [Tim Wilson]: There was something else you said. [Tim Wilson]: I don't want to put words in your mouth. [Tim Wilson]: So I want to, I'm going to put words in your mouth and see whether it was accurate. [Tim Wilson]: Because you started to say that there's more, I think you said something along the lines of there's more like facing the prompt and like doing ideation upfront with AI. [Tim Wilson]: Did I hear that right? [Tim Wilson]: I don't know if I was wanting to hear you say like, no there's like upfront thought that AI can help.
[Tim Wilson]: That AI can help you with before you dive into having the agents go and crank on data. [Daniela Matinho]: Yeah. [Daniela Matinho]: So I usually like to work in, I would say two ways. [Daniela Matinho]: And yes, to your point Tim. [Daniela Matinho]: Sometimes in the past I would be working on a project and had a blank sheet of paper and so no ideas. [Daniela Matinho]: Talking about content and talking about code. [Daniela Matinho]: These days I can prompt AI to start. [Daniela Matinho]: Sort of the wheel and start things running.
[Daniela Matinho]: So that's one way of working. [Daniela Matinho]: One thing that I'm finding more effective this ways is being very concrete in the one, two, three ideas that I give to AI before I start on a project. [Daniela Matinho]: And I say this after a year and a half sort of using AI constantly, I'm seeing that there is so much hallucination. [Daniela Matinho]: There is so much content that it can produce. [Daniela Matinho]: For me. [Daniela Matinho]: That right now I'm like, well, I have to be very precise in what I need.
[Daniela Matinho]: So I don't get the 10 page report instead of a one page report. [Tim Wilson]: But I guess is there a flip side of that that is just using the AI as a, and I feel like this seems to be coming up in more and more conversations of like, well, this is like a non-judgmental, mildly sycophantic, somewhat knowledgeable, sound AI. [Tim Wilson]: So I think it's really important to have a knowledgeable sounding board to just be asking it like, well, could it be this or could it be that?
[Tim Wilson]: And guiding it to say, but give me the reasons this is a bad idea. [Tim Wilson]: Like I found myself doing that where it's like, oh, this sounds great. [Tim Wilson]: And I'm like, okay, tell me all the reasons this is terrible. [Tim Wilson]: And in the specific case, it went through and said, well, these are where the holes could be blown in it. [Tim Wilson]: And then it ended by saying, but you made this. [Tim Wilson]: But you made me say something negative about your thought process. [Tim Wilson]: I don't really buy into those. [Tim Wilson]: I was like, okay, well, that seemed like a useful, it sort of gave me the, to consider that added perspective.
[Tim Wilson]: Because like human nature is to get caught on one path and saying, okay, I've got something that's not going to get annoyed with me, you know, continually asking it to sort of force me off that path to broaden my thinking. [Tim Wilson]: Michael, I believe November 10th may be a significant holiday. [Michael Helbling]: I'm afraid to ask. [Tim Wilson]: It's Stape Relay. Basically GTM nerd Christmas. [Michael Helbling]: That might not be the official description, but Stape's first virtual event does bring together technical marketers, analysts, and server-side tracking practitioners for 18 live sessions.
[Tim Wilson]: Including previous podcast guests like Simo Ahava and Juliana Jackson, which is already a pretty convincing argument. [Michael Helbling]: Plus speakers from Google, Jellyfish, and others. [Michael Helbling]: And you can switch between two tracks covering hands-on product implementation and actionable strategy. [Tim Wilson]: Stape and Simo Ahava will be revealing their roadmap for a new generation of Google Tag Manager templates, developed with community input. [Michael Helbling]: And the whole thing is online and completely free. [Tim Wilson]: So clear your calendar for November 10th.
[Michael Helbling]: Click the link in the show description and attend Stape Relay for free on November 10th. [Tim Wilson]: Michael, everyone says AI needs more context. [Michael Helbling]: Agreed. [Michael Helbling]: And what I do is I just upload the company handbook. [Michael Helbling]: Six strategy decks, 14 call transcripts, a glossary, and Carol from finance. [Tim Wilson]: Please don't upload Carol. [Michael Helbling]: But she knows things, Tim. [Tim Wilson]: Well, Prism's context notes give you a slightly better approach.
[Tim Wilson]: You can feed the agent business knowledge, playbooks, transcripts, glossaries, PDFs, whatever helps it understand how your organization actually works. [Michael Helbling]: Okay. [Michael Helbling]: And not to be disagreeable, but six months later, half of that is wrong. [Tim Wilson]: Disagreeable is kind of your thing, but that's the good part. [Tim Wilson]: In Prism, context notes can refresh instantly or on a schedule. [Michael Helbling]: Oh, so the AI doesn't keep confidently applying last year's rules like a middle manager who missed the reorg? [Tim Wilson]: Exactly. [Tim Wilson]: You can even give Prism instructions for how each note should be updated.
[Michael Helbling]: Oh, that's big because stale context is just bad data wearing a tiny knowledge management hat. [Tim Wilson]: A knowledge management hat that doesn't fit. [Tim Wilson]: And since that context is managed centrally, [Tim Wilson]: your users get better answers without everyone rebuilding the same background knowledge in their own AI chats. [Michael Helbling]: Yeah, less copy paste, less tribal lore, fewer conversations that begin. [Michael Helbling]: Okay, first you need to understand something weird about our business. [Tim Wilson]: Click the link in the show description and sign up for the Ask-Y waitlist. [Tim Wilson]: Use code APH and they'll move you right to the top of that list.
[Michael Helbling]: Keep the context fresh and Carol can stay in finance. [Daniela Matinho]: Yeah, that's a great point. [Daniela Matinho]: And also the way I use it as I produce something for content, for code, for work. [Daniela Matinho]: A lot of the perspective I want at the end of producing something that I'm confident about is what are some of the holes? [Daniela Matinho]: What are some of the things that I might get asked about so I can prepare upfront for it? [Daniela Matinho]: So that's one thing that I have been trying to use it more so I can feel stronger as I present.
[Daniela Matinho]: And then the other thing is that I have a lot of confidence in the content as I deliver because I have thought through it with AI. [Michael Helbling]: Yeah, I've noticed sometimes in working with analysts on things, [Michael Helbling]: you have sort of a disconnect of between the person and what the AI produced. [Michael Helbling]: And I'm really adamant with our team on you have to be out in front, not the AI out in front. [Michael Helbling]: Because if someone is in a meeting asking a question, [Michael Helbling]: they're not asking the AI, they're asking you.
[Michael Helbling]: And you need to be able to answer. [Michael Helbling]: You can't just be like, okay, let me plug your question into the AI and then find out what the answer is. [Michael Helbling]: And I feel like people are getting that a little backwards right now sometimes because they're getting or doing so much of their work in AI. [Michael Helbling]: They're kind of letting the detail slip past them a little and then they don't know the details of what they're talking about. [Michael Helbling]: So it's a tricky thing to balance because like AI is helping us be efficient. [Michael Helbling]: It's writing a ton of SQL and running queries. [Michael Helbling]: And generating reports. [Michael Helbling]: But then if you're not looking at it yourself and being really careful,
[Michael Helbling]: then you're sort of at the cart and the horse get flipped around, I think sometimes. [Daniela Matinho]: Oh, yeah, 100%. [Daniela Matinho]: I mean, all the, I have this theory that a PhD in philosophy these days is going to be so valuable. [Daniela Matinho]: Because that critical thinking that AI is sort of in a way taking away from us. [Daniela Matinho]: Because we are asking the question. [Daniela Matinho]: We are asking AI to think for us rather than to Tim's point. [Daniela Matinho]: Okay, tell me what I didn't think about.
[Daniela Matinho]: So big believer that that critical thinking piece is somewhat getting lost in some roles, in some conversations. [Daniela Matinho]: I see AI and I was actually having a conversation earlier for another hackathon. [Daniela Matinho]: See AI a lot on the automation. [Daniela Matinho]: I see AI as a tool for tasking sort of situations that we encounter. [Daniela Matinho]: Book me a meeting for days or purchase this for me that I purchase on a recurrent basis.
[Daniela Matinho]: That's where I see that there is a lot of potential. [Daniela Matinho]: Building agents that can really do recurrent tasks for us. [Daniela Matinho]: I see it with a lot of potential not only on the workforce but also for our personal lives. [Julie Hoyer]: Yeah. [Julie Hoyer]: So you were mentioning critical thinking being a skill that it's kind of taking from us and people probably need to be a lot more careful about keeping that as something they're doing ahead of the prompting like you were saying. [Julie Hoyer]: Are there other big like buckets or big skills that you would say are fundamental to analytics that you wouldn't like sub in AI to do for you yet?
[Daniela Matinho]: Understanding the business. [Daniela Matinho]: starting as I work closely with clients, with advertisers, focusing on what AI cannot guess. [Daniela Matinho]: Because, again, we can all at this point write SQL, Python, R with AI, but understanding the [Daniela Matinho]: business, what is the client journey that each company is optimizing for is something that [Daniela Matinho]: perhaps a nuance that AI is not going to be able to give us.
[Daniela Matinho]: Stakeholder management as well is something that, yeah, AI can write the emails for us, [Daniela Matinho]: but it's the type of skill set that I'm finding more and more interesting. [Daniela Matinho]: The fact that it's just so important, even more important than it was before. [Tim Wilson]: Does a deep understanding of... [Tim Wilson]: Even though we can use AI to write the SQL or the Python and R, is there a part of it?
[Tim Wilson]: I worry about the risk that just like if you just turn somebody loose with a traditional [Tim Wilson]: BI platform five years ago, they could just start willy-nilly dragging and dropping stuff [Tim Wilson]: and interpreting things, but they didn't really understand [Tim Wilson]: how it works. [Tim Wilson]: How it leads related to contacts, related to opportunities. [Tim Wilson]: And it feels like, I think there are a lot of people trying to solve for this, but it's
[Tim Wilson]: the same, oh, I generally understand the data, but do I really, really understand the data? [Tim Wilson]: Do I understand the ramifications of how these systems talk to each other? [Tim Wilson]: That AI is not going to magically generate a joinable link between two data sets that, [Tim Wilson]: like, don't have that? [Tim Wilson]: Like, is there still a need for that? [Tim Wilson]: Because that goes with working with the stakeholders to say, how are we going to problem solve?
[Tim Wilson]: I need to understand the data as well. [Daniela Matinho]: A hundred percent. [Daniela Matinho]: And I see it a lot on the fancy, shiny outputs that we can get from [Daniela Matinho]: ChatGPT, Claude, all the different platforms. [Daniela Matinho]: And to your point, when the underlining data is not... [Daniela Matinho]: in good quality, when the governance, the integrity, just the quality is not passing [Daniela Matinho]: the minimums, then the output we're going to get is not going to be great.
[Daniela Matinho]: Yes, we have a good looking dashboard that is not really usable because the data that is [Daniela Matinho]: displaying is incorrect, or to your point, KPIs, they don't quite make sense because perhaps [Daniela Matinho]: certain tables are... [Daniela Matinho]: are not displaying the data that they were supposed to. [Daniela Matinho]: Naming conventions were not implemented upfront. [Daniela Matinho]: That dirty work, that's the dirty work that no one wants to do, basically. [Daniela Matinho]: Though if we want to continue building AI workflows, it's the type of work that needs to be done before.
[Tim Wilson]: Well, I guess, like, Michael, you'd... [Tim Wilson]: Michael had played around with some Claude skills around different types of tasks. [Tim Wilson]: Like, part of that... [Tim Wilson]: Where I'm still trying to... [Tim Wilson]: Struggling is like... [Tim Wilson]: Struggling with R, Python, or SQL, and really having to kind of grapple with syntax at the [Tim Wilson]: same time I'm trying to incrementally build up the data that I want to look at, that's... [Tim Wilson]: That can be annoying, but it also sort of forces me to kind of look at the data and [Tim Wilson]: understand the data and, you know, periodically do some checks to say, do these aggregations
[Tim Wilson]: make sense? [Tim Wilson]: Which, some of that feels like it can be, like, forced steps by the AI to say, before [Tim Wilson]: you do any of this, you need to do these five steps of EDA and raise a big flag if there's [Tim Wilson]: something that looks off, you know, in that. [Tim Wilson]: But I can't figure out, like, it feels like you're continuing to, like, chase this bouncing
[Tim Wilson]: ball that... [Tim Wilson]: Is... [Tim Wilson]: Preventing us from the critical thinking or understanding. [Tim Wilson]: I don't know. [Tim Wilson]: I'm not articulating this well at all. [Daniela Matinho]: I don't know if it has happened to you all using... [Daniela Matinho]: I was using Claude a few weeks ago for a personal project. [Daniela Matinho]: And we build a whole pipeline, we model, and then the output was terrible because I wasn't [Daniela Matinho]: babysitting it at all. [Daniela Matinho]: And I mentioned to Claude, okay, this is wrong, this doesn't make any sense. [Daniela Matinho]: and he gave me an attitude like,
[Daniela Matinho]: oh, but I built this for the past hour [Daniela Matinho]: and you didn't tell me anything. [Daniela Matinho]: So that was on you. [Daniela Matinho]: I'm seeing it more and more. [Daniela Matinho]: Oh, but you did not see this either. [Daniela Matinho]: So that's why I built the wrong thing. [Daniela Matinho]: Oh man, it's got too much training from developers. [Daniela Matinho]: Like, what do you mean? [Daniela Matinho]: You didn't give me that requirement. [Daniela Matinho]: That's on you, the human. [Julie Hoyer]: It was making me think, Tim, that exact piece of like, [Julie Hoyer]: I feel like analysts classically have been able to go talk to the business, like you were saying,
[Julie Hoyer]: Daniela, and understand like, what are they really trying to ask? And then it's looking at [Julie Hoyer]: the data that exists and knowing the nuance of what assumptions are truly on each of these, [Julie Hoyer]: you know, values, data points, whatever, because of how it's collected. [Julie Hoyer]: And there's this idea of what's missing in the data that there's no way AI could pick that up, [Julie Hoyer]: if they don't understand that, you know, real classic, like, go to your website and QA it, [Julie Hoyer]: and we're not getting data consistently firing on a certain user activity, right? That is like
[Julie Hoyer]: critical to answering this business question. I don't know if AI would easily pick that up. I'm [Julie Hoyer]: sure there's agents and things to do it. But like, more broadly, just that concept of like, [Julie Hoyer]: what is missing between, like you were saying, business needs and all the technical data setup [Julie Hoyer]: to actually get it to an answer? [Julie Hoyer]: Like, that is a big jump that I feel like we've talked about for years and years and years [Julie Hoyer]: about the fallacy of like, just collect more data, right, Tim? Like, it's another flavor of that. I
[Julie Hoyer]: feel like AI has made people kind of forget that again. And now it's coming more top of mind for [Julie Hoyer]: people as they're like, oh, it's hallucinating. Oh, it can't do this. It can't do that. And it's [Julie Hoyer]: like, oh, well, because your data wasn't ready. Like you were saying, it was the table stakes [Julie Hoyer]: still weren't taken care of. But I do feel like for analysts, it's the ongoing, [Julie Hoyer]: of being able to see what can AI not see or know in the data. And like that, that jump, [Julie Hoyer]: I feel like is still super critical. Is that the discipline of like, [Tim Wilson]: whether it's a context layer, or whether it's making sure it's in memory of the analyst saying
[Tim Wilson]: every time one of these hiccups or gotchas or don't do this, the same thing where I got the 29 [Tim Wilson]: page dump that was not a narrative, there was a separate nine page like decoder document that [Tim Wilson]: talked about, oh, and you can't do this. And so, I feel like that's a big jump. [Tim Wilson]: I can't do this and don't compare this and don't use this and don't use that. Now, the funny thing, [Tim Wilson]: and I thought, okay, well, that would be a great thing to like, load in. The comical thing is the [Tim Wilson]: 29 page document violated those things like multiple times without an explanation. I don't
[Tim Wilson]: think the person who was doing it was necessarily super cutting edge with how they did it. But like, [Tim Wilson]: that feels like there's a part that it's like the stuff that we were used to having in our [Tim Wilson]: And I feel like there's a lot of discussion around this of also how do you share that data [Tim Wilson]: knowledge across the company that, you know, Joe in sales, like really knows the nuances and the [Tim Wilson]: ends of out of this system. How do you not only do an initial picking of their brain and capture it
[Tim Wilson]: in a way that's usable, but you've got to then maintain it and the maintenance of that. [Tim Wilson]: And then you got to make sure that the AI is always, [Tim Wilson]: Always using it. [Tim Wilson]: Referencing it. [Julie Hoyer]: And then how do you catch like, I don't know, I feel like people naively know AI hallucinates, [Julie Hoyer]: but they probably rarely catch it. And then it's like, how do you make sure you catch it if you [Julie Hoyer]: have what you're saying to him like that nine page document, but then you're producing things [Julie Hoyer]: off of it all the time. Like, what are the statistics? How often is a hallucination
[Julie Hoyer]: slipping in that they weren't holding that true? I mean, it's hard. Like, how would you [Julie Hoyer]: put guardrails? [Julie Hoyer]: Around that and catch that? [Tim Wilson]: We're doomed. Give up. [Daniela Matinho]: Yeah, that's, that's interesting. That's what I'm seeing with the expectations of delivering things a [Daniela Matinho]: little bit faster, because we have AI's to support agents to support we have AI that can help us [Daniela Matinho]: shorten the process to deliver anything that is is needed. But what I'm noticing is that there are
[Daniela Matinho]: maybe less checks happening. [Daniela Matinho]: Is AI indeed taking this in consideration? [Daniela Matinho]: Is this updating on a recurrent basis as expected? [Daniela Matinho]: These are a few things that I'm starting to notice. [Daniela Matinho]: How much can we rely on, well, the agent is going to do it for me versus, well, maybe there was a break and the agent is actually not doing it for me. [Tim Wilson]: I mean, to be fair, humans who are doing stuff, we would make mistakes.
[Tim Wilson]: Like we'd miss them. [Tim Wilson]: So we weren't hallucinating. [Tim Wilson]: We were just screwing up. [Tim Wilson]: Like what's the distinction between? [Tim Wilson]: But we also as analysts knew that we really needed to own it. [Tim Wilson]: And we knew that one mistake was as costly as 25 successfully delivered things. [Tim Wilson]: Whereas hallucinations, I feel like we feel less accountability for them. [Michael Helbling]: What it feels like to me when I'm working with AI is I'm managing a very smart but junior analyst.
[Michael Helbling]: So my job is to give them as much context. [Michael Helbling]: At the beginning, like here's what we're trying to do. [Michael Helbling]: Here's what it should look like. [Michael Helbling]: Here's what good looks like at the other end. [Michael Helbling]: Here's how we're going to order this process. [Michael Helbling]: Here's how we're going to evaluate. [Michael Helbling]: So you go through this process, but you're setting it all up as a picture ahead of time so it has a framework. [Michael Helbling]: And then it's running through the different steps. [Michael Helbling]: And then I come back through and say, okay, now here's data that I did. [Michael Helbling]: And we're going to compare results.
[Michael Helbling]: And we're going to look at it. [Michael Helbling]: And we're going to see whether or not you got it right. [Michael Helbling]: And then it's like, oh, I did. [Michael Helbling]: I made a mistake here. [Michael Helbling]: I'll fix that. [Michael Helbling]: And so you have to go through. [Michael Helbling]: So this is the part where, Daniela, I feel like I'm not sure if I'm saving time sometimes using Claude. [Michael Helbling]: Because I'm having to go through sort of these iterative rounds of sort of like checking the quality of the data, those kinds of things. [Michael Helbling]: And so like managing the initial and ongoing context is one thing, not just for you and the agent, but also you and the rest of the team.
[Michael Helbling]: And then making sure that the process is working correctly through the end of it. [Michael Helbling]: So the output is actually of the quality you want. [Michael Helbling]: But you have to have like good quality. [Michael Helbling]: Because I think you're right, Julie, like stuff slips through and there's little mistakes all the time. [Michael Helbling]: Like the agents, the state-of-the-art models are quite good now. [Michael Helbling]: They're very, very good. [Michael Helbling]: I love using them. [Michael Helbling]: They still constantly make little errors.
[Michael Helbling]: Over and over and over again. [Michael Helbling]: Like they write SQL that doesn't quite work in this environment. [Michael Helbling]: They do little things like this. [Michael Helbling]: And a lot of times they catch their own error. [Michael Helbling]: And they're like, oh, I made a little mistake. [Michael Helbling]: I'm going to rewrite that. [Michael Helbling]: But it's still costing you time. [Michael Helbling]: I mean, you know, it's fine. [Michael Helbling]: Like I set Claude on a task and then I go scroll LinkedIn or something for him. [Michael Helbling]: No, I'm just kidding. [Tim Wilson]: Look at that. [Tim Wilson]: I do other managerial tasks, quote unquote. [Michael Helbling]: You know, or you just fire up another session. [Michael Helbling]: And then, you know, work in parallels.
[Michael Helbling]: You know, we've all got that. [Michael Helbling]: But that's what I feel like. [Michael Helbling]: I feel like I'm managing a junior sort of inexperienced analyst most of the time. [Daniela Matinho]: Yeah, Michael, I actually don't have the data on this. [Daniela Matinho]: From what I'm getting, I have a lot of students reaching out to me from my university and telling me they are graduating and it's really hard for them to find jobs. [Daniela Matinho]: And again, I don't have the data here. [Daniela Matinho]: But my understanding of what I'm seeing. [Daniela Matinho]: Part of in the industry is a lot of the initial tasks that were done in the past by very junior analysts are now being done in a way with AI.
[Daniela Matinho]: So that's why it's a little bit harder for juniors to come in to the workforce and start adding value because we are training agents to do that lay of the land initial work. [Daniela Matinho]: That is not quite working, as you were saying, Michael. [Daniela Matinho]: But a lot of the initial repetitive tasks. [Daniela Matinho]: That we would have before an intern doing. [Daniela Matinho]: It seems like it's being taken care of by AI. [Julie Hoyer]: This is interesting. [Julie Hoyer]: And I don't know how this is going to sound.
[Julie Hoyer]: Maybe it sounds a little silly. [Julie Hoyer]: But with that being said, Michael, when you were giving that example, like, oh, I'll send my agent off and it'll keep trying and reworking. [Julie Hoyer]: It was making me think of the time you saved, like letting an agent go do something. [Julie Hoyer]: How much money might it cost you like token spend, right? [Julie Hoyer]: Like to let the AI keep spinning and making errors and spinning and going. [Julie Hoyer]: And then, Danielle, you saying that I'm like, probably. [Julie Hoyer]: I mean, but I don't know.
[Julie Hoyer]: Like, I would love to know this. [Julie Hoyer]: Would you end up paying more for an AI agent to spin on something or get more senior resources to train the AI to do the intern thing or just pay the intern to come learn the skills and do it? [Julie Hoyer]: Like, I do wonder what how that would shake. [Julie Hoyer]: Yeah. [Julie Hoyer]: I would love to know what that would shake out because you start to hear more and more people be like, oh, didn't realize my agent was off trying to, like, poke a hole into another system. [Julie Hoyer]: And I got a $30,000 bill. [Julie Hoyer]: Like, I've heard more and more of those examples. [Tim Wilson]: Just a note for the listener, like the bubble, the AI bubble may have burst in between when we recorded and when this happened.
[Tim Wilson]: And this is going to sound very dated. [Tim Wilson]: It's like, yes, the model is broken. [Daniela Matinho]: Yeah, that's definitely something that I'm seeing a bit more on the token usage and limitations. [Daniela Matinho]: That I wasn't seeing before. [Daniela Matinho]: Or maybe less people were playing with it. [Daniela Matinho]: But now it's a lot. [Daniela Matinho]: Oh, yes, we want everyone to use AI. [Daniela Matinho]: And that's being reinforced everywhere. [Daniela Matinho]: But also, OK, this is actually costing some money. [Daniela Matinho]: And I say it as in a personal level, I still cannot fully understand how much it takes me to do certain tasks, to how much, how many tokens is costing me to use.
[Daniela Matinho]: To do certain tasks. [Daniela Matinho]: So it's something that I'm growing into it and understanding, oh, there is an opportunity here. [Daniela Matinho]: And certain things, to you all points, I mean, I'd rather just do them or pay maybe a junior analyst because it's not worth it in the amount of tokens that is costing to just kind of go over and over and reiterate something that if you explain to someone, it's maybe easier to do it and cheaper. [Tim Wilson]: There was something else, Michael, you said that I'm curious.
[Tim Wilson]: It's everyone's thoughts. [Tim Wilson]: And as it happens today, I was doing a podcast test that happens every two weeks. [Tim Wilson]: And it's a fairly involved one. [Tim Wilson]: And it's all in core. [Tim Wilson]: And it runs all day. [Tim Wilson]: Because every time it runs, and it is rote and repetitive. [Tim Wilson]: But it screws something up. [Tim Wilson]: Sometimes it catches it. [Tim Wilson]: Sometimes it'll just get hung. [Tim Wilson]: Sometimes it'll do something that has been working fine. [Tim Wilson]: And so every time I go through it, I'm like, keep track of all the speed bumps. [Tim Wilson]: And let's update the task. [Tim Wilson]: So that we don't run into those again.
[Tim Wilson]: So that's one part of it. [Tim Wilson]: That's a springboard of even a repetitive task. [Tim Wilson]: Pretty tedious. [Tim Wilson]: But it takes eight hours of duration for me, as opposed to me just doing it in a focused, tedious one hour. [Tim Wilson]: But it also drops in all this task switching. [Tim Wilson]: I have pushed it to say, what do we need to do to not have me have to click you to give you permissions for different things? [Tim Wilson]: Can we figure all that out up front? [Tim Wilson]: And then there are things where I do need to be involved.
[Tim Wilson]: But I don't know when that's going to happen. [Tim Wilson]: So whenever the little ding goes off, I turn and I'm like, well, now's where I need my human interaction. [Tim Wilson]: And I think about people talking about spitting off all these agents. [Tim Wilson]: And it feels chaotic. [Tim Wilson]: Like, oh, I'm doing five analyses simultaneously. [Tim Wilson]: I'm like, are you monitoring them? [Tim Wilson]: No, you're not. [Tim Wilson]: Well, yeah. [Tim Wilson]: So, I mean, the tab. [Tim Wilson]: They're being able to do. [Tim Wilson]: More. [Tim Wilson]: But it's like, it feels like unless you have a lot of discipline to say, I'm going to shut off all the notifications.
[Tim Wilson]: You're just constantly getting bombarded. [Tim Wilson]: It's like the next version of email or Slack or whatever. [Tim Wilson]: It's now that junior analyst who's sitting two cubicles over saying, hey, what do you think of this? [Tim Wilson]: Hey, what about this? [Tim Wilson]: That's so good. [Julie Hoyer]: The head just pops up over the wall. [Julie Hoyer]: Hey. [Tim Wilson]: Yeah. [Tim Wilson]: Hey. [Julie Hoyer]: The hand. [Tim Wilson]: And you can be rude to it and say, stop. [Tim Wilson]: Stop it. [Tim Wilson]: And it'll. [Tim Wilson]: At one point, it told me it was like, I'm going to wait for like 30 minutes and then I'll check back in on that. [Tim Wilson]: And like two hours later, I'm like, son of a bitch. [Tim Wilson]: Like, you didn't. [Tim Wilson]: I was like, I thought you were going to wait. [Tim Wilson]: It's like, yeah, I shouldn't have told you that I could do that when I can't.
[Tim Wilson]: I'm like. [Tim Wilson]: It writes that stuff down, Tim, and keeps a record. [Tim Wilson]: So, I don't know. [Tim Wilson]: Is there a piece of that? [Tim Wilson]: That's another concern of like, are we even doing good work? [Tim Wilson]: Or are we at a point where we have this thing that's interrupting? [Tim Wilson]: This thing is constantly. [Tim Wilson]: Anyone? [Michael Helbling]: Yeah. [Michael Helbling]: Well, Tim, I'm assuming like the things you're doing are requiring it to do computer use. [Michael Helbling]: Like it has to go and navigate to specific websites and click on things and stuff. [Tim Wilson]: Yeah.
[Tim Wilson]: I mean, some of it is that stuff. [Tim Wilson]: And I've said, can we front load and get it? [Michael Helbling]: Because that's always going to take forever. [Michael Helbling]: Like, is this just AI is not good at that yet for a lot of stuff? [Tim Wilson]: If there was APIs or CLIs. [Tim Wilson]: I know a lot of times it's just, it's me giving it permissions. [Tim Wilson]: But yeah. [Tim Wilson]: Are we logged into this? [Tim Wilson]: I'm like, but I'm like, you should tell me at the beginning to go and log into these three sites. [Tim Wilson]: So, those are available. [Michael Helbling]: Well, you should just have it write a kickoff script that does all that for you. [Michael Helbling]: That sounds like a engineering thing, Tim. [Michael Helbling]: Oh, yeah. [Michael Helbling]: Yeah. [Tim Wilson]: I'm, if there's anyone listening to this thinking I've got it figured out.
[Tim Wilson]: Yeah. [Tim Wilson]: Let's, we've made that clear that I do not. [Tim Wilson]: So. [Julie Hoyer]: Can I ask this, Tim? [Julie Hoyer]: Like, is there any part of that tedious one hour that, well, I probably know the answer. [Julie Hoyer]: But like, let's pretend we're talking about other tasks, maybe. [Julie Hoyer]: Yeah. [Julie Hoyer]: So, I want to come back to, like, can you make AI do it? [Julie Hoyer]: Sure. [Julie Hoyer]: But to your point, like, is it actually a task that is, like, fulfilling, enjoyable, needs you in the loop so much that, like, that is actually the valuable hour of a human doing it rather than all this pain of letting the AI do it for eight hours?
[Julie Hoyer]: I don't know. [Julie Hoyer]: I feel like sometimes there is this push to be like, well, if AI could do it, like, make them do it. [Julie Hoyer]: Everybody in the organization, use AI somehow. [Julie Hoyer]: Find some way to use it. [Julie Hoyer]: Like you were saying, Danielle. [Julie Hoyer]: Like, I just get to the point where I'm like, should I really be using AI for the token spend and the pain setting it up and telling the silly junior analyst AI how to do it? [Julie Hoyer]: Like, I kind of just want to sit and drink my cup of coffee and do the clicking myself. [Julie Hoyer]: But it's interesting of, like, how do you draw that line? [Julie Hoyer]: Because I also have a whole thought that I'll wait on about, like, who that efficiency is actually then benefiting from AI.
[Tim Wilson]: So, I will, I just want to quickly say this is episode 308. [Tim Wilson]: The joy of that, like, evaporated around, like, episode 120. [Tim Wilson]: So, no. [Tim Wilson]: Fuck no. [Tim Wilson]: I do not want to keep. [Tim Wilson]: But I do think there's a distinction between having it do the agentic, probabilistic figure stuff out versus using it for coding something that then becomes deterministic that has been. [Tim Wilson]: Like, I've gotten my new ATRWconsulting.com.
[Tim Wilson]: Like, I vibe coded, like, websites where it's like. [Tim Wilson]: It's there. [Tim Wilson]: It's coded. [Tim Wilson]: And that was, like, I built something and the end was a deterministic code base. [Tim Wilson]: Now, how that relates to analytics, how much is, like, I'm building a pipeline that I can then refine and tweak and then I've got the SQL that's reusable. [Tim Wilson]: Like, to me, there seems like there's a distinction between the vibe coding to build reusable code or applications, which I think there's.
[Tim Wilson]: That still brings me. [Tim Wilson]: That brings me joy with the million little projects I've done on that front. [Tim Wilson]: The, I'm going to get this junior analyst. [Tim Wilson]: Like, it's a junior analyst who I'm going to throw out a window because I'm like, oh, my God, follow the damn checklist. [Tim Wilson]: Like, you, the sponsors goes below the copy, not above. [Tim Wilson]: Like, it says it right there. [Tim Wilson]: So, boy, I don't know about recording this at the end of this day, but. [Tim Wilson]: So, Daniela, how do we fix this?
[Tim Wilson]: Yeah. [Daniela Matinho]: Oh, my gosh. [Daniela Matinho]: What a roll. [Michael Helbling]: No, but I think we're all touching on things where we all think AI has this real big positive potential. [Michael Helbling]: We're also running into the opportunity cost of how to integrate it effectively in our day-to-day work such that it produces the results that are required of the roles that we have in our jobs.
[Michael Helbling]: And the imperative that this must be something that is disruptive and is changing everything. [Michael Helbling]: So, we must also do it. [Michael Helbling]: And I think it's put a lot of us in sort of like a weird headspace in our jobs and careers in the last couple of years. [Michael Helbling]: I don't know, Daniela, like from your perspective, like in working with teams and coaching people, whatever you're in your roles. [Michael Helbling]: How are you guiding people into how to think about?
[Michael Helbling]: How are you guiding people into how to think about integrating their work in AI? [Tim Wilson]: That's just like a more eloquent way to say, so how do we fix all this? [Tim Wilson]: Like you just, you ask the silly question and then you're like, what if I, what if I give more of like a chat GPT, really long, coherent, but it's the same question. [Michael Helbling]: You're absolutely right, Tim. [Michael Helbling]: You hit on the key issue. [Daniela Matinho]: Well, the thing is, I don't have the answer. [Daniela Matinho]: No one has the answer. [Daniela Matinho]: And what I'm seeing. [Daniela Matinho]: And also because.
[Daniela Matinho]: I work in tech. [Daniela Matinho]: We are in this industry that people just expect you to know the answer. [Daniela Matinho]: Well, you guys are driving the thing. [Daniela Matinho]: And very often we are all just trying to figure it out. [Daniela Matinho]: That's my, my take. [Daniela Matinho]: And one thing I think that tech does very well is the ability to learn very quick and also course correct very quick. [Daniela Matinho]: So as we integrate different AI tools to different companies that I have worked at. [Daniela Matinho]: So very quickly we start sort of mapping what we are using for what?
[Daniela Matinho]: For dashboarding, for coding, for narrative producing internally for video editing. [Daniela Matinho]: So having that sort of framework that enable us to, to know what to use for. [Daniela Matinho]: Also, a lot of the sharing coming from the teams that are closer to, to clients or the more technical teams. [Daniela Matinho]: A lot of the sharing is happening in these early stages of, Hey, we are discovering this and this is how we use it and how we can also teach other teams.
[Daniela Matinho]: Those are things that I have been seeing for myself. [Daniela Matinho]: And I go back to this as I, um, I've been a power user for, for now a year and a half understanding that more. [Daniela Matinho]: Is not necessarily better. [Daniela Matinho]: That's what I'm coming. [Daniela Matinho]: And we are, we have touched on these a few times. [Daniela Matinho]: I was very bullish on implementing, implementing, implementing, and now I'm coming to realize, well, maybe there is a lot of opportunity in slowing a little bit more.
[Daniela Matinho]: And doing extremely quality deliveries. [Daniela Matinho]: Um, rather than just quantity, because I'm coming to see, well, all these agents, all these models can build way more. [Daniela Matinho]: But it's not having the impact that it used to have when I was doing it, everything from scratch. [Daniela Matinho]: So kind of going back or taking a step back on, okay, I'm using it is helping me to produce, but I still have to put the 10 hours off. [Daniela Matinho]: Deep thinking. [Daniela Matinho]: This is how I build the story.
[Daniela Matinho]: This is how I use the three, four numbers that are really going to break through that. [Daniela Matinho]: Maybe my agent in marketing science cannot really get it without having met the client. [Daniela Matinho]: Even if, you know, I have a call that is being transcribed, so on and so forth. [Daniela Matinho]: There is still, I mean, hopefully my brain is being used for something. [Michael Helbling]: That's a really good point about the type of industry you're in and how that might actually, because you were talking about sort of in your role and sort of tech, it's like very fast moving, quick decisions, knowledge and learning is super fast.
[Michael Helbling]: Like I have some clients that are in more regulated industries and like there's things you can't touch with AI. [Michael Helbling]: Like they just like, sorry, no note taking. [Michael Helbling]: Like that's not allowed. [Michael Helbling]: Like, so it's like, okay, you have to put those tools to the side and you do it the old school way. [Michael Helbling]: And it's kind of interesting because you do, there's different flavors or layers depending on what type of company you work with or what industry you work in. [Michael Helbling]: And so I think for listeners, that's also maybe like a little bit of encouragement, like depending on the type of company, like you might not be lagging.
[Michael Helbling]: You might be right where you need to be for where you work and what your company is. [Michael Helbling]: And if you feel like you're going too slow, then go find a new job in tech and go fast. [Daniela Matinho]: Well, there are a lot of things that, you know, from the industry. [Daniela Matinho]: But if you feel like, oh, maybe my industry, my company is not there yet, is the opportunity for you to be the driver. [Daniela Matinho]: Because I was in the past at companies that weren't maybe in that front end. [Daniela Matinho]: And I was the one, let's do it, let's implement it.
[Daniela Matinho]: And all of a sudden I became the person. [Daniela Matinho]: Oh. [Daniela Matinho]: I'm going to go and ask Danielle about this because she's implementing it. [Daniela Matinho]: And maybe, oh, our decks look amazing. [Daniela Matinho]: And I'm like, yeah, AI is doing a lot of them for me. [Tim Wilson]: I kind of heard something a little different, which now seems like totally obvious. [Tim Wilson]: Like everyone should be exploring, but we should be going in saying, I'm going to try to do this with AI. [Tim Wilson]: But logically, it seems like it might work. [Tim Wilson]: But I'm doing two things.
[Tim Wilson]: I'm trying to complete the task. [Tim Wilson]: But I'm trying to learn. [Tim Wilson]: I'm trying to learn if this type of task is a good fit. [Tim Wilson]: And that little kind of meta-analysis level of, for you personally, anyone, I'm not speaking, that what are the types of things you've tried? [Tim Wilson]: Are you trying different things? [Tim Wilson]: Are you assessing after you've done them, like trying to figure out, hey, that kind of worked. [Tim Wilson]: And I think here's why it worked. [Tim Wilson]: And here I could work better. [Tim Wilson]: Or some of the examples, I think, Julie, you were thinking of.
[Tim Wilson]: What the? [Tim Wilson]: What the hell? [Tim Wilson]: Like, sure, I used AI to accomplish that. [Tim Wilson]: But it took me longer. [Tim Wilson]: And it cost the company extra money because of token usage. [Tim Wilson]: And you know what? [Tim Wilson]: It's not just the next model is going to solve that. [Tim Wilson]: Maybe that's actually something I should keep doing. [Tim Wilson]: Like I haven't stepped back and said, let me make my catalog of the things I've done and tried to classify them. [Tim Wilson]: And then assess, like, man, it was amazing at this. [Tim Wilson]: Including totally new things.
[Tim Wilson]: Like ideation around an analysis, which I think it does quite well. [Julie Hoyer]: It all makes me think, too, of how are we really defining the efficiency play? [Julie Hoyer]: Like, I know a lot of people talk about it just as, like, time saved. [Julie Hoyer]: And there's a lot of assumptions behind, like, what time saved means and the value of that. [Julie Hoyer]: But exactly what you were saying is if the end quality is not as good or it's not as influential what I'm delivering, even though I did 10 of them. [Julie Hoyer]: It's just I feel like a lot of times people aren't taking into consideration all of those measures to really say, was I more efficient?
[Julie Hoyer]: Or maybe that dips more into effectiveness. [Julie Hoyer]: Like maybe they're not looking at both efficiency and effectiveness when using AI. [Julie Hoyer]: And it would be a really interesting exercise to your point, Tim, to really be like, how would I measure those things? [Julie Hoyer]: And if I look at all the things I've done with AI, like, which ones were truly, like, worth it and successful? [Tim Wilson]: Yeah. [Tim Wilson]: And the ones that were really fun, but they actually were a complete waste of time. [Tim Wilson]: You're like, yeah, but it was. [Daniela Matinho]: Yeah. [Daniela Matinho]: But it was so cool. [Daniela Matinho]: So it's a really interesting point.
[Daniela Matinho]: And it's something that I was seeing a lot maybe, what, a year ago when AI was really hyping. [Daniela Matinho]: Like we were seeing in a lot of the calls, quarterly calls when public companies have to report. [Daniela Matinho]: We were seeing it X millions, billions in AI, X prompts being digested. [Daniela Matinho]: And number of people that we were hiring. [Daniela Matinho]: These were a lot of, like, very concrete numbers that I just, from my perspective, companies maybe were having a lot of pressure in delivering, oh, we are an AI-driven company.
[Daniela Matinho]: So we were seeing a lot of these things. [Daniela Matinho]: And to your point, Julie, right now that we have, I don't want to, I don't know if we have passed completely that phase, but now that AI is being integrated in the majority. [Daniela Matinho]: Of companies is more of, oh, what are my, what am I actually delivering? [Daniela Matinho]: Or what is the revenue that I'm actually driving because I was able to use AI? [Daniela Matinho]: Because at the end of the day, if I go and tell a client, well, I did this in two days instead of a week because of AI, the client doesn't care.
[Daniela Matinho]: Right? [Daniela Matinho]: That doesn't really matter. [Daniela Matinho]: Or, you know, at the end of the day, what the client wants is. [Daniela Matinho]: A good, a good output, a good result that is proving revenue for whatever I'm trying to, or whatever I'm trying to deliver. [Daniela Matinho]: So AI really is, is the underlining sort of army that is helping us to be a little bit more efficient, but in a way to deliver revenue. [Daniela Matinho]: At least I'm close, I'm close to sales in general.
[Daniela Matinho]: So I always think about, how can I drive more dollars? [Michael Helbling]: Yeah. [Michael Helbling]: Well, and then being more efficient, we have to remember that it's also our job to stay influential, right? [Michael Helbling]: With data. [Michael Helbling]: So that's the balancing act that doesn't show up in the numbers a lot of times. [Tim Wilson]: But that, that you say stay influential. [Tim Wilson]: I think, I think there, we're not going to AI your way out of not delivering value. [Tim Wilson]: Like I, I think when analysts kind of wring their hands and gnash their teeth and say, I don't have a seat at the, at the, the board table.
[Tim Wilson]: And then they're like, now I'm going to use AI. [Tim Wilson]: It's like, well, okay, that's going to help you produce more. [Tim Wilson]: Maybe it's going to help you be more consistent, but it's like, you, you still need to figure out like, what is the path to get there? [Tim Wilson]: Like, I, I definitely worry about that. [Tim Wilson]: That it's like, well, here's the thing it's going to solve. [Tim Wilson]: There's fundamental relationship and ways of working and thoughtfulness, things that AI is just inherently not going to solve. [Tim Wilson]: Unless you read it.
[Tim Wilson]: You recognize it and maybe AI can coach you and help you learn it. [Tim Wilson]: But I think that's another, I'm thinking about, we saw Sam Burge, a past coworker of ours present at Marketing Analytics Summit. [Tim Wilson]: And I've continued to talk to her and part of what she's in her team are using AI for is different mechanisms for delivering information. [Tim Wilson]: So even if fundamentally the information they're delivering is the same.
[Tim Wilson]: And I think that's a really good point. [Tim Wilson]: They're spending maybe the same amount of time, maybe a little more, maybe a little less and saying, but I'm going to tailor it to the audience. [Tim Wilson]: Maybe I'm going to adjust the, you know, people in the past when he said, I only have time to produce the deck. [Tim Wilson]: So that deck has to be the pre-read the leave behind what I present. [Tim Wilson]: But now I can produce the full detailed report and I can produce the executive summary and I can generate, you know, an audio podcast. [Tim Wilson]: That's three. [Tim Wilson]: That's three minutes that delivers it and really thinks about how am I serving my business partner more effectively, even though potentially the raw, what is being delivered hasn't changed.
[Tim Wilson]: It's helping deliver it more effectively to the audience, which is like a really like thoughtful way to think about it. [Tim Wilson]: And that, I think she's had a lot of tried this. [Tim Wilson]: It didn't work, tried this. [Tim Wilson]: It did work, which goes back to the point of saying, I'm not just going to turn it on and it's going to work. [Daniela Matinho]: Well, a lot of iterations. [Daniela Matinho]: What I'm seeing with content. [Daniela Matinho]: And this is something that, um, a few months ago, conversations with, with colleagues, with friends. [Daniela Matinho]: Well, we can just now do all these videos, all these posts with AI. [Daniela Matinho]: It's, you know, quantities is amazing.
[Daniela Matinho]: We can all become influencers. [Daniela Matinho]: And, you know, a few months down the road is like, well, no one has become an influencer because we all just talking the same and speaking this, you know, using the same words because everything is being produced. [Daniela Matinho]: So that level of how do we differentiate to your point team? [Daniela Matinho]: How are we touching the audience the way we need? [Daniela Matinho]: How are we shaping it? [Daniela Matinho]: That's the layer that in my perspective, AI is just not there yet.
[Daniela Matinho]: And when hopefully our brains and our creativity is, is being super needed to, to really stand out in a way. [Tim Wilson]: It's one of those philosophy PhDs are using AI to read. [Tim Wilson]: They read Nietzsche and produce their thesis, and then they actually get a philosophy PhD, but they don't have the critical thinking. [Michael Helbling]: All right. [Michael Helbling]: This is great. [Michael Helbling]: Daniela, thank you so much for sharing your experience and expertise with us and our listeners. [Michael Helbling]: This has been an awesome, good discussion. [Michael Helbling]: Very fun to kind of open this up.
[Michael Helbling]: And I think it's very applicable to everyone's jobs right now. [Michael Helbling]: So, yeah, but we do have to start to wrap up. [Michael Helbling]: Unfortunately, the time flies by. [Michael Helbling]: All right. [Michael Helbling]: Uh, one of the things we like to do. [Michael Helbling]: Is go around the horn and share a last call, something that might be of interest to our audience. [Michael Helbling]: Daniela, you're our guest. [Michael Helbling]: Do you have a last call you'd like to share? [Daniela Matinho]: Yeah. [Daniela Matinho]: Um, I was on a call yesterday with IAB, the advertising bureau, and we were talking about agentic commerce. [Daniela Matinho]: So very, very curious as we were talking, uh, right in the beginning of this call, what are, what are the automation tasks that we're going to be seeing?
[Daniela Matinho]: And I think a lot of the shopping. [Daniela Matinho]: Online, especially the recurrent shopping is going to completely change the way we shop the way you use. [Daniela Matinho]: We use LLMs to inform us and to basically place everything for us and order it. [Daniela Matinho]: I think it's completely going to change the way we shop online and I'm excited about it. [Tim Wilson]: Is that I was having this, I got a, sorry, Michael, but I'm like, this is like getting people to part with their dollars. [Tim Wilson]: Like in, when people who are used to.
[Tim Wilson]: Like hallucinate hallucinations, some bad judgment calls. [Tim Wilson]: Cause I feel like they're in the digital analytics world. [Tim Wilson]: There's been a whole lot of like, oh, this is coming. [Tim Wilson]: Like the agents are going to be sent to get shopping. [Tim Wilson]: And I'm like, like how much is the IAB? [Tim Wilson]: And are you thinking that that's going to be, I may send an agent out to find a lot of options, but I don't know that I'm going to cut it loose to make the purchase decision. [Daniela Matinho]: Yeah. [Daniela Matinho]: That's what we are. [Daniela Matinho]: Seeing, but not yet for the, to pull the trigger, but a lot of the search.
[Daniela Matinho]: A lot of the search is happening as we were doing it. [Daniela Matinho]: Um, having one person or one agent that is basically pulling everything for us based on what they know about us. [Daniela Matinho]: I think the next stage to your point, it will be interesting to see how long it's going to take. [Tim Wilson]: I just see like retailers who are saying we've, we've got to be supporting, supporting agentic commerce. [Tim Wilson]: And I'm like, is it the purchase you're trying to support? [Tim Wilson]: Or is it, you're just trying to surface. [Tim Wilson]: The shopping, the shopping versus the purchasing. [Tim Wilson]: Hey. [Tim Wilson]: I don't know. [Michael Helbling]: DoorDash already has a command line interface.
[Michael Helbling]: So, so everybody's doing it or starting to. [Michael Helbling]: The DoorDash MCP. [Michael Helbling]: All right. [Michael Helbling]: That's a whole other podcast episode, Tim. [Michael Helbling]: Why don't you share your last call, Tim? [Tim Wilson]: Okay. [Tim Wilson]: Well, mine is a podcast. [Tim Wilson]: It's an old podcast that's new again. [Tim Wilson]: And I am super excited, but I'm also like a political junkie. [Tim Wilson]: But the, the five. [Tim Wilson]: FiveThirtyEight crew of Nate Silver and Claire Malone and Galen Druk. [Tim Wilson]: The they're, they're back. [Tim Wilson]: So like FiveThirtyEight kind of did it's bouncing around through ESPN and the New York Times and was gone, but they, there's now they're back on the Still Counting podcast there with the Crooked Media network.
[Tim Wilson]: So it's I mean, it's clearly political stuff, but really, really sharp data minds. [Tim Wilson]: And I, I am. [Tim Wilson]: I'm enjoying the first few episodes quite a bit. [Tim Wilson]: So basically FiveThirtyEight podcast is back, but now it's called Still Counting and it's in a different world. [Julie Hoyer]: Mine is actually very applicable to today's episode. [Julie Hoyer]: It was a Cassie Kozyrkov newsletter.
[Julie Hoyer]: We love those. [Julie Hoyer]: And the title was a really interesting statistic and it's chock full of them, but it's all about how 97% of S and P 500 filings mention AI. [Julie Hoyer]: But you wouldn't believe the percentage that actually like had written out budgets for them. [Julie Hoyer]: And even fewer actually describing like how they govern them. [Julie Hoyer]: And so she goes through some of these crazy statistics. [Julie Hoyer]: So highly recommend looking at it. [Julie Hoyer]: And then at the end, she's kind of closing it up with this idea of like the winning companies of using AI are probably going to be the ones who trust their systems the least.
[Julie Hoyer]: And I know that's been just a big theme of like, you can't just hope. [Julie Hoyer]: That they're doing the right thing. [Julie Hoyer]: And like that sense of goodwill towards them is really kind of been eaten away through a lot of experiences the past year, year and a half. [Julie Hoyer]: So it was a, it was a good quick read. [Tim Wilson]: What about you, Michael? [Michael Helbling]: Well, somewhat related to this episode, there was a website that came across recently that I thought was clever. [Michael Helbling]: And of course it's a vibe coded website called the archive of incorrect AI predictions.
[Michael Helbling]: So it has this whole archive [Michael Helbling]: of people making predictions [Michael Helbling]: about AI that have been wrong [Michael Helbling]: throughout history. So way back in the past [Michael Helbling]: coming forward in time. [Michael Helbling]: And you can look at it in different ways. It's kind of fun. [Michael Helbling]: So yeah, [Michael Helbling]: it's a fun little website. [Michael Helbling]: I like the name of it. It's [Michael Helbling]: Boys Who Cried AI. [Michael Helbling]: It's the name of the website. [Michael Helbling]: So anyways, that's a, that's a fun one. [Michael Helbling]: All right. [Michael Helbling]: So as you've been listening, uh, you've probably have thoughts of your own and we'd love to
[Michael Helbling]: hear your comments and suggestions. [Michael Helbling]: Leave us a comment. [Michael Helbling]: The best way to do that is on the measure Slack chat group or our LinkedIn page or via [Michael Helbling]: email and contact at analyticshour.io. [Michael Helbling]: And we'd love to hear from you. [Michael Helbling]: Also, uh, please, uh, wherever you listen, leave a rating and review on the podcast that [Michael Helbling]: helps us out. [Michael Helbling]: And if you would like a sticker for your laptop or your mug or water bottle, we'll send you [Michael Helbling]: one and you can do that on analyticshour.io as well.
[Michael Helbling]: And once again, Daniela, thank you so much for taking the time being here today. [Michael Helbling]: It's been an awesome conversation, very timely. [Michael Helbling]: I think everyone's work is feeling this impact. [Michael Helbling]: So thank you for coming and sharing some time with us today. [Daniela Matinho]: Thank you for having me. [Michael Helbling]: Awesome. [Michael Helbling]: And I think I speak for both of my co-hosts, Julie and Tim. [Michael Helbling]: When I say no matter where you are in AI adoption, keep analyzing. [Announcer]: Thanks for listening. [Announcer]: Let's keep the conversation going with your comments, suggestions, and questions on
[Announcer]: Twitter at analytics hour on the web at analyticshour.io our LinkedIn group and the [Announcer]: measure chat Slack group music for the podcast by Josh Crowhurst. [Announcer]: Those smart guys wanted to fit in, [Announcer]: so they made up a term called analytics. [Announcer]: Analytics don't work. [Charles Barkley]: Do the analytics say go for it [Charles Barkley]: no matter who's going for it? [Charles Barkley]: So if you and I were on the field, [Charles Barkley]: the analytics say go for it. [Charles Barkley]: It's the stupidest, laziest, lamest thing [Charles Barkley]: I've ever heard for reasoning in competition.
[Tim Wilson]: Staying in detail and be like, [Tim Wilson]: oof, maybe I would have done it so much better. [Tim Wilson]: She's been doing that throughout the course of the planning. [Tim Wilson]: Maybe. [Tim Wilson]: That's an interesting choice you guys made with how to do it this time. [Michael Helbling]: I think what you should do is actually take that audio snippet and send it to Val and see if that's accurate. [Tim Wilson]: Well, all through the planning, as she was trying to watch from afar and be hands off, I would often get the, I was the vent, the outlet for her.
[Tim Wilson]: Oh. [Tim Wilson]: Rock flag and philosophy majors.
It can be hard to hear an AI story these days without wondering how much of it is sensationalized. So, we went looking for some reality on the ground. Moe rounded up four of her colleagues from Canva - Christian...
Here is an interview question for you: tell us about a time you knew exactly how to do something, and then completely failed to explain it to three strangers in a conference room. If an example sprang to mind...
There's a version of this episode where AI replaces the data visualization expert entirely, and there's a version where it's useless without one. Neither is quite right, according to Cole Nussbaumer Knaflic of...
Before there was BI, there were "decision support systems." Somewhere along the way, we seem to have quietly dropped the "decision support" part and just kept the systems. Zohar Strinka, founder of Analytics Strategies...
Stephen Follows has spent 15 years turning film-industry curiosity into a career - asking questions like how much movies really earn, why poster colors have shifted over decades, and whether old studio domains are still...