Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Value Driven Data Science

Value Driven Data Science artwork

Episode 122: [Value Boost] 4 Questions Every Data Scientist Should Ask About AI Strategy

Sep 9, 202612 minAudio
ShareXLinkedInFacebookWhatsAppEmail

Choosing which AI tools to buy is the easy part of an AI strategy. The hard part - and the part most organisations are ignoring - is everything that sits beneath the tools, from platform and inference decisions to hardware and sovereignty. In this Value Boost episode, Victor Coimbra joins Dr Genevieve Hayes to share his framework for cutting through the noise of AI tool selection to the strategic decisions that will actually determine whether an organisation's AI future succeeds or fails. You'll discover: The four layers of an agentic AI strategy and why most organisations only think about one [01:53] The three symptoms that signal an organisation's AI strategy is breaking down [05:52] The four questions that reveal what an organisation's AI strategy is actually missing [08:21] The one question data scientists should lead with when advising stakeholders on AI [10:43] Guest Bio Victor Coimbra is a Partner and CTO at Artefact, the world’s largest pure-play AI consulting firm and co-founded the firm’s Latin American operations. In 2024, he was recognised in the Forbes 30 Under 30 Brazil list for his outstanding contributions to AI innovation. Links Connect with Victor on LinkedIn Victor's article on the four-layer framework Artefact website Connect with Genevieve on LinkedIn Be among the first to hear about the release of each new podcast episode by signing up HERE

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show.

[00:00:00] Dr Genevieve Hayes: Hello, and welcome back to Value-Driven Data Science, where data professionals become strategic experts. I'm Dr. Genevieve Hayes, and I'm here again with Victor Coimbra, Global AI and Data Engineering Lead at AI consultancy Artifact. Last week, Victor and I discussed how the shift to hybrid human AI agentic operating models fundamentally changes what organizations need from their data professionals. [00:00:32] Today, in this Value Boost episode, you'll learn a four-layer framework you can use to diagnose where an organization's AI strategy is breaking down and discover where the real strategic decisions of the next two years will be made. Welcome back, Victor. [00:00:50] Victor Coimbra: Very glad to be here again [00:00:52] Dr Genevieve Hayes: Many organizations currently have AI strategies that read a lot like shopping lists. [00:00:58] To them, AI is just another software decision to be approved. Their biggest concerns are whether to choose Anthropic versus OpenAI, and how many Copilot seats they should add to their existing Microsoft bill. However, when it comes to rolling out agentic AI, the application layer is just the tip of the iceberg, and without considering what's below the surface, these organizations are setting themselves up for a world of pain. [00:01:27] Fortunately, this pain is something data scientists are well-placed to help their stakeholders avoid, but only if they know the right questions to ask. Victor, you recently wrote an article in which you outlined a four-layer framework that describes what organizations should really be considering when building their agentic AI capabilities. [00:01:49] Can you walk us through what each of those layers involves? [00:01:53] Victor Coimbra: For sure. So I like to think as a iceberg, as you said, doctor, the first layer is the application layer, and that's what everyone sees, when you look at the iceberg [00:02:03] so what tool I'm using. Do I am using Claude? Do I'm using CoWork? Do I'm using Copilot, and that's easy to see, but like in any iceberg the true nature of the work and the strategic impact is gonna be on the other layers that are below the sea. And the first layer below the sea is what we call the AI platform. [00:02:26] And in here lives three big things. The harness, the context, and the observability layer. That's your AI platform. Is where you're gonna decide how to manage the power of the LLMs with the harness. Is how you're gonna structure the context of your company, and context in here is super important. [00:02:45] Context is not the same thing as data. Data is maybe 10% of the context that you produce in your company. And finally, the observability layer, where I gonna understand the token lineage, where people actually spend the token in, the token out. And these three things help us to understand the full picture of what you are actually doing on your company. [00:03:09] So we are still on the very surface. This is still something that we deep dive a bit on the second layer. But there is two more below this that's become harder, that's your inference layer, and in here is how you control your inference token, is that you're gonna use open source model, you're gonna build your own models, you're gonna use something that's besides the LLM. [00:03:29] For example, visual models, word models. And then the fourth layer, and the most important one is the hardware layer, how do I control my hardware? And what I mean by hardware is how much do I have the power to actually produce intelligence, [00:03:45] and you're gonna see that most of the advanced big tech companies like SpaceX, Meta OpenAI, Microsoft, Google, they all invest a huge amount of money on data centers, and the reason for that is because they want to control this hardware layer, it's gonna be almost the electricity layer. [00:04:04] For some companies, might not be the goal,. But for a company that wants to be very tech-driven and AI-driven, at least have a strategy of how to handle these four layers, for some layers, you can outsource But other layers you need to at least outsource in a conscious way. [00:04:22] And today, I see that is most ignore and people don't take this as a conscious decision and only choose to see the tip of the iceberg and not actually deep dive on everything that's below the sea [00:04:36] Dr Genevieve Hayes: Why do you think that's the case that all these companies are focusing on the tip of the iceberg given that those bottom three layers are clearly critical from a strategic point of view? [00:04:47] Victor Coimbra: I think is where it shines the most, and I think we naturally are driven by short-term gains. And that's the truth of humanity, and I think [00:04:56] corporate tends to always look on the short term, and the true value is on these long-term gains. And technology prove us that, I have this phrase that I really like that's never bet against technology, because in the long term, technology always win, [00:05:13] and sometimes we like to bet against technology because we like short-term gains, but true transformation takes time, and true transformation takes effort for you to deep dive, open the black box, and actually see what's going on down there [00:05:30] Dr Genevieve Hayes: As I mentioned in the intro, if an organization builds their entire AI strategy at the application layer, then they're setting themselves up for failure. When that inevitable failure does occur, is there any way a data scientist can use this four-layer framework to diagnose where an organization's AI strategy is breaking down? [00:05:52] Victor Coimbra: The first thing is to identify when the company's failing. And I think there's three main symptoms that proves that probably you are take the wrong decision, the wrong layer. The first symptom is that people are not adopting your tool or your technology, like they are actually not using. [00:06:10] And sometimes we think it's because of the people that we have, but actually it's a totally different opposite way. Probably the tool that we got is not the best thing for the job. The second symptom is a governance problem, and we're gonna see a lot this in the very near future some leakage, some people upload something that should not, some decision that AI took that should not be revised by a human. [00:06:35] It's very important for us to remember that AI cannot get fired, and I think sometimes we forgot that, and I think that's the second symptom, when we start to have security and governance problem. And the third symptom, and usually is when people start to act, is the explosion of cost and tokens. [00:06:51] Uber, for example, spent in three or four months, the entire budget for the entire year in tokens, this is probably the first three signs of the apocalypse, the first three signs that probably the company's, is taking the wrong decisions. And when a data scientist identify those, the first thing is to understand, okay, on these four layers, where we don't have anything, and they're gonna figure out probably on the first layer below the sea, that's the AI platform, you probably don't have anything, [00:07:20] because nobody thought about building a AI platform in the first place. They just thought about buying license. And then they're gonna start to dig and th- gonna start to figure out that nobody thought about other models besides LLMs, and there is, there is fine-tuning. There is ability to open source models. [00:07:37] There is ability for to explore new things, and this is gonna figure out that this takes and needs computing, and I think that's where people go to the fourth layer. So I think that's super important, and I think the first symptom for leaders that are listening to us as well, is that when your team stop to work because Claude got out of service, you probably have a problem right now, [00:08:01] because your entire operation cannot be hostage of a thing or a solution that you do not control today. So you have to have at least a strategy of how to tackle that. [00:08:13] Dr Genevieve Hayes: In your article, you reduced each of the four layers down to a single strategic question. What are those four questions? [00:08:21] Victor Coimbra: The first question, the first application layer is for you to understand what people actually use AI. And I think on the previous episodes that we discussed, that's where the big question is. Identify the archetypes of usage that AI itself how do I use AI? [00:08:37] The second one, the platform layer for me, is do I have visibility where my token is being spent? And I bet 80% of the companies do not know, they buy tokens, but they, do not know where they are spending these tokens. [00:08:53] It's the same thing as you buy electricity or petroleum or oil, and you never know when to use it, it's the same rational, I don't buy electricity if I not know where exactly I'm gonna use. [00:09:03] And then the third one is, do I control my intelligence? Do I control what the models are actually producing? And you're gonna figure out that probably nobody actually control that. You are probably waiting for Claude to update their models to be better, and you're actually not thinking about a true intelligence strategy [00:09:22] it's not about use the most intelligent model, but about to use the best model to the best task. Sometimes it's Claude, sometimes it's an open source model, sometimes a model that some random person built in the academic world across the globe. Do you control your output of intelligence? [00:09:39] Probably not. And for me, the fourth one is, in the hardware layer, is do I control where my intelligence is created? And probably control part, that's when the intelligence created inside of your human employees, much more creative intelligence. But you probably do not control where your intelligence created when we're talking about agents and AI. [00:10:03] And here there's a big topic about sovereignty. If you do not control where your intelligence is created, tomorrow there is probably a regulation, or there's probably a new law that's very new that can block it, the, your entire operation from scratch, because we don't know where this actually is produced. [00:10:24] It's the same thing as electricity, you need to control, as a company, where do you produce probably the most important resource that you're gonna use that is intelligence, I think that's the main questions for each one of those layers. [00:10:36] Dr Genevieve Hayes: And if a data scientist only had time to ask their stakeholders one of those four questions which would you tell them to lead with and why? [00:10:43] Victor Coimbra: The first one, and the reason for that is because it's where they're gonna be most impact in terms of actual work. I think it gonna take some time for leaders and companies to understand the other layers. But today, if you need to ask one question, the question should be where and how should I use AI on the best way possible? [00:11:07] Because you're gonna see that different ways of using AI are gonna require different skill sets that you as a data scientist needs to develop. [00:11:15] Dr Genevieve Hayes: And that's it for today's conversation with Victor. If you haven't already, listen to our previous episode where we discussed what hybrid human AI agentic organizations mean for data science. Thanks for joining me again, Victor. [00:11:31] Victor Coimbra: Thank you very much. The pleasure is mine [00:11:33] Dr Genevieve Hayes: And for those in the audience, thanks for listening. I'm Dr. Genevieve Hayes, and this has been Value-Driven Data Science.

Open the transcript file

Hosts and Guests

Hosts

  • Genevieve Hayes Host Feed-stated

Guests

  • Victor Coimbra Guest Feed-stated

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Episode 132: [Value Boost] Making a Causal Argument Without Experiments

    Nov 18, 202615 minAudio

    Almost every analysis a data scientist presents makes a claim about cause and effect, whether they say so or not. But most organisations don't have the data, time or budget to validate these claims by running a formal...

    Credit the people on this episode

  2. Episode 131: Building a Causal Mindset in a Predictive World

    Nov 11, 202627 minAudio

    A machine learning model can tell you which of your customers are likely to cancel. It can't tell you what would happen if you intervened to change that. Prediction and causation are two completely different problems,...

    Credit the people on this episode

  3. Episode 130: [Value Boost] The Digital Twin Blueprint

    Nov 4, 20269 minAudio

    Building a digital twin without a clear objective is like showing up at a construction site without a blueprint. You might produce something, but it's unlikely to be structurally sound, and it will certainly cost more...

    Credit the people on this episode

  4. Episode 129: Building Digital Twins Executives Can Trust

    Oct 28, 202627 minAudio

    When the cost of failure is low, experimentation is often the best way to make decisions. But when a wrong call could cost millions or endanger lives, you need an entirely different approach. This is where digital twins...

    Credit the people on this episode

  5. Episode 128: [Value Boost] The Three Types of Memory Every AI Agent Needs

    Oct 21, 202613 minAudio

    One of the most frustrating things about working with AI is when it seems to suddenly forget everything it knew about you and your work. This is annoying when using AI chatbots. When building AI systems, it can be...

    Credit the people on this episode