Data Engineering Podcast

Data Engineering Podcast artwork

Why Metadata Still Matters: From Data Catalogs to AI-Ready Semantic Hubs

1 hr 25 minAudio

Summary In this episode, I spoke with Christian Bremeau and Simon Dynin about their decades-long work in metadata management and how that experience shaped the evolution of Meta Integration Technology and its direct-to-customer MetaKarta platform. We explored the recurring challenges of fragmented data ecosystems, why every tool inevitably creates its own metadata silo, and why organizations still need a vendor-neutral system to stitch lineage, semantics, and change history across databases, ETL/ELT pipelines, BI tools, and cloud platforms. Christian and Simon explained why deep lineage, versioning, configuration management, and deterministic understanding of transformations matter far more than a polished catalog UI when teams need to trust numbers, assess impact, or satisfy audits. We also dug into the growing importance of metadata in the age of AI. Christian and Simon shared their view that metadata management is becoming foundational for both BI and agentic systems, especially as organizations struggle with semantic drift, tool sprawl, and non-deterministic AI behavior. They introduced Metacarta’s new Semantic Hub as a way to reconcile business logic across legacy and modern tools, compile shared semantics into downstream systems, and provide consistent answers for both human analysts and AI agents. Along the way, we discussed metadata ops, the organizational processes needed to make metadata programs succeed, and the practical situations where a cross-platform metadata system is - or isn’t - the right fit. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Today’s episode is sponsored by Parallel - where agents find answers. Most engineers today closely follow new model releases, but don’t pay attention to their agent’s most important tool: web search. Parallel develops enterprise-grade infrastructure for agents to retrieve high-quality context from the web. Their core products are a suite of APIs for retrieving high quality information from the web with Pareto-optimal quality, cost, and speed. Whether you work on voice agents that need 200 millisecond latency, chat bots that balance speed, depth, and quality, or long-horizon agents to do thorough, overnight research for you, Parallel is a single platform for all your agentic research. Get started for free at dataengineeringpodcast.com/parallel Your host is Tobias Macey and today I'm interviewing Christian Bremeau and Simon Dynin about his long history of dealing with metadata at Meta Integration Technology, and how they are adapting to the age of AI Interview Introduction How did you get involved in the area of data management? Can you start by sharing some of the highlights of your career in metadata and how the industry's focus on that subject has changed over the years? What are the major sources of friction and fragmentation in metadata management in your experience? There are numerous metadata management, metadata integration, data catalog, and other governance/discovery/semantic platforms in the ecosystem. How do you explain the differentiating factors of your system to people who are evaluating their options? One of the constant challenges in data systems is the tension between data locality and data consistency. What is your approach to managing a canonical, reliable definition of a data asset, while making it available and accessible to all of the consuming systems? There was a surge of interest in metadata systems in the 2018 - 2022 time-frame as the industry invested in cloud data warehouses, ELT, and all of the other "modern data stack" patterns. Now AI is driving a new surge of interest. What are the categorical differences between these periods in your experience? What are the new stressors that you anticipate from the continued growth in LLM-mediated and agentic data interactions? Can you describe the overall architecture of the MetaKarta platform and how it has evolved in the latest version? For a team who is trying to tame their data estate, what does the adoption of MetaKarta look like? How does it change their day-to-day work? You're promoting the idea of "MetaDataOps" as a dedicated concern for an organization. What are the skills and activities of someone who is focused on that capability? What are the most interesting, innovative, or unexpected ways that you have seen MetaKarta used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Metadata and MetaKarta? When is MetaKarta the wrong choice? What do you have planned for the future of MetaKarta? Contact Info Christian LinkedIn Simon LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Links Meta Integration Technology MetaKarta) Cognos Cognos Framework Manager Informatica ER Studio Talend Data Catalog ETL == Extract, Transform, Load ELT == Extract, Load, Transform ThoughtSpot The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

Tobias MaceyHello, and welcome to the Data Engineering Podcast, the show about modern data management. Today's episode is sponsored by Parallel, where agents find answers. Most engineers today closely follow new model releases, but don't pay attention to their agent's most important tool, web search. Parallel develops enterprise grade infrastructure for agents to retrieve high quality context from the web. Their core products are a suite of APIs for retrieving high quality information from the web with Pareto optimal quality, cost, and speed. Whether you work on voice agents that need two hundred millisecond latency, chatbots that balance speed, depth, and quality, or long horizon agents to do thorough overnight research for you, Parallel is a single platform for all of your agentic research.

Get started for free today at dataengineeringpodcast.com/parallel. Your host is Tobias Maci, and today I'm interviewing Christian Brameau and Simon Dinan about their long history of dealing with metadata at Medicarda and how they're adapting to the age of AI. So, Christian, can you start by introducing yourself?

Christian BremeauSure. This is Christian Bremmaud. I'm the founder of Meta Integration Technology, which is the name of a company, now offering direct, to customers under the brand Metacarta. So we'll tell you more about it and how all this came together.

Tobias MaceyAnd how about you, Simon?

Simon DyninYeah, Christian and I have been doing it for probably too long. We're kind of partners in crime. I'm taking care of technical side of the house. I'm CTO of the company. And yes, it's been almost like one, two, pushing three decades.

Tobias MaceyGoing back to you, Christian, do you remember how you first got started working in the overall space of data and some of the highlights of your career working in metadata?

Christian BremeauRight. So that's going to be a long story to tell a little bit. That first question is going to take quite some time. But I think it's important to understand where we're coming from, to see where we're going on that. So I could have done many things around in my careers, but no, just after college, my first job was about metadata. And that was, you know, over thirty years ago, as it happens. And I was in Europe at the time in France and working on a lot of European project about metadata standards.

Basically, that was all about exchanging metadata between different tools and all about having a common metadata repository in Europe. And that involves standards such as ECMA in Europe and obviously ECMA works with ISO and ISO standards on metadata. That got us to work a lot with The United States and I came in US in '92 and worked again on standards. And that's exactly,

the bootstrap of the metadata with the company. So the company Meta Integration Technology was created in '97. So as Simon was pointing out, it's going to be soon three decades we've been doing this. And the name Meta integration is definitely metadata integration, and that's the spirit behind the name. I know meta took another connotation these days, but we were first using the name meta integration

out there and really it went on on standards initially and we got heavily involved with the OMG group, the object management group, but had important standards known as UML for object modelling, unified modelling language at the time, XMI, and CWM, the common warehouse meta model. And here already you see and feel the spirit of having a common meta model between vendors. And that involved at the times key companies like Oracle, the Oracle Analytic Group, SAS, IBM,

all these key vendors wanted to have a common meta model to share metadata between vendors. But as you can imagine, they each had their own twist and having a real common meta model was very hard to achieve. Not in some areas. You know, we can all agree what a common meta model is for relational database. All good. But as soon as you dive into analytics, everybody gets into their own little twist and flavors, you know. So that really

got into the thinking that the metadata standards didn't really work at the time. And one could question it in the future and we'll get on that a little bit later because that's a common thread happening. So we created metadata integration and part of our first line of products was all about what we call model bridges. So basically metadata converters between tools and that really is how we started. We started actually

on government contracts also, and that keeps the value of the company because we never took any venture capital. This is a 100% employee owned company today. And that is very important, as some call us the Switzerland of metadata because we're not controlled by any external body. We don't have any bank accounts in Switzerland. If anybody is thinking this way, that's not the case. But that's the neutrality of the ecosystem. So what happened at that point? Metadata don't work. We have this model bridge software, meta integration model bridge, MIMB,

and it can basically move metadata between tools because it can convert between different meta models. And in those days, the BI tools were business objects like business objects at something called a universe. Very important in what we're going to come to a bit later because they had some sort of a semantic, common semantic layers so that the reports on top were sharing common metrics, common dimensions and everything. And in those days was another competitor close to you in Canada, Tobias, was Cognos.

And Cognos created an equivalent tool called Cognos Framework Manager. Same idea. A common metadata layer in the middle. So that all reports used on that. And these guys changed us. We do owe to the Cognos company in Canada. They came to us and they said metadata is important. This new tool we're going to have with a good metadata layer needs to be able to import export from any tool around.

And that's how it started. They say we would like to put your model bridges OEM inside our software. So when when you use Cognos Framework Manager as a BI, you know, semantic modelling tool, if you wish, at the beginning, we need to be able to import from any metadata and export from any metadata. For example, a nice model from Erwin, you know, nice data model with the dimensions, facts, the business names, bring it in. And we need to also be able to forward engineer not just to our reports. We want all this knowledge to go outside.

So metadata integration took life within Cognos Framework Manager. We signed an OEM agreement with them at the time, and that was at the very beginning of, you know, twenty five years ago. And it spiraled out of that. Basically, everybody wanted to have metadata integration, the ability to integrate metadata with their tool. So Cognos was the first one, but it follows suit with with others with slightly different needs. The next one was Informatica

Informatica for the data integration tool, power center, but also the data catalogue, the ability to bring metadata and get lineage from any tools. And then was the first data modelling. I believe it was ER Studio from Unbucketero, Hidera a little bit late. And these are still extremely active OEM agreements today. You use ER Studio and they have what they call the meta wizard. When you see import export menu in these tools, what people don't know, it's meta integration inside

giving all the import export from any technology. And it kept going, you know, business object came in a bit later, which got acquired by IBM. So I had an OEM agreement with IBM. The initial, you know, group within CWM, so SAS, also Oracle. Now Oracle took a different twist. Some vendors wanted to build the data catalogue and they wanted to have the import export of metadata from us, you know, whether it's a data catalog, a BI design tool or a DI design tool. But some wanted

the entire data catalog from us, you know, the full metadata management solution rebranded for them. And that was the case of Oracle Enterprise Metadata Management. This is the entire meta integration metadata management platform rebranded with the Oracle sticker on it. And others did exactly the same. Today, the very popular Click Talend data catalogue, Talend acquired Click, is actually our software rebranded with hundreds of customers

using this at extreme large scale. You know, I'm not sure I can mention customers in this in this since they are OEM partners, but the largest bank in Canada, largest customers in in EMEA, in APAC, all over the world as this is internationalized. And it kept going all the way to Microsoft. So Microsoft Purview metadata import is our software as well inside. And I mentioned many others I didn't mention, like Erwin, the import export of Erwin from Quest is also our software.

So that kept going and the meta integration brand, meta integration OEM for just the connectors and the data lineage or the entire platform rebranded. And then some companies came to us and said, would like to buy directly from you because we would like a vendor neutral solution. And that started with some other large companies like Walmart basically came to us and wanted the technology for them. But also others lately in the insurance business like GAIG

or Canal Insurance. Same story. You know, we have many, many vendors. We know what we're using today for BI. We don't know what you're going to use tomorrow. We need something like this. And eventually that created the Meta Carter brand, which is our brand directly to customers from Meta integration, basically. So that's the name Meta Carter. So Meta Carter Carter is the map. It's the enterprise map of metadata, the enterprise architecture,

any tool, you know, out there. That's the key of what we do is really be vendor neutral. So that was a long introduction. I know that. But it's very essential to understand. We breathe and drink metadata from the beginning. We are vendor neutral. We have to support all these vendors. And that's basically how we got into that.

Tobias MaceyThat's definitely a long history and metadata has gained a lot of visibility in recent years, particularly starting around 2018 when we had the fragmentation of the ecosystem away from a lot of the monolithic vertically integrated platforms like Informatica, etcetera, and into a lot of the cloud data warehouses, the introduction of tools like DBT and ELT workflows, and a lot of the investment in the various open source components that caused the knowledge of the data estate to be fragmented across all of them. And so then there was a push to try and stitch that back together through these various metadata systems.

And I'm wondering if you can talk through some of the key pieces of friction and fragmentation that exist in the data ecosystem in particular and some of the ways that your long history maybe shows that everything old is new again or maybe some of the new tensions that are arising up because of the proliferation of new tools and new systems. And I'll I'll pause on the AI piece until later.

Christian BremeauSounds good. Yeah. The the story has evolved. And, as it evolved, there are some ups and downs and some keywords that didn't help. One pair of keywords that didn't help is called data catalog. And I'm going to tell you why, because it's inevitable that if you are a software vendor, large or small, you cannot not have a metadata repository at the heart of your product. Okay, so no matter what, if you take a product from

Informatica, as you mentioned, or Microsoft or even a smaller vendor, a data modelling tool or DI vendor must have a metadata repository to share metadata between different tools of their offering. Right. So a shared metadata repository was essential for metadata management. But then the branding and the data catalog twist didn't help because then, you know, everybody said you need a data catalog.

But then people say, I already have a data catalogue. Oh, wait, I've got many data catalogue. Then started the words of catalogue of catalogue and all these kind of activities that were and it didn't help. And the answer is it is inevitable. You must have every tool. If you buy a vendor that doesn't have at least core metadata management to share metadata between its tools, you're not going to be there. So each vendor's got to have its metadata repository, its own data catalogue.

But then sometimes you need also a higher level of catalogue that is across and vendor independent. Sometimes you don't need it because if you're single vendor, you're fine. You know, you don't need necessary support for all these different BI tools. But the larger your company is, you know, every time we talk to a large company and we ask the question, so what what database do you use? What DI or BI tool do you use? And the usual answer, name one, we probably got it somewhere, you know, and that's probably what it is. And when it comes to enterprise architecture

and data lineage, there is no tool left behind. You need to basically support all of them. And there is a legacy architecture. There is a future architecture. There is a hybrid architecture. And that is very, very important. Now these tools are not created equal. A lot of data catalogues got produced as, I would say, a database of metadata. It's a cheap product to put on the market and you can put a quick UI even faster and better now, AI generated, and it's going to look like a million bucks and it's going to be great. It's going to be fantastic.

But the devil is in the details to do it right. And then they got a hard time to what I call retrofit the metadata foundation, you know, on top of a pretty UI and high level systems. No, you got to build very strong metadata management foundation and metadata management foundations. It's all about version and configuration management and putting it all together. Many of these data catalogs do not have a true metadata version and configuration inside.

And that's a big, big deal because it's all about change management. It's always changing. So first of all, bringing the metadata from the database, the DI, the the first time it's going to take time. Right. Because, you know, you can remember the first time working with Boeing, their Cognos server was on its knee during the days responding to users. So we had to harvest metadata at night when he was less busy and, you know, pass all this. But then there are technique of change management. You don't want to redo that all the time.

You know, maybe only 5% of the reports have changed and a few have been added. So you're not detect change, put it together efficiently and then what is known as metadata stitching, which may sound easy if it's a relational database, you know, schema table column, all good, except true metadata stitching is the type the same. People didn't realize that that column, they changed the type on you. There is no errors in UBI report because they don't care. They don't know. But your number is wrong now. And metadata stitching is all about that.

So that's part of the ingredients of true metadata management. And that basically is the big, big difference from this perspective. And what also changed and the differentiator between all the vendors is related to deployment and pricing. And that's also a huge differentiator. A lot of the modern data catalogues or metadata management solutions are cloud only, which is great, which is fine. But on the vendor cloud, we call it, you know, vendor managed

on the vendor cloud. But some companies do not want to work on that particular vendor cloud or sometimes they are highly secure government or banks or things who really, besides the big guys like Amazon, Google, and even then they have private secure cloud, They want their own deployment that we call. Customer managed versus, you know, managed by by the vendor itself. And that's a big, big difference. So we differentiate ourselves

by being independent on the deployment. You can do your self deployment or you can use our cloud. And it turns out even through our OEM partners like ClickTalent Data Catalog, 80% of the customers are basically deploying on their own cloud. I'm not going to use the word on premise because it's private cloud or it's their own system, but they are very, very independent. And that brings to the last factor, which is pricing.

With pure cloud management, you know, vendor cloud management comes the side effect known as pricing and capacity pricing. And that has hurt this market very, very, very much. It starts by the notion of named user. So basically, you pay by named user with your email address and you need somebody or somebody is out of a company and you've got to manage all these name users. It becomes quickly expensive, but it's everywhere. You have to pay per connector. You add a connector. It's more you pay the volume per objects and it adds up. So what became the first very affordable 50 ks POC,

it's becoming very hard to go with. So we adopted the exact opposite approach. So not only for deployment, you can deploy privately, you know, on your own and and you do not depend on that. You have your storage, you have control and you have a probably more affordable storage if you're a large company than we can provide for you. And again, things are becoming very, very big. We have customers like the large bank in Canada was mentioning, reaching three terabytes of metadata

underneath with version and configuration management. It gets big, you know. So what we have is predictable pricing. We don't charge per name users. We have notion of concurrent users among unlimited users. So in other words, you can be on the block and users can come and go. Nobody should be denied access to the catalogue. Or then you have a lot more users than you thought, but that's good news. This means your metadata management is successful and then you can buy more concurrent users. We don't charge per connector.

We don't charge per volume. So what predictable pricing leads to, as much as I said, the majority, maybe two third, are self deployed versus on our cloud. Most of them buy two years of predictable pricing in advance. The only reason they might increase in price is if they have a huge adoption and have a lot of concurrent users. And that's really a big, big differentiator right there. So again, a long response, but because there is a lot to say

from, you know, a metadata repository, a data catalog, a metadata management. Remember Gartner had a metadata management quadrant, then it kind of disappeared. Now it's reappearing again because data catalog and then it was AI catalog versus or not, but we'll get into that later probably.

Tobias MaceyOne of the other aspects of this overall ecosystem, as you pointed out, there are a lot of Johnny come latelys where they say, yes, we do metadata. We'll bring all of your information about your data estate into one place, but maybe it's lacking some of the more nuanced and detail oriented aspects of true metadata capabilities such as versioning and change management like you were referencing. Another one of the huge challenges is the fact that, as you pointed out, every

engine that works with the data needs to have some concept of its own metadata for how its own internal representation works, how it keeps track of state, how it keeps track of data assets. But you also need to be able to bring that to a more global view in order for things to be able to actually work across those different engines. I think that's another constant struggle that we have in the industry of every engine wants to grow more and more of its capability to own more of the metadata,

but it's never going to be complete unless you have system where that is the sole purpose is to be the cross cutting repository of knowledge of the data estate for an organization. I'm curious how you are addressing some of the challenges of the locality versus consistency that arises with all of those engines needing to own some portion of the metadata, but then also needing to have information about the metadata of the other engines that they don't touch.

Christian BremeauRight. Right. So first of all, there is another differentiator that I forgot to mention on the first part, which is how do you get the data lineage within each of those, you know? And we brought a lot of what we ask for is what we call data lineage by design. So meaning that you have to prove where it comes from, from the design. You know, you can have an ETL tool, an ELT tool. You can have a script. You can have a very complex BI tool.

And from the source of your data, which can be really on the far left of your architecture, can even come from a mainframe, a COBOL, it can be in a file to its life in a shot in a BI tool. On the right, there is a lot of data transformation, each specific in every tool. And when the number is not right, you have to prove where it's coming from. So back to the previous one first is one of the key aspect is to prove the lineage on each of these tools.

And what we have developed all these years are pretty much emulators, debuggers of each of these tools, you know, like a power center, like a power BI on the right side or like a Tableau or like a very complex sparks for Python script, you know. So we have compilers, parses, get the runtime information, have a full emulation to understand exactly where it goes. So if that number is this, is we can prove the formulas on each of those.

And even the OEM partners we have, people get shocked. You know, why? Why is Informatica Power Center sometimes calling our bridges? Well, they have a source code of power center over cloud data integration. But within a transformation, you can have 10 pages of SQL for for for Teradata or SQL Snowflake SQL or Python script that you need to emulate in order to have the details of a transformation. So we're not just observing logs.

You know, sometimes we have to look at logs, runtime logs for information. We have true debuggers, emulators, parsers of all this transformation. And that will be very important later when we get into our new semantic hub is basically we understand what's going on inside each of these BI transformation to calculate metrics and what the formulas are. That is very important. So now the locality, the data flows in the enterprise

from many, many levels. And, you know, there are often three or four different technologies on the left side of your main data warehouse. And then it goes into staging areas on the right marts, you know, and then it goes to your reports. We get all of them. We stage all of them and we take care of the automation. What happens as an illustration with this locality of the data is that many times it's actually a pass through.

You know, you wouldn't be saying you see that and you're not going to document it on 20 of these systems. So we have the data lineage and the stitching that puts all these together. We have automatic detection of this pass through. So if you document, for example, in the data warehouse, if you document early in the file system, something like an SSN has been detected and we get it through or something very important out there.

We have automatic documentation because we detect the pass through and as long as it has not been transformed, every time you get on these local systems, you have that definition that has been done automatically for you. And it turns out, despite all the transformation, more than half of the data keep passing proof from the, you know, bronze to silver to sometimes staging and goes all the way. And maybe the transformation is at the end, nothing else. So we have sometimes

this if you have a business glossary term that has been done an automatic pass through on that. Simon, you want to add a little bit on that topic of locality and pass through and automatic documentation?

Simon DyninYeah, I think you described it perfectly. At the end of the day, Tobias can ask about, yeah, we have a DBT on the scene, we have cloud, data warehouses, snowflakes, data bricks, you know, they all bring their own kind of metadata management systems. And yes, we need to put this Humpty Dumpty together. But you said that when you look at this tapestry of relationship,

and yes, you need to have a system that is capable to analyze kind of this data graph kind of with the speed of thought. This graph has billions of notes and we need to traverse it to give you an answer. And we're capable of doing it. But when you find what's wrong, then you ideally would like to jump back to the source, to the design document, to some kind of a script

that produced this transformation. And this is where we differ. This is what's kind of paramount for a lot of organizations. Instead of kind of tracing lineage through logs, okay, which is very effective, And we're doing very well for outliers, you know, for things that we cannot get from design. Because for us design, it's kind of initial step. We take parametrization

and build kind of runtime out of it. Runtime image, runtime lineage. Going through logs is fine, but you never know where this log kind of query, logged query came from. And this is a big, big problem. This problem actually going to get even worse when AI comes to the scene in spades.

Christian BremeauLog can be polluted in many ways because, you know, sometimes by just observing log is the only solution versus emulating the design, is a lot of things are happening in the log and it's not always easy to trace what happens. So first of all, some jobs are running on regular basis. It's not because they're running every night, but the lineage, the amount of time it runs affects it. Sometimes they're a backup and restore and the lineage come back, but that is a one time restore of an external database because of a merger or something. You know, these kind of things happen.

So lineage by design is very important because when there is an audit and you combine all these lineage with version and configuration management where, you know, the customer gets an audit and they made a decision on that BI report number. Okay, but that was at the end of Q3 last year. Okay, where was it coming from? So if you don't have this lineage that basically is also combined with version and configuration management where you can prove

at the time it was coming from this information from the ECTL, that's where they multiplied by 1.1% at that and that changed the number. That is the kind of critical information that I expected in metadata management.

Tobias MaceyOne of the other ways that metadata has manifested in recent years is the idea of the semantic layer as a distinct product where you have this one technological system that acts as the system of record to say, this is what this number means and maybe a little bit about how it got computed versus a more holistic approach like what you're doing where you have more of that organizational context by virtue of having the full scope of where the data came from and how it got there.

Also, the idea of the metric system, there are a number of different, very narrowly scoped aspects that, to my understanding, ultimately boil down to it's a system of record for metadata. And I'm wondering what are your thoughts on just some of the ways that the industry keeps trying to define these point solutions rather than doing a more holistic approach of tackling all of the metadata because of the complexities involved?

Christian BremeauRight. Basically, the metadata management and the metadata were input for managing your DI, managing your BI. Now, obviously, the most important or equally important to the driving factor now is the metadata to serve as a context for AI, for LLM. And that is, you know, obviously the semantic out of that. So metadata management is obviously very well positioned for that.

But, you know, a lot of the AI and LLM only were looking at the data. Then obviously everybody realized having all the metadata about the data will help them understand a lot more what's going on. But as you pointed out, it's not just the metadata, you know, the the definitions and everything you know about this table and this column and and all of this. I think the end to end whole story of consumption and production of this data to be used. That provides the true context for AI.

And we are in a great position for that because having the lineage, we understand all the transformation where it came from, how it got into this data and that was how it's consumed outside. So we have all the joints, all the lookups, we have all the transformations. So we can provide a very rich context to AI to really answer the other questions you want to get into. But let's not forget that. It was not invented here, you know, as I was mentioning at the beginning of a podcast, you know, having all this semantic layer to be used only by reporting at the time was already the cup of tea of a business object universe or Cognos FM at the time.

So we basically, in this new version of a product of a meta carta we just released, which is version 12 of our product that shows our history as well. This version 12 include a very, very includes a very critical product called Semantic Hub and the Semantic Hub as a semantic language inside. So that's a platform to collect the semantic models inside and to drive both AI and BI on the other side.

Now, because of where we're coming from, we have a very rich way to not start from scratch, to not just start from just, you know, the table and column. We have the whole context coming in where it's coming from. And we have the ability to bring this metadata from the problems people have already suffered with. You know, they have already, you know, with BI, with whether you use Power BI or Tableau, you know, it's been a sprawl of reports where everybody duplicated

everything and all this semantic has been multiple interpretation of the same matrix. So we can bring these things in the semantic hub to reconcile them and then push them for a single implementation on the other side. So Simon will describe a little bit more of a semantic hub, but the use of metadata management is a platform that's got to drive a semantic hub out there, but we're not a semantic layer that you have to go through, you know, to get out there. We have a different approach. We bring the metadata

from all these original tools, whether it's a BO universe, if you have some wonderful metrics out there, whether it's a Cognos or MicroStrategy, or the latest Tableau and Power BI. We can bring all this in. You can reconcile and then we can compile that into the semantic layer of your tool, which can be a semantic layer of Snowflake, that can be metrics views of Databricks, that can be analytic views of Oracle.

We've been doing it all in what we do. But now you have a metric that is calculated and enforced in that database layer. But both the BI tools, you know, a new BI report, we can produce a semantic model for Power BI. We can also compile it into Tableau and UAI LLM are all sharing that same metric value. And depending on the context, you can have also different implementation. But at least you no longer have

the you resolve two problems. You resolve the sprawl with different interpretations, you know, and you resolve also all these precious metrics. Some people are still using business object and Cognos today because they've been incapable of moving these metrics away from others. And Simon can tell a little bit more about our use of a semantic hub, but that's definitely the innovation we bring with MetaCarta version 12.

Simon DyninYes, this semantic story is exciting, and it's been kind of exciting for many years. Yeah, I came to the scene maybe thirty plus years ago with this business object defining not only what the real table is, but how to join it, how to describe it, and end users can start consuming it. BI kind of took off. Now we have this Power BI beauty, Looker,

ThoughtSpot, etcetera, etcetera. But there is plenty of legacy. And at some point, I hope Christian will tell us that people who are struggling with legacy, like MicroStrategy, BO, Cognos, and they want to reuse what they have. Now they have an opportunity. But what happened probably ten years ago, plus, yes, semantic layer companies came to the scene. Yes, people allow you to

kind of the translation between your raw data and kind of your logical representation of your enterprise in one place, create a server and expose the servers through API and suddenly, yeah, sometimes API, sometimes spreadsheets, sometimes kind of other applications can access it. And, but the promise that you will have your kind of semantics in one place. Like you guys said, metric stores. But it requires you to have this intermediate server and everything goes through it. I think this is a very honorable solution.

If we kind of look what happened lately, this is where kind of cloud data warehouses, like Snowflake and Databricks. And in the past, even Oracle with analytic views, Oracle analytic views, let's not forget, they kind of pioneered this, you know, sometime back, but semantic views from Snowflake or Databricks, metric views, It's like kind of a new kid on the block. Know, they try to achieve the same thing. They try to have a place

where you can have logic, all your business kind of logic in one place. You can define as many as possible kind of KPIs if you can, and allowing people just to ask a question without being concerned with joins, you know, basic complexity of your physical data representation. Okay, it's all kind of wonderful, but what are the alternatives? And we could have thought about it, you know, on one side,

there are challenges that there is a competition. There is a semantic modeling war right now. On one side, Microsoft controls their semantic modeling, kind of in Power BI type of assets. On another side, let's say Snowflake would like to be the king of the hill. Matter of fact, they don't want to work together. Microsoft Power BI is not open for business easily to

Snowflake and the same for Databricks. Okay, it means there is a crack in the system, unfortunately these vendors need to work together, but it's not going to happen. But imagine the world where you can design the complexity of your enterprise and we're taking it very seriously this word complexity. Business is not simple. You can have real world questions to ask and it requires you to

have a meta model capable to capture the complexity of the world and this is what we're promising. Okay, this is quite unique. And again, it comes from our background. We spent twenty plus years kind of building this common meta model of BI, common meta model of data warehousing, common meta model of data integration. Imagine And if you take this common meta model of BI, this is a kind of a semantic model. We understand all types of kind of metrics,

you know, that you can imagine. And you have this powerful system, okay, and you have it as a visual development environment. Obviously, there are kind of agentic skills that you can use to describe what you want and we will do it as well, but it's for business analyst, you don't need to be kind of SQL guru to use our system. This is very important. But then when you have your semantic model in place, and by the way, it's much easier to build when you're building it on top of data catalog,

because you already have your kind of metadata of your database or file system in place. You already kind of understand joins because you kind of know the lineage inside of kind of BI and DI. And you already have descriptions because you may be imported them or maybe you kind of using data catalog as you're supposed to with a glossary, documenting

particular physical assets like tables and columns. And then you can just drag and drop it into the semantic model and automagically you're going to build the core, the foundation. And then you need to kind of define your metrics, or if you're importing it from BI, you're going to get them already, because we can import, you know, reports, whole your kind of estate, understand what's essential, and bring it into this kind of semantic

model that you require. But the magic comes next. You have the semantic model that you can query from SQL. Sorry, you can query from AI agents, okay? And the difference is kind of very significant. Remember in semantic layer, okay, you of, you need to ask a question using their language type of a thing, and they can translate it into a database,

ask the database, and send you a result back. In our case, you know, you just need to give us a business kind of question. We're going to give you kind of just SQL. You use the SQL, and again, we are kind of an MCP server. And then you can use another MCP server of the database to reliably kind of ask the question in terms of SQL directly and do whatever you want. We're not involved kind of in data per se,

we're still involved in metadata. And you have full control, security, full privacy available to you. Okay, this is a very unique architectural difference, okay, that takes us to the next level. But in addition to being able to provide SQL information to agents, and we believe this is the kind of a future of

metadata cataloging, data cataloging as they call today. You need to use kind of wealth of information in your catalog to help agents to produce a deterministic, sequel. Because let me contrast it what's happening in the world today. A lot of systems, they do have an understanding of relationships, sometimes between tables they have descriptions,

and they give it to you as a context, and then they rely on the LLM to produce SQL. LMs are capable to produce SQL, but they're very creative. Sometimes they produce SQL that does not compile, sometimes they produce SQL that is wrong, does not deliver what's required. We are guaranteeing that the SQL we produce will be done according to the semantic model you created. It will run. And we support kind of not only kind of modern data warehouses, we support even legacy like Postscript

warehouses, Oracle, etcetera. Okay, I hope you're getting the gist. But in addition, imagine you're running your traditional BI analytics. Okay, no problem. We can generate your Tableau Power BI metadata as well. And if you attach to a particular database semantic layer, let's say, know, snowflake semantic views. We can generate them for you as well. And we can... And you can kind of... Because at the end of the day, this is a kind of a true

deterministic way to get something from even Genie Databricks or kind of Snowflake, agentic environments. You know, you can ask them a lot of questions. They unfortunately try to get to the database and make mistakes, but if they have this well structured semantically correct semantic layer, okay, that we generate, then suddenly they can give you deterministic answer that will cover 100 of the questions that people are asking.

I hope it gives you kind of a picture overview, the first kind of taste of what we're cooking.

Tobias MaceyAbsolutely. And so you've brought in the subject of AI, and that is definitely unavoidable in any conversation regarding technology these days. And I'm wondering if you can talk to some of the new and interesting stressors that the AI and agentic machine speed interaction with both consuming and producing metadata has on systems like what you're building and some of the ways that your Medicarda product is oriented towards addressing some of those challenges in the market.

Simon DyninOkay. Okay. We're jumping into AI. Loving it. Okay, let's contrast it with what we kind of used to. Kind of a typical data catalog many years ago was basically kind of a town or municipal library. Kind of you had a dedicated librarian, let's call her, Sarah, and some kind of an elite IT engineer. And they catalogue every dataset, file this

Dewey Decimal cards and maybe dusted shelves once a week. It was kind of beautiful, quiet, deliberate, predictable. Now picture kind of changes dramatically. Instead of quiet librarians, you have thousand hyperactive research assistants who are drinking Red Bull, probably, and never sleep. This is kind of what autonomous agents and LLMs are doing to metadata management.

And the first kind of headache is going to be related to kind of asset sprawl, kind of sorcerer apprentice sprawl. The human analyst can create probably scratch pad tables, you know, maybe a few, maybe a dozen a week. LLM agent can kind of create, you know, hundreds by breakfast, you know, baseband, sometimes 50 temporary views, synthetic joints and abandon them after they're

done just with a single kind of a slack question. And in this case, if you keep collecting this information, you're Sarah, you know, you would like to clean your... Have your library kind of in in good shape. Suddenly it looks like, I don't know, episodes of hoarders or something. Basically you aren't kind of garbage collecting at machine speed and

your catalog will choke. This is one of the challenges, stressors that we are thinking about, you know, and trying to kind of provide solutions. Second issue is that we lost our kind of train tracks, you know, and I'm talking about kind of non deterministic of data lineage. Traditional data lineage was kind of very mechanical. You have a table A plus table B, kind of equal table C with clear, some kind of

ETL or SQL script. You can trace it backwards with kind of your eyes closed, you know, people knew what they're talking about. And LMs don't just kind of run on these tracks, they drive kind of off road buggies, you know, because they kind of think probabilistically instead of kind of humans. And when kind of compliance auditor walks in six months later and asks, why did you merge European

user IDs with marketing churn data? You can point to kind of a real DAG from Spark, you know, you have to explain an algorithm was Tuesday morning, a long time ago. This is another kind of a challenge that we're preparing ourselves. Imagine we will be able to trace the lineage, understand that it was a prompt and this was a decision,

but still it becomes very messy. It means we're preparing for lineage that will be a web of relationships and we need to untangle it. I think another one is obviously speed, know. This is what we... Because traditionally, you know, we're all kind of involved in software development. We know that LLMs these days they can speed code very quickly. Okay? And they can as well, you know, SQL, you know, or

the transformations very quickly. Means kind of all catalogs that were built like yellow pages updated, let's say overnight. Remember Christian was telling kind of stories of Boeing, yeah, because resources not available during the day. You know, the servers are too busy. But agents use will need to use catalogs as kind of their kind of eyes and ears, kind of in real time. And if an agent has to kind of pause for,

I don't know, three seconds to fetch schema definition via kind of sluggish interface of ours, know, the whole pipeline grinds to a halt. It means catalogs has to be nimble. And again, our experience and our capability to kind of allow people to traverse, you know, billions of objects, you know, with the speed of thought, you know, come very handy in this case. And kind of... Think the the last thing that kind of that come to mind when we're

talking about this subject is jigsaw puzzle problem. Everybody knows it. Loose lips sink ships. In data, we always relied on a strict column level access control. Okay? This is what we used to. But agents are incredible at connecting kind of innocent dots. You know, sometimes they even can get data from places where you kind of do not expect. When you ask agent not to have access to,

I don't know, executive salaries, but it can read anonymized travel receipts, corporate calendar invites, and batch swipe kind of logs, you know. It can kind of put something together, you know, that you don't expect. It means kind of basically secret can be out easily. You know, the security kind of will become paramount and data catalogs, you know, need to kind of help. And I'm here, I'm talking about kind of futures. I'm talking about problems

that we kind of thinking about. But the world is changing. The world is changing kind of every day and we are adapting. And yes, we used to kind of build data catalogs to help people to find things. Now we have to build them to keep machines from running off the rail.

Tobias MaceySo digging into the Medicarda product itself that you're delivering, I guess, what are the aspects of the current time in the world that we have that prompted you to go direct to consumer after such a long history of working with these larger vendors. And I'm wondering if you can explain some of the design and architecture of the Medicarda platform and how it fits into the... I hate to use the term modern data stack, but the set of technologies that a majority of people are using today.

Christian BremeauSure. I'll address these remaining questions. Again, the word data catalog is something we don't like to use. It is actually pejorative to the whole purpose of a metadata management platform. It's just not just about cataloging data. It's way more than that. The reason we offered Metacarta in conjunction with OEM vendors is because there is a need for both. You know, some people want a solution from one vendor and focusing on a vendor.

And even if they need to support multiple tools, they want to focus on the metadata management platform or data catalog of that vendor. And that's just fine. And some others want a more vendor neutral solution. And as I said, we got pushed by quite a few large companies and is, you know, pretty much making the analogies. You provide wonderful Lego bricks to to all these large vendors. But we too are big and we want to have access to your Lego bricks pretty much. So we got pushed in doing that.

We are absolutely dedicated to both. So meta integration as a Switzerland of metadata, when there is an improvement that we do for for Microsoft and Power BI or for now Salesforce Informatica, Power Center or we do a new improvement, it's for everybody. Our ecosystem for the past thirty years is whatever new connectors, new lineage or new, it's equally available to everybody, to all OEM partners. You'll see it within Informatica,

within Salesforce, within Erwin, within any tool and so are the direct. So that is the motivation is whatever you need and what fits you will be there. But there is no version for one versus a version for another. Whatever latest improvement or news or new lineage or new feature is available to everybody. And that's what has kept this company going. It's the, you know, neutrality, the ecosystem and desire to maintain that for all. So that's what pushed Medacart.

And what what is the architecture and and what has evolved in the latest version of all this? Well, besides the the classic thing you expect on a new version. So first of all, for some reason, we decided to redesign the entire UI. Even UIs have evolved. You know, we have seen colorful UI and things and now there is a more cleaner look, simplified look, you know, and you see pretty much a similar look when you're looking at Snowflake or Microsoft or any of these products.

And when you look and feel, it's exactly the same. It feels very irated, you know, focused. Of course, you upgrade to the latest of for security reason, you have to keep up with all open source software. You know, it's not just Java, Tomcat, all these things have to be keeping up. But the real new aspect of this version is the addition of a semantic hub as a platform to drive both AI and BI to give the same answer.

And our approach to the semantic hub and its language inside is a visual semantic hub with a semantic compiler, again, that generates the semantic layers, that generate the BI and make sure both AI and BI are looking from the same. So the big difference with the classic people in the semantic space in this new architecture is we are not a layer. We're not an MDX emulator. We're not a layer that you call, that you depend upon.

The key aspect of what we provide is at the end, you don't depend on us. We are pretty much the platform for the development of the metadata that we push, but you're not a real time bottleneck that you depend upon to do that. Basically, we're not an MDX layer, we're not a layer. And also the approach compared to perhaps other data catalog is all about deterministic SQL. We don't use AI to generate the SQL. We compile

the semantic layer into the database. We compile into the semantic layers of BI tools and we generate deterministically the SQL for AI so that they all give the same answer. Big differentiator in all this. So yes, that's what's new. And suddenly the semantic hub is where we believe the big future is, you know, to share a semantic model and platform between AI and BI.

Tobias MaceyAnother keyword that you're pushing out is the idea of metadata ops and that metadata management should be a dedicated concern for organization and not an afterthought or a bolt on to another role. And I'm curious if you can talk to some of the key skills and activities that are associated with that capability and some of the ways that organizations should be thinking about the scope and staffing for that particular functionality?

Christian BremeauBasically, metadata ops is all about the roles and responsibilities and processes on top of the tooling, the metadata management tooling and the semantic hub on top of that. You know, you can give the fastest car and the most capable off road vehicle to anyone. If you don't have the methods and processes, you're most likely going to fail at some point. And I think this is something we now believe is important. But when we were not direct to customers, we've been kind of ignoring because

we were just supplying the technology and platform and the tools and everything. But to have a total success, you also need to have the roles and responsibilities. And some of them are very key. You know, part of the success and failures of metadata management and data catalogs in general are often related to that. You know, some people have a very inexpensive product to start with and then they cannot scale. But part of the other problem they've been facing, they didn't put the right resources and processes on top of that to make it happen.

Now, what has happened since then is DevOps and DataOps have been around and understood, especially DevOps, I would say, you know, with the cloud infrastructure. DataOps have been specialized sometimes with some tools. But what metadata ops is really about all these processes around because it's again to support change in the platform we just explained before. It is to integrate with the existing processes. Of course, this data warehouse, if you have snowflakes or data breaks or anything, they keep changing.

The tables keep changing, you know, and the ETL and everything keep changing. So we need to integrate in these processes as they deploy new tables, new views, we automatically generate and deploy continuous deployments of these semantic views, the semantic models on top of that generate so that you can keep up with that. And there are processes you need to integrate with to make it happen. And that's where metadata ops. You know, it is very important. And we're now investing

a lot in describing. And frankly, it helps all these services companies as well as those who use internal resources to actually be successful with the tools. Back again, you can have the best tools and technologies if you don't use it right, if you don't have the processes to make it happen. And if you do not integrate well with your existing data warehouse processes because you already have a lot of data integration and deployment process, you're most likely going to fail and even more at the pace required for AI today. That's definite.

Tobias MaceyFor teams who either already have a substantial data estate and some form of metadata management and also for teams who are just getting started, what does the adoption path for the Medicarda product look like and how does it change their day to day work?

Christian BremeauRight. Well, the adoption path, part of our success is our quick time from install, deploy, harvest and get your first results. You know, a lot of the data catalogs and metadata management have been given sometimes the analogy of the data warehouse took forever to install or like the jokes of some, if you manage to install it, you deserve a license and even more to harvest metadata and all of this. Our key to success from the practice perspective is it's very easy and quick to install even when you do your self deployment.

And that is very important because it gets around all the red tape of security if you use a third party cloud that you need to get approved. You know, if you're in your secure environment, that's already very, very fast. And then you can harvest, you can stitch and you can get going very quickly. That is a big change. Some classic, you know, metadata management solutions compared to ours is the ability to quickly deploy,

quickly integrate with the existing processes and the existing tooling system.

Tobias MaceyAnd as you are working with your customers and some of the early adopters of the Medicarda product? What are some of the most interesting or innovative or unexpected ways that you're seeing it used?

Christian BremeauSome aspects we didn't really anticipate as a top value. And one of them is to help the professional services and even their own company managing change. And that's why metadata management is very important when you have true version and configuration management. As I was telling before, it's a constantly evolving system. So people want to split a table in the data warehouse into multiple tables. They want to change a type. They want to change this. What's the impact of change?

If you have a true version and configuration management, not only you have versions of the past of a configuration and the system, but you can also do predictability in the future. So you can say, hey, how about if we change this? What's the impact? We can clearly say which jobs, which ETL and DI jobs have to change and where at which line of code. Point in the design of ETL, we even have the opening tool to show exactly where and same in the BI.

So some of things we didn't see is how much valuable this can be used in cost productivity and impact of change. How long is it going to take and what's going happen to make it work? You know, that's that's one illustration that is very important and only true version and configuration management system based metadata management can solve this kind of thing. Another one is on reporting and compliance based as a metadata management repository.

What surprised us is how many people go straight to the metadata to the lineage to produce their report, the completeness. For example, again, having glossaries of different standards, you know, you have in health care, in finance, many different standards and they can map and automatically generate the compliance report for health care, for FCC directly from the metadata management repository. These are examples of usage that comes

into that.

Tobias MaceyAs you've both been going on this long and I'm sure very interesting and convoluted journey of tackling metadata wherever it may live and bringing this Medicarda product direct to consumers, What are some of the most interesting or unexpected or challenging lessons that you've learned along the way?

Christian BremeauWell, how more and more strategic these things have become. Thirty years ago, you had to explain metadata and even metadata management. And now it's at the heart of any platform out there. And AI has really materialized the critical need for metadata management. And again, it gets back to shared semantic between all of these systems. And suddenly nobody saw really AI taking that importance. But that became very, very key and a new return on investment

for metadata management because it's been cycles, you know, where people could not find the resources for data cataloging and metadata management. Now you cannot afford to go in in the AI and LLM without proper context and metadata management. So it's not really a surprise or unexpected, but it's the advance in technology that made it useful, even more.

Tobias MaceyAnd for people who are trying to come to grips with their overall data estate and looking at various metadata solutions, what are the cases where Medicarda is the wrong choice?

Simon DyninKind of from our experience, if you're taking kind of a, I don't know, blood oath to a single vendor, if you run kind of exclusively off on one monolithic BI stack, is there an intention to change? Yes. Probably you can survive with the vendor's ecosystem. Just use their native model and accept the kind of wallet garden that you're living in. You know, some people kind of very secretly love managing heavy runtime middleware. If you want another kind of live query proxy,

definitely don't need us. MetaCarta kind of will not kind of wake you up at 02:00 in the morning. Obviously, if you're really small, kind of you, I know, your data stack is three program, you know, Oracle tables in a dream, you know, probably any data catalog could be kind of too much. Yep. I think a lot of people actually have kind of this fantasy of thriving on semantic anarchy, you know, things will work out eventually.

Yeah, we're going to figure out the gross margin, going to be eventually the same kind of into different tools, you know, and we can mitigate any problems. Yeah, I don't think we will kind of feed kind of this type of company mold. Christian, anything kind of comes to mind from your side?

Christian BremeauNo, that's exactly what you said. I mean, basically single vendor and small businesses, not necessarily, you know, well suited for this kind of thing. You mentioned a third one, people who really wanna hack the whole thing A to Z, then you go some open source solution or something. What surprised me on the second one on single vendor is that despite that thing, they're single vendor, then they realize there are some other tools out there. You know, I was mentioning a few companies in the insurance business and we work with very big one like GIG and smaller

one with Canal Insurance. Canal Insurance is a very much a Microsoft job. You know, they're very happy with Power BI and they use SSIS, SSRS, Power BI, SQL Server, ADLS, everything. But there are always something outside. So, for example, in the insurance business, there are external application that I use like the big, big insurance applications. And guess what's going on inside? They have BOD, that has BODS, pardon me, business subject data services from SAP that is doing the data movement in these external applications they use in the industry. And the reporting is done in IBM Cognos.

So despite the fact they are a smaller, you know, insurance company, despite the fact they are really dedicated to Microsoft and rightfully, they needed to have lineage and integration with things you would not suspect. And that's how they came to us. But that's that's, you know, classic. Unfortunately, if you were controlling 100% of your IT stacks, that would be right. But the reality, you have to use also external tools out there. You know, that's for sure. Also, if you only interested in the technical aspect or

the business aspect, you know, part of the key success of our product is to integrate the business user experience with a technical user experience because they have really different needs. Our approach to the business users, you don't you do not impose a data catalog or a metadata management tool to a business user. They're too busy doing something else. So we took a completely different approach. Whatever BI tool you use, Power BI, whether you use even business object or whether you use,

you know, Tableau, we have a browser extension in these tools. And basically as they are in these BI reports, they see on the right from the metadata management the definitions. So they are in there say, oh, I didn't know that. Where is it coming from? And they have information right in their BI tool, this business user experience. And where did they get that from? And then the business user comes straight to the source. They don't understand whether DI or an ETL tool or even an enterprise architecture.

Oh, we do capture this information. And then they ask the second question. Do we have any of the BI reports? Yes, there are five others, two in Tableau, three in that, and you can directly open into. That's the kind of business user interactions that are expected. They don't want to hear the word metadata management or data catalogue, but then they appreciate what's inside and they come there as a vendor neutral portal

to open the right report at the right place, see where it's coming from. Is there something else, you know, kind of thing. But at the same time, when they don't like the number on that report, they want to be able to talk with IT and talking with IT, they can give a link, a URL from that same data catalogue where the guy in IT can debug basically the ETL job or the stored procedures at the right lane line or, you know, the semantic model in Power BI to help. So if you're not interested,

if you're focusing only on the technical side, only on the business side, we're not necessarily the right choice. But if we want to put both community together, we definitely excel at that. And they need to have one repository with a different look and feel and different interaction to them.

Tobias MaceyAnd as you continue to build and improve and iterate on the Medicarda product as well as your core business, what are some of the things you have planned for the near to medium term or any particular projects or problem areas that you're excited to explore?

Christian BremeauWell, we're just starting on the Symantec hub despite the huge thirty years experience of bringing all this together. But I think we are planning a lot of evolution on the Semantic Hub Compiler, keeping up with the evolution of the technologies. It's very important, think, as Simon pointed out a little bit earlier, companies like Snowflake and Databricks do understand the importance of the semantic layer and they are investing in it. And to the surprise of many,

you will find out that the Oracle analytic layers are way more capable than where, you know, Snowflake or Databricks is. So all these companies are going to evolve and we're going to evolve with them in our semantic compiler so that you don't have to worry about keeping up with these evolutions of all these semantic layers because we generate and we compile into it. We are responsible to keep up for you with the evolution of all these semantic layers,

both on the input and on the output so that you can focus on something in the middle, the semantic app, which is way more powerful than, let's say, where the open semantic interface standard is right now because we have to capture things from the old systems, what business object or Cognos or many of these BI tools could do, which is not what Snowflake or Databricks could do today. But we got to evolve with them. And by the way, we are supporting the open semantic interface, but it's something that's going to evolve a lot. We had to do a lot more because we do conversions

between BI tools. So what we are going to be working on, it's all about the Semantic Hub compiler and all about doing more and more and keeping up as all these platforms are going to evolve dramatically on top of that.

Simon DyninI would love to piggyback on kind of Christian statements about our plans and reinforce them with some examples. Definitely hyperscalers will invent something new tomorrow, okay? But you don't want to redo your semantic layer, semantic model, Okay, because this is a key. It means you want to write once and target whatever the hyperscalers and then tomorrow. Okay, this is an engine that we're building. It means it has to be comprehensive. It has to handle anything under the sun and be...

Could be designed for kind of a future targeting. Yes. We definitely planning kind of to go from BI dashboards to kind of agentic context. Yeah. Imagine you've been running kind of legacy micro strategy, you know, BO, Cognos, Looker, whatever, you know, and it's time kind of to maybe do something new, but transition is painful. Imagine you can kind of bring it into our environment easily with a click and then use kind of MCP kind of frontier,

kind of to get all the juices left kind of from this system. Okay, you can kind of build a conversational AI technology. Okay, conversational kind of BI technology. You can modernize kind of this legacy system and suddenly, you know, you can calculate net kind of retention and it wouldn't kind of guess and create messy sequel. It will kind of pull a compiled auditable kind of semantic

contract. And again, it's gonna happen in milliseconds. And I think this is the two major kind of goals for us for our near future.

Tobias MaceyAre there any other aspects of the work that you're doing on Medicarda, the work that you're doing at Meta Integration Technologies or this overall ecosystem of metadata management that we didn't discuss yet that you'd like to cover before we close out the show?

Christian BremeauI think you raised excellent questions. A lot of topics have been covered. I think we made it pretty clear that the Semantic Hub is our bread and butter. We do believe in governed semantic models, and that's the golden assets. And we believe and we lived it for the past thirty years that the technology evolved and our job is to keep up with it and so that people can focus on their assets. Hence, the metadata ops,

which is basically govern and manage your metrics and your semantic and let the technology evolve. But you don't want to have to be recompiling. Sorry, I didn't mean to say recompiling, but rewriting your code and keep up. That is something that should not be your job. You should be focusing on the real goal, the real assets, which are you metadata, you semantic and you complex metrics and never underestimate

the complexity. Those who've done very complex BI tools in the past understand that well. You know, frankly, for example, the analogy of Oracle I was mentioning before, all this started from Oracle analytic tools, which pushed the analytics into the database. That was like fifteen, twenty years ago, you know, and now, you know, Snowflake Databricks, excellent, good evolution on the cloud, but it's going to evolve. And we believe that we want to be helping on that.

Simon DyninI think in another avenue, we considering to pursue and we believe that tools in our space need to be kind of more proactive. They need to kind of close the loop with what's called kind of active compilation. You know, we kind of create a kind of image enterprise IDE. Kind of right now, compilers, they take your definitions and, you know, in output kind of native

SQL kind of warehouse code. Then probably the next step kind of is event driven compilation. If an engineer, you know, alters kind of underlying physical ERP table or kind of some kind of upstream DBT model drift happens, you know. Compiler kind of shouldn't wait for an analyst to notice kind of kind of the broken chain on Monday morning. It should not only... It it should kind of flag the logic mismatch,

try to recover from the system automagically. I I think it would be exciting to happen and I think in general we live in exciting times.

Tobias MaceyAll right. Well, for anybody who wants to get in touch with the both of you and the work that you're doing. I'll have you add your preferred contact information to the show notes. As the final question, I'd like to get your perspectives on what you see as being the biggest gap in the tooling or technology that's available for data management today.

Christian BremeauThe biggest gap in many tools is the level of details and proof in like in the lineage. You know, a lot of tools have high level lineage, high level level of functionality for metadata management. But the devil is in the details. You know, if you have to go in a high level of depth in this complexity, that's the gap that we see in many, many systems, in tools and technology in general.

Now that's for the metadata management aspect of data management. With respect to the gap in pure data management itself, I'm not too sure about this because that's our focus is really in the metadata management, to be honest. Data management has other challenges with the pace of changes these days. Maybe Simon, you want to add on the data management aspect as well?

Simon DyninWe spent decades and billions of dollars kind of perfecting the plumbing to move petabytes at lighting speed and only to recognize nobody taught the pipes what net revenue means. Does it include taxes or not? Okay. And the biggest gap is that modern tools are obsessed with syntax, and they kind of still almost clueless about semantics. And they will happily kind of tell you where the table lives, who queried it,

you know, but semantics still is a key. I think automating this process, it around AI agents, kind of the key to high performance kind of sports cars that kind of we are building. You know, we're not the engine, you know, Medicarta and Symantec Hub is gonna full blown sports cars that you can drive and have fun.

Tobias MaceyAll right. Well, thank you both for taking the time today to join me and share the work that you've been doing for a long time now on metadata management and integration and all of the associated challenges, both technical and organizational that come out of that. So I appreciate all the time and effort that you've put into helping all of our ecosystems work more effectively with data and now bringing that direct to consumers. So I wish you the best of luck with that latest chapter, and I hope you enjoy the rest of your day. Thank you, Tobias.

Simon DyninThank you. It was a pleasure and honor.

Tobias MaceyThank you for listening, and don't forget to check out our other shows. Podcast.net covers the Python language, its community, and the innovative ways it is being used. And the AI engineering podcast is your guide to the fast moving world of building AI systems. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. And if you've learned something or tried out a project from the show, then tell us about it. Email hosts@dataengineeringpodcast.com with your story. Just to help other people find the show, please leave a review on Apple Podcasts and tell your friends and coworkers.

Open the transcript file

Hosts and Guests

Hosts

Guests

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Entity Resolution at Scale: Graphs, LLMs, and Real-World Data Messiness

    59 minAudio

    Summary In this episode, I spoke with Steven Renwick, CEO and co-founder of Tilores, about the realities of entity-oriented data processing and the practical role of LLMs in entity resolution. We covered the confusing...

    Credit the people on this episode

  2. How Amazon Uses Telemetry and AI to Automate Physical Logistics

    50 minAudio

    Summary In this episode, I spoke with Keerti Kulkarni about the data and AI systems behind physical logistics automation at Amazon. She explained how her team supports dock-door operations across more than 100 North...

    Credit the people on this episode

  3. Reducing Data Debt with Agile Ledger Architecture

    50 minAudio

    Summary In this episode Christopher Doidge talks about his Agile Ledger Architecture (ALA) approach to data warehousing and how it aims to reduce data debt while shortening the path from raw data to trustworthy business...

    Credit the people on this episode

  4. What Context Really Means in Data Engineering and AI

    53 minAudio

    Summary In this episode Soham Mazumdar, co-founder and CEO of Wisdom.ai, talks about what “context” really means in data engineering and AI systems. He explores why context has become such an overloaded term, spanning...

    Credit the people on this episode

  5. Specialized AI for Data Engineers: Inside Astronomer’s Otto

    46 minAudio

    Summary In this episode Yetunde Dada discusses Otto, Astronomer’s AI agent for Airflow, and the broader challenge of making agentic tooling actually useful for data engineers. She explored why generic coding assistants...

    Credit the people on this episode