
Adding ML layer to Search: Hybrid Search Optimizer with Daniel Wrigley and Eric Pugh
Vector Podcast website: https://vectorpodcast.com Haystack US 2025: https://haystackconf.com/2025/ Federated search, Keyword & Neural Search, ML Optimisation, Pros and Cons of Hybrid search It is fascinating and funny how things develop, but also turn around. In 2022-23 everyone was buzzing about hybrid search. In 2024 the conversation shifted to RAG, RAG, RAG. And now we are in 2025 and back to hybrid search - on a different level: finally there are strides and contributions towards making hybrid search parameters learnt with ML. How cool is that? Design: Saurabh Rai, https://www.linkedin.com/in/srbhr/ The design of this episode is inspired by a scene in Blade Runner 2049. There's a clear path leading towards where people want to go to, yet they're searching for something. 00:00 Intro 00:54 Eric's intro and Daniel's background 02:50 Importance of Hybrid search: Daniel's take 07:26 Eric's take 10:57 Dmitry's take 11:41 Eric's predictions 13:47 Doug's blog on RRF is not enough 16:18 How to not fall short of the blind picking in RRF: score normalization, combinations and weights 25:03 The role of query understanding: feature groups 35:11 Lesson 1 from Daniel: Simple models might be all you need 36:30 Lesson 2: query features might be all you need 38:30 Reasoning capabilities in search 40:02 Question from Eric: how is this different from Learning To Rank? 42:46 Carrying the past in Learning To Rank / any rank 44:21 Demo! 51:52 How to consume this in OpenSearch 55:15 What's next 58:44 Haystack US 2025 YouTube: https://www.youtube.com/watch?v=quY769om1EY
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
