Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Vector Podcast

Vector Podcast artwork

Evgeniya Sukhodolskaya - Data Advocate, Toloka - Data at the core of all the cool ML

Jan 28, 20231 hr 27 minAudio
  • Apple Podcasts

Toloka’s support for Academia: grants and educator partnerships https://toloka.ai/collaboration-with-educators-form https://toloka.ai/research-grants-form These are pages leading to them: https://toloka.ai/academy/education-partnerships https://toloka.ai/grants Topics: 00:00 Intro 01:25 Jenny’s path from graduating in ML to a Data Advocate role 07:50 What goes into the labeling process with Toloka 11:27 How to prepare data for labeling and design tasks 16:01 Jenny’s take on why Relevancy needs more data in addition to clicks in Search 18:23 Dmitry plays the Devil’s Advocate for a moment 22:41 Implicit signals vs user behavior and offline A/B testing 26:54 Dmitry goes back to advocating for good search practices 27:42 Flower search as a concrete example of labeling for relevancy 39:12 NDCG, ERR as ranking quality metrics 44:27 Cross-annotator agreement, perfect list for NDCG and Aggregations 47:17 On measuring and ensuring the quality of annotators with honeypots 54:48 Deep-dive into aggregations 59:55 Bias in data, SERP, labeling and A/B tests 1:16:10 Is unbiased data attainable? 1:23:20 Announcements This episode on YouTube: https://youtu.be/Xsw9vPFqGf4 Podcast design: Saurabh Rai: https://twitter.com/srvbhr

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Vishakha Gupta - ApertureData, Hybrid Search Across Knowledge Graph, Vector And Unstructured Data

    Nov 12, 202648 minAudio

    Vishakha Gupta is the co-founder and CEO of ApertureData (https://www.aperturedata.io). In this episode we talk about hybrid search that combines a knowledge graph, vector search and storage for text, documents, images,...

    • Apple Podcasts

    Credit the people on this episode

  2. Berlin Buzzwords 2026 - Charlie Hull on role of Search in AI era

    Aug 3, 202629 minAudio

    This episode was recorded LIVE during Berlin Buzzwords 2026 Charlie's site and blog: https://thesearchjuggler.com/ LinkedIn: https://www.linkedin.com/in/charliehullsearch/ Search Relevance Slack (7000 search...

    • Apple Podcasts

    Credit the people on this episode

  3. Berlin Buzzwords 2026 - Julien Nioche on Greenops and new open source project SPRUCE

    Aug 3, 202625 minAudio

    This is a LIVE discussion with Julien Nioche, the creator of SPRUCE, open source greenops framework. We recorded LIVE at Berlin Buzzwords 2026. Cameraman is Zoom's AI and it goes a little crazy at times zooming on...

    • Apple Podcasts

    Credit the people on this episode

  4. Berlin Buzzwords 2026 - Trey Grainger & Doug Turnbull, Role of Search in modern AI and new course

    Jun 21, 202639 minAudio

    This episode was recorded LIVE at the Berlin Buzzwords 2026 YouTube version: https://youtu.be/acOGVynTVpM The cameraman is pure Zoom's AI ;) The Course: "AI-Powered Search: Modern Retrieval for Humans & Agents"...

    • Apple Podcasts

    Credit the people on this episode

  5. Beyond Hyperspace - Ohad Levi on Hardware Accelerated Search and Agentic Memory

    Jun 15, 202657 minAudio

    In this episode we sat down with Ohad Levi, co-founder and CEO of Hyperspace, to discuss the harware-accelerated search product he has built to address the search latency problem. Ohad also shares his thoughts on...

    • Apple Podcasts

    Credit the people on this episode