Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

Crazy Wisdom

Crazy Wisdom artwork

Episode #522: The Hardware Heretic: Why Everything You Think About FPGAs Is Backwards

Jan 12, 202653 minAudio
ShareXLinkedInFacebookWhatsAppEmail

In this episode of the Crazy Wisdom podcast, host Stewart Alsop sits down with Peter Schmidt Nielsen, who is building FPGA-accelerated servers at Saturn Data. The conversation explores why servers need FPGAs, how these field-programmable gate arrays work as "IO expanders" for massive memory bandwidth, and why they're particularly well-suited for vector database and search applications. Peter breaks down the technical realities of FPGAs - including why they "really suck" in many ways compared to GPUs and CPUs - while explaining how his company is leveraging them to provide terabyte-per-second bandwidth to 1.3 petabytes of flash storage. The discussion ranges from distributed systems challenges and the CAP theorem to the hardware-software relationship in modern computing, offering insights into both the philosophical aspects of search technology and the nuts-and-bolts engineering of memory controllers and routing fabrics. For more information about Peter's work, you can reach him on Twitter at @PTRSCHMDTNLSN or find his website at saturndata.com. Timestamps 00:00 Introduction to FPGAs and Their Role in Servers 02:47 Understanding FPGA Limitations and Use Cases 05:55 Exploring Different Types of Servers 08:47 The Importance of Memory and Bandwidth 11:52 Philosophical Insights on Search and Access Patterns 14:50 The Relationship Between Hardware and Search Queries 17:45 Challenges of Distributed Systems 20:47 The CAP Theorem and Its Implications 23:52 The Evolution of Technology and Knowledge Management 26:59 FPGAs as IO Expanders 29:35 The Trade-offs of FPGAs vs. ASICs and GPUs 32:55 The Future of AI Applications with FPGAs 35:51 Exciting Developments in Hardware and Business Key Insights 1. FPGAs are fundamentally "crappy ASICs" with serious limitations - Despite being programmable hardware, FPGAs perform far worse than general-purpose alternatives in most cases. A $100,000 high-end FPGA might only match the memory bandwidth of a $600 gaming GPU. They're only valuable for specific niches like ultra-low latency applications or scenarios requiring massive parallel I/O operations, making them unsuitable for most computational workloads where CPUs and GPUs excel. 2. The real value of FPGAs lies in I/O expansion, not computation - Rather than using FPGAs for their processing power, Saturn Data leverages them primarily as cost-effective ways to access massive amounts of DRAM controllers and NVMe interfaces. Their server design puts 200 FPGAs in a 2U enclosure with 1.3 petabytes of flash storage and terabyte-per-second read bandwidth, essentially using FPGAs as sophisticated I/O expanders. 3. Access patterns determine hardware performance more than raw specs - The way applications access data fundamentally determines whether specialized hardware will provide benefits. Applications that do sparse reads across massive datasets (like vector databases) benefit from Saturn Data's architecture, while those requiring dense computation or frequent inter-node communication are better served by traditional hardware. Understanding these patterns is crucial for matching workloads to appropriate hardware. 4. Distributed systems complexity stems from failure tolerance requirements - The difficulty of distributed systems isn't inherent but depends on what failures you need to tolerate. Simple approaches that restart on any failure are easy but unreliable, while Byzantine fault tolerance (like Bitcoin) is extremely complex. Most practical systems, including banks, find middle ground by accepting occasional unavailability rather than trying to achieve perfect consistency, availability, and partition tolerance simultaneously. 5. Hardware specialization follows predictable cycles of generalization and re-specialization - Computing hardware consistently follows "Makimoto's Wave" - specialized hardware becomes more general over time, then gets leapfrogged by new specialized solutions. CPUs became general-purpose, GPUs evolved from fixed graphics pipelines to programmable compute, and now companies like Etched are creating transformer-specific ASICs. This cycle repeats as each generation adds programmability until someone strips it away for performance gains. 6. Memory bottlenecks are reshaping the hardware landscape - The AI boom has created severe memory shortages, doubling costs for DRAM components overnight. This affects not just GPU availability but creates opportunities for alternative architectures. When everyone faces higher memory costs, the relative premium for specialized solutions like FPGA-based systems becomes more attractive, potentially shifting the competitive landscape for memory-intensive applications. 7. Search applications represent ideal FPGA use cases due to their sparse access patterns - Vector databases and search workloads are particularly well-suited to FPGA acceleration because they involve searching through massive datasets with sparse access patterns rather than dense computation. These applications can effectively utilize the high bandwidth to flash storage and parallel I/O capabilities that FPGAs provide, making them natural early adopters for this type of specialized hardware architecture.

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Episode #583: Five Materials, One Print: How MIT Printed a Working Motor for 50 Cents

    Nov 20, 20261 hr 9 minAudio

    In this episode of the Crazy Wisdom Podcast, host Stewart Alsop sits down with Luis Fernando Velásquez-García, a principal research scientist at MIT's Microsystems Technology Laboratories, to explore the fascinating...

    Credit the people on this episode

  2. Episode #582: He Started by Cutting the Grass: How an Argentine Factory Taught Itself to Build Cars

    Nov 16, 202648 minAudio

    In this episode of the Crazy Wisdom Podcast, host Stewart Alsop sits down with Candelaria Tornquist, founder of Reklus, an artisan automobile manufacturer located north of Buenos Aires. Candelaria shares her fascinating...

    Credit the people on this episode

  3. Episode #581: When Technique Becomes Autonomous: A Quant's Guide to Civilizational Decay

    Nov 13, 202657 minAudio

    In this episode of the Crazy Wisdom Podcast, host Stewart Alsop sits down with his first anonymous guest, world destroyar, for a wide-ranging conversation that moves from algorithms and financial markets to surveillance...

    Credit the people on this episode

  4. Episode #580: Is the West in Winter? Spengler, Nick Land and the Faustian Spirit

    Nov 9, 20261 hr 3 minAudio

    In this episode of the Crazy Wisdom Podcast, host Stewart Alsop welcomes his good friend Rainer Franz, who is building Stockfish for Politics, for a wide-ranging philosophical conversation that spans Western...

    Credit the people on this episode

  5. Episode #579: Sun Tzu in the Server Room: Cyber Warfare Without Firing a Bullet

    Nov 2, 20261 hr 4 minAudio

    Stewart Alsop sits down with his friend John Mercer for a wide-ranging conversation that explores the surprising connections between ancient warfare and modern cybersecurity. Drawing on analogies from Roman and Greek...

    Credit the people on this episode