Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

DEV

DEV artwork

Why Your GPU Is Loafing: Optimizing Deep Learning Training at Scale

Jul 17, 20268 minAudio

Provisioning a powerful GPU only to watch utilization flatline is one of the most common - and costly - frustrations in deep learning. This episode of Development digs into the systemic reasons why large-scale training runs underperform, drawing on this in-depth guide to optimizing GPU utilization for large-scale deep learning models. Rather than hunting for a single silver-bullet fix, the episode frames GPU performance as an interconnected system where small inefficiencies compound - and where targeted, methodical changes add up fast. Here's what the episode covers: Data pipeline bottlenecks: Why a slow or single-threaded data loader is often the first and most impactful culprit - and how parallel workers in PyTorch and TensorFlow keep the GPU fed between batches. Storage layer choices: How the difference between spinning drives, SSDs, and network-attached storage quietly shapes throughput, especially on large datasets. Batch size trade-offs: Why bigger isn't always better - very large batches can hurt generalization and destabilize training, and why incremental tuning beats guesswork. Mixed precision training: How using 16-bit floats for most operations while preserving 32-bit precision where it counts can meaningfully boost throughput and reduce memory pressure with minimal risk using modern framework APIs. Data and model parallelism: The distinction between splitting data across GPUs versus splitting the model itself, and when each approach is the right tool - including what to watch out for when scaling to multi-node setups. Profiling and system balance: Why skipping built-in profiling tools leaves optimization as guesswork, and how CPU capacity, RAM, and network bandwidth all feed into the GPU performance equation. The episode closes with a strong case for disciplined, documented iteration - changing one variable at a time and recording outcomes - as the practice that separates engineers who consistently improve training runs from those who spin their wheels. For more on the data infrastructure side of ML performance, check out the Development episode Zero-Copy Data Pipelines: What Apache Arrow Actually Does for ML. DEV

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Hosts and Guests

Nobody is credited on this episode yet.

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. The U.S. Has More Data Centers Than the Next 14 Countries Combined

    Nov 20, 20265 minAudio

    The global data center map looks lopsided by any measure - but the full picture is more striking than a simple site count suggests. This episode of DEV.co digs into a newly published graphic analysis showing just how...

    Credit the people on this episode

  2. From $30 Million to $5 Billion: The AI App Explosion by the Numbers

    Nov 18, 20265 minAudio

    The generative AI app market has posted one of the most dramatic growth curves in consumer software history - and the numbers behind that curve reveal a story far more nuanced than the headline figure suggests. This...

    Credit the people on this episode

  3. AI Adoption: Why the Numbers Are So Far Apart

    Nov 15, 20265 minAudio

    A single statistic can make AI feel like a done deal - nearly every organization already on board, the window to act almost closed. But a competing figure from the U.S. Census Bureau tells a starkly different story....

    Credit the people on this episode

  4. 83% of Developers Use AI Tools — But Only 6.6% Actually Trust Them

    Nov 13, 20265 minAudio

    Widespread AI adoption among software developers is no longer a trend to watch - it's the current reality. But a sweeping Stack Overflow survey of nearly 31,000 developers reveals a striking contradiction at the heart...

    Credit the people on this episode

  5. Private RAG Over Microsoft 365: Permissions, Delta Queries, and What Actually Ships

    Nov 11, 20265 minAudio

    Enterprise teams keep hitting the same wall when grounding LLMs on SharePoint and OneDrive: a working prototype that fails the moment security asks who actually owns access to the returned content. This episode of...

    Credit the people on this episode