Artificial intelligence is running into a problem that has little to do with algorithms. Models grow faster than hardware can feed them data, and the electricity used by data centers keeps rising. The International Energy Agency estimates that data centers used about 415 TWh in 2024, around 1.5% of the world’s electricity, and projects about 945 TWh by 2030 in its base case [1].
This series asks a strange question with a serious purpose: what could the networks of fungi, which grow under every forest floor with no brain and no plan, teach the people who design AI hardware? Before any mushroom appears, we need to understand the problem it would be solving, and that is what this first part is about. Part 2 compares the brain, the mycelium and the chip. Part 3 pushes the idea to its limit: human neurons wired the way fungi wire a forest.
- In today’s chips, fetching a number from memory costs hundreds to thousands of times more energy than computing with it. AI workloads suffer from this more than most.
- Moore’s law still adds transistors, but since the mid-2000s it no longer brings automatic gains in speed or efficiency. The big gains now come from architecture.
- The brain shows another way to organize computing: memory and processing in the same place, and activity only where it is needed.

1 · The von Neumann bottleneck: moving data costs more than computing
In 1945 John von Neumann described a computer split into two parts: a unit that calculates and a memory that holds both instructions and data [2]. Almost every computer since has followed that plan. In 1977, accepting the Turing Award, John Backus named its central weakness. The processor and the memory talk through a narrow channel, and every piece of data must pass through it on the way in and on the way out. “I propose to call this tube the von Neumann bottleneck,” he wrote [3].
For decades the bottleneck was tolerable. Today it dominates the cost of computing. In a 45-nanometer chip, adding two 32-bit integers costs about 0.1 picojoules, while reading 64 bits from main memory costs 1,300 to 2,600 picojoules [4]. Fetching the data takes thousands of times more energy than using it.

The gap keeps widening. Amir Gholami and colleagues at Berkeley looked at twenty years of server hardware. Peak computing power grew 3.0 times every two years, the bandwidth of DRAM memory only 1.6 times, and the links between chips 1.4 times [5]. Over two decades that adds up to roughly 60,000 times more compute against about 100 times more memory bandwidth. William Wulf and Sally McKee called this mismatch the memory wall back in 1995 [6].

Large language models feel this more than most software. When a model writes text, every new token requires reading all of its weights from memory, and each weight is used for only a couple of arithmetic operations. That is roughly one operation per byte read. An NVIDIA H100 offers about 989 trillion dense 16-bit operations per second and 3.35 TB/s of memory bandwidth [7], so it needs around 295 operations per byte to keep its arithmetic units busy (my arithmetic, a standard “roofline” estimate). Serving many users at once raises the ratio, but the basic picture holds: an expensive chip that spends much of its time waiting for data.
Phones and laptops are not exempt. A 2018 study of Google workloads on consumer devices found that, on average, 62.7% of total system energy went into moving data between memory and the processor [8].
2 · Moore’s law did not die. It changed shape
In 1965 Gordon Moore observed that the number of components on a chip was doubling every year [9]. In 1975 he predicted that the pace would slow to a doubling every two years [10]. The forecast held up remarkably well. Intel’s 4004 of 1971 had 2,300 transistors. NVIDIA’s B200, announced in 2024, has 208 billion, split across two pieces of silicon [11]. That is about 26 doublings in 53 years, close to one every two years.

What ended was a side effect. For thirty years, shrinking transistors also made them faster and more efficient without heating the chip more. This effect, called Dennard scaling after a 1974 paper by Robert Dennard and colleagues, broke down in the mid-2000s [12]. Clock speeds stalled at a few gigahertz, and the industry turned to more cores, more parallelism and specialized chips.
The performance of a single processor core tells the story. It grew about 52% a year between 1986 and 2003 and has grown about 3.5% a year since 2015 [13]. Meanwhile the cost per transistor stopped falling at around the 28-nanometer node, according to an industry slide presented by Google in 2023 [14]. Power per chip keeps climbing: an H100 can draw 700 W and a B200 1,000 W [7, 11].

The figure above puts numbers on the pressure. Electricity is one limit. Heat is another: a rack of the newest AI servers draws 120 to 132 kW, ten times a conventional rack, which makes liquid cooling mandatory [15]. Reliability is a third. During a 54-day window of training Meta’s Llama 3 405B model on more than 16,000 GPUs, the job was interrupted unexpectedly 419 times, about eight times a day, and roughly 78% of those interruptions were traced to hardware [16]. Memory is the fourth. While it writes, a model keeps a “KV cache”, a working memory of every token in the conversation so far. For a 70-billion-parameter model and a single 128,000-token conversation, that cache takes about 43 GB, more than half of an H100’s memory, on top of roughly 140 GB for the model’s weights, which already have to be spread over at least two GPUs (my calculation from the model’s published architecture [17]).
In 2020, Charles Leiserson and colleagues summed up the consequence in Science. When the silicon stops handing out improvements, they have to come from “the top”: software, algorithms and hardware architecture [18].
3 · The brain’s answer: 86 billion neurons on about 20 watts
There is one machine that already runs intelligence on a tiny power budget. The human brain contains about 86 billion neurons [19] and, in the neocortex alone, on the order of 150 trillion synapses [20]. It accounts for about 2% of body weight and about 20% of the body’s oxygen consumption at rest [21], roughly 20 W [22]. A single H100 draws as much power as about 35 brains. Training GPT-3 was estimated by Google researchers at 1,287 MWh [23]; a brain running at 20 W uses about 3.5 MWh in twenty years. The comparison is only an illustration, but it conveys the distance.
Neurons are slow. An electrical spike lasts about a millisecond and travels from about half a meter to more than a hundred meters per second, depending on whether the fiber is insulated with myelin [24]. A chip completes billions of cycles per second. The brain’s advantage comes from how it is organized:
- Memory and computation in the same place. A synapse stores the strength of a connection and also processes the signal that crosses it. There is no memory bus.
- Sparse, event-driven activity. The energy cost of a spike is so high that probably fewer than 1% of cortical neurons can be substantially active at the same time [25]. Whatever is quiet costs almost nothing.
- Local learning. A synapse strengthens or weakens depending on the relative timing of the two neurons it connects, within a window of about 20 milliseconds [26]. Artificial networks learn by backpropagation, a global error signal that the brain does not seem to use in the same form [27].
Engineers already copy these ideas. IBM’s TrueNorth chip (2014) packed a million artificial spiking neurons into a chip that drew 63 milliwatts in a real-time video task [28]. Intel’s Hala Point (2024) combines 1,152 Loihi 2 chips with up to 1.15 billion neurons and a maximum power of 2,600 W [29]. IBM’s NorthPole (2023) places memory next to every core and, on an image-recognition benchmark, processed about 25 times more frames per joule than a GPU built on a comparable 12-nanometer process [30]. An IBM analog chip performs matrix multiplications inside the memory itself, with an energy efficiency that IBM estimates at up to 14 times that of comparable digital hardware [31].
Two caveats. An artificial neuron is not a biological one, so counting them side by side is only an illustration. And neuromorphic chips win on specific tasks: on large problems, spiking networks still trail conventional deep networks and are harder to train.
4 · Two claims to retire
- “Moore’s law is dead.” Transistor counts keep growing. What ended were the automatic gains in speed and efficiency that used to come with each new generation [13].
- “A brain equals an exaflop,” or “neurons are a million times more efficient.” Spikes and floating-point operations are not comparable units. There is not even a reliable figure for how much a brain computes: Joe Carlsmith’s careful review judges anything from about 10¹³ to 10¹⁷ operations per second plausible for matching the brain’s performance, with 10¹⁵ more likely than not to be enough [32].
What comes next
If the next improvement in AI hardware comes from organization rather than from smaller transistors, it is worth studying every network that has solved the problem of moving resources cheaply. The brain is the obvious one. The other grows under our feet. It has no neurons at all, yet it builds transport networks that are cheap, robust and able to repair themselves.

That network is the mycelium of fungi, and it is the subject of Part 2: Thinking like a mycelium.
Notes on methods
The cover is a collage made by computer for this series, not a photograph: an hourglass and a chip modeled in 3D with Blender and engraved by a program, and a mycelium network grown by an algorithm. The charts were drawn from the sources cited in each caption. Two figures are my own arithmetic: the “roofline” estimate for the H100 (989 dense TFLOPS divided by 3.35 TB/s) and the cache size of Llama 3.1 70B (2 × 80 layers × 8 key-value heads × 128 dimensions × 2 bytes ≈ 0.33 MB per token). Figures supplied by manufacturers are presented as such.
References
- International Energy Agency. Energy and AI. April 2025. iea.org
- von Neumann, J. “First Draft of a Report on the EDVAC” (1945). Reprinted in IEEE Annals of the History of Computing 15(4), 1993. DOI
- Backus, J. “Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs.” Communications of the ACM 21(8): 613–641, 1978. DOI
- Horowitz, M. “Computing’s energy problem (and what we can do about it).” IEEE International Solid-State Circuits Conference (ISSCC), 2014. DOI
- Gholami, A., Yao, Z., Kim, S., Hooper, C., Mahoney, M. W. and Keutzer, K. “AI and Memory Wall.” IEEE Micro, 2024. DOI
- Wulf, W. A. and McKee, S. A. “Hitting the Memory Wall: Implications of the Obvious.” ACM SIGARCH Computer Architecture News 23(1): 20–24, 1995. DOI
- NVIDIA. H100 Tensor Core GPU, datasheet. nvidia.com
- Boroumand, A. et al. “Google Workloads for Consumer Devices: Mitigating Data Movement Bottlenecks.” ASPLOS, 2018. DOI
- Moore, G. E. “Cramming more components onto integrated circuits.” Electronics 38(8), 1965. Reprinted in Proceedings of the IEEE 86(1), 1998. DOI
- Moore, G. E. “Progress in digital integrated electronics.” IEEE International Electron Devices Meeting, 1975. Context at the Computer History Museum. computerhistory.org
- NVIDIA. “NVIDIA Blackwell Platform Arrives to Power a New Era of Computing.” Press release, 18 March 2024. nvidianews.nvidia.com
- Dennard, R. H. et al. “Design of ion-implanted MOSFET’s with very small physical dimensions.” IEEE Journal of Solid-State Circuits 9(5): 256–268, 1974. DOI
- Hennessy, J. L. and Patterson, D. A. Computer Architecture: A Quantitative Approach, 6th ed., 2019, fig. 1.1. See also “A New Golden Age for Computer Architecture,” Communications of the ACM 62(2), 2019. DOI
- Tom’s Hardware. “Chips aren’t getting cheaper: the cost per transistor stopped dropping a decade ago at 28nm.” December 2023, reporting a Google slide shown at IEDM 2023. tomshardware.com
- Supermicro. SRS-GB200-NVL72 rack specifications (about 132 kW); SemiAnalysis estimates about 120 kW for the same rack. supermicro.com
- Llama Team, AI at Meta. “The Llama 3 Herd of Models.” arXiv:2407.21783, 2024 (interruption counts in Table 5). arXiv
- Meta. Llama 3.1 model card, 2024. github.com
- Leiserson, C. E., Thompson, N. C., Emer, J. S. et al. “There’s plenty of room at the Top: What will drive computer performance after Moore’s law?” Science 368: eaam9744, 2020. DOI
- Azevedo, F. A. C. et al. “Equal numbers of neuronal and nonneuronal cells make the human brain an isometrically scaled-up primate brain.” Journal of Comparative Neurology 513(5): 532–541, 2009. DOI
- Pakkenberg, B. et al. “Aging and the human neocortex.” Experimental Gerontology 38: 95–99, 2003. DOI
- Raichle, M. E. and Gusnard, D. A. “Appraising the brain’s energy budget.” PNAS 99(16): 10237–10239, 2002. DOI
- Balasubramanian, V. “Brain power.” PNAS 118(32): e2107022118, 2021. DOI
- Patterson, D. et al. “Carbon Emissions and Large Neural Network Training.” arXiv:2104.10350, 2021. arXiv
- Purves, D. et al. Neuroscience, 2nd ed. “Increased Conduction Velocity as a Result of Myelination.” NCBI Bookshelf. ncbi.nlm.nih.gov
- Lennie, P. “The cost of cortical computation.” Current Biology 13(6): 493–497, 2003. DOI
- Bi, G. and Poo, M. “Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type.” Journal of Neuroscience 18(24): 10464–10472, 1998. DOI
- Lillicrap, T. P., Santoro, A., Marris, L., Akerman, C. J. and Hinton, G. “Backpropagation and the brain.” Nature Reviews Neuroscience 21: 335–346, 2020. DOI
- Merolla, P. A. et al. “A million spiking-neuron integrated circuit with a scalable communication network and interface.” Science 345(6197): 668–673, 2014. DOI
- Intel. “Intel Builds World’s Largest Neuromorphic System to Enable More Sustainable AI.” Press release, 17 April 2024. newsroom.intel.com
- Modha, D. S. et al. “Neural inference at the frontier of energy, space, and time.” Science 382(6668): 329–335, 2023. DOI
- Ambrogio, S. et al. “An analog-AI chip for energy-efficient speech recognition and transcription.” Nature 620: 768–775, 2023. DOI
- Carlsmith, J. How Much Computational Power Does It Take to Match the Human Brain? Open Philanthropy, 2020. openphilanthropy.org
