ZeRO-2 (Shard Gradients + Optimizer)
Mathematics for AI and Machine Learning
From the book
This pictogram belongs to the Mathicon figure library for Zero2. It follows the same visual conventions as the book diagrams even when it is not tied to a single printed mind-map node.
What this drawing shows
What you see. Shows model state with gradients and optimizer states sharded across ranks, representing ZeRO stage 2.
In the book. Look up ZeRO-2 (Shard Gradients + Optimizer) in the manuscript index or chapter mind maps.
Where to read next
Look up ZeRO-2 (Shard Gradients + Optimizer) in the book index to find the matching figure or discussion. The pictogram follows the same visual conventions as the printed diagrams.
