For physically different kinds of memory there are significant differences in the time to read or write the contents of a particular location in memory, the amount of information that is read or written on a given occasion, the total volume of information that can be stored, and the unit costs of storing a given amount of information. To optimize its use and to achieve greater efficiency and economy, memory is organized in a hierarchy with the highest performance and in general the most expensive devices at the top, and with progressively lower performance and less costly devices in succeeding layers. The contents of a typical memory hierarchy, and the way in which data moves between adjacent layers, might be organized as follows.

1. 1. Register—A single word held in each register of the processor; typically a word contains 4 bytes. This is sometimes not thought of as part of the hierarchy.

2. 2. Cache—Groups of words within the cache; typically a single group in the cache will hold 64 words (say 256 bytes), and there will be, say, 1024 such groups, giving a total cache of 256 Kbytes. Single words pass between the cache and registers within the processor. All transfers into and out of the cache are controlled entirely by hardware. Modern processors with a multi-core architecture (see multi-core processor) will have two levels of cache, L1 and L2, where an L1 cache is associated with each core and the L2 cache links the whole device to main memory.

3. 3. Main memory—Words within the main (random-access) memory. On a very high performance system, groups of words corresponding to a group within the cache are transferred between the cache and the main memory in a single cycle of main memory. On lower-performance systems the size of the group of words in the cache is larger than the width of the memory bus, and the transfer takes the form of a sequence of memory cycles. The algorithm that controls this movement is implemented entirely in hardware. Main memory sizes are very variable—from 256 Mbytes on a small system up to tens or even hundreds of Gbytes on a high-performance system.

4. 4. Online backing store—Blocks of words held on permanently connected backing store. There may be two somewhat distinct forms of activity here:

   1. (a) swapping device—pages (of say 4 Kbytes) or segments (up to many Mbytes) of memory held on a swapping device are transferred as complete units between their backing-store home and a page frame or segment area in main memory, under the control of an algorithm implemented by the software of the operating system but with hardware assistance to indicate when pages or segments are to be moved;

   2. (b) backing store—complete files, or clearly identifiable subsections of large files, are moved between the backing-store device and the main memory in response to explicit actions by the programmer, usually by a supervisor call to the operating system.

5. 5. Demountable storage—Complete files, backed up onto removable disks or magnetic tape within the file store system and the archiving system. Complete files are transferred in either direction. The creation of backup copies and the reinstatement of a backed-up file may be automatic, or may require direct intervention by the end user. The backup medium is typically an external hard disk, a solid-state drive, or a modified form of a video system, often connected via plug-in, plug-out system such as USB.

6. 6. Read-only library—Complete files, and collections of associated files relating to a single application, held on read-only devices such as DVD, or on a device with some form of write-protection control. Complete sets of files are read into the system from the read-only device, but for obvious reasons there are never any transfers from the system to the device.