What is processor cache

At the opposite end of the spectrum, we have direct-mapped caches. A direct-mapped cache is a cache where each cache block can contain one and only one block of main memory.

This type of cache can be searched extremely quickly, but since it maps to memory locations, it has a low hit rate. In between these two extremes are n- way associative caches.

An eight-way associative cache means that each block of main memory could be in one of eight cache blocks. The next two slides show how hit rate improves with set associativity. Keep in mind that things like hit rate are highly particular — different applications will have different hit rates. So why add continually larger caches in the first place? Because each additional memory pool pushes back the need to access main memory and can improve performance in specific cases.

Each stair step represents a new level of cache. Larger caches are both slower and more expensive. At six transistors per bit of SRAM 6T , cache is also expensive in terms of die size, and therefore dollar cost. The performance impact of adding a CPU cache is directly related to its efficiency or hit rate; repeated cache misses can have a catastrophic impact on CPU performance. The following example is vastly simplified but should serve to illustrate the point.

Imagine that a CPU has to load data from the L1 cache times in a row. The L1 cache has a 1ns access latency and a percent hit rate. It, therefore, takes our CPU nanoseconds to perform this operation.

Haswell-E die shot click to zoom in. The repetitive structures in the middle of the chip are 20MB of shared L3 cache. Now, assume the cache has a 99 percent hit rate, but the data the CPU actually needs for its th access is sitting in L2, with a cycle 10ns access latency.

A cache hit means your system was able to successfully retrieve the information you needed from this storage. The concept of a processor cache falls within a more general computer science process called locality of reference. Generally, these locations are near each other. This is done through instructions written as loops and subroutine calls. There are two ways data moves from main memory to the cache memory of a computer.

With temporal locality , the computer knows that information will soon be used, so it's stored in cache memory to make retrieval easier. Early CPUs used only one level of cache, but as technology evolved, it became necessary to separate these memory retrieval areas so that systems could keep up. The three levels are:. By monitoring the cache memory in the microprocessor , you can take a look at the hit ratio to see where performance may be lagging. This is done by upgrading your CPU and cache chips.

Of course, the easiest way to do this is to just buy a new computer — but if it otherwise performs perfectly, it may be worth a partial upgrade. However, if you have an older motherboard, it may have slots that allow you to just slip in a higher-capacity L2 or L3 cache. The way a cache memory in microprocessor is mapped has also evolved over the years.

However, this tended to slow things down even though it reduced the risk of data loss. That data is stored in the processor cache, then later sent to RAM at scheduled intervals.

If data is old or missing, RAM may grab those updates from the cache to minimize risks, but otherwise, it remains in the cache to keep the computer at peak operating speed. There are three different types of configurations:. If you know anything about random access memory , or RAM, you know that it temporarily stores information. Each memory cell in DRAM has a circuit with a transistor and a capacitor, with the capacitor storing every data bit.

To keep data, DRAM has to refresh every few milliseconds, since information tends to leak from capacitors. Power it back up again, and your cache begins collecting information from scratch.

Virtual memory is something the operating system generates to prevent data loss due to a lack of physical memory at a given time. This saves fetching the instructions and data repeatedly from RAM — a relatively slow process which might otherwise keep the CPU waiting. Transfers to and from cache take less time than transfers to and from RAM. The more cache there is, the more data can be stored closer to the CPU.

Not a lot of physical space is allocated for cache.