Cashmere: Introduction


The Cashmere system provided shared memory with a release-consistency model using advanced network interfaces. This document describes the implementation of Cashmere on the Memory Channel network.

The Memory Channel (MC) network provides applications with a global address space using memory mapped transfer regions. A region can be mapped into a process's address space as either a transmit or a receive region. Data written into a transmit region will appear in any receive regions with the same global identifier. In the figure below, the lined region is mapped as both transmit and receive regions on node 1 and a receive region on node 2. The gray region is mapped as a receive region on node 1 and a transmit region on node 2.

Cashmere creates two areas within a process's address space. The higher area contains the process's private copy of shared memory. The lower area contains MC regions used to maintain memory consistency between processes. For every shared memory page there is a page allocated in the process's private copy of shared memory and another allocated in the MC area. If a process is located on the home node for a shared page, the page is mapped into a MC receive region. If a process is not located on the home node, then the page is mapped into a MC transmit region.

All modifications to shared memory are made using write instructions to the private copy of shared memory, see figure 1. These private writes are duplicated into the MC area. If the process is on the page's home node, the duplicate writes are kept local. If the process is not on the home node, the duplicate writes are transmitted through the Memory Channel to the MC receive region on the home node.