Cashmere: Program Initialization

Initialization Sample

The following program demonstrates how the csm_init_* calls should be used.
  struct { int values[4096]; } *pGlobalData1;
  struct { int others[256]; } *pGlobalData2;
  void main(int arg, char **argv)
          csm_memory_page_round(sizeof(*pGlobalData1)) +
      csm_init_start(&argc, &argv);
      if (csm_pid == 0) {
          pGlobalData1 = csm_malloc(sizeof(*pGlobalData1));
          csm_distribute(&pGlobalData1, sizeof(pGlobalData1));
          pGlobalData2 = csm_malloc(sizeof(*pGlobalData2));
          csm_distribute(&pGlobalData2, sizeof(pGlobalData2));
      // computation using global variable...
      pGlobalData1->values[0] += pGlobalData2->others[csm_pid];

The user start's the program. The csm_init_memory_size call sets the maximum bytes of shared memory that will be used. Calling csm_init_memory_size is optional. The maximum memory size can also be set with the -m:size Cashmere command line option. If no size is specified with either csm_init_memory_size of -m:size, the default is 512MB.

The csm_init_start call initializes the Cashmere protocol and forks the child processes. By default, the number of processes started is equal to the number of processors in the cluster. The number of processes can be modified by either setting the csm_num_pid in the application code or using the -p:processes command line option. The number of processes can also be modified by setting csm_num_nid or using the -n:nodes command line option to set the maximum number of nodes used in the cluster or by setting csm_num_cid or using the -c:cpus command line option to set the maximum number of computer processors used on each node. The system is smart enough to figure out exactly how many nodes a cluster has and how many processors are in each node.

After csm_init_start has been called, it sets the following global variables:

  • csm_num_pid: Number of compute processes executing.
  • csm_num_nid: Number of nodes executing compute processes.
  • csm_num_cid: Number of CPUs executing compute processes on the process' node.
  • csm_pid: Process' process identifier. Values of csm_pid are contiguous running from 0 to csm_num_pid-1. The only exception is that protocol processors have csm_pid ranging from CSM_MAX_PROCS to CSM_MAX_PROCS+csm_num_nid-1.
  • csm_nid: Process' node identifier. Ranges from 0 to csm_num_nid-1.
  • csm_cid: Process' CPU identifier. Ranges from 0 to csm_num_cid-1. Protocol processors have a csm_cid of csm_num_cid.
  • csm_num_locks: Number of locks available to the application. Lock identifiers range from 0 to csm_num_locks-1.
  • csm_num_barriers: Number of barriers available to the application.
  • csm_num_flags: Number of flags available to the application.
  • csm_num_shared_memory: Number of bytes of shared memory available to the application.
  • csm_num_shared_pages: Number of pages of shared memory available to the application.

    The application calls csm_init_complete after it has finished initialization. Between the call to csm_init_start and csm_init_complete, only the process with csm_pid = 0 can touch shared memory.

    Applications allocate shared memory by calling csm_malloc. Pointers that reside in private memory but point into shared memory can be distributed to all applications by using the csm_distribute call. Calls to csm_distribute are valid only between the call to csm_init_start and csm_init_complete.

    Finally, csm_exit terminates a process. Before exiting, process 0 (csm_pid == 0), csm_exit waits until all other processes have finished. and outputs runtime statistics.

    [API] [TOC]