CSC 2/458

Parallel and Distributed Systems

Spring 2019.

Possible topics for student-led classes

The following are listed in no particular order, but I’ve highlighted in red the ones I’d particularly like to see someone take.  Also note that this is in no way an exhaustive list; feel free to suggest a topic of your own! 

The HPCS languages
Fortress (Sun), X10 (IBM), and Chapel (Cray).  These three languages, developed under a DARPA-sponsored program a decade ago, have proven very influential.  Among other things, they include transactions, control abstraction, and explicit locality management.  You might also want to cover Habañero Java, which is derived from X10. 

Partitioned global address space (PGAS) languages
These are arguably the most mature parallel langauges for high-end scientific computing.  You’ll want to cover co-array Fortran, UPC, and Titanium. 

Parallel functional languages
Several people have argued that functional languages are the ideal notation in which to express parallel programs, becuase the lack of side effects makes parallelization easy.  Languages you might want to cover include modern (concurrent) Haskell, pH (an earlier parallel Haskell), Erlang, Scala, or F#. 

CUDA, OpenCL, and OpenACC
The future of high-end computing appears to belong, increasingly, to data-parallel machines.  Present, compare, and contrast these leading data-parallel notations. 

Other parallel languages and notations
Compare and contrast some subset of HPF, Java, C#, Cilk, Ada, TBB, CNC, Ruby, ...

Race detectors
If data races are bugs, we need better tools to find them.  Survey hardware and software techniques to identify data races and/or mitigate their impact. 

Deterministic parallel computing
In the extreme case, data races can be eliminated by ensuring that a program or execution is always deterministic—oblivious to decisions by the underlying thread scheduler.  Explore either languages (e.g., Deterministic Parallel Java) or hardware/software systems to tame conventional languages. 

Model checking
Once suitable only for toy problems, model checking has become a first-class tool for proving the correcness of real-world parallel algorithms.  Explain its capabilities and limitations. 

GPGPU architecture
GPUs don’t look like CPUs.  Survey the state of the art from NVIDIA, AMD, and Intel. 

Graph runtimes
Survey systems for large-scale graph computations, such as Pregel, GraphLab, Grappa, Giraph, and Hive.  The article by McCune et al. is a good starting point for information. 

TensorFlow
In the tradition of MapReduce and Pregel, Google has introduced the TensorFlow system for scalable machine learning.  Teach us all about it!  Alternatively, explore the various machine-learning hardware designs currently under development. 

The top 500 list
Learn about the world’s biggest machines, and tell us what they’re like. 

Advanced topics in large-scale cache coherence—
beyond the confines of a single bus.  Compare and contrast several large-scale commercial machines.  Alternatively, present the token coherence scheme of Martin, Hill, and Wood, which provides a highly optimized implementation of sequential consistency, or the Amoeba and SPATL projects, from the group of Prof. Sandhya Dwarkadas

Software distributed shared memory (S-DSM)
Compare Ivy, Munin, Treadmarks, HLRC, Cashmere, etc.

Advanced topics in locking and synchronization
Possibilities include

CORBA and .NET
These are competing standards for distributed object-oriented computing.  How do they compare? 

ssh, firewalls, proxies, tunneling, VPN, etc. 
What are the tradeoffs in the design of security mechanisms for remote computer access? 

Distributed file systems
There are lots of these around, from the well-established, feature-rich AFS to the experimental OceanStore and FarSite.  So how come most of us still use NFS and SMB? 

Peer-to-peer sharing, overlay networks, distributed hash tables, etc.
This stuff was really hot a few years back.  Was it really good for anything other than stealing movies? 

Active devices and storage-area networks
Increasingly people are proposing that storage devices be connected directly to the network, the way processors and printers are.  Network-attached devices tend to have quite a bit of processing power.  Can we do cool stuff by downloading application code? 

System area networks and software-defined networking
These are very high bandwidth, very low latency, and—increasingly—software-configurable networks, suitable for supercomputers and data-center clusters. Look into Infiniband, GSN, and Cray's proprietary networks. 

The GRID
Another hot topic from a few years back, the GRID sought to connect high-end computational centers (e.g., the National Labs) into a global system, with convenient naming, authentication, remote access, data sharing, resource allocation, etc.  See the book (2nd edition) by Foster and Kesselman. 

Last Change: 01 May 2019 / Michael Scott's email address