Parallel and Distributed Systems
Possible topics for student-led classes
The following are listed in no particular order, but I’ve highlighted in
red the ones I’d particularly like to see someone
Also note that this is in no way an exhaustive list; feel free to
suggest a topic of your own!
- The HPCS languages
- Fortress (Sun), X10 (IBM), and Chapel (Cray). These three
languages, developed under a DARPA-sponsored program a decade ago,
have proven very influential.
Among other things, they include
transactions, control abstraction, and explicit locality
You might also want to cover Habañero Java, which is derived from
- Partitioned global address space (PGAS)
- These are arguably the most mature parallel langauges for
high-end scientific computing. You’ll want to cover
co-array Fortran, UPC, and Titanium.
- Parallel functional languages
- Several people have argued that functional languages are the
ideal notation in which to express parallel programs, becuase the
lack of side effects makes parallelization easy. Languages
you might want to cover include modern (concurrent) Haskell,
pH (an earlier parallel Haskell), Erlang, Scala, or F#.
- CUDA, OpenCL, and OpenACC
- The future of high-end computing appears to belong,
increasingly, to data-parallel machines. Present, compare, and
contrast these leading data-parallel notations.
- Other parallel languages and notations
- Compare and contrast some subset of HPF, Java, C#, Cilk, Ada, TBB,
CNC, Ruby, ...
- Race detectors
- If data races are bugs, we need better tools to find them.
Survey hardware and software techniques to identify data races
and/or mitigate their impact.
- Deterministic parallel computing
- In the extreme case, data races can be eliminated by ensuring
that a program or execution is always
deterministic—oblivious to decisions by the
underlying thread scheduler. Explore either languages (e.g.,
Deterministic Parallel Java) or hardware/software systems to tame
- Model checking
- Once suitable only for toy problems, model checking has become a
first-class tool for proving the correcness of real-world parallel
algorithms. Explain its capabilities and limitations.
- GPGPU architecture
- GPUs don’t look like CPUs.
Survey the state of the art from NVIDIA, AMD, and Intel.
- Graph runtimes
- Survey systems for large-scale graph computations, such as
Pregel, GraphLab, Grappa, Giraph, and Hive. The article by McCune
et al. is a good starting point for information.
- In the tradition of MapReduce and Pregel, Google has introduced
the TensorFlow system for scalable machine learning.
Teach us all about it!
Alternatively, explore the various machine-learning hardware designs
currently under development.
- The top 500 list
- Learn about the world’s biggest machines, and tell us what
- Advanced topics in large-scale cache coherence—
- beyond the confines of a single bus. Compare and contrast
several large-scale commercial machines.
Alternatively, present the token coherence scheme of Martin,
Hill, and Wood, which provides a highly optimized implementation of
sequential consistency, or the Amoeba and SPATL projects, from the
group of Prof. Sandhya Dwarkadas.
- Software distributed shared memory (S-DSM)
- Compare Ivy, Munin, Treadmarks, HLRC, Cashmere, etc.
- Advanced topics in locking and synchronization
- Possibilities include
- scheduler-conscious synchronization
- timeout capability
- NUMA adaptivity
- CORBA and .NET
- These are competing standards for distributed object-oriented
computing. How do they compare?
- ssh, firewalls, proxies, tunneling, VPN, etc.
- What are the tradeoffs in the design of security mechanisms for
remote computer access?
- Distributed file systems
- There are lots of these around, from the well-established,
feature-rich AFS to the experimental OceanStore and FarSite. So how
come most of us still use NFS and SMB?
- Peer-to-peer sharing, overlay networks, distributed
hash tables, etc.
- This stuff was really hot a few years back. Was it really
good for anything other than stealing movies?
- Active devices and storage-area networks
- Increasingly people are proposing that storage devices be
connected directly to the network, the way processors and printers are.
Network-attached devices tend to have quite a bit of processing power.
Can we do cool stuff by downloading application code?
- System area networks and software-defined networking
- These are very high bandwidth, very low latency,
suitable for supercomputers and data-center clusters.
Look into Infiniband, GSN, and Cray's proprietary networks.
- The GRID
- Another hot topic from a few years back, the GRID sought to connect
high-end computational centers (e.g., the National Labs) into a
global system, with convenient naming, authentication, remote access,
data sharing, resource allocation, etc. See the book (2nd edition)
by Foster and Kesselman.
28 April 2017 /