Main Page/hybrid prog

This is the document page for hybrid programming proposal for NekCEM (some notes taken from DOE Exascale workshop).

Programming Model Approaches

• hybrid/evolutionary: MPI+ __ ?
  • MPI for inter-node prog, since # notes and inter-node concerns not expected to change dramatically
    • support for hybrid programming/interoperability
    • purer one-sided communications; active messages
    • asynchronous collectives
  • something else for intra-node
    1. OpenMP (Shared memory, aka Global Address Space)
       • introduction of locality-oriented concepts?
       • efforts in OpenMP 3.0 ?
    2. PGAS languages (Partitioned Global Address Space)
       • already support a notation of locality in a shared namespace
       • UPC (Unified Parallel C)/CAF need to relax strictly SPMD execution model
    3. Sequoia
       • support a strong notation of vertical locality

• unified/holistic: __ ?
  • a single notation for inter- and intra-node programming?
  • traditional PGAS languages: UPC, CAF, Titanium
    • require extension to handle nested parallelism, vertical locality
  • HPCS languages: Chapel, X10, Fortress(?)
    • designed with locality and post-SPMD parallelism in mind
  • other candidates: Charm++, Global Arrays, Parallel X, ...

• others
  • mainstream multi-core/GPU language: (sufficient promise to be funded?)
  • domain-specific language
    • fit your problem?
    • should focus on more general solutions
  • functional languages
    • never heavily adopted in mainstream or HPC
    • copy-on-write optimization and alias analysis?
  • parallel scripting languages?

Expectation

• parallelism: nested, dynamic, loosely-coupled, data-driven (i.e. post-SPMD programming/execution models)
  • to take advantage of architecture
  • to better support load balancing and resilience

• locality: concepts for vertical control as well as horizontal (i.e. locality within a node rather than simply between nodes)

Tools: Debuggers, perf. analysis

• challenges
  • need aggregation to hide details
  • need to report info in user's terms
• good area for innovation (e.g. execution visualization to understand mapping of code to hardware)