Scaling Implicit Parallelism via Dynamic Control Replication
This program is tentative and subject to change.
We present dynamic control replication, a run-time program analysis that enables scalable execution of implicitly parallel programs on large machines through a distributed and efficient dynamic dependence analysis. Dynamic control replication distributes dependence analysis by executing multiple copies of an implicitly parallel program while ensuring that they still collectively behave as a single execution. By distributing and parallelizing the dependence analysis, dynamic control replication supports efficient, on-the-fly computation of dependences for programs with arbitrary control flow at scale. We describe an asymptotically scalable algorithm for implementing dynamic control replication that maintains the sequential semantics of implicitly parallel programs.
An implementation of dynamic control replication in the Legion runtime delivers the same programmer productivity as writing in other implicitly parallel programming models, such as Dask or TensorFlow, while providing better performance (11.4X and 14.9X respectively in our experiments), and scalability to hundreds of nodes. We also show that dynamic control replication provides good absolute performance and scaling for HPC applications, competitive in many cases with explicitly parallel programming systems.
This program is tentative and subject to change.
Mon 1 Mar Times are displayed in time zone: Eastern Time (US & Canada) change
11:10 - 12:10: Session 3. Compilers, Analysis, SynthesisMain Conference | |||
11:10 - 11:25 Talk | Synthesizing Optimal Collective Algorithms Main Conference Zixian CaiAustralian National University, Zhengyang LiuUniversity of Utah, Saeed MalekiMicrosoft Research, Madan MusuvathiMicrosoft Research, Todd MytkowiczMicrosoft Research, Jacob Nelson, Olli SaarikiviMicrosoft Research, Redmond | ||
11:25 - 11:40 Talk | Parallel Binary Code Analysis Main Conference Xiaozhu MengRice University, Jonathon AndersonRice University, John Mellor-CrummeyRice University, Mark W. KrentelRice University, Barton P. MillerUniversity of Wisconsin - Madison, Srđan MilakovićRice University | ||
11:40 - 11:55 Talk | Compiler Support for Near Data Computing Main Conference Mahmut Taylan KandemirPenn State University, USA, Jihyun RyooPenn State University, USA, Xulong TangUniversity of Pittsburgh, USA, Mustafa KarakoyTUBITAK-BILGEM, Turkey | ||
11:55 - 12:10 Talk | Scaling Implicit Parallelism via Dynamic Control Replication Main Conference Michael BauerNVIDIA, Wonchan LeeNVIDIA, Elliott SlaughterSLAC National Accelerator Laboratory, Zhihao JiaCarnegie Mellon University, Mario Di RenzoSapienza University of Rome, Manolis PapadakisNVIDIA, Galen ShipmanLos Alamos National Laboratory, Patrick McCormickLos Alamos National Laboratory, Michael GarlandNVIDIA, Alex AikenStanford Univeristy |