Faster Parallel Traversal of Scale Free Graphs at Extreme Scale with Vertex Delegates

Roger Pearce, Maya Gokhale, Nancy M. Amato

Research output: Chapter in Book/Report/Conference proceedingConference contribution

44 Scopus citations

Abstract

© 2014 IEEE. At extreme scale, irregularities in the structure of scale-free graphs such as social network graphs limit our ability to analyze these important and growing datasets. A key challenge is the presence of high-degree vertices (hubs), that leads to parallel workload and storage imbalances. The imbalances occur because existing partitioning techniques are not able to effectively partition high-degree vertices. We present techniques to distribute storage, computation, and communication of hubs for extreme scale graphs in distributed memory supercomputers. To balance the hub processing workload, we distribute hub data structures and related computation among a set of delegates. The delegates coordinate using highly optimized, yet portable, asynchronous broadcast and reduction operations. We demonstrate scalability of our new algorithmic technique using Breadth-First Search (BFS), Single Source Shortest Path (SSSP), K-Core Decomposition, and Page-Rank on synthetically generated scale-free graphs. Our results show excellent scalability on large scale-free graphs up to 131K cores of the IBM BG/P, and outperform the best known Graph500 performance on BG/P Intrepid by 15%
Original languageEnglish (US)
Title of host publicationSC14: International Conference for High Performance Computing, Networking, Storage and Analysis
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages549-559
Number of pages11
ISBN (Print)9781479955008
DOIs
StatePublished - Nov 2014
Externally publishedYes

Fingerprint

Dive into the research topics of 'Faster Parallel Traversal of Scale Free Graphs at Extreme Scale with Vertex Delegates'. Together they form a unique fingerprint.

Cite this