Simplifying Scalable Graph Processing with a Domain-Specific Language

Sungpack Hong, Semih Salihoglu, Jennifer Widom, Kunle Olukotun

Research output: Chapter in Book/Report/Conference proceedingConference contribution

32 Scopus citations

Abstract

Large-scale graph processing, with its massive data sets, requires distributed processing. However, conventional frameworks for distributed graph processing, such as Pregel, use non-traditional programming models that are well-suited for parallelism and scalability but inconvenient for implementing non-trivial graph algorithms. In this paper, we use Green-Marl, a Domain-Specific Language for graph analysis, to intuitively describe graph algorithms and extend its compiler to generate equivalent Pregel implementations. Using the semantic information captured by Green-Marl, the compiler applies a set of transformation rules that convert imperative graph algorithms into Pregel's programming model. Our experiments show that the Pregel programs generated by the Green-Marl compiler perform similarly to manually coded Pregel implementations of the same algorithms. The compiler is even able to generate a Pregel implementation of a complicated graph algorithm for which a manual Pregel implementation is very challenging.
Original languageEnglish (US)
Title of host publicationProceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization - CGO '14
PublisherAssociation for Computing Machinery (ACM)
ISBN (Print)9781450326704
DOIs
StatePublished - 2014
Externally publishedYes

Fingerprint

Dive into the research topics of 'Simplifying Scalable Graph Processing with a Domain-Specific Language'. Together they form a unique fingerprint.

Cite this