Skip to end of metadata
Go to start of metadata

These are the notes for the skype conference call held on Wednesday April 10 2013, on the progress and activities of the Social Network Benchmark task force (11:00-12:30).

Participants:

  • Barry Biship (AD), Norbert Martínez (UPC); Orri Erling (OGL); Peter Boncz (VU); Alex Averbuch (NEO), Andrey Gubichev (TUM) 

Summary:

Facebook released LinkBench benchmark

  • NEO, UPC, VU performed cursory evaluation of operations/access patterns. Outcome: not typical of graph access patterns (nothing beyond “retrieve edges for a node”)
  • UPC looks at distributions:
    • more realistic than in SIB, with better average clustering coefficient, diameter and hop plot
    • some problems with the generator and the edge count not related to node count. It could be a problem with the generator, or an invalid configuration file, but perhaps we should wait for the paper (it is very probable there are bugs in the LinkBench code)

Chinese GRADES paper mentions benchmark based on Social Network use case:

  • Uses real "Chinese Twitter" dataset
  • Compares with S3G2, finds it lacking (please include details as to how it was lacking, I haven't read the paper yet)

Orri / Analytics

  • Orri implemented a number of the algorithms from the Pregel paper in Virtuoso 
  • Open questions
    • Which analytics algorithms can be made deterministic
    • By choosing deterministic algorithms that are implementable in a BSP model, perhaps we can ensure determinism in distributed/parallel environments
    • How do we verify correctness of vendor implementations?
      • Enforce exact results
      • Check if results are "close enough"
      • ..?

Action Points:

  1. 3SG2 going forward
    1. UPC already working on code base
    2. Duc to go to UPC to help (when? what are the goals for his visit?)
    3. Improve distributions - based on LinkBench, “Chinese Twitter”
  2. “Twitter” dataset from China
    1. Get dataset
    2. Find the code for their fork of YCSB
  3. Things to do before next TUC
    1. Norbert & Peter to prepare talks on state of SN task force
    2. Work on a comprehensive list of queries for Social Networks, so industry partners can help scrutinize them
      1. Create new wiki page for this - clean up old pages and add to new page - Peter to create page, Alex to migrate content with help from Norbert (and Orri?)
  4. Regarding other benchmark frameworks/projects
    1. LinkBench: someone (Peter?) to contact authors and possibly invite to LDBC
      1. Ask for early access to paper too
    2. Alex looking into integration of YCSB & LinkBench
    3. MIT/Intel working on graph benchmarking - Peter to contact
    4. “Chinese Twitter” used YCSB - look for code
    5. “Chinese Twitter” dataset: can we get it, use to compare with and improve S3G2
  • No labels