Skip to content

Commit

Permalink
[Doc] fix the giant graph tutorial (dmlc#591)
Browse files Browse the repository at this point in the history
* fix.

* fix.
  • Loading branch information
zheng-da authored Jun 1, 2019
1 parent 350b485 commit 0554824
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 2 deletions.
23 changes: 22 additions & 1 deletion tutorials/models/5_giant_graph/2_giant.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
#
# The graph store has two parts: the server and the client. We need to run
# the graph store server as a daemon before training. We provide a script
# ```run_store_server.py`` <https://github.com/zheng-da/dgl-1/blob/sampling-example/examples/mxnet/sampling/run_store_server.py>`__
# ```run_store_server.py`` <https://github.com/dmlc/dgl/blob/master/examples/mxnet/sampling/run_store_server.py>`__
# that runs the graph store server and loads graph data. For example, the
# following command runs a graph store server that loads the reddit
# dataset and is configured to run with four trainers.
Expand Down Expand Up @@ -335,6 +335,27 @@
#
# |image2|
#
# Scale to giant graphs
# ---------------------
#
# Finally, we would like to demonstrate the scalability of DGL with giant
# synthetic graphs. We create three large power-law graphs with
# `RMAT <http://www.cs.cmu.edu/~christos/PUBLICATIONS/siam04.pdf>`__. Each
# node is associated with 100 features and we compute node embeddings with
# 64 dimensions. Below shows the training speed and memory consumption of
# GCN with neighbor sampling.
#
# ====== ====== ================== ===========
# #Nodes #Edges Time per epoch (s) Memory (GB)
# ====== ====== ================== ===========
# 5M 250M 4.7 8
# 50M 2.5B 46 75
# 500M 25B 505 740
# ====== ====== ================== ===========
#
# We can see that DGL can scale to graphs with up to 500M nodes and 25B
# edges.
#
# .. |image0| image:: https://s3.us-east-2.amazonaws.com/dgl.ai/tutorial/sampling/arch.png
# .. |image1| image:: https://s3.us-east-2.amazonaws.com/dgl.ai/tutorial/sampling/NUMA_speedup.png
# .. |image2| image:: https://s3.us-east-2.amazonaws.com/dgl.ai/tutorial/sampling/whole_speedup.png
Expand Down
7 changes: 6 additions & 1 deletion tutorials/models/5_giant_graph/README.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,9 @@ Training on giant graphs
<https://github.com/dmlc/dgl/tree/master/examples/pytorch/sampling>`__:
we can perform neighbor sampling and control-variate sampling to train
graph convolution networks and its variants on a giant graph.

* **Scale to giant graphs** `[tutorial] <5_giant_graph/2_giant.html>`__
`[MXNet code] <https://github.com/dmlc/dgl/tree/master/examples/mxnet/sampling>`__
`[Pytorch code]
<https://github.com/dmlc/dgl/tree/master/examples/pytorch/sampling>`__:
We provide two components (graph store and distributed sampler) to scale to
graphs with hundreds of millions of nodes.

0 comments on commit 0554824

Please sign in to comment.