[Example] Add implementation of InfoGraph (dmlc#2644)

* Add implementation of unsupervised model * [Doc] Update Implementor's information * [doc] add index of infograph * [Feature] QM9_v2 Dataset Support * fix a typo * move qm9dataset from data to examples * Update qm9_v2.py * Infograph -> InfoGraph * add implementation and results of semi-supervised model * Update README.md * Update README.md * fix a typo * Remove the duplicated links. * fix some typos * fix typos * update model.py * update collate fn * remove unused functions * Update model.py * add device option * Update evaluate_embedding.py * Update evaluate_embedding.py * Update unsupervised.py * Fix typos * fix bugs * Update README.md Co-authored-by: Mufei Li <[email protected]>
feixian15 · Mar 9, 2021 · 3317522 · 3317522
1 parent a1f59c3
commit 3317522
Show file tree

Hide file tree

Showing 9 changed files with 1,202 additions and 34 deletions.
diff --git a/examples/README.md b/examples/README.md
@@ -13,6 +13,7 @@ The folder contains example implementations of selected research papers related
 | [Graph Convolutional Networks for Graphs with Multi-Dimensionally Weighted Edges](#mwe) | :heavy_check_mark:  |                                  |                           |                    | :heavy_check_mark: |
 | [SIGN: Scalable Inception Graph Neural Networks](#sign)      | :heavy_check_mark:  |                                  |                           |                    | :heavy_check_mark: |
 | [Strategies for Pre-training Graph Neural Networks](#prestrategy) |                     |                                  | :heavy_check_mark:        |                    |                    |
+| [InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization](#infograph) | | | :heavy_check_mark: | | |
 | [Graph Neural Networks with convolutional ARMA filters](#arma) | :heavy_check_mark:  |                                  |                           |                    |                    |
 | [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](#appnp) | :heavy_check_mark:  |                                  |                           |                    |                    |
 | [Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks](#clustergcn) | :heavy_check_mark:  |                                  |                           | :heavy_check_mark: | :heavy_check_mark: |
@@ -87,125 +88,100 @@ The folder contains example implementations of selected research papers related
 - <a name="grand"></a> Feng et al. Graph Random Neural Network for Semi-Supervised Learning on Graphs. [Paper link](https://arxiv.org/abs/2005.11079). 
     - Example code: [PyTorch](../examples/pytorch/grand)
     - Tags: semi-supervised node classification, simplifying graph convolution, data augmentation
-
 - <a name="hgt"></a> Hu et al. Heterogeneous Graph Transformer. [Paper link](https://arxiv.org/abs/2003.01332).
     - Example code: [PyTorch](../examples/pytorch/hgt)
     - Tags: dynamic heterogeneous graphs, large-scale, node classification, link prediction
-
 - <a name="mwe"></a> Chen. Graph Convolutional Networks for Graphs with Multi-Dimensionally Weighted Edges. [Paper link](https://cims.nyu.edu/~chenzh/files/GCN_with_edge_weights.pdf).
     - Example code: [PyTorch on ogbn-proteins](../examples/pytorch/ogb/ogbn-proteins)
     - Tags: node classification, weighted graphs, OGB
-
 - <a name="sign"></a> Frasca et al. SIGN: Scalable Inception Graph Neural Networks. [Paper link](https://arxiv.org/abs/2004.11198).
     - Example code: [PyTorch on ogbn-arxiv/products/mag](../examples/pytorch/ogb/sign), [PyTorch](../examples/pytorch/sign)
     - Tags: node classification, OGB, large-scale, heterogeneous graphs
-
 - <a name="prestrategy"></a> Hu et al. Strategies for Pre-training Graph Neural Networks. [Paper link](https://arxiv.org/abs/1905.12265).
     - Example code: [Molecule embedding](https://github.com/awslabs/dgl-lifesci/tree/master/examples/molecule_embeddings), [PyTorch for custom data](https://github.com/awslabs/dgl-lifesci/tree/master/examples/property_prediction/csv_data_configuration)
     - Tags: molecules, graph classification, unsupervised learning, self-supervised learning, molecular property prediction
-
 - <a name="gnnfilm"></a> Marc Brockschmidt. GNN-FiLM: Graph Neural Networks with Feature-wise Linear Modulation. [Paper link](https://arxiv.org/abs/1906.12192).
     - Example code: [Pytorch](../examples/pytorch/GNN-FiLM)
     - Tags: multi-relational graphs, hypernetworks, GNN architectures
-
 - <a name="gxn"></a> Li, Maosen, et al. Graph Cross Networks with Vertex Infomax Pooling. [Paper link](https://arxiv.org/abs/2010.01804).
     - Example code: [Pytorch](../examples/pytorch/gxn)
     - Tags: pooling, graph classification
-
 - <a name="dagnn"></a> Liu et al. Towards Deeper Graph Neural Networks. [Paper link](https://arxiv.org/abs/2007.09296).
     - Example code: [Pytorch](../examples/pytorch/dagnn)
     - Tags: over-smoothing, node classification
 
 ## 2019
 
+
+- <a name="infograph"></a> Sun et al. InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization. [Paper link](https://arxiv.org/abs/1908.01000). 
+    - Example code: [PyTorch](../examples/pytorch/infograph)
+    - Tags: semi-supervised graph regression, unsupervised graph classification
 - <a name="arma"></a>  Bianchi et al. Graph Neural Networks with Convolutional ARMA Filters. [Paper link](https://arxiv.org/abs/1901.01343).
     - Example code: [PyTorch](../examples/pytorch/arma)
     - Tags: node classification
-
 - <a name="appnp"></a> Klicpera et al. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. [Paper link](https://arxiv.org/abs/1810.05997).
     - Example code: [PyTorch](../examples/pytorch/appnp), [MXNet](../examples/mxnet/appnp)
     - Tags: node classification
-
 - <a name="clustergcn"></a> Chiang et al. Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks. [Paper link](https://arxiv.org/abs/1905.07953).
     - Example code: [PyTorch](../examples/pytorch/cluster_gcn), [PyTorch-based GraphSAGE variant on OGB](../examples/pytorch/ogb/cluster-sage), [PyTorch-based GAT variant on OGB](../examples/pytorch/ogb/cluster-gat)
     - Tags: graph partition, node classification, large-scale, OGB, sampling
-
 - <a name="dgi"></a> Veličković et al. Deep Graph Infomax. [Paper link](https://arxiv.org/abs/1809.10341).
     - Example code: [PyTorch](../examples/pytorch/dgi), [TensorFlow](../examples/tensorflow/dgi)
     - Tags: unsupervised learning, node classification
-
 - <a name="diffpool"></a> Ying et al. Hierarchical Graph Representation Learning with Differentiable Pooling. [Paper link](https://arxiv.org/abs/1806.08804).
     - Example code: [PyTorch](../examples/pytorch/diffpool)
     - Tags: pooling, graph classification, graph coarsening
-
 - <a name="gatne-t"></a> Cen et al. Representation Learning for Attributed Multiplex Heterogeneous Network. [Paper link](https://arxiv.org/abs/1905.01669v2).
     - Example code: [PyTorch](../examples/pytorch/GATNE-T)
     - Tags: heterogeneous graphs, link prediction, large-scale
-
 - <a name="gin"></a> Xu et al. How Powerful are Graph Neural Networks? [Paper link](https://arxiv.org/abs/1810.00826).
     - Example code: [PyTorch on graph classification](../examples/pytorch/gin), [PyTorch on node classification](../examples/pytorch/model_zoo/citation_network), [PyTorch on ogbg-ppa](https://github.com/awslabs/dgl-lifesci/tree/master/examples/property_prediction/ogbg_ppa), [MXNet](../examples/mxnet/gin)
     - Tags: graph classification, node classification, OGB
-
 - <a name="graphwriter"></a> Koncel-Kedziorski et al. Text Generation from Knowledge Graphs with Graph Transformers. [Paper link](https://arxiv.org/abs/1904.02342).
     - Example code: [PyTorch](../examples/pytorch/graphwriter)
     - Tags: knowledge graph, text generation
-
 - <a name="han"></a> Wang et al. Heterogeneous Graph Attention Network. [Paper link](https://arxiv.org/abs/1903.07293).
     - Example code: [PyTorch](../examples/pytorch/han)
     - Tags: heterogeneous graphs, node classification
-
 - <a name="lgnn"></a> Chen et al. Supervised Community Detection with Line Graph Neural Networks. [Paper link](https://arxiv.org/abs/1705.08415).
     - Example code: [PyTorch](../examples/pytorch/line_graph)
     - Tags: line graph, community detection
-
 - <a name="sgc"></a> Wu et al. Simplifying Graph Convolutional Networks. [Paper link](https://arxiv.org/abs/1902.07153).
     - Example code: [PyTorch](../examples/pytorch/sgc), [MXNet](../examples/mxnet/sgc)
     - Tags: node classification
-
 - <a name="dgcnnpoint"></a> Wang et al. Dynamic Graph CNN for Learning on Point Clouds. [Paper link](https://arxiv.org/abs/1801.07829).
     - Example code: [PyTorch](../examples/pytorch/pointcloud/edgeconv)
     - Tags: point cloud classification
-
 - <a name="scenegraph"></a> Zhang et al. Graphical Contrastive Losses for Scene Graph Parsing. [Paper link](https://arxiv.org/abs/1903.02728).
     - Example code: [MXNet](../examples/mxnet/scenegraph)
     - Tags: scene graph extraction
-
 - <a name="settrans"></a> Lee et al. Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks. [Paper link](https://arxiv.org/abs/1810.00825).
     - Pooling module: [PyTorch encoder](https://docs.dgl.ai/api/python/nn.pytorch.html#settransformerencoder), [PyTorch decoder](https://docs.dgl.ai/api/python/nn.pytorch.html#settransformerdecoder)
     - Tags: graph classification
-
 - <a name="wln"></a> Coley et al. A graph-convolutional neural network model for the prediction of chemical reactivity. [Paper link](https://pubs.rsc.org/en/content/articlelanding/2019/sc/c8sc04228d#!divAbstract).
     - Example code: [PyTorch](https://github.com/awslabs/dgl-lifesci/tree/master/examples/reaction_prediction/rexgen_direct)
     - Tags: molecules, reaction prediction
-
 - <a name="mgcn"></a> Lu et al. Molecular Property Prediction: A Multilevel Quantum Interactions Modeling Perspective. [Paper link](https://arxiv.org/abs/1906.11081).
     - Example code: [PyTorch](https://github.com/awslabs/dgl-lifesci/tree/master/examples/property_prediction/alchemy)
     - Tags: molecules, quantum chemistry
-
 - <a name="attentivefp"></a> Xiong et al. Pushing the Boundaries of Molecular Representation for Drug Discovery with the Graph Attention Mechanism. [Paper link](https://pubs.acs.org/doi/10.1021/acs.jmedchem.9b00959).
     - Example code: [PyTorch (with attention visualization)](https://github.com/awslabs/dgl-lifesci/tree/master/examples/property_prediction/pubchem_aromaticity), [PyTorch for custom data](https://github.com/awslabs/dgl-lifesci/tree/master/examples/property_prediction/csv_data_configuration)
     - Tags: molecules, molecular property prediction
-
 - <a name="rotate"></a> Sun et al. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. [Paper link](https://arxiv.org/pdf/1902.10197.pdf).
     - Example code: [PyTorch](https://github.com/awslabs/dgl-ke/tree/master/examples), [PyTorch for custom data](https://aws-dglke.readthedocs.io/en/latest/commands.html)
     - Tags: knowledge graph embedding
-
 - <a name="mixhop"></a> Abu-El-Haija et al. MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing. [Paper link](https://arxiv.org/abs/1905.00067).
     - Example code: [PyTorch](../examples/pytorch/mixhop)
     - Tags: node classification
-
 - <a name="sagpool"></a> Lee, Junhyun, et al. Self-Attention Graph Pooling. [Paper link](https://arxiv.org/abs/1904.08082).
     - Example code: [PyTorch](../examples/pytorch/sagpool)
     - Tags: graph classification, pooling
-
 - <a name="hgp-sl"></a> Zhang, Zhen, et al. Hierarchical Graph Pooling with Structure Learning. [Paper link](https://arxiv.org/abs/1911.05954).
     - Example code: [PyTorch](../examples/pytorch/hgp_sl)
     - Tags: graph classification, pooling
-
 - <a name='hardgat'></a> Gao, Hongyang, et al. Graph Representation Learning via Hard and Channel-Wise Attention Networks [Paper link](https://arxiv.org/abs/1907.04652).
     - Example code: [Pytorch](../examples/pytorch/hardgat)
     - Tags: node classification, graph attention
-
 - <a name='ngcf'></a> Wang, Xiang, et al. Neural Graph Collaborative Filtering. [Paper link](https://arxiv.org/abs/1905.08108).
     - Example code: [Pytorch](../examples/pytorch/NGCF)
     - Tags: Collaborative Filtering, Recommendation, Graph Neural Network 

diff --git a/examples/pytorch/grand/README.md b/examples/pytorch/grand/README.md
@@ -2,16 +2,15 @@
 
 This DGL example implements the GNN model proposed in the paper [Graph Random Neural Network for Semi-Supervised Learning on Graphs]( https://arxiv.org/abs/2005.11079).
 
-Paper link: https://arxiv.org/abs/2005.11079
-
 Author's code: https://github.com/THUDM/GRAND
 
-Contributor: Hengrui Zhang ([@hengruizhang98](https://github.com/hengruizhang98))
+## Example Implementor
+
+This example was implemented by [Hengrui Zhang](https://github.com/hengruizhang98) when he was an applied scientist intern at AWS Shanghai AI Lab.
 
-## Dependecies
+## Dependencies
 - Python 3.7
 - PyTorch 1.7.1
-- numpy
 - dgl 0.5.3
 
 ## Dataset

diff --git a/examples/pytorch/infograph/README.md b/examples/pytorch/infograph/README.md
@@ -0,0 +1,164 @@
+# DGL Implementation of InfoGraph
+This DGL example implements the model proposed in the paper [InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization](https://arxiv.org/abs/1908.01000).
+
+Author's code: https://github.com/fanyun-sun/InfoGraph
+
+## Example Implementor
+
+This example was implemented by [Hengrui Zhang](https://github.com/hengruizhang98) when he was an applied scientist intern at AWS Shanghai AI Lab.
+
+## Dependencies
+
+- Python 3.7
+- PyTorch 1.7.1
+- dgl 0.6.0
+
+## Datasets
+
+##### Unsupervised Graph Classification Dataset:
+
+ 'MUTAG', 'PTC', 'IMDBBINARY'(IMDB-B), 'IMDBMULTI'(IMDB-M), 'REDDITBINARY'(RDT-B), 'REDDITMULTI5K'(RDT-M5K) of dgl.data.GINDataset.
+
+| Dataset         | MUTAG | PTC   | RDT-B  | RDT-M5K | IMDB-B | IMDB-M |
+| --------------- | ----- | ----- | ------ | ------- | ------ | ------ |
+| # Graphs        | 188   | 344   | 2000   | 4999    | 1000   | 1500   |
+| # Classes       | 2     | 2     | 2      | 5       | 2      | 3      |
+| Avg. Graph Size | 17.93 | 14.29 | 429.63 | 508.52  | 19.77  | 13.00  |
+
+**Semi-supervised Graph Regression Dataset:**
+
+QM9 dataset for graph property prediction (regression)
+
+| Dataset | # Graphs | # Regression Tasks |
+| ------- | -------- | ------------------ |
+| QM9     | 130,831  | 12                 |
+
+The 12 tasks are:
+
+| Keys  | Description                                |
+| ----- | :----------------------------------------- |
+| mu    | Dipole moment                              |
+| alpha | Isotropic polarizability                   |
+| homo  | Highest occupied molecular orbital energ   |
+| lumo  | Lowest unoccupied molecular orbital energy |
+| gap   | Gap between 'homo' and 'lumo'              |
+| r2    | Electronic spatial extent                  |
+| zpve  | Zero point vibrational energy              |
+| U0    | Internal energy at 0K                      |
+| U     | Internal energy at 298.15K                 |
+| H     | Enthalpy at 298.15K                        |
+| G     | Free energy at 298.15K                     |
+| Cv    | Heat capavity at 298.15K                   |
+
+## Arguments
+
+##### 	Unsupervised Graph Classification:
+
+###### Dataset options
+
+```
+--dataname        str      The graph dataset name.                Default is 'MUTAG'.
+```
+
+###### GPU options
+
+```
+--gpu              int     GPU index.                             Default is -1, using CPU.
+```
+
+###### Training options
+
+```
+--epochs           int     Number of training periods.            Default is 20.
+--batch_size       int     Size of a training batch.              Default is 128.
+--lr               float   Adam optimizer learning rate.          Default is 0.01.
+--log_interval     int     Interval bettwen two evaluations.	  Default is 1.
+```
+
+###### Model options
+
+```
+--n_layers         int     Number of GIN layers.                  Default is 3.
+--hid_dim          int     Dimension of hidden layers.            Default is 32.
+```
+
+##### 	Semi-supervised Graph Regression:
+
+###### Dataset options
+
+```
+ --target          str     The regression Task.                   Default is 'mu'.
+ --train_num       int     Number of supervised examples.         Default is 5000.
+```
+
+###### GPU options
+
+```
+--gpu              int     GPU index.                             Default is -1, using CPU.
+```
+
+###### Training options
+
+```
+--epochs           int     Number of training periods.            Default is 200.
+--batch_size       int     Size of a training batch.              Default is 20.
+--val_batch_size   int     Size of a validation batch.            Default is 100.
+--lr               float   Adam optimizer learning rate.          Default is 0.001.
+```
+
+###### Model options
+
+```
+--hid_dim          int     Dimension of hidden layers.            Default is 64.
+--reg              int     Regularization weight.                 Default is 0.001.
+```
+
+## How to run examples
+
+Training and testing unsupervised model on MUTAG.
+
+ (As graphs in these datasets are quite small and sparse, moving graphs from cpu to gpu would take a longer time than training, we recommend using **cpu** for these datasets).
+
+```bash
+# MUTAG:
+python unsupervised.py --dataname MUTAG --n_layers 4 --hid_dim 32
+```
+
+Replace 'MUTAG' with dataname in ['MUTAG', 'PTC', 'IMDBBINARY', 'IMDBMULTI', 'REDDITBINARY', 'REDDITMULTI5K'] if you'd like to try other datasets.
+
+Training and testing semi-supervised model on QM9 for graph property 'mu' with gpu.
+
+```bash
+# QM9:
+python semisupervised.py --gpu 0 --target mu
+```
+
+Replace 'mu' with other target names above.
+
+## 	Performance
+
+The hyperparameter setting in our implementation is identical to that reported in the paper.
+
+##### Unsupervised Graph Classification:
+
+|      Dataset      | MUTAG |  PTC  | RDT-B | RDT-M5K | IMDB-B | IMDB-M |
+| :---------------: | :---: | :---: | :---: | ------- | ------ | ------ |
+| Accuracy Reported | 89.01 | 61.65 | 82.50 | 53.46   | 73.03  | 49.69  |
+|        DGL        | 89.88 | 63.54 | 88.50 | 56.27   | 72.70  | 50.13  |
+
+* REDDIT-M dataset would take a quite long time to load and evaluate. 
+
+##### Semisupervised Graph Regression on QM9:
+
+Here we only provide the results of 'mu', 'alpha', 'homo'.
+
+
+
+|      Target       |   mu   | alpha  |  homo  |
+| :---------------: | :----: | :----: | :----: |
+|   MAE Reported    | 0.3169 | 0.5444 | 0.0060 |
+| The authors' code | 0.2411 | 0.5192 | 0.1560 |
+|        DGL        | 0.2355 | 0.5483 | 0.1581 |
+
+* The source of QM9 Dataset has changed so there's a gap between the MAE reported in the paper and that we reprodcued.
+* See this [issue](https://github.com/fanyun-sun/InfoGraph/issues/8) for authors' response.