More detail explanation of data and fix comments.

textsum/README.md

@@ -27,6 +27,7 @@ for example vocabulary format. In <b>How To Run</b> below, users can use toy
 data and vocab provided in the data/ directory to run the training by replacing
 the data directory flag.
 
+
 <b>Experiment Result</b>
 
 8000 examples from testset are sampled to generate summaries and rouge score is
@@ -73,10 +74,13 @@ Install TensorFlow and Bazel.
 
 ```shell
 # cd to your workspace
-# clone the code to your workspace and create empty WORKSPACE file.
-# move the data to your workspace. If don't have full dataset yet, copy
-# the toy data from the data/ directory from code directory and rename
-# the files.
+# 1. Clone the textsum code to your workspace 'textsum' directory.
+# 2. Create an empty 'WORKSPACE' file in your workspace.
+# 3. Move the train/eval/test data to your workspace 'data' directory.
+#    In the following example, I named the data training-*, test-*, etc.
+#    If your data files have different names, update the --data_path.
+#    If you don't have data but want to try out the model, copy the toy
+#    data from the textsum/data/data to the data/ directory in the workspace.
 ls -R
 .:
 data  textsum  WORKSPACE

textsum/data.py

@@ -70,11 +70,15 @@ def NumIds(self):
     return self._count
 
 
-def ExampleGen(recordio_path, num_epochs=None):
-  """Generates tf.Examples from path of recordio files.
+def ExampleGen(data_path, num_epochs=None):
+  """Generates tf.Examples from path of data files.
+
+    Binary data format: <length><blob>. <length> represents the byte size
+    of <blob>. <blob> is serialized tf.Example proto. The tf.Example contains
+    the tokenized article text and summary.
 
   Args:
-    recordio_path: CNS path to tf.Example recordio
+    data_path: path to tf.Example data files.
     num_epochs: Number of times to go through the data. None means infinite.
 
   Yields:
@@ -86,7 +90,7 @@ def ExampleGen(recordio_path, num_epochs=None):
   while True:
     if num_epochs is not None and epoch >= num_epochs:
       break
-    filelist = glob.glob(recordio_path)
+    filelist = glob.glob(data_path)
     assert filelist, 'Empty filelist.'
     random.shuffle(filelist)
     for f in filelist: