upload evaluation version and some pretrained models

README.md

@@ -1,16 +1,16 @@
-# HR-Depth: High Resolution Self-Supervised Depth Estimation
+# HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation
 
 This is the official implementation for training and testing depth estimation using the model proposed in
 
->HR-Depth: High Resolution Self-Supervised Depth Estimation
+>HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation
 >
 >Xiaoyang Lyu, Liang Liu, Mengmeng Wang, Xin Kong, Lina Liu, Yong Liu*, Xinxin Chen and Yi Yuan.
 
 This paper has been accepted by AAAI 2021.
 
 ![Qualitative_Result](./images/hr_depth.gif)
 
-Our paper, code and models will be released soon.
+**Note:** We temporarily release the evaluation version and some pretrained models of our paper. The training codes are modified according to [Monodepth2](https://github.com/nianticlabs/monodepth2), and we will release them soon. 
 
 # Quantitative Results
 
@@ -20,4 +20,83 @@ Our paper, code and models will be released soon.
 
 ## Lite-HR-Depth Results
 
-![Quantitative_result_2](./images/Quantitative_result_lite.png)
+![Quantitative_result_2](./images/Quantitative_result_lite.png)
+
+# Usage
+
+## Requirements
+
+Assuming a fresh Anaconda distribution, you can install the dependencies with:
+
+```shell
+conda install pytorch=1.5.0 torchvision=0.6.0 -c pytorch
+conda install opencv=4.2
+pip install scipy=1.4.1
+```
+
+## Pretrained Model
+
+We provided pretrained model as follow:
+
+| Model Name                                                   | Resolution      | Dataset | Supervision | Abs_Rel | $\delta<1.25$ | $\delta<1.25^2$ | $\delta<1.25^3$ |
+| ------------------------------------------------------------ | --------------- | ------- | ----------- | ------- | ------------- | --------------- | --------------- |
+| [HR_Depth_CS_K_MS_$640\times192$](http://hr-depth-pretrain-model.s3.amazonaws.com/HR_Depth_CS_K_MS_640x192.zip) | $640\times192$  | CS+K    | MS          | 0.104   | 0.893         | 0.964           | 0.983           |
+| [HR_Depth_K_MS_$1024\times320$](http://hr-depth-pretrain-model.s3.amazonaws.com/HR_Depth_K_MS_1024x320.zip) | $1024\times320$ | K       | MS          | 0.101   | 0.899         | 0.966           | 0.983           |
+| [HR_Depth_K_M_$1280\times384$](http://hr-depth-pretrain-model.s3.amazonaws.com/HR_Depth_K_M_1280x384.zip) | $1280\times384$ | K       | M           | 0.104   | 0.894         | 0.966           | 0.984           |
+| [Lite_HR_Depth_K_T_$1280\times384$](http://hr-depth-pretrain-model.s3.amazonaws.com/Lite_HR_Depth_K_T_1280x384.zip) | $1280\times384$ | K       | T           | 0.104   | 0.893         | 0.967           | 0.985           |
+
+## KITTI training data
+
+You can download the entire [KITTI_raw dataset]() by running:
+
+```shell
+wget -i splits/kitti_archives_to_download.txt -P kitti_data/
+```
+
+Then unzip with
+
+```shell
+cd kitti_data
+unzip "*.zip"
+cd ..
+```
+
+<font color=red>**Warning:**</font> <font color=red>The size of this dataset is about 175GB, so make sure you have enough space to unzip them.</font>
+
+## KITTI evaluation
+
+`--data_path`:  **path of KITTI dataset**
+`--load_weights_folder`: **path of models**
+`--HR_Depth`: **inference by HR-Depth**
+`--Lite_HR_Depth`: **inference by Lite-HR-Depth**
+
+To prepare the ground truth depth maps run:
+
+```shell
+python export_gt_depth.py --data_path ./kitti_RAW
+```
+
+assuming that you have placed the KITTI RAW dataset in the default location of `./kitti_data`.
+
+If you want to generate ground truth map, please run:
+
+```shell
+python export_gt_depth.py --data_path ./kitti_RAW --split eigen	
+```
+
+We also provide ground truth value in `splits/eigen/gt_depths.npz`.
+
+For HR-Depth:
+
+```shell
+python evaluate_depth.py --data_path ./kitti_RAW --load_weights_folder ./models/HR_Depth_CS_K_MS_640x192 --HR_Depth
+
+python evaluate_depth.py --data_path ./kitti_RAW --load_weights_folder ./models/HR_Depth_K_M_1280x384 --HR_Depth 
+```
+
+For Lite-HR-Depth:
+
+```shell
+python evaluate_depth.py --data_path ./kitti_RAW --load_weights_folder ./models/Lite_HR_Depth_K_T_1280x384 --Lite_HR_Depth
+```
+

datasets/__init__.py

@@ -0,0 +1 @@
+from .kitti_dataset import KITTIRAWDataset, KITTIOdomDataset, KITTIDepthDataset

datasets/kitti_dataset.py

@@ -0,0 +1,123 @@
+from __future__ import absolute_import, division, print_function
+
+import os
+import skimage.transform
+import numpy as np
+import PIL.Image as pil
+
+from kitti_utils import generate_depth_map
+from .mono_dataset import MonoDataset
+
+
+class KITTIDataset(MonoDataset):
+    """Superclass for different types of KITTI dataset loaders
+    """
+    def __init__(self, *args, **kwargs):
+        super(KITTIDataset, self).__init__(*args, **kwargs)
+
+        self.K = np.array([[0.58, 0, 0.5, 0],
+                           [0, 1.92, 0.5, 0],
+                           [0, 0, 1, 0],
+                           [0, 0, 0, 1]], dtype=np.float32)
+
+        self.full_res_shape = (1242, 375)
+        self.side_map = {"2": 2, "3": 3, "l": 2, "r": 3}
+
+    def check_depth(self):
+        line = self.filenames[0].split()
+        scene_name = line[0]
+        frame_index = int(line[1])
+
+        velo_filename = os.path.join(
+            self.data_path,
+            scene_name,
+            "velodyne_points/data/{:010d}.bin".format(int(frame_index)))
+
+        return os.path.isfile(velo_filename)
+
+    def get_color(self, folder, frame_index, side, do_flip):
+        color = self.loader(self.get_image_path(folder, frame_index, side))
+
+        if do_flip:
+            color = color.transpose(pil.FLIP_LEFT_RIGHT)
+
+        return color
+
+
+class KITTIRAWDataset(KITTIDataset):
+    """KITTI dataset which loads the original velodyne depth maps for ground truth
+    """
+    def __init__(self, *args, **kwargs):
+        super(KITTIRAWDataset, self).__init__(*args, **kwargs)
+
+    def get_image_path(self, folder, frame_index, side):
+        f_str = "{:010d}{}".format(frame_index, self.img_ext)
+        image_path = os.path.join(
+            self.data_path, folder, "image_0{}/data".format(self.side_map[side]), f_str)
+        return image_path
+
+    def get_depth(self, folder, frame_index, side, do_flip):
+        calib_path = os.path.join(self.data_path, folder.split("/")[0])
+
+        velo_filename = os.path.join(
+            self.data_path,
+            folder,
+            "velodyne_points/data/{:010d}.bin".format(int(frame_index)))
+
+        depth_gt = generate_depth_map(calib_path, velo_filename, self.side_map[side])
+        depth_gt = skimage.transform.resize(
+            depth_gt, self.full_res_shape[::-1], order=0, preserve_range=True, mode='constant')
+
+        if do_flip:
+            depth_gt = np.fliplr(depth_gt)
+
+        return depth_gt
+
+
+class KITTIOdomDataset(KITTIDataset):
+    """KITTI dataset for odometry training and testing
+    """
+    def __init__(self, *args, **kwargs):
+        super(KITTIOdomDataset, self).__init__(*args, **kwargs)
+
+    def get_image_path(self, folder, frame_index, side):
+        f_str = "{:06d}{}".format(frame_index, self.img_ext)
+        image_path = os.path.join(
+            self.data_path,
+            "sequences/{:02d}".format(int(folder)),
+            "image_{}".format(self.side_map[side]),
+            f_str)
+        return image_path
+
+
+class KITTIDepthDataset(KITTIDataset):
+    """KITTI dataset which uses the updated ground truth depth maps
+    """
+    def __init__(self, *args, **kwargs):
+        super(KITTIDepthDataset, self).__init__(*args, **kwargs)
+
+    def get_image_path(self, folder, frame_index, side):
+        f_str = "{:010d}{}".format(frame_index, self.img_ext)
+        image_path = os.path.join(
+            self.data_path,
+            folder,
+            "image_0{}/data".format(self.side_map[side]),
+            f_str)
+        return image_path
+
+    def get_depth(self, folder, frame_index, side, do_flip):
+        f_str = "{:010d}.png".format(frame_index)
+        depth_path = os.path.join(
+            self.data_path,
+            folder,
+            "proj_depth/groundtruth/image_0{}".format(self.side_map[side]),
+            f_str)
+
+        depth_gt = pil.open(depth_path)
+        depth_gt = depth_gt.resize(self.full_res_shape, pil.NEAREST)
+        depth_gt = np.array(depth_gt).astype(np.float32) / 256
+
+        if do_flip:
+            depth_gt = np.fliplr(depth_gt)
+
+        return depth_gt