Hendricks, Lisa Anne, et al. "Localizing Moments in Video with Natural Language." ICCV (2017).
@inproceedings{hendricks17iccv,
title = {Localizing Moments in Video with Natural Language.},
author = {Hendricks, Lisa Anne and Wang, Oliver and Shechtman, Eli and Sivic, Josef and Darrell, Trevor and Russell, Bryan},
booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
year = {2017}
}
License: BSD 2-Clause license
I will release code to replicate my models shortly. For now please see "eval.py"
To access the dataset, please look at the json files in the "data" folder. The json files contain a list of data points which are stored as dict structures with the following fields:
annotation_id: Annotation ID for data point
description: Description for a specific video segment
video: Video ID
times: Ground truth time points marked by annotators. There are four times corresponding to four different annotators. For each time, the first number indicates the GIF in which the moment start, and the second number indicated the GIF in which the moment ends. A time of [3,3] indicates that the moment started on second 3*5=15 and ended on moment (3+1)*5=20.
download_link: A download link for the video
num_segments: How many GIFs were shown to annotators. Some videos are a little shorter than 25 seconds, so were split into five GIFs instead of six.
Use the script download_videos.py: python download_videos.py --download --video_directory DIRECTORY
There are some videos which have been removed from Flickr (~3% of the original videos). You may access them at my website: https://people.eecs.berkeley.edu/~lisa_anne/didemo/data/missing_videos/missing_videos.zip
Please contact me if you find more videos are missing. I originally downloaded the videos provided in the YFCC100M dataset (10th entry for lines in "video_licenses.txt"), but many of these links no longer work. The script I used to download the videos should work, but let me know if you run into any issues. If you have a better understanding of the FlickrAPI (or are familiar with YFCC100M) and know why I can no longer download videos using the original download links, let me know.
You can view the Creative Commons licenses and all metadata in "video_licenses.txt".