Skip to content

Commit

Permalink
add notes to shell script
Browse files Browse the repository at this point in the history
  • Loading branch information
MaxVanDijck committed Mar 17, 2022
1 parent 0b8fe3e commit 8ed9925
Showing 1 changed file with 32 additions and 1 deletion.
33 changes: 32 additions & 1 deletion imagenet/extract_ILSVRC.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,47 @@ mkdir imagenet
#
# Extract the training data:
#
# Create train directory; move .tar file; change directory
mkdir imagenet/train && mv ILSVRC2012_img_train.tar imagenet/train/ && cd imagenet/train
# Extract training set; remove compressed file
tar -xvf ILSVRC2012_img_train.tar && rm -f ILSVRC2012_img_train.tar
#
# At this stage imagenet/train will contain 1000 compressed .tar files, one for each category
#
# For each .tar file:
# 1. create directory with same name as .tar file
# 2. extract and copy contents of .tar file into directory
# 3. remove .tar file
find . -name "*.tar" | while read NAME ; do mkdir -p "${NAME%.tar}"; tar -xvf "${NAME}" -C "${NAME%.tar}"; rm -f "${NAME}"; done
#
# This results in a training directory like so:
# imagenet/train/
# ├── n01440764
# │ ├── n01440764_10026.JPEG
# │ ├── n01440764_10027.JPEG
# │ ├── ......
# ├── ......
#
# Change back to original directory
cd ../..
#
# Extract the validation data and move images to subfolders:
#
mkdir imagenet/val && mv ILSVRC2012_img_val.tar imagenet/val/ && cd imagenet/val && tar -xvf ILSVRC2012_img_val.tar
# Create validation directory; move .tar file; change directory; extract validation .tar; remove compressed file
mkdir imagenet/val && mv ILSVRC2012_img_val.tar imagenet/val/ && cd imagenet/val && tar -xvf ILSVRC2012_img_val.tar && rm -f ILSVRC2012_img_val.tar
# get script from soumith and run; this script creates all class directories and moves images into corresponding directories
wget -qO- https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh | bash
#
# This resilts in a validation directory like so:
#
# imagenet/val/
# ├── n01440764
# │ ├── ILSVRC2012_val_00000293.JPEG
# │ ├── ILSVRC2012_val_00002138.JPEG
# │ ├── ......
# ├── ......
#
#
# Check total files after extract
#
# $ find train/ -name "*.JPEG" | wc -l
Expand Down

0 comments on commit 8ed9925

Please sign in to comment.