CSE 40535 Term Project by Alden Kane. Uses color and motion detection features from underwater video feed for frame-by-frame swimmer localization and detection.
Source code by Alden Kane. See "Resources and References," as well as code comments, for pertinent citations.
I used a GoPro Hero7 Black (1920 x 1080 at 30 or 60 FPS) for data collection. I collected approximately 1.5 hours of underwater video at Rolf's Acquatic Center and Rockne Memorial Pool at the University of Notre Dame.
Library Requirements, using Python 3.7.5 Release:
numpy==1.17.4 opencv-contrib-python==4.1.2.30
Steps for Running Program:
..1. Clone Repository
..2. Ensure all libraries are installed
..3. Set cam = cv2.VideoCapture('') location of video to test
..4. From the top-level directory:
python3 uw_swimmer_detection.py
Test Samples
..* A test video, at ../dataSet/swim3/swim3.1-12-of-14.mp4 has been provided for running the code
Testing by Alden Kane showed an Intersection over Union (IoU) of approximately 0.81 ± 0.12 on the train and validation sets. The test set, sourced from Rockne Memorial Pool, showed lower classificaiton accuracy due to poor overhead illumination in the pool.
Install npm modules in project or root directory
npm install parse --save
npm install btoa --save
Usage:
//require function from module
const sendImage = require('./parsePi.js');
//call function with filePath as argument
sendImage(filePath);
Function Arguments:
filePath --> takes relative file path as a string.
Reads JSON file of from
{ "swimDetected": "Boolean"
"numberSwimmers": "Integer"
"drownDetected": "Boolean" }
Image class should show new instance with image file as base64 encoded string and argument values appended on Parse server
System improvements can be generated using:
..* A convolutional neural network (CNN) for swimmer classification. This entails annotating a data set large enough for feature, and then integrating it with color and motion detection in a majority-voting system.
..* A more diverse data set of swimmers with different skin pigment, age, and swimming style.
..* Improved sensor selection. Selecting embedded cameras with lower resolution and frame rates can enhance run time, save on computational cost, and allow for scaleable architectures for implementation in commercial pools.
..* Dr. Adam Czajka (Notre Dame Computer Vision Research Lab), for his keen insight and expertise
..* Dr. Adrian Rosebrock (PyImageSearch) for his useful CV tutorials