From 64be6804e067bbf63bb7f551efc5c906c6d280a1 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 16 Jul 2020 21:54:37 +0900 Subject: [PATCH 01/38] Update README.md --- README.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index cfdfd8ec..1d55d821 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,16 @@ [![Build Status](https://travis-ci.com/PurdueCAM2Project/Embedded2.svg?branch=master)](https://travis-ci.com/PurdueCAM2Project/Embedded2) # Embedded Computer Vision 2 -## Personal Protective Equipment (PPE) Usage Detection -#### The system obfuscates faces after detection and classification to protect individual privacy -#### Features: +## Personal Protective Equipment (PPE) Usage +#### The system detects and classifies PPE usage. Afterward, it obfuscates faces to protect individual privacy + +## Features: * Mobilenet-v2 based SSD performs face detection * CNN performs classification of detected faces to determine if PPE is being used * Faces are encrypted using AES * Runs on Jetson Nano in real time -#### Primary scripts: +## Primary scripts: `jetson/goggles/goggleClassifier.py --directory=path/to/imagefolder` `jetson/face_detector.py --trained_model=path/to/ssd_model.pth --classifier=path/to/trained_classifier.pth --cuda` * goggleClassifier.py is how we train our goggle classifier. The model is saved into a .pth file that is loaded as the trained_model of face_detector.py. face_detector.py will detect your face and classify whether you are wearing goggles, glasses, or neither. @@ -17,3 +18,8 @@ * Only include --cuda with face_detector if you have a GPU * We have been using ssd300_WIDER_100455.pth as the SSD model. The classifier model will be any other .pth file stored on the [Drive](https://drive.google.com/drive/u/1/folders/1ZeKVygo-RyIDL_EnxeYJR8tk-xqzgi3Z). * We recommend using [Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) for this project. Once you have it installed you can run `conda env create -f environment.yml` from the Embedded2 directory for the necessary packages. + +## Builds: +* Travis CI + +## License: From 2fba5a2a290c414d7e181a0558c224ce9e88fadd Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 16 Jul 2020 22:21:05 +0900 Subject: [PATCH 02/38] Update README.md --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 1d55d821..dd33d8ee 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ # Embedded Computer Vision 2 ## Personal Protective Equipment (PPE) Usage -#### The system detects and classifies PPE usage. Afterward, it obfuscates faces to protect individual privacy +#### The system detects and classifies PPE usage. Afterward, it obfuscates faces to protect individual privacy. ## Features: * Mobilenet-v2 based SSD performs face detection @@ -18,6 +18,7 @@ * Only include --cuda with face_detector if you have a GPU * We have been using ssd300_WIDER_100455.pth as the SSD model. The classifier model will be any other .pth file stored on the [Drive](https://drive.google.com/drive/u/1/folders/1ZeKVygo-RyIDL_EnxeYJR8tk-xqzgi3Z). * We recommend using [Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) for this project. Once you have it installed you can run `conda env create -f environment.yml` from the Embedded2 directory for the necessary packages. +* Image is sent to detector. Detector is either blazeface, retinaface or ssd. Cuda is enabled and calssifier is activiated. Encrypted images are outputted after detection and classification. ## Builds: * Travis CI From 060f3ab2be04f9199babdfdeedb58629e24e3652 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 21 Jul 2020 23:11:35 +0900 Subject: [PATCH 03/38] Update README.md --- README.md | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index dd33d8ee..98f8d927 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,36 @@ [![Build Status](https://travis-ci.com/PurdueCAM2Project/Embedded2.svg?branch=master)](https://travis-ci.com/PurdueCAM2Project/Embedded2) # Embedded Computer Vision 2 -## Personal Protective Equipment (PPE) Usage -#### The system detects and classifies PPE usage. Afterward, it obfuscates faces to protect individual privacy. +The system is used to detect the usage of Personal Protection Equipment (PPE), specifically goggles, in labs that require them. The system is run in real time on Jetson Nano and uses a Rasspberry Pi camera to record footage in the lab. To ensure individual privacy is protected, the system obfuscates faces after detection and classification. Images are stored in a remote storage drive and image metadata are stored on a SQL database. -## Features: -* Mobilenet-v2 based SSD performs face detection +### Features: +* Retinaface based SSD performs face detection * CNN performs classification of detected faces to determine if PPE is being used * Faces are encrypted using AES +* Image metadata is stored on a SQL database server +* Images are transfered to remote computer using SFTP * Runs on Jetson Nano in real time +# Table of Contents +- [Installation](#Installation) +- [Usage](#Contributing) +- [Credits](#Credits) +- [License](#License) + +## Installation +1. Clone the project and enter the folder +```shell +$ git clone https://github.com/PurdueCAM2Project/Embedded2.git +$ cd Embedded2 +``` +2. The classiifier model (.pth file) can be found on [Drive](https://drive.google.com/drive/u/1/folders/1ZeKVygo-RyIDL_EnxeYJR8tk-xqzgi3Z). Downloadand place it in the ```Embedded2/src/jetson``` directory. +3. There is a requirement.txt file with all the necessary dependencies. We, however, recommend using Conda for this project. Once you have conda installed, run the following command to setup the enviroment with necessary dependicies. +```shell +$ conda env create -f environment.yml +``` +4. Add the Embedded2 folder to PYTHONPATH by adding the following line in your .bashrc file: +```export PYTHONPATH=/path/Embedded2``` + ## Primary scripts: `jetson/goggles/goggleClassifier.py --directory=path/to/imagefolder` `jetson/face_detector.py --trained_model=path/to/ssd_model.pth --classifier=path/to/trained_classifier.pth --cuda` From 2742eef8da36951669d2d6e4e8c64ad1ba1498a9 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 21 Jul 2020 23:40:40 +0900 Subject: [PATCH 04/38] Update README.md --- README.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 98f8d927..3417f637 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,7 @@ The system is used to detect the usage of Personal Protection Equipment (PPE), s - [Installation](#Installation) - [Usage](#Contributing) - [Credits](#Credits) +- [Builds](#Builds) - [License](#License) ## Installation @@ -31,15 +32,22 @@ $ conda env create -f environment.yml 4. Add the Embedded2 folder to PYTHONPATH by adding the following line in your .bashrc file: ```export PYTHONPATH=/path/Embedded2``` -## Primary scripts: +## Usage: +We recommend using [Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) for this project. Once you have it installed you can run `conda env create -f environment.yml` from the Embedded2 directory for the necessary packages. + +1. Make sure that the image folder is in Pytorch [Imagefolder](https://pytorch.org/docs/stable/torchvision/datasets.html?highlight=imagefolder#torchvision.datasets.ImageFolder) structure. + +2. Run the following script. `jetson/goggles/goggleClassifier.py --directory=path/to/imagefolder` `jetson/face_detector.py --trained_model=path/to/ssd_model.pth --classifier=path/to/trained_classifier.pth --cuda` -* goggleClassifier.py is how we train our goggle classifier. The model is saved into a .pth file that is loaded as the trained_model of face_detector.py. face_detector.py will detect your face and classify whether you are wearing goggles, glasses, or neither. -* The image folder should be in Pytorch [Imagefolder](https://pytorch.org/docs/stable/torchvision/datasets.html?highlight=imagefolder#torchvision.datasets.ImageFolder) structure. -* Only include --cuda with face_detector if you have a GPU +* goggleClassifier.py trains our goggle classifier. The model is saved into a .pth file that is loaded as the trained_model of face_detector.py. +* face_detector.py detects the face and classifies whether the person is wearing goggles, glasses, or neither. * We have been using ssd300_WIDER_100455.pth as the SSD model. The classifier model will be any other .pth file stored on the [Drive](https://drive.google.com/drive/u/1/folders/1ZeKVygo-RyIDL_EnxeYJR8tk-xqzgi3Z). -* We recommend using [Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) for this project. Once you have it installed you can run `conda env create -f environment.yml` from the Embedded2 directory for the necessary packages. -* Image is sent to detector. Detector is either blazeface, retinaface or ssd. Cuda is enabled and calssifier is activiated. Encrypted images are outputted after detection and classification. +* Only include --cuda with face_detector if you have a GPU + + 3. The image is sent to one of the three options of detectors: blazeface, retinaface or ssd. Make sure that cuda is enabled and calssifier is activiated. The encrypted images are outputted after detection and classification. + +## Credits: ## Builds: * Travis CI From 20577db58cf376a7f0d22b227fc175ad447abba0 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 21 Jul 2020 23:41:49 +0900 Subject: [PATCH 05/38] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 3417f637..233c0e89 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ We recommend using [Conda](https://docs.conda.io/projects/conda/en/latest/user-g * We have been using ssd300_WIDER_100455.pth as the SSD model. The classifier model will be any other .pth file stored on the [Drive](https://drive.google.com/drive/u/1/folders/1ZeKVygo-RyIDL_EnxeYJR8tk-xqzgi3Z). * Only include --cuda with face_detector if you have a GPU - 3. The image is sent to one of the three options of detectors: blazeface, retinaface or ssd. Make sure that cuda is enabled and calssifier is activiated. The encrypted images are outputted after detection and classification. + 3. The image is sent to one of the three types of detector: blazeface, retinaface or ssd. Make sure that cuda is enabled and calssifier is activiated. The encrypted images are outputted after detection and classification. ## Credits: From ed775fe5d32ebb908654d18b3299285f10f91789 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 21 Jul 2020 23:50:30 +0900 Subject: [PATCH 06/38] Update README.md --- README.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 233c0e89..58ff14c2 100644 --- a/README.md +++ b/README.md @@ -33,11 +33,11 @@ $ conda env create -f environment.yml ```export PYTHONPATH=/path/Embedded2``` ## Usage: -We recommend using [Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) for this project. Once you have it installed you can run `conda env create -f environment.yml` from the Embedded2 directory for the necessary packages. +1. We recommend using [Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) for this project. Once you have it installed, you can run `conda env create -f environment.yml` from the Embedded2 directory for the necessary packages. -1. Make sure that the image folder is in Pytorch [Imagefolder](https://pytorch.org/docs/stable/torchvision/datasets.html?highlight=imagefolder#torchvision.datasets.ImageFolder) structure. +2. Make sure that the image folder is in Pytorch [Imagefolder](https://pytorch.org/docs/stable/torchvision/datasets.html?highlight=imagefolder#torchvision.datasets.ImageFolder) structure. -2. Run the following script. +3. Run the following script. `jetson/goggles/goggleClassifier.py --directory=path/to/imagefolder` `jetson/face_detector.py --trained_model=path/to/ssd_model.pth --classifier=path/to/trained_classifier.pth --cuda` * goggleClassifier.py trains our goggle classifier. The model is saved into a .pth file that is loaded as the trained_model of face_detector.py. @@ -45,9 +45,10 @@ We recommend using [Conda](https://docs.conda.io/projects/conda/en/latest/user-g * We have been using ssd300_WIDER_100455.pth as the SSD model. The classifier model will be any other .pth file stored on the [Drive](https://drive.google.com/drive/u/1/folders/1ZeKVygo-RyIDL_EnxeYJR8tk-xqzgi3Z). * Only include --cuda with face_detector if you have a GPU - 3. The image is sent to one of the three types of detector: blazeface, retinaface or ssd. Make sure that cuda is enabled and calssifier is activiated. The encrypted images are outputted after detection and classification. + 4. The image is sent to one of the three types of detector: blazeface, retinaface or ssd. Make sure that cuda is enabled and calssifier is activiated. The encrypted images are outputted after detection and classification. ## Credits: +* [Crontabs](https://github.com/robdmc/crontabs) ## Builds: * Travis CI From bf64721843d521effab9357d6e2bce0fb614db47 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 21 Jul 2020 23:54:23 +0900 Subject: [PATCH 07/38] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 58ff14c2..f4ef3651 100644 --- a/README.md +++ b/README.md @@ -38,6 +38,7 @@ $ conda env create -f environment.yml 2. Make sure that the image folder is in Pytorch [Imagefolder](https://pytorch.org/docs/stable/torchvision/datasets.html?highlight=imagefolder#torchvision.datasets.ImageFolder) structure. 3. Run the following script. + `jetson/goggles/goggleClassifier.py --directory=path/to/imagefolder` `jetson/face_detector.py --trained_model=path/to/ssd_model.pth --classifier=path/to/trained_classifier.pth --cuda` * goggleClassifier.py trains our goggle classifier. The model is saved into a .pth file that is loaded as the trained_model of face_detector.py. From 4a242e96861cc5c9a511eda193e3375c87eb588d Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 23 Jul 2020 21:59:44 +0900 Subject: [PATCH 08/38] Update README.md --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index f4ef3651..ccdc5600 100644 --- a/README.md +++ b/README.md @@ -39,10 +39,10 @@ $ conda env create -f environment.yml 3. Run the following script. -`jetson/goggles/goggleClassifier.py --directory=path/to/imagefolder` -`jetson/face_detector.py --trained_model=path/to/ssd_model.pth --classifier=path/to/trained_classifier.pth --cuda` -* goggleClassifier.py trains our goggle classifier. The model is saved into a .pth file that is loaded as the trained_model of face_detector.py. -* face_detector.py detects the face and classifies whether the person is wearing goggles, glasses, or neither. +`scripts/goggle_classifier.py --directory=path/to/imagefolder` +`scripts/face_extractor.py --trained_model=path/to/ssd_model.pth --classifier=path/to/trained_classifier.pth --cuda` +* goggle_classifier.py trains our goggle classifier. The model is saved into a .pth file that is loaded as the trained_model of face_extractor.py. +* face_extractor.py detects the face and classifies whether the person is wearing goggles, glasses, or neither. * We have been using ssd300_WIDER_100455.pth as the SSD model. The classifier model will be any other .pth file stored on the [Drive](https://drive.google.com/drive/u/1/folders/1ZeKVygo-RyIDL_EnxeYJR8tk-xqzgi3Z). * Only include --cuda with face_detector if you have a GPU From 3b4a3879705573ccf664523b17568cb3e813aaac Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 23 Jul 2020 22:21:18 +0900 Subject: [PATCH 09/38] Create README.md --- scripts/README.md | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 scripts/README.md diff --git a/scripts/README.md b/scripts/README.md new file mode 100644 index 00000000..e854f870 --- /dev/null +++ b/scripts/README.md @@ -0,0 +1,6 @@ +Description of files and the functions used in the files + +# automatic_notification.py +## send_email +The function first retrieves the current timestamp. Then, it forms an email message consisting of information about the sender and the receiver, the email's subject, body messages and timestamp. It connects to a server in order to send the mail. After the mail is sent, the server session is quit. + From c46211014f1f55be5f44a66a816bd49d1a7cdea5 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 23 Jul 2020 22:40:57 +0900 Subject: [PATCH 10/38] Update README.md --- scripts/README.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/scripts/README.md b/scripts/README.md index e854f870..4866c019 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -1,6 +1,19 @@ Description of files and the functions used in the files # automatic_notification.py +The file uses Cron.schedule() to run the send_email function every day. ## send_email The function first retrieves the current timestamp. Then, it forms an email message consisting of information about the sender and the receiver, the email's subject, body messages and timestamp. It connects to a server in order to send the mail. After the mail is sent, the server session is quit. +# face_extractor.py +The file parses an argument of input directory, output directory, trained model, images, rate and horizontal orientation. It calls get_images or get_Videos to get the files. Then, it calls crop_faces_from_images or crop_faces_from_videos to crop and save the face images. +## get_images +The function gets filenames of images from the input directory and returns a list of the image filenames. +## get_Videos +The function gets filenames of videos from the input directory and returns a list of the video filenames. +## crop_and_save_img +The function runs the frame through FaceDetector to create a bounding box around the face and crop the image. Then, it saves the cropped face image. +## crop_faces_from_images +The function iterates through the image files and crops and saves face images. +## crop_faces_from_videos +The function first iterates through the video files. For each video, the function iterates through the video frames and crops and saves face images. If the video is shot horizontally, the function flips it so that it's in the right orientation. From 618c8323e16296b94f80e467731b3f5933c447d0 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 23 Jul 2020 22:44:07 +0900 Subject: [PATCH 11/38] Update README.md --- scripts/README.md | 31 +++++++++++++++++++++++++------ 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/scripts/README.md b/scripts/README.md index 4866c019..9d0313fa 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -2,18 +2,37 @@ Description of files and the functions used in the files # automatic_notification.py The file uses Cron.schedule() to run the send_email function every day. -## send_email +#### send_email The function first retrieves the current timestamp. Then, it forms an email message consisting of information about the sender and the receiver, the email's subject, body messages and timestamp. It connects to a server in order to send the mail. After the mail is sent, the server session is quit. # face_extractor.py The file parses an argument of input directory, output directory, trained model, images, rate and horizontal orientation. It calls get_images or get_Videos to get the files. Then, it calls crop_faces_from_images or crop_faces_from_videos to crop and save the face images. -## get_images +#### get_images The function gets filenames of images from the input directory and returns a list of the image filenames. -## get_Videos +#### get_Videos The function gets filenames of videos from the input directory and returns a list of the video filenames. -## crop_and_save_img +#### crop_and_save_img The function runs the frame through FaceDetector to create a bounding box around the face and crop the image. Then, it saves the cropped face image. -## crop_faces_from_images +#### crop_faces_from_images The function iterates through the image files and crops and saves face images. -## crop_faces_from_videos +#### crop_faces_from_videos The function first iterates through the video files. For each video, the function iterates through the video frames and crops and saves face images. If the video is shot horizontally, the function flips it so that it's in the right orientation. + +# goggle_classifier.py + +## MapDataset + +#### __init__ + +#### __getitem__ + +#### __len__ + +#### get_model + +#### load_data + +#### train_model + +#### get_metrics + From 48dab585aed1b64720189b7175bc196f453a01c8 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Fri, 24 Jul 2020 00:09:25 +0900 Subject: [PATCH 12/38] Update README.md --- scripts/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/scripts/README.md b/scripts/README.md index 9d0313fa..e86dbbfc 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -20,14 +20,14 @@ The function first iterates through the video files. For each video, the functio # goggle_classifier.py -## MapDataset - +### MapDataset +This class contains custom dataset for applying different transforms to training and validation data. It consists of the __init__, __getitem__ and __len__ functions. #### __init__ - +This function initializes variables regarding the dataset. #### __getitem__ - +This function gets the item and returns the mapping of the item in the dataset. #### __len__ - +This function returns the number of elements in the dataset. #### get_model #### load_data From 0746f68b8c7cc492c2a4730523ebbfb7a0648c55 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Fri, 24 Jul 2020 00:43:48 +0900 Subject: [PATCH 13/38] Update README.md --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index dc6f9c69..8a488e69 100644 --- a/README.md +++ b/README.md @@ -52,5 +52,3 @@ $ conda env create -f environment.yml ## Builds: * Travis CI - -## License: \ No newline at end of file From e38686009b8156d1696dbbfd6e4596747df53254 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 30 Jul 2020 23:55:29 +0900 Subject: [PATCH 14/38] Update README.md --- scripts/README.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/scripts/README.md b/scripts/README.md index e86dbbfc..8c50329f 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -28,11 +28,17 @@ This function initializes variables regarding the dataset. This function gets the item and returns the mapping of the item in the dataset. #### __len__ This function returns the number of elements in the dataset. -#### get_model -#### load_data +#### classifier_transforms +Data augmentation options used to train and validate the classifier -#### train_model +#### mobilenet_v2_layer_parameter +#### get_model +This function first initializes Mobilenet and freezes relevant layers. Then, it returns the pretrained Mobilenet model with the relevant layers frozen. +#### load_data +This function first loads the image data from the data location specified. Then, it uses MapDataset to perform data augmentations, creating variables regarding 'train' and 'val'. Fianlly, it returns these new variables: 'train' and 'val' Dataloader, sizes of 'train' and 'val' datasets and the names of dataset classes +#### train_model +This function first initalizes hyperparemeters used to train the model. For each epoch, it trains and validates the data. As it iterates through the data, it goes through forward propagation unless the phase is 'train'; in this case, it will go through backward propagation. It prints the loss and accuracy of train or val. There are checkpoints every 10 epochs; these checkpoints help us make the comparison among the trained model and the overfit and underfit models. Finally, it returns the trained model. #### get_metrics From d866e5a12609348ec9a9dbdc075ff69e1b955191 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Fri, 31 Jul 2020 00:02:41 +0900 Subject: [PATCH 15/38] Update README.md --- scripts/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/README.md b/scripts/README.md index 8c50329f..e7d72d9c 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -19,7 +19,7 @@ The function iterates through the image files and crops and saves face images. The function first iterates through the video files. For each video, the function iterates through the video frames and crops and saves face images. If the video is shot horizontally, the function flips it so that it's in the right orientation. # goggle_classifier.py - +The file first parses the argument for training the Mobilenet classifier. It then initializes the TensorBoard writer. It other loads a pretrained model or calls get_model to use a pretrained Mobilenet model with layers frozen. Then, it performs training and validation augmentations and then outputs the results from the training. ### MapDataset This class contains custom dataset for applying different transforms to training and validation data. It consists of the __init__, __getitem__ and __len__ functions. #### __init__ @@ -41,4 +41,4 @@ This function first loads the image data from the data location specified. Then, #### train_model This function first initalizes hyperparemeters used to train the model. For each epoch, it trains and validates the data. As it iterates through the data, it goes through forward propagation unless the phase is 'train'; in this case, it will go through backward propagation. It prints the loss and accuracy of train or val. There are checkpoints every 10 epochs; these checkpoints help us make the comparison among the trained model and the overfit and underfit models. Finally, it returns the trained model. #### get_metrics - +It prints statistics from final epoch of training. From 447c70801eeba5d8351b974062b0cc9e40eb1fe9 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 4 Aug 2020 02:04:51 +0900 Subject: [PATCH 16/38] Update README.md --- scripts/README.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/scripts/README.md b/scripts/README.md index e7d72d9c..819e74db 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -32,13 +32,11 @@ This function returns the number of elements in the dataset. #### classifier_transforms Data augmentation options used to train and validate the classifier -#### mobilenet_v2_layer_parameter - #### get_model This function first initializes Mobilenet and freezes relevant layers. Then, it returns the pretrained Mobilenet model with the relevant layers frozen. #### load_data This function first loads the image data from the data location specified. Then, it uses MapDataset to perform data augmentations, creating variables regarding 'train' and 'val'. Fianlly, it returns these new variables: 'train' and 'val' Dataloader, sizes of 'train' and 'val' datasets and the names of dataset classes #### train_model -This function first initalizes hyperparemeters used to train the model. For each epoch, it trains and validates the data. As it iterates through the data, it goes through forward propagation unless the phase is 'train'; in this case, it will go through backward propagation. It prints the loss and accuracy of train or val. There are checkpoints every 10 epochs; these checkpoints help us make the comparison among the trained model and the overfit and underfit models. Finally, it returns the trained model. +This function first initializes hyperparemeters used to train the model. For each epoch, it trains and validates the data. As it iterates through the data, it goes through forward propagation unless the phase is 'train'; in this case, it will go through backward propagation. It prints the loss and accuracy of train or val. There are checkpoints every 10 epochs; these checkpoints help us make the comparison among the trained model and the overfit and underfit models. Finally, it returns the trained model. #### get_metrics -It prints statistics from final epoch of training. +It prints statistics from the final epoch of training. From 5069c70c954e90bccebb9c0d6e01216651861b44 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 4 Aug 2020 02:05:27 +0900 Subject: [PATCH 17/38] Update README.md --- scripts/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/README.md b/scripts/README.md index 819e74db..66217f41 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -1,4 +1,4 @@ -Description of files and the functions used in the files +Description of files in the 'scripts' folder and the functions used in the files # automatic_notification.py The file uses Cron.schedule() to run the send_email function every day. From 2422052feb242dbfed15d0ca980b7db9879feb3c Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 4 Aug 2020 02:10:01 +0900 Subject: [PATCH 18/38] Update README.md --- scripts/README.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/scripts/README.md b/scripts/README.md index 66217f41..8c617206 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -19,9 +19,9 @@ The function iterates through the image files and crops and saves face images. The function first iterates through the video files. For each video, the function iterates through the video frames and crops and saves face images. If the video is shot horizontally, the function flips it so that it's in the right orientation. # goggle_classifier.py -The file first parses the argument for training the Mobilenet classifier. It then initializes the TensorBoard writer. It other loads a pretrained model or calls get_model to use a pretrained Mobilenet model with layers frozen. Then, it performs training and validation augmentations and then outputs the results from the training. +The file first parses the argument for training the Mobilenet classifier. It then initializes the TensorBoard writer. It either loads a pretrained model or calls get_model to use a pretrained Mobilenet model with layers frozen. Then, it performs training and validation augmentations and then outputs the results from the training. ### MapDataset -This class contains custom dataset for applying different transforms to training and validation data. It consists of the __init__, __getitem__ and __len__ functions. +This class contains custom dataset for applying different transforms to training and validation data. It consists o __init__, __getitem__ and __len__ functions. #### __init__ This function initializes variables regarding the dataset. #### __getitem__ @@ -29,8 +29,9 @@ This function gets the item and returns the mapping of the item in the dataset. #### __len__ This function returns the number of elements in the dataset. + #### classifier_transforms -Data augmentation options used to train and validate the classifier +This is a dictionary of data augmentation options used to train and validate the classifier. #### get_model This function first initializes Mobilenet and freezes relevant layers. Then, it returns the pretrained Mobilenet model with the relevant layers frozen. @@ -39,4 +40,4 @@ This function first loads the image data from the data location specified. Then, #### train_model This function first initializes hyperparemeters used to train the model. For each epoch, it trains and validates the data. As it iterates through the data, it goes through forward propagation unless the phase is 'train'; in this case, it will go through backward propagation. It prints the loss and accuracy of train or val. There are checkpoints every 10 epochs; these checkpoints help us make the comparison among the trained model and the overfit and underfit models. Finally, it returns the trained model. #### get_metrics -It prints statistics from the final epoch of training. +This function prints statistics from the final epoch of training. From 76a31e8d8cd87213f57a09557edc6b97fc029b63 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 4 Aug 2020 02:31:25 +0900 Subject: [PATCH 19/38] Update README.md --- README.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 8a488e69..6759293e 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ The system is used to detect the usage of Personal Protection Equipment (PPE), s $ git clone https://github.com/PurdueCAM2Project/Embedded2.git $ cd Embedded2 ``` -2. The classiifier model (.pth file) can be found on [Drive](https://drive.google.com/drive/u/1/folders/1ZeKVygo-RyIDL_EnxeYJR8tk-xqzgi3Z). Downloadand place it in the ```Embedded2/src/jetson``` directory. +2. The classiifier model (.pth file) can be found on [Drive](https://drive.google.com/drive/folders/1QfS7YiuCxK-K93dnEYIMoeHAU65Cs14n). Download and place it in the ```Embedded2/src/jetson``` directory. 3. There is a requirement.txt file with all the necessary dependencies. We, however, recommend using Conda for this project. Once you have conda installed, run the following command to setup the enviroment with necessary dependicies. ```shell $ conda env create -f environment.yml @@ -32,14 +32,13 @@ $ conda env create -f environment.yml 4. Add the Embedded2 folder to PYTHONPATH by adding the following line in your .bashrc file: ```export PYTHONPATH=/path/Embedded2``` ## Usage: -1. We recommend using [Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) for this project. Once you have it installed, you can run `conda env create -f environment.yml` from the Embedded2 directory for the necessary packages. 2. Make sure that the image folder is in Pytorch [Imagefolder](https://pytorch.org/docs/stable/torchvision/datasets.html?highlight=imagefolder#torchvision.datasets.ImageFolder) structure. 3. Run the following script. `scripts/goggle_classifier.py --directory=path/to/imagefolder` -`scripts/face_extractor.py --trained_model=path/to/ssd_model.pth --classifier=path/to/trained_classifier.pth --cuda` +`scripts/face_extractor.py --trained_model=path/to/_model.pth --classifier=path/to/trained_classifier.pth --cuda` * goggle_classifier.py trains our goggle classifier. The model is saved into a .pth file that is loaded as the trained_model of face_extractor.py. * face_extractor.py detects the face and classifies whether the person is wearing goggles, glasses, or neither. * We have been using ssd300_WIDER_100455.pth as the SSD model. The classifier model will be any other .pth file stored on the [Drive](https://drive.google.com/drive/u/1/folders/1ZeKVygo-RyIDL_EnxeYJR8tk-xqzgi3Z). From 370e1b4812eb2efc80a689ce0cd79bf2eef32224 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 4 Aug 2020 02:31:50 +0900 Subject: [PATCH 20/38] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 6759293e..0a73886a 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ [![Build Status](https://travis-ci.com/PurdueCAM2Project/Embedded2.svg?branch=master)](https://travis-ci.com/PurdueCAM2Project/Embedded2) -# Embedded Computer Vision 2 +# Embedded Computer Vision 2020 The system is used to detect the usage of Personal Protection Equipment (PPE), specifically goggles, in labs that require them. The system is run in real time on Jetson Nano and uses a Rasspberry Pi camera to record footage in the lab. To ensure individual privacy is protected, the system obfuscates faces after detection and classification. Images are stored in a remote storage drive and image metadata are stored on a SQL database. ### Features: From e62cebeba429652a4f432b335e0c0e0dce65c669 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 4 Aug 2020 02:34:23 +0900 Subject: [PATCH 21/38] Update README.md --- README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 0a73886a..6014051d 100644 --- a/README.md +++ b/README.md @@ -24,8 +24,8 @@ The system is used to detect the usage of Personal Protection Equipment (PPE), s $ git clone https://github.com/PurdueCAM2Project/Embedded2.git $ cd Embedded2 ``` -2. The classiifier model (.pth file) can be found on [Drive](https://drive.google.com/drive/folders/1QfS7YiuCxK-K93dnEYIMoeHAU65Cs14n). Download and place it in the ```Embedded2/src/jetson``` directory. -3. There is a requirement.txt file with all the necessary dependencies. We, however, recommend using Conda for this project. Once you have conda installed, run the following command to setup the enviroment with necessary dependicies. +2. The classifier model (.pth file) can be found on [Drive](https://drive.google.com/drive/folders/1QfS7YiuCxK-K93dnEYIMoeHAU65Cs14n). Download and place it in the ```Embedded2/src/jetson``` directory. +3. There is a requirement.txt file with all the necessary dependencies. We, however, recommend using Conda for this project. Once you have conda installed, run the following command to setup the enviroment with necessary dependencies. ```shell $ conda env create -f environment.yml ``` @@ -33,9 +33,9 @@ $ conda env create -f environment.yml ```export PYTHONPATH=/path/Embedded2``` ## Usage: -2. Make sure that the image folder is in Pytorch [Imagefolder](https://pytorch.org/docs/stable/torchvision/datasets.html?highlight=imagefolder#torchvision.datasets.ImageFolder) structure. +1. Make sure that the image folder is in Pytorch [Imagefolder](https://pytorch.org/docs/stable/torchvision/datasets.html?highlight=imagefolder#torchvision.datasets.ImageFolder) structure. -3. Run the following script. +2. Run the following script. `scripts/goggle_classifier.py --directory=path/to/imagefolder` `scripts/face_extractor.py --trained_model=path/to/_model.pth --classifier=path/to/trained_classifier.pth --cuda` @@ -44,7 +44,7 @@ $ conda env create -f environment.yml * We have been using ssd300_WIDER_100455.pth as the SSD model. The classifier model will be any other .pth file stored on the [Drive](https://drive.google.com/drive/u/1/folders/1ZeKVygo-RyIDL_EnxeYJR8tk-xqzgi3Z). * Only include --cuda with face_detector if you have a GPU - 4. The image is sent to one of the three types of detector: blazeface, retinaface or ssd. Make sure that cuda is enabled and calssifier is activiated. The encrypted images are outputted after detection and classification. + 3. The image is sent to one of the three types of detector: blazeface, retinaface or ssd. Make sure that cuda is enabled and calssifier is activiated. The encrypted images are outputted after detection and classification. ## Credits: * [Crontabs](https://github.com/robdmc/crontabs) From 920959e6a4b8efb6bf780bf4b4337448e51d067d Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 4 Aug 2020 02:35:28 +0900 Subject: [PATCH 22/38] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 6014051d..35a04928 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ $ conda env create -f environment.yml `scripts/face_extractor.py --trained_model=path/to/_model.pth --classifier=path/to/trained_classifier.pth --cuda` * goggle_classifier.py trains our goggle classifier. The model is saved into a .pth file that is loaded as the trained_model of face_extractor.py. * face_extractor.py detects the face and classifies whether the person is wearing goggles, glasses, or neither. -* We have been using ssd300_WIDER_100455.pth as the SSD model. The classifier model will be any other .pth file stored on the [Drive](https://drive.google.com/drive/u/1/folders/1ZeKVygo-RyIDL_EnxeYJR8tk-xqzgi3Z). +* We have been using ssd300_WIDER_100455.pth as the SSD model. The classifier model will be any other .pth file stored on the [Drive](https://drive.google.com/drive/folders/1QfS7YiuCxK-K93dnEYIMoeHAU65Cs14n). * Only include --cuda with face_detector if you have a GPU 3. The image is sent to one of the three types of detector: blazeface, retinaface or ssd. Make sure that cuda is enabled and calssifier is activiated. The encrypted images are outputted after detection and classification. From 0a6726a816824f3c183a7cc953ccdfe7229a3e0e Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 4 Aug 2020 23:44:23 +0900 Subject: [PATCH 23/38] Create README.md --- src/jetson/models/Retinaface/README.md | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 src/jetson/models/Retinaface/README.md diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md new file mode 100644 index 00000000..ceea01d5 --- /dev/null +++ b/src/jetson/models/Retinaface/README.md @@ -0,0 +1,10 @@ +# net.py + +# retinaface.py + +# data +This folder consists of __init__.py and config.py. + +__init__.py imports the configurations from the models.Retinaface class. + +config.py defines a library of configurations, including inference configuration and MobileNetV1 and Resnet50 backbone configurations for training. From 39c351d03c47074f5364b7f06f97ad391ae2ebc7 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Tue, 4 Aug 2020 23:56:38 +0900 Subject: [PATCH 24/38] Update README.md --- src/jetson/models/Retinaface/README.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index ceea01d5..5c1dd1e2 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -1,4 +1,14 @@ # net.py +## conv_bn +## conv_bn_no_relu +## conv_bn1X1 +## conv_dw +## SSH +### __init__ +### forward +## FPN +## MobileNetV1 + # retinaface.py @@ -8,3 +18,15 @@ This folder consists of __init__.py and config.py. __init__.py imports the configurations from the models.Retinaface class. config.py defines a library of configurations, including inference configuration and MobileNetV1 and Resnet50 backbone configurations for training. + +# layers + +## functions +### prior_box.py +This file defines a class PriorBox. The class consists of __init__ and forward functions. +#### __init__ +This function + +## modules +### multibox_loss.py +This file defines a class MultiBoxLoss. The class consists of __init__ and forward functions. From 0ec76afdbaddc2574140ce018aaa5a50fcfc8512 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Wed, 5 Aug 2020 00:01:13 +0900 Subject: [PATCH 25/38] Update README.md --- src/jetson/models/Retinaface/README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index 5c1dd1e2..46150a93 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -11,6 +11,11 @@ # retinaface.py +## ClassHead +## BboxHead +## LandmarkHead +## RetinaFace +## load_model # data This folder consists of __init__.py and config.py. From 7956e3a4832a6b7f1b8961c6e72aa9b3d2007d45 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Wed, 5 Aug 2020 00:05:51 +0900 Subject: [PATCH 26/38] Update README.md --- src/jetson/models/Retinaface/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index 46150a93..59ffcf34 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -1,4 +1,5 @@ # net.py +This file describes functions about returning convolutional layers. ## conv_bn ## conv_bn_no_relu ## conv_bn1X1 @@ -11,6 +12,7 @@ # retinaface.py +This file describes functions about making layers describing facial images. ## ClassHead ## BboxHead ## LandmarkHead From 4d29bc495a7f093f0e7b54bd2c6e861f0829b1e5 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 18:15:39 +0900 Subject: [PATCH 27/38] Update README.md --- src/jetson/models/Retinaface/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index 59ffcf34..2f7b32c4 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -32,7 +32,9 @@ config.py defines a library of configurations, including inference configuration ### prior_box.py This file defines a class PriorBox. The class consists of __init__ and forward functions. #### __init__ -This function +This function initializes the variables of priorbox according to the configuration of the training models. Furthermore, for each source feature map, it calculates the priorbox coordinates. +#### forward +This function first loops through the feature maps to compute the anchor using dense face localizations, min_size and image_size. Then, it passes the anchors to torch land and returns the forward pass of the prior box tensor. ## modules ### multibox_loss.py From 9d68f73757a348d22bd04dc919055324a7779f7c Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 18:30:43 +0900 Subject: [PATCH 28/38] Update README.md --- src/jetson/models/Retinaface/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index 2f7b32c4..d14443f9 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -31,11 +31,11 @@ config.py defines a library of configurations, including inference configuration ## functions ### prior_box.py This file defines a class PriorBox. The class consists of __init__ and forward functions. -#### __init__ -This function initializes the variables of priorbox according to the configuration of the training models. Furthermore, for each source feature map, it calculates the priorbox coordinates. -#### forward -This function first loops through the feature maps to compute the anchor using dense face localizations, min_size and image_size. Then, it passes the anchors to torch land and returns the forward pass of the prior box tensor. +The __init__ function initializes the variables of priorbox according to the configuration of the training models. Furthermore, for each source feature map, it calculates the priorbox coordinates. +The forward function first loops through the feature maps to compute the anchor using dense face localizations, min_size and image_size. Then, it passes the anchors to torch land and returns the forward pass of the prior box tensor. ## modules ### multibox_loss.py This file defines a class MultiBoxLoss. The class consists of __init__ and forward functions. +The __init__ function initializes variables related to comparing ground truth boxes and priorboxes. +The forward function computes the SSD weighted loss: the calculations include confidence target indices, localization target and hard negative mining. From 045ac6c2bb8fae21f3e20e332868af9633dae6fb Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 18:32:19 +0900 Subject: [PATCH 29/38] Update README.md --- src/jetson/models/Retinaface/README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index d14443f9..09b07801 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -31,11 +31,15 @@ config.py defines a library of configurations, including inference configuration ## functions ### prior_box.py This file defines a class PriorBox. The class consists of __init__ and forward functions. + The __init__ function initializes the variables of priorbox according to the configuration of the training models. Furthermore, for each source feature map, it calculates the priorbox coordinates. + The forward function first loops through the feature maps to compute the anchor using dense face localizations, min_size and image_size. Then, it passes the anchors to torch land and returns the forward pass of the prior box tensor. ## modules ### multibox_loss.py This file defines a class MultiBoxLoss. The class consists of __init__ and forward functions. + The __init__ function initializes variables related to comparing ground truth boxes and priorboxes. + The forward function computes the SSD weighted loss: the calculations include confidence target indices, localization target and hard negative mining. From 9e628d88136e3e348442de032fecc1937895c4c7 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 18:42:55 +0900 Subject: [PATCH 30/38] Update README.md --- src/jetson/models/Retinaface/README.md | 40 ++++++++++++++------------ 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index 09b07801..30e23eed 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -1,23 +1,27 @@ -# net.py +## net.py This file describes functions about returning convolutional layers. -## conv_bn -## conv_bn_no_relu -## conv_bn1X1 -## conv_dw -## SSH -### __init__ -### forward -## FPN -## MobileNetV1 - - -# retinaface.py +#### conv_bn +This funcion returns one layer of batch normalized convolution layer (filter=3x3) applying relu activation. +#### conv_bn_no_relu +This function returns one layer of batch normalized convolution layer (filter=3x3) without applying relu activation. +#### conv_bn1X1 +This function returns one layer of batch normalized convolution layer (filter=1x1) applying relu activation. +#### conv_dw +This function returns layer of batch normalized depthwise convolution layer applying relu activation +#### SSH +This class defines single stage headless face detector. +#### FPN +This class defines feature pyramid network. +#### MobileNetV1 +This class defines MobileNetV1 model used as backbone for face detection. + +## retinaface.py This file describes functions about making layers describing facial images. -## ClassHead -## BboxHead -## LandmarkHead -## RetinaFace -## load_model +#### ClassHead +#### BboxHead +#### LandmarkHead +#### RetinaFace +#### load_model # data This folder consists of __init__.py and config.py. From fd5845e08ae9e270487f86ab6adb2bc4685f6d05 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 19:25:57 +0900 Subject: [PATCH 31/38] Update README.md --- src/jetson/models/Retinaface/README.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index 30e23eed..cf8aff71 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -7,14 +7,21 @@ This function returns one layer of batch normalized convolution layer (filter=3x #### conv_bn1X1 This function returns one layer of batch normalized convolution layer (filter=1x1) applying relu activation. #### conv_dw -This function returns layer of batch normalized depthwise convolution layer applying relu activation +This function returns layer of batch normalized depthwise convolution layer applying relu activation. #### SSH This class defines single stage headless face detector. + +The __init__ function initializes batch normalized convolutional layers using the number of input and output channels, the stride and the leakiness. + +The forward function receives tensor outputted by backbone. Using this input, it returns output tensor after applying network layers(batch normalized convolutional) and operations. #### FPN This class defines feature pyramid network. + +The __init__ function initializes batch normalized convolutional layers using the list of input channels, the number of output channels, the stride and the leakiness. + +The forward function first initalizes 3 different batch normalized convolutional layers(output1, output2, output3); each initalized layer uses respective input channel from the input channel list. Then, output2 is updated, involving the interpolation of output3 and the merge of output2. Afterward, output1 is updated, involving the interpolation of output2 and the merge of output1. Finally, the forward function returns output1, output2 and output3. #### MobileNetV1 This class defines MobileNetV1 model used as backbone for face detection. - ## retinaface.py This file describes functions about making layers describing facial images. #### ClassHead From fc867a229272b66c67d0716752262bcd95b1c06a Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 19:32:40 +0900 Subject: [PATCH 32/38] Update README.md --- src/jetson/models/Retinaface/README.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index cf8aff71..0cf77064 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -22,6 +22,10 @@ The __init__ function initializes batch normalized convolutional layers using th The forward function first initalizes 3 different batch normalized convolutional layers(output1, output2, output3); each initalized layer uses respective input channel from the input channel list. Then, output2 is updated, involving the interpolation of output3 and the merge of output2. Afterward, output1 is updated, involving the interpolation of output2 and the merge of output1. Finally, the forward function returns output1, output2 and output3. #### MobileNetV1 This class defines MobileNetV1 model used as backbone for face detection. + +The __init__ function first initalizes different stages of sequential batch normalized convolution layers and batch normalized depthwise convolution layer. It also initializes operational functions. + +The forward function receives input tensor x: x is either a single image tensor or a batch of them. The forward function applied the stagges of network layers and performs operations. Finally, it returns the output tensor after passing through MobileNetV1 backbone. ## retinaface.py This file describes functions about making layers describing facial images. #### ClassHead @@ -41,7 +45,7 @@ config.py defines a library of configurations, including inference configuration ## functions ### prior_box.py -This file defines a class PriorBox. The class consists of __init__ and forward functions. +This file defines a class PriorBox. The __init__ function initializes the variables of priorbox according to the configuration of the training models. Furthermore, for each source feature map, it calculates the priorbox coordinates. @@ -49,7 +53,7 @@ The forward function first loops through the feature maps to compute the anchor ## modules ### multibox_loss.py -This file defines a class MultiBoxLoss. The class consists of __init__ and forward functions. +This file defines a class MultiBoxLoss. The __init__ function initializes variables related to comparing ground truth boxes and priorboxes. From f0a079911788f0111e68b174e6ea21974a19a167 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 19:34:39 +0900 Subject: [PATCH 33/38] Update README.md --- src/jetson/models/Retinaface/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index 0cf77064..e041bc4b 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -29,6 +29,7 @@ The forward function receives input tensor x: x is either a single image tensor ## retinaface.py This file describes functions about making layers describing facial images. #### ClassHead +This class #### BboxHead #### LandmarkHead #### RetinaFace From 49bb99b666f779ee9c1fc7f21791033bbf938707 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 22:00:14 +0900 Subject: [PATCH 34/38] Update README.md --- src/jetson/models/Retinaface/README.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index e041bc4b..31429ae4 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -29,9 +29,23 @@ The forward function receives input tensor x: x is either a single image tensor ## retinaface.py This file describes functions about making layers describing facial images. #### ClassHead -This class +This class describes functions for the classification of the image. + +The __init__ function adds layers on top of feature extractor for classification. + +The forward function receives tensor from the face extractor and returns reshaped tensor after passing it through a 1x1 conv layer. #### BboxHead +This class describes functions for creating a bounding box around the face and calculating face coordinates. + +The __init__ function adds layers on top of feature extractor for finding face coordinates. + +The forward function receives tensor from the face extractor and returns reshaped output tensor after passing it through a 1x1 conv layer. #### LandmarkHead +This class describes functions for calculating face landmark coordinates. + +The __init__ function adds layers on top of feature extractor for finding face landmark coordinates. + +The forward function receives tensor from the face extractor and returns reshaped output tensor after passing it through a 1x1 conv layer. #### RetinaFace #### load_model From 7d6a0c1076176b970cf124686ca48af38d40f310 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 22:05:03 +0900 Subject: [PATCH 35/38] Update README.md --- src/jetson/models/Retinaface/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index 31429ae4..b499dd42 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -1,5 +1,5 @@ ## net.py -This file describes functions about returning convolutional layers. +This file describes functions that return batch normalized convolutional layers and defines classes Single Stage Headless face detector, Feature Pyramid Network and MobileNetV1. #### conv_bn This funcion returns one layer of batch normalized convolution layer (filter=3x3) applying relu activation. #### conv_bn_no_relu From 0b24bd00072718fcacd1bc817856c15a17ed2929 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 22:07:17 +0900 Subject: [PATCH 36/38] Update README.md --- src/jetson/models/Retinaface/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index b499dd42..20b6203c 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -48,6 +48,7 @@ The __init__ function adds layers on top of feature extractor for finding face l The forward function receives tensor from the face extractor and returns reshaped output tensor after passing it through a 1x1 conv layer. #### RetinaFace #### load_model +This function returns trained model loaded to desired device. # data This folder consists of __init__.py and config.py. From b69a80e026eded2f5e5947552e02fea75c0f5ad9 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 22:40:43 +0900 Subject: [PATCH 37/38] Update README.md --- src/jetson/models/Retinaface/README.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index 20b6203c..8db6cbfe 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -27,7 +27,8 @@ The __init__ function first initalizes different stages of sequential batch norm The forward function receives input tensor x: x is either a single image tensor or a batch of them. The forward function applied the stagges of network layers and performs operations. Finally, it returns the output tensor after passing through MobileNetV1 backbone. ## retinaface.py -This file describes functions about making layers describing facial images. +This file describes the RetinaFace model and the classes and functions used to operate on the facial images. + #### ClassHead This class describes functions for the classification of the image. @@ -47,6 +48,18 @@ The __init__ function adds layers on top of feature extractor for finding face l The forward function receives tensor from the face extractor and returns reshaped output tensor after passing it through a 1x1 conv layer. #### RetinaFace +This class defines the RetinaFace model. + +The __init__ function + +The _make_class_head function adds layer on top of retinaface for classification. + +The _make_bbox_head function adds layer on top of retinaface for outputing bounding box coordinates. + +The _make_landmark_head function adds layer on top of retinaface for outputing facial landmark coordinates. + +The forward function + #### load_model This function returns trained model loaded to desired device. From b97d84d616d355a16783ba2b13bf91aa44294e54 Mon Sep 17 00:00:00 2001 From: lee2998p <59533465+lee2998p@users.noreply.github.com> Date: Thu, 6 Aug 2020 23:05:26 +0900 Subject: [PATCH 38/38] Update README.md --- src/jetson/models/Retinaface/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/jetson/models/Retinaface/README.md b/src/jetson/models/Retinaface/README.md index 8db6cbfe..17d28453 100644 --- a/src/jetson/models/Retinaface/README.md +++ b/src/jetson/models/Retinaface/README.md @@ -50,7 +50,7 @@ The forward function receives tensor from the face extractor and returns reshape #### RetinaFace This class defines the RetinaFace model. -The __init__ function +The __init__ function initalizes variables and configurations used for training. The _make_class_head function adds layer on top of retinaface for classification. @@ -58,7 +58,7 @@ The _make_bbox_head function adds layer on top of retinaface for outputing bound The _make_landmark_head function adds layer on top of retinaface for outputing facial landmark coordinates. -The forward function +The forward function first receives the input image(s) and passes it(them) through FPN (feature pyramid network). Then, it operates on the facial images with classes SSH (single stage headless face detector), ClassHead, BboxHead and LandmarkHead. Finally, it returns bounding box and landmark coordinates and class confidences. #### load_model This function returns trained model loaded to desired device.