🚀 Temporal-Action-Localization (TAL)

Temporal Action Localization using GRU-Splitted Network and Learn-to-Rank

This repository implements a new network based on Gated Recurrent Unit (GRU) with novel post-processing methods for Temporal Action Localization (TAL).

• Introduces the GRU-Splitted model, a new output layer design for GRU.
• Uses linear interpolation to generate action proposals with precise temporal boundaries.
• Employs a Learn-to-Rank (LTR) module for ranking the generated proposals.
• Evaluated on Thumos14, achieving 27.52% mAP at IoU 0.7, 5.82% higher than state-of-the-art.

---

📌 Table of Contents

1. Overview
2. Framework
3. Data Preparation
4. Training & Evaluation
5. Ranking Proposals
6. Qualitative Results
7. Reference

---

🧠 Overview

Temporal Action Localization (TAL) involves detecting action instances in untrimmed videos with precise start and end times.

This method combines:

• GRU-Splitted: GRU with a new output design for improved temporal modeling.
• Linear Interpolation: To refine proposal boundaries.
• Learn-to-Rank (LTR): For ordering proposals based on likelihood.

---

🛠 Framework

Figure 1: Overview of GRU-Splitted network for Temporal Action Localization.

---

📂 Data Preparation

• We use I3D features from the Thumos14 dataset, prepared via RecapNet.
• Download the features and place them in the data/ folder.

⚡ Ensure the folder structure matches the training scripts.

---

🛠 Training & Evaluation

Training the Probability Prediction Model

1. Run the Model script to train the GRU-Splitted network.
2. The trained model will be saved in Trained_Model/.

Generating Proposals

• Run Eval_Gen to generate action proposals.
• To compute precise temporal boundaries, run Eval_Gen_Interpolation.

Evaluation Metrics

• Compute AR@AN and R@100-tIoU using Eval_Metric.

---

📊 Ranking Proposals

To rank the generated proposals more effectively:

1. Train one of the ranking modules: Model_prop or Model_prop_LTR.
2. Extract features from the generated proposals using modified prop_feat or prop_feat_all scripts.
3. Evaluate the ranked proposals with Eval_Gen_Prop.

⚡ The ranking approach is inspired by BSN, with modifications for this method.

---

📸 Qualitative Results

Generated proposals visualized on sample videos:

These figures show action proposals generated by the GRU-Splitted network.

---

📚 Reference

Please cite the following paper if you use this method:

@article{keshvarikhojasteh2023temporal,
  title={Temporal action localization using gated recurrent units},
  author={Keshvarikhojasteh, Hassan and Mohammadzade, Hoda and Behroozi, Hamid},
  journal={The Visual Computer},
  volume={39},
  number={7},
  pages={2823--2834},
  year={2023},
  publisher={Springer}
}