NiceShot_AI: Python Computer Vision Tool

NiceShot AI is a Python tool powered by computer vision to analyze gameplay videos. With the integration of cutting-edge tools like YOLO, OpenCV, and FFmpeg, NiceShot AI is designed to automatically detect, track and clip key gameplay events from Call of Duty: Black Ops 6.

Simple demo showcasing tool results: (https://youtu.be/nFs7VJxT-Ig)

---

Key Events Detected by the Model

• Kill
• Death: When a player gets killed during gameplay.
• Medal: When a medal earned by the player pops up during gameplay.

---

Model Description

YOLOv8n by Ultralytics. Fine-tuned on custom collected & annotated dataset of gameplay videos for Call of Duty: Black Ops 6. Model detects kill, death and medal pop up events with good confidence. Model is expected to generalize well on modern Call of Duty games as their core gameplay does not vastly change. Model is also tested on one Call of Duty: Black Ops 7 gameplay video providing initial very good detection results.

---

Tool Features

• Robust to Scalability: Uses configurable variables instead of hard-coded values, enabling it to adapt to different game models and events without massive changes. (still to be tested)
• Event Auto-Clipping: Clipping kill & death events from the CoD gameplay video.
• Clipping Kill Streaks: Clipping multiple consecutive kills during gameplay by concatenating all unique kills detections within time threshold between each detection and the following.
• Extracting Best Kill Clips: Extracting hot kill clips where multiple medals pop up during the event.
• Clips Export in 16:9 & TikTok formats
• Generating Kills Highlight Reel: Concatenating best or all extracted kill clips into one video with simple fade in & out transition edits between clips in both vertical & horizontal formats.
• Analyzing BO6 videos in bulk from a Twitch channel: Downloads and analyzes CoD BO6 streams from a Twitch channel performing bulk analysis of gameplay videos.

---

Session Analysis

• Events Timestamping & CSV Output: Timestamps detected events and dumps into a CSV file with 2 columns [Timestamp, Event] for further gameplay data analysis and inspections.

Below is an analysis report I prepared in PowerBI from the ".csv" data file, by creating DAX measures to calculate relevant statistics.

---

Advanced Tool Features

• Accurate Event Clipping: Using EasyOCR to avoid counting frames of KILLCAMS and SPECTATING.
• Custom Montage Lengths: Allowing for creating compilations of any length.

---

Limitations

• An event may get detected more than one time or not detected at all: From my testing, an event getting detected more than once is more likely than not getting detected at all which means more tuning should be done on the tracker algorithm.
• Special kill events are not detected: Model is not trained on detecting player's "grenade kill", "mine/trap kill", etc. Just face to face gun kills are counted.

---

Installation

To get started with NiceShot_AI, download & install ffmpeg from the official website: https://www.ffmpeg.org/download.html first and add it to your PATH.

Second: Install the Dependencies

Create a Python virtual environment (optional, but recommended). My Python version is 3.10.11

python -m venv venv
venv\Scripts\activate

Install torch cuda. What worked for me is cuda 12.1:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Install dependencies:

pip install -r requirements.txt

---

Run the tool

from src.niceshot_ai.detector import EventDetector

detector = EventDetector('yolov8n-cod_bo6.pt', # YOLO Model Path
                    'ffmpeg.exe', # FFMPEG Path (You can find it where you installed FFMPEG)
                    "video.mp4", # CoD BO6 gameplay video or Twitch Channel Link
                    total_hours=0.15, # Total hours to analyze of the video (each video in case analyzing a Twitch Channel)
                    save_clips=True, # Save clips locally
                    add_to_csv=True, # Add events and timestamps to a CSV file.
                    output_dir='output', # Output directory where all clips, highlights and CSV file are saved
                    csv_file=f"timestamps.csv", # CSV file path
                    frames_to_skip=5, # Frames to skip during analysis (The more, the faster the analysis is finished)
                    frame_idx_start=0, # Starting frame
                    create_montage=True, # Create a highlight reel of kills
                    max_workers=6, # Default
                    max_videos=3, # Only useful if passing a Twitch channel as it gets the most recent specified number of videos
                    montage_length_sec=50, # Total duration of the generated highlight reel in seconds
                    vertical_format=True, # Output clips in vertical format
                    advanced_detection=True # Use OCR to confirm some events
                    ) 

detector.detect_events()

---

Processing Speed

Tested on my new laptop with the following specs:

• CPU: core i9 14th gen
• GPU: RTX5070 8GB
• RAM: 32GB

Tested on my old laptop with the following specs:

• CPU: core i7 10th gen
• GPU: GTX1650 4GB
• RAM: 16GB

Device	Frame Inference
New laptop	Up to 170 FPS
Old laptop	Up to 60 FPS

Advanced Detection with OCR

This is run only to confirm an event after it's detected. Not through the whole video frames. It can cause the processing speed to fall down from 170 FPS to 30 FPS (on new laptop) temporarily until event is confirmed. It can definitely be turned off, however this will cause a kill event during "SPECTATING" to be counted.

---