Merged
Conversation
Signed-off-by: Charlie Huang <charliehuang09@gmail.com>
Signed-off-by: Charlie Huang <charliehuang09@gmail.com>
Signed-off-by: Charlie Huang <charliehuang09@gmail.com>
Signed-off-by: Charlie Huang <charliehuang09@gmail.com>
Signed-off-by: Charlie Huang <charliehuang09@gmail.com>
Signed-off-by: Charlie Huang <charliehuang09@gmail.com>
Signed-off-by: Charlie Huang <charliehuang09@gmail.com>
Signed-off-by: Charlie Huang <charliehuang09@gmail.com>
Signed-off-by: Charlie Huang <charliehuang09@gmail.com>
Signed-off-by: Charlie Huang <charliehuang09@gmail.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an NVIDIA VPI-based AprilTag detector backend (defaulting to the PVA backend) and switches existing integration/runtime entrypoints to use it, with a small robustness tweak in pose estimation.
Changes:
- Introduce
NvidiaAprilTagDetector(VPI) and link localization against VPI. - Switch
main_bot_mainand the AprilTag integration test to use the NVIDIA detector instead of the OpenCV detector. - Skip detections with unknown tag IDs in
SquareSolver.
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| src/test/integration_test/apriltag_detect_test.cc | Uses NvidiaAprilTagDetector in the integration test path. |
| src/main_bot_main.cc | Uses NvidiaAprilTagDetector for all three camera localization threads. |
| src/localization/square_solver.cc | Filters out detections whose tag IDs aren’t present in the field layout. |
| src/localization/nvidia_apriltag_detector.h | Defines NVIDIA/VPI detector API + defaults. |
| src/localization/nvidia_apriltag_detector.cc | Implements VPI AprilTag detection and VPI resource management. |
| src/localization/CMakeLists.txt | Adds VPI discovery and links localization to VPI. |
| src/camera/cv_camera.cc | Removes a backup-image debug log line. |
| scripts/deploy.sh | Deploys constants with --delete to keep remote in sync. |
| constants/misc/apriltag2.jpg | Adds a new AprilTag image asset. |
You can also share your feedback on Copilot code review. Take the survey.
| int max_detections = 16, bool verbose = false); | ||
| NvidiaAprilTagDetector( | ||
| int image_width, int image_height, const nlohmann::json& intrinsics, | ||
| VPIAprilTagDecodeParams params = {NULL, 0, 1, // NOLINT |
Comment on lines
15
to
20
| NvidiaAprilTagDetector::NvidiaAprilTagDetector(int image_width, | ||
| int image_height, | ||
| nlohmann::json intrinsics, | ||
| const nlohmann::json& intrinsics, | ||
| VPIAprilTagDecodeParams params, | ||
| VPIBackend backend, | ||
| int max_detections, bool verbose) |
| detection.tag_id = detections[i].id; | ||
| detection.timestamp = timestamped_frame.timestamp; | ||
| detection.confidence = detections[i].decisionMargin; | ||
| detection.confidence = 1; |
src/localization/CMakeLists.txt
Outdated
| @@ -1,4 +1,5 @@ | |||
| cmake_minimum_required(VERSION 3.10) | |||
|
|
|||
| find_package(VPI REQUIRED) | |||
| @@ -11,12 +11,14 @@ namespace localization { | |||
| // Nvidia's apriltag detection algorithim | |||
Comment on lines
12
to
13
| // Supports either VPA or CPU backend | ||
| // VPA and CPU are both slower than GPU we learned through benchmarks, but they can be used to offload some computation from the GPU |
| #include <fmt/chrono.h> | ||
| #include <vpi/Array.h> | ||
| #include <vpi/Stream.h> | ||
| #include <vpi/algo/ConvertImageFormat.h> |
Comment on lines
37
to
+46
| cv::Mat gray; | ||
|
|
||
| if (timestamped_frame.frame.channels() == 1) { | ||
| gray = timestamped_frame.frame; | ||
| } else if (timestamped_frame.frame.channels() == 3) { | ||
| cv::cvtColor(timestamped_frame.frame, gray, cv::COLOR_BGR2GRAY); | ||
| } | ||
|
|
||
| if (input_ == nullptr) { | ||
| (vpiImageCreateWrapperOpenCVMat(gray, 0, &input_)); | ||
| CHECK(!vpiImageCreateWrapperOpenCVMat(gray, 0, &input_)); |
Comment on lines
+50
to
+54
| if (!localization::kapriltag_layout.GetTagPose(detection.tag_id) | ||
| .has_value()) { | ||
| LOG(WARNING) << "Got invalid tag"; | ||
| continue; | ||
| } |
Signed-off-by: Charlie Huang <charliehuang09@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.