Implementing Vision
Integrating Vision into Robot Code
Vision integration means reading NetworkTables data, feeding AprilTag measurements into odometry, and using vision feedback for control.
Vision-corrected odometry is what makes accurate autonomous and teleop assists possible.
Vision Implementation Strategy
Four steps get Limelight data into your robot's odometry without letting bad readings wreck the pose estimate.
🚀 Implementation Sequence
LimelightHelpers Library
First, import the Limelight helper library available on GitHub. It contains pre-built NetworkTables wrappers that provide clean access to vision data without manual NetworkTables subscriptions.
Limelight Subsystem
Next, create a new subsystem to pull values using the Limelight helper tool. The pose estimator needs three things from it: pose, timestamp, and standard deviation (how much we trust the reading). Pose and timestamp come straight from LimelightHelpers; the trust formula we have to write ourselves.
CTRE Pose Estimator
Once we have those three values, pass them into the CTRE pose estimator, which has built-in methods for exactly this. The vision subsystem needs a reference to the pose estimator so it can add measurements.
Wire it up in Robot
The drivetrain owns the pose estimator. In the OpMode model you wire the camera up once in Robot's constructor (e.g. Limelight.registerAll(drivetrain, "limelight")), passing it the drivetrain so it can add measurements.
Why This Approach?
- Library First: LimelightHelpers abstracts NetworkTables complexity.
- Validation Layer: The Limelight subsystem filters bad measurements before they make it to your pose estimate
- Dynamic Trust: Standard deviations adjust based on measurement quality, preventing bad data from degrading odometry
Standard Deviation & Filtering
Trusting vision data correctly is just as important as receiving it. We use dynamic standard deviations plus a few filters so only good measurements reach the odometry.
Formula for Workshop
We use a simple formula based on tag count and distance. As the robot gets further from tags, the standard deviation increases (trust decreases). More tags visible decreases the standard deviation (trust increases).
Suggested Filtering Strategies
Beyond the formula, we apply several filters to reject bad data entirely:
- •Field Boundary Check: Reject poses that are outside the field perimeter.
- •Ambiguity Filter: For single-tag detections, reject if the ambiguity score is too high (indicating the tag might be flipped).
- •Z-Height Check: Reject poses where the robot is calculated to be flying or underground.
Camera Setup & Calibration
Calibration matters: if the camera's mounting offsets or lens distortion are wrong, every pose it reports is wrong too.
Limelight Camera Configuration
Set up your Limelight camera with proper positioning, focus, and calibration.
Change Pipeline to AprilTag
Access the Limelight web interface and switch the active pipeline to AprilTag mode. This enables 3D pose estimation using AprilTags for accurate robot localization.
Adjust Exposure
In the camera settings, reduce the exposure as low as possible while still reliably detecting AprilTags. Lower exposure reduces motion blur and improves tag detection accuracy during fast robot movement.
Set Camera Offsets
Accurately measure and enter your camera's position and angle relative to the robot's center. This transform is critical for converting camera detections into accurate field-relative robot poses. Follow the Limelight documentation for detailed instructions.
Camera Calibration
Use a Limelight calibration board to calibrate your camera. This corrects for lens distortion and improves pose accuracy, especially at the edges of the field of view. Follow the Limelight Calibration Guide for detailed instructions.
Reading Limelight Data
Limelight publishes vision data to NetworkTables. The LimelightHelpers library (published by Limelight on GitHub) gives you a clean API for reading this data without direct NetworkTables access. Limelight distributes it against the current-season WPILib packages, so the workshop repo carries a copy migrated to the 2027 org.wpilib.* packages — that copy is what the code below uses.
LimelightHelpers.java
LimelightHelpers
The workshop's copy of LimelightHelpers, migrated to the WPILib 2027 packages. This is the exact file the Limelight subsystem below uses to retrieve pose estimates and raw vision measurements.
Loading file...
Limelight.java
Limelight Code
Subsystem that pulls robot pose from LimelightHelpers, validates the estimate, models measurement noise from tag distance/count, and feeds pose+timestamp+std devs to a consumer (e.g., your drivetrain pose estimator). Caches the last valid estimate and exposes getters for logging.
Loading file...
Robot.java (vision wiring)
Vision is wired up once in Robot's constructor: Limelight.registerAll(drivetrain, ...) creates each camera and registers its per-loop update() on the scheduler. Shared subsystems like the drivetrain live on Robot as public final fields, and each OpMode reaches them through the Robot reference it is constructed with.
Loading file...
The camera isn't a Mechanism
The Limelight owns no actuators, so it isn't a Mechanism — it's a plain class whose update() is registered on the scheduler with Scheduler.getDefault().addPeriodic(camera::update), wired up once from Robot's constructor via Limelight.registerAll(drivetrain, ...). Its job is to read LimelightHelpers / NetworkTables, validate the pose estimate, model the measurement noise (std devs) from tag distance and count, and feed drivetrain.addVisionMeasurement(pose, timestamp, stdDevs).
Utils.fpgaToCurrentTime(...) was removed (Phoenix 6 now shares the WPILib timebase), so vision timestamps go straight to the pose estimator without any conversion.
Workshop Code Implementation
Everything above comes straight from the 3-Limelight branch of the Workshop-Code repository, which integrates the Limelight with the swerve drive and odometry. Reference the full working project and adapt it for your own robot.
Vision Best Practices
Do
- Validate vision data before using it
- Account for latency (automatically done)
- Use appropriate standard deviations
- Test different exposures (lower is better)
- Log vision data for debugging
Don’t
- Trust vision measurements blindly
- Ignore latency compensation
- Use vision as only odometry source
- Forget to tune camera settings
- Skip testing in match conditions
Additional Resources
What's Next?
Up Next: Dynamic Flywheel