A · Multimodal
OpenAI · GPT-6 Astra
Checking server configuration…
01 Data intake
From sources to useful frames.
Import source footage, sample frames and select the data worth keeping.
Your project starts here
Build a useful collection.
Create a session, then add images or video from your machine. Imported files stay on this computer.
01Import
Keep source footage intact.
02Sample
Extract frames with provenance.
03Select
Choose frames for annotation.
Active session
0source files
0frames
0selected
Your sources
Source files
Import images or sample frames from video.
Drop images or videos
or choose files · originals stay local
Cancelling skips queued files. The active file finishes so its result stays clear. Keep this page open until it completes.
Source library 0
Images are ready immediately. Open the source library to extract frames or suggest passages from a video.
Choose what to review
Frame collection 0
Select images for annotation. Open a thumbnail to inspect its source.
No frames in this session
Review signals and inspection limits
Saved signals guide inspection. No detections does not mean a negative image.
Compare local models
Compare local detectors on the same frames. Results are saved with their model configuration.
Choose detectors
Checking models…Choose up to two models. Availability checks local files and dependencies; the runtime is loaded only when you start a run.
Loading model availability…
Install runtime or pretrained weights
The optional CPU runtime and official pretrained weights are installed separately. These commands download software and model weights; your source data stays local. No download starts from this page.
Run these commands from the project directory, then refresh
models. For a custom storage directory, add
--data-dir '/path/to/storage' to the download
command. See the README for installation details and model
licenses.
A missing or invalid trained checkpoint needs its saved training output or a workspace backup; these commands only download official weights.
Saved comparisons
Inspect outputs side by side, including completed frames from cancelled runs.
A shared view of the same data.
Choose your frames and models above to create a comparison. Saved results can be reopened even when the models are unavailable.
Display filters apply to saved detections only. Each model's native thresholds still apply; changing this slider does not rerun inference.
Counts and count differences are inspection signals. No validated labels are attached to this visual comparison. Quality evaluation measures full-image and tiled detection against frozen, human-validated labels. Use it to compare detection quality and processing time on the same images.
Run provenance and model settings
Configuration recorded when each model ran, including runtime and checkpoint identity. Timing is per frame, with one warm-up inference per run excluded. Tiled runs also record their crop layout and duplicate suppression settings.
Review queue
Inspect selected frames, resolve proposals and record your review.
Browse and filter selected frames
The queue includes selected frames. Selection for datasets is unchanged.
Low scores, uncertain recommendations and possible omissions are review hints, not proven errors. No saved target detections does not prove that the image is empty.
How to interpret model disagreement
Loading review queue…
No frames match this filter.
Build your ground truth.
Select images or extracted video frames in Data intake, then open a frame in the review queue to annotate the project's classes.
This frame changed since you opened it. Your unsaved edits are still here. Reload the latest revision before saving again.
Draw a box around each visible target object. Select a box to move it or drag its corners. Focus the canvas to zoom with the wheel; hold Space and drag to pan. Coordinates always refer to the original image.
Local edit history starts from the saved revision.
Keyboard shortcuts and local history
- V / B / H
- Select, draw or pan
- Space + drag
- Temporarily pan; the middle mouse button also pans
- + / −
- Zoom in or out
- 0 / 1 / F
- Fit image, actual pixel size or focus selected box
- Ctrl/Cmd + Z
- Undo a local edit
- Ctrl/Cmd + Shift + Z or Y
- Redo a local edit
- Delete / Backspace
- Remove the selected box
- Escape
- Cancel the current gesture
Shortcuts apply in the editor, outside text fields and dialogs. View changes do not modify labels. Up to 100 local edits can be undone; saving, reloading or changing frames starts a new history. Human validation remains explicit.
Proposals to review
Proposals stay separate from your labels until you accept them. Every decision needs a saved human review.
Matching trained classes or explicitly mapped official detector classes are imported. Importing does not validate any labels.
Multimodal review
Checking model availability…
Reviews the selected image and up to eight candidate crops with the chosen model. It can question a class or flag uncertainty; it does not find new box coordinates. You remain responsible for missed objects and final validation.
Multimodal review currently supports the original person / car definitions. Use manual annotation, a matching trained checkpoint or explicitly mapped official detector proposals for these classes.
Set up the local vision model
Install Ollama, then run the following command in a terminal. This downloads a multi-gigabyte model and uses local memory; IRIS never starts the download.
ollama pull qwen3-vl:4b-instruct
Start Ollama on its default loopback address, then refresh availability. Manual annotation and detector proposals work without it.
Set IRIS_DASHSCOPE_API_KEY and your workspace
URL in IRIS_DASHSCOPE_BASE_URL in the server
environment, restart IRIS, then refresh availability. API
keys are never entered in this browser.
Preview the exact outbound images and the cost limit before approving an API request. Preparing a preview stays local.
Review this API request
Provider pricingSaved revisions and provenance
Generate proposals · Local detector, one image or a batch
Generate new boxes with an available local detector. No model is downloaded here. Proposals never replace saved human labels or validate a frame.
Scores depend on the detector and are not calibrated probabilities of correctness. Check the whole image for missed targets even when no box is returned.
Available operations and class support
Preview checks saved inputs without inference. Creating runs the local detector and saves its raw output and separate proposals. Save or discard unsaved edits first.
Saved proposal runs
Cloud proposals · DINO-X · Preview cost, then review every box
DINO-X-1.0 generates boxes from your saved images and class prompts. Images are sent to an external provider only after your approval. Proposals remain separate from human labels.
Open this panel to check the provider.
Provider key
The key is stored by the local server. Its value is never returned to this page.
Class prompts
Leave empty for saved taxonomy defaults. Custom prompts map exact class IDs to text. The preview shows the prompts that will be sent.
Save or discard edits before previewing. Scores are not calibrated probabilities; inspect the whole image for missed objects, including empty results.
Preview is local and free. It checks saved inputs and identifies reusable or already submitted requests.
Must cover the estimate. Reused results and polling do not submit another paid request.
Saved DINO-X batches
Cancellation stops local work; a submitted request may still complete or incur a charge. Preparing remaining images restores the settings for a new preview. Saved successes are reused, submitted requests are polled, and unknown delivery outcomes are blocked.
Local batch review · Prepare proposals for several frames
Review 1–8 existing boxes per frame with a local vision model. Choose saved annotations or a saved detector run; this review classifies candidate boxes and does not discover new objects. Every proposal still needs your correction and validation.
Checking eligibility runs no inference. Starting queues the eligible frames for local processing; images stay on this machine. Unsaved editor changes are never included.
Batch history
Creates a new linked batch after checking current saved labels. Successful frames and existing proposals are kept; they are never replayed.
Validated data / Frozen versions
Build a release from reviewed images across your project, then use that exact version for training and evaluation.
Prepare a dataset
Choose reviewed data, assign scene groups, then freeze a release.
Saved releases
Saved versions remain unchanged as annotations evolve.
Validate frames in Annotation, then freeze your first release above.
Inspect frozen classes and identifiers
| Class | Boxes | IRIS class ID | Export category ID | Official COCO source ID |
|---|
Export category IDs belong to this saved COCO-format dataset. They are independent of optional mappings to the official COCO detector categories.
Saved class definitions
Local download of frozen images, reviewed labels, split files and recorded source/reviewer metadata. Up to 256 MiB.
Frozen frame revisions and provenance
Train a local detector
Choose a detector and how deeply to fine-tune it on a frozen release.
Training runs
Follow local runs, inspect saved progress and open a completed model.
0
Start a bounded local run once a dataset release and local model are ready.
Optimizer steps
0 / 0
Latest training loss
Not recorded
Recorded active time
Not recorded
Trained model ready
Compare its predictions or evaluate it on held-out images.
Saved progress & continuation
The latest two recovery states are retained for continuation. They cannot be used for detection; a completed run publishes a separate model checkpoint.
Continue from saved progress
Optimization history
Training loss describes optimization on the training frames. It is not detection accuracy and does not prove improvement. Use Quality evaluation to measure detection quality on held-out frames.
Training configuration and provenance
Prepare for another machine
Package a trained detector and compare its outputs on another machine.
Package library
Saved packages
Train a checkpoint, then complete a full-image evaluation in Quality evaluation to prepare a portable package.
Run & measure on the destination
Unzip on the target machine and follow README.md to install the pinned runtime. From the package directory:
python run.py inspect python run.py predict image.png --output ../prediction.json python run.py measure --output ../measurement.json --repeats 3
Measure runs the detector on the bundled images. One warm-up is excluded. Recorded prediction time excludes image decoding, model loading and package verification; inspect the report for separate timings. Exact output parity is distinct from detection quality and speed.
Immutable package identity and contract
Import target-machine measurements
Check the runner report, review parity and timings, then save it explicitly.
Review checked measurements
Checked parity, timings and declared environment
Saved target-machine measurements
Evaluate frozen validation frames across the current project. Human-validated boxes are the reference; the active session does not restrict this view.
Measure before you choose
Compare one or two detectors on the same held-out validation data.
Evaluation runs
Saved metrics, paired comparisons and the frames behind each error.
0
Complete an evaluation to inspect detection quality and record your choice of reference model.
All quality metrics and timing
Tiled forward times sum all tile passes. Total includes image decoding, crops and merging; weight loading and warmup are excluded. COCO AP keeps its 100-detection limit; the precision/recall counts use all saved predictions above threshold.
Quality and labeled support by class
AP and mAP are shown as percentages. A class without labeled examples has N/A AP and is excluded from mAP; inspect its support and false positives. Small or unrepresentative datasets do not establish real-world performance.
Explore saved errors
How to read these changes
A new miss is a labeled object matched by the baseline and missed by the candidate; a recovered label is the reverse. False-positive changes compare counts, not individual objects. Miss and false-positive ordering uses the larger count of either model.
Inspect errors
Dashed blue: human labelsGreen: matched detectionOrange: false positiveRed dashed: missed label
Evaluation protocol and provenance
Final test audit
After making your validation choices, evaluate the same models, inference modes, tiling settings and thresholds on the reserved test split. Keep test results for reporting; use validation to select a reference. The configuration is copied unchanged.
Reference model
Your recorded choice, with validation evidence and decision history.
No reference has been chosen.
Reference decision history
One sequence. Two tracking histories.
Reuse saved detector outputs. Each tracker keeps its own local IDs.
Freeze selected video frames
In Data intake, select at least two saved frames from one video. The sequence keeps their source order and records omitted frames as unknown gaps.
Calculate a detector cache
A new calculation runs only when you press the button. The saved score floor is 0.001; detector filtering and detection caps still apply.
Run a tracking comparison
Checking tracker runtime…
Default ByteTrack and BoT-SORT profiles · high 0.5 / low 0.1 · lost buffer 30 analyzed updates · no learned ReID · one replay pass · maximum 500 available frames.
Tracking replay
Solid boxes: detector observations. Dashed boxes: predictions without a current observation. Unconfirmed observations are labeled.
IDs are local to each lane. Matching numbers across lanes do not identify the same object. Replay overlays do not score quality. Reference-based metrics appear in the separate quality panel.
Display settings never rerun tracking or change its input. Trails break on missing observations and missing source frames. Prediction age uses analyzed updates; seconds are shown only when source time is known.
Continuity inspection
A loss followed by a different ID is not proof of the same real object. Occlusion, crossing and identity continuity require independent review.
Saved profiles, timing and provenance
Replay timing excludes detector inference. One pass leaves repeatability unchecked; elapsed time does not establish tracking quality.
Tracking quality
Compare the saved lanes against one frozen human reference revision.
Choose a completed tracking comparison to inspect or calculate its quality reports.
Only confirmed, measured observations and complete human-reviewed, scorable frames enter the metrics. Playback filters do not change the calculation. Opening a saved report creates no new report and runs no detector or tracker.
Review each native detector class mapping. “Ignore” removes that class from the evaluated scope. Suggestions use frozen taxonomy metadata; class names alone do not establish a mapping.
Detection and identity-event counts cover evaluated frames only. Excluded predictions, unconfirmed observations and unassigned detections below are diagnostics across all available frames. Undefined rates are shown explicitly. These metrics do not select a winner.
Frame matches, errors and continuity events
Switches and fragments are counted only inside contiguous evaluated source frames. Gaps and excluded frames reset continuity. Class-confusion pairs remain false positives and false negatives.
Excluded frames and reasons
Protocol limits and provenance
Pipeline cost
Measure a fresh detector and one tracker lane on the same verified saved frames.
Choose a completed tracking comparison to prepare a cost measurement or read its saved runs.
This measures a saved-frame pipeline. Camera capture, transport, video demux, UI display and database publication are outside its timing scope. It does not establish camera latency or tracking quality.
Prepare a fresh local measurement
The frozen detector recipe, weights and tracker profile are reused. A fresh process loads the model, warms up separately and resets the tracker for every repetition. Calculation begins only when you press Measure. Maximum 500 available frames.
Import a completed cost report
Import the JSON produced by the measurement CLI against this same frozen comparison. Hardware and execution are declarations from the file; import does not authenticate them or execute a model.
Choose a report file, then press Import explicitly.
Only completed reports publish measurements. A cancelled or failed attempt retains its task status.
Measured stages
All durations below come from this fresh run. Whole-pipeline and stage calls are measured on each processed frame. Nested detector/tracker timers overlap their enclosing calls and must not be added again.
Memory evidence
RSS is sampled at measurement boundaries. Process lifetime high-water includes setup and warmup; it is a different scope. CUDA values are PyTorch allocator peaks on the selected actual device, reset after warmup for each repetition, and do not represent whole-device memory.
Repetitions, processed frames and simulated scheduling
Hardware, warmup, protocol and provenance
Compare tracking settings
Explore a bounded set of profiles against saved human references. Keep the baseline until evidence supports a later decision.
Open Tracking to load study sources.
Prepare a bounded profile study
1. Freeze the data roles
Development and validation both inform tuning. Test sequences stay reserved and are never replayed or scored by this study. Related takes and overlapping frames cannot cross roles.
Create a frozen temporal dataset
Select sequences, assign their roles and pin the reviewed reference revision. A single development sequence is allowed; it cannot establish performance on another take.
2. Choose a baseline and candidates
Explicitly map native detector classes to the frozen reference taxonomy. Mapping suggestions use saved taxonomy metadata.
Suggestions explore individual settings and the alternate tracker. Include the candidates you want and inspect their parameters below. No profile is applied automatically.
3. Set the work budget
Repetitions measure tracker cost and repeatability; they do not add ground-truth frames. Detector inference, pipeline latency, memory and camera cadence are not measured here.
Run uses exactly this preview. Changing any setting requires a new preview. Cancellation, deadline or failure leaves no successful partial report.
Quality uses the first replay pass. Tracker adapter timings include association and camera-motion compensation; these nested timings must not be added again. Source-image reading, setup and full replay wall time have separate scopes. No detector or full-pipeline performance claim is made.
Source coverage, timing details and profile evidence
Selected-object continuity
Choose one object and inspect what happens when observations disappear or tracker IDs change.
This compares two diagnostic policies on saved observations. It does not replay an application’s complete selection logic. Geometric recovery is a hypothesis about the same object, not proof of identity.
Click a confirmed observed box to set the initial selection. The list below provides the same action by keyboard. Predictions and estimated boxes cannot be selected.
Visible selection rules
Guarded geometry applies confidence, class and geometry checks even when the numeric ID stays the same. Multiple plausible candidates cause abstention. Pending recovery never renews the last accepted observation. Clear the source-seconds limit to use available updates only; unknown timestamps also use that fallback.
Optional human reference assessment
Without a reference, the report describes behavior only.
This uses the saved first-pass observations. It launches no detector or tracker and changes no model, profile, reference or dataset role.
Behavior on the available frames
Assessment against the chosen human identity
Selected policy timeline
Frozen rules, source and reference provenance
Portable pipeline bundle
Package a saved detector and tracker profile for integration in another project.
Open Tracking to load saved sources.
Experimental package. Weights, settings and compatibility contracts are frozen together. New bundles include a standalone image/video runtime and a Python integration example. Quality and speed on your target device require separate tests.
Choose the detector target independently of the source device. The tracker uses CPU.
Includes unchanged checkpoint bytes, frozen settings, an integrity inspector, a standalone runtime, a Python example and license notices. No clips, source images, annotations or selected person identities are included.
Exact file inventory and compatibility contract
A download is published only after the complete package passes integrity checks.
Inspect the bundle without loading the model. Version, provenance and license details are saved below.
Run in another project
Extract into a new folder, then follow its README to prepare the CPU or CUDA environment. Choose input and output paths outside that folder.
iris pipeline extract pipeline.zip --to ./new-runtime python -B ./new-runtime/run.py check-runtime python -B ./new-runtime/run.py video /path/to/clip.mp4 --output /path/to/new-results.json
The package also includes example.py for timestamped frames and explicit selection events. Run reports remain separate from this saved package; they do not establish tracking quality.
Frozen manifest, provenance and license notices
Temporal reference
Human identities belong to this frozen sequence. Review is declared frame by frame.
Choose a frozen sequence to begin.
Reference provenance
A newer revision exists or this draft conflicts with saved evidence. Your draft is retained. Reload the latest revision to discard this draft, or keep it open while resolving the conflict.
Start a draft from saved tracking observations
Choose a lane and explicitly map its native classes. Only measured observations are copied; predictions are excluded. Every proposal remains uncertain and unreviewed until you inspect it. Seeding replaces the current draft and is available before the first saved reference.
Inspect each source frame
Draw or adjust boxes in original pixel coordinates. Missing source frames remain unknown.
Select a box to move it or drag its corner handles. At higher zoom, scroll to inspect small targets. A visibility record has no inferred box.
Human review records your declaration. Partial review preserves uncertain objects. Complete confirmation sets every object on this frame to certain and requires assigned identities and known visibility.
Independent evidence
Freeze a reviewed reference, tune candidate approaches, then compare their saved results.
Create an independent reference
Freeze reviewed labels and separate scene groups for tuning and evaluation. Candidate inputs contain the image and class definitions; reference boxes remain separate. Only validated manual or reviewed imported labels are eligible.
This records your declaration; IRIS cannot verify reviewer independence automatically. A saved reference is immutable.
Scene groups
Assign whole scenes to one role. Choose 1–25 images per role; both roles must contain images. Related scenes and shared pixels cannot cross roles.
Saved benchmarks
Reopen the frozen protocol, candidate configurations and measured results.
Prepare your candidates
Choose an approach and freeze its settings. Preview each trial before running it.
B · Segmentation
SAM 3 · local text-to-box
Checking local setup…
C · Combined
Astra → SAM 3 → Astra
Checking combined setup…
A uses an external multimodal API. B uses local SAM 3 with class prompts; this version produces boxes without masks. C uses Astra to prepare class prompts, SAM to generate boxes, then Astra to review the candidates. The local detector control stays available.
Add a frozen candidate configuration
Prepare up to 8 configurations during tuning. Preparing or saving settings sends no image to a provider.
C · One fixed pass through three stages
- Astra plans. The image and saved class definitions produce one SAM text prompt per class.
- SAM grounds locally. The generated prompts produce native candidate boxes above the frozen SAM threshold.
- Astra reviews. The image and candidate IDs, boxes and labels are sent for acceptance, rejection or reclassification. Coordinates are preserved; Astra cannot add boxes.
Up to two external requests per image, one SAM image encoding and one evaluation per class prompt. No iterative loop. Manual class prompts belong to approach B.
Frozen classes sent to Astra
Exact model: gpt-6-astra · image detail: original
Images are resized as configured and converted to PNG without source metadata. Reference boxes, review notes and independence notes are not sent. Multimodal proposals have no detector confidence score.
Checking local setup…
Freeze a configuration before installation if needed. SAM runs locally through an isolated runtime with the official weights and CUDA. Nothing is installed, downloaded or tested automatically.
One text prompt per saved class
These are the independent reference's frozen class definitions. Review each proposed text prompt: the same prompt is used for every image in this configuration. A prompt does not guarantee class recognition.
Native SAM scores are not calibrated or comparable with detector confidence. This adapter uses SAM's native box output; masks are not calculated or saved.
Use tuning trials to inspect candidates before locking the reference for evaluation.
Prepare an explicit trial
Inspect the frozen local work plan and runtime identity
Review outgoing requests
This conservative estimate is a planning bound, not a guaranteed invoice cap. The provider's billing record is authoritative. Images leave this computer; the request does not establish a processing region or zero data retention.
This configuration accepts saved evidence through the recorded-trials import API. Open an existing trial below to inspect its source receipts or measure human corrections. Preview and Run are unavailable because importing evidence does not execute a model.
Results by configuration and role
Results compare candidate output with the frozen reference. Human corrections are separate records; they never replace the reference. Missing times and unavailable costs remain unmeasured.
Compare the saved evidence and record your interpretation.
Compare approaches and keep a report
Compare every frozen configuration and every saved trial in one scene role. Failed and missing evidence stays visible. No trial is run and no winning approach is selected here.
Current saved evidence · refreshable comparison
Record your analysis
The report freezes the displayed role, all its saved trials, image evidence and correction revisions at preview time. Author conclusions are separate from measurements.
This declaration is not verified automatically. Simulations do not establish real model quality, speed or cost. One complete run cannot establish repeatability.
Report preview · not yet saved
Inspect all evidence frozen in this preview
Immutable reports
Objective
Author conclusion
This saved snapshot stays unchanged when live results or corrections change. The JSON contains full report evidence; the standalone HTML contains a readable summary without images or external resources.
Evidence retained in this immutable report
From results to a decision
Your next experiment deserves a record.
Bring together a frozen dataset, its models and a completed evaluation. Add what you set out to test, choose a few examples and write your own conclusion.
Uses saved results. No training or inference is started.The question
Your conclusion
Written by the reviewer. Metrics do not select a winner automatically.The experiment at a glance
Frozen when this report was createdQuality measurements
Results by scene
Deployment target measurements
Selected examples
Examples were chosen by the report author. They do not represent the whole dataset. Dashed white boxes are reviewed labels; solid colored boxes are saved predictions above the evaluation threshold.
No example images were included in this report. The metrics still cover the complete evaluated split.
Download this report
A standalone HTML file with the frozen results and your current notes.
Unchecked: metrics and notes only. Images are embedded only when you choose to include them. No data is sent to an external service.
Reproducibility and recorded reference decisions
Project jobs
Saved work across this project. You can keep working while jobs run; external requests are identified in their details.