[general]
name=Bambi - QGIS Integration
qgisMinimumVersion=3.22
description=Detect and track wildlife in drone videos/photos with geo-referencing
version=7.0.0
author=Christoph Praschl, Anna Maschek, David C.Schedl
email=christoph.praschl@fh-hagenberg.at

about=A comprehensive QGIS plugin for detecting, tracking, and geo-referencing wildlife in aerial drone recordings (videos or still photos) from DJI Enterprise drones with thermal and/or RGB cameras.
    Features include:
    - Frame extraction and undistortion from drone videos (SRT timestamps) or photo collections (EXIF)
    - Animal detection using YOLO models (auto-downloaded from HuggingFace)
    - Multi-object tracking with built-in, BoxMOT, or geo-referenced backends; import of pre-computed TRex tracklets
    - Demographic classification of tracked animals: cross-modal RGB/thermal track matching, DINOv3 features, occlusion/species/sex classifiers aggregated per animal by vote, and a size-based juvenile flag (requires a Hugging Face token for the gated DINOv3 model)
    - Segmentation tool: SAM3 / SAM 3.1 on the extracted frames by text or clicked point prompts, per frame or tracked across a sequence, through the Roboflow API, transformers, or Meta's sam3 package; masks geo-referenced, added as layers and exported as GeoJSON
    - Geo-referencing of detections, tracks, and SAM3 segmentations onto a DEM (real-world UTM coordinates)
    - Flight route visualization and perpendicular distance sampling for transect-based surveys
    - Survey analytics: kernel-density heatmaps, line-transect distance-sampling density/abundance estimation (from detections or tracks), and transect-based population estimation (naive, bootstrap, zero-inflated negative binomial)
    - Per-frame camera field-of-view footprints and coverage areas
    - Georeferenced map products: per-frame GeoTIFFs, true orthomosaics, and ALFS light-field mosaics
    - DEM import: automatic download (Austria), GeoTIFF conversion, or flat surface meshes for aquatic surveys
    - Calibration wizards for camera intrinsics (single-camera SfM, stereo RGB+thermal) and per-flight pose corrections
    - Companion tools: result video creator, radiometric thermal image viewer (DJI Thermal SDK), randomized transect flight planner, and interactive map-canvas inspectors
    - Key-frame based labelling tool: review detections/tracks on extracted frames (thermal or RGB), draw and edit track bounding boxes with species/sex/age/occlusion classes, interpolate between key frames, and propagate boxes of one or several tracks across frames via DEM geo-referencing
    - Transect splitting tool: split a flight into named transects (frame ranges) on the extracted frames, with a flight-route overview map, flight-path length measurement and an "end after X metres" helper
    - Results stored in GeoPackages that carry their own meaning: every detection has an id the later steps refer to, user-defined fields travel through the whole pipeline, and each step records what it produced and what it invalidated
    - Export to COCO, YOLO, MOT, TRex tracklets, GeoJSON (animals or segmentations), Camtrap DP and Darwin Core Archive (GBIF publishing)
    - Several flights per QGIS project, each with its own target folder, configuration and layer group
    - Built-in dependency manager to install all required and optional packages from within QGIS
    All results are automatically added as styled layers to the QGIS project, grouped per flight.

changelog=Version 7.0.0
    A survey is more than a count: what tells you about a population is which animals are in it. This release adds the demographic step, following "When One Modality Is Not Enough: Multimodal Sex and Life-Stage Classification of Red Deer from Aerial RGB-Thermal Video". Tracked animals are classified by species and sex, juveniles are flagged by size, and the two cameras are used together rather than separately - because they fail in opposite conditions. In colour a deer under canopy blends into the ground; in thermal it is an unmistakable blob but the fur colour is gone and antlers show only while they are still growing and warm. Which sensor carries the sex cue therefore changes with the season, and reading both at once is what stays reliable across the year.
    The classifiers read features from Meta's DINOv3, which is a gated model on Hugging Face: request access once and enter a read token in the new Classification tab. Nothing else in the plugin changes if you do not use it.
    Added:
    - Segmentation tool: Source selector - the extracted frames, or the P6 orthomosaic. The mosaic is cut into overlapping tiles, each tile is segmented, and a prompt's masks are merged back into one map, so a tree canopy seen in ten frames is one polygon with one id instead of ten duplicates. The mosaic's GeoTIFF transform geo-references the masks directly; no DEM step. Thermal mosaics are stretched to 8-bit for the model.
    - Segmentation tool (toolbar): SAM3 moved out of the Processing tab into a window of its own, because prompting is interactive. It shows the extracted frames of either camera, takes text prompts or clicked points (left = on the object, right = beside it, several objects per frame), and segments the current frame, a frame range, or tracks the objects across the range as one clip with the video tracker. Three backends: the Roboflow API (text on images), facebook/sam3 through transformers (everything, CPU or GPU), and Meta's official sam3 package, which is the only way to run SAM 3.1 - Meta's March 2026 update with object multiplexing for ~7x faster many-object video tracking, published on Hugging Face as a bare checkpoint without a transformers integration. Geo-referencing, the QGIS layers and the GeoJSON export of the masks live in the window too; results merge per prompt, so a re-run of "deer" leaves "boar" alone. The SAM3 configuration tab is gone - backend, model and prompts are remembered per machine, not per flight.
    - ALFS-PY 3.0: the PyTorch renderer is no longer a separate package. One framework now carries three render engines (ModernGL, PyTorch, Vulkan) and two ray casters (Embree, Warp), and the Dependency Manager picks them from two dropdowns instead of a PyTorch checkbox. The choice is saved in the QGIS project and exported to the renderer on every step that renders, so a project renders the same way whoever opens it - and Install fetches exactly what that choice needs. A machine still carrying the old AlfsTorch package is told, and Install removes it: it provided the same import name and would otherwise silently override the selection.
    - Species classification has a published default model (cpraschl/bambi-species-classification on Hugging Face): red deer, roe deer and wild boar, downloaded on first use like the occlusion and sex classifiers. Its class names map onto the project's species without configuration. The default crop padding is 0.25 - a square at 1.5x the longer box side, which is what all the published classifiers were trained on; a project embedded at another padding simply embeds again into a new run.
    - Classification (Processing tab): occlusion, species, sex and age classifiers on tracked animals, each its own step so they can be run separately or repeated one at a time, plus a size-based age estimate where no classifier is configured. Species and sex are decided per animal by a vote across its frames, which is what makes a noisy per-frame call safe - an antler resolves only from some angles, so many frames of a true male look female and the majority still recovers him. The margin behind every call is kept, so a borderline animal can be reviewed, and the vote can be repeated at a different quorum without re-running anything.
    - C1 Match RGB and Thermal Tracks: works out which thermal track and which RGB track are the same animal, by registering the two views onto each other and comparing where the boxes sit. An animal both cameras saw is a confirmed animal; a track only one camera saw is either an animal the other sensor cannot make out, or noise. The run log reports the confirmed count, and when nothing matches it names which gate rejected everything and how far the closest candidate was - so "there were no animals" is distinguishable from "the gate is wrong for this resolution". Matched pairs can be added to QGIS as a line layer.
    - C2 Compute DINOv3 Embeddings: describes every animal's crop once, so all three classifiers reuse the same features. Vectors are written beside the frames, one file per frame, and are reusable outside the plugin. A re-run embeds only what is missing, so an interrupted run resumes rather than starting again, and changing the crop settings starts a new set without discarding the old one.
    - C6 Age Classification: a juvenile cannot be told from an adult female by appearance at survey resolution - which is why the sex classifier's second class is female/juvenile - so size settles it. An animal is called a juvenile only if it sits far below its cohort and has a clear gap to the next animal up; in any herd someone is smallest, and that alone is not evidence. Sizes are compared only within one flight, because how tightly boxes fit varies between recordings. Needs no models.
    - Classification configuration tab: the Hugging Face token (kept in the QGIS settings, not the project file, because a project gets shared), a "Check access" button that answers before a long run starts, the classifier table, crop and voting settings, and the cross-modal matching gates.
    - "Download models" in the Classification tab fetches every classifier set to Default, so the class mapping can be set up before anything is run - asking someone to run a classifier once before they can configure it is backwards. The classifier files are a few megabytes each; the DINOv3 model is downloaded separately on the first embedding run.
    - Class mapping per classifier: connects the classes a model returns to the values used in this project. The mapping follows the class order rather than the names, since a model returns positions - so renaming a label never re-points it, and a model that does not name its classes at all still works. Classes are read from the model where possible, probed for their count where the model carries no names, and defined by hand where the model is not available yet.
    - Sex and life-stage classifiers per species: the cue is species-specific - antlers, for red deer - so a model fitted on one species says nothing useful about another. A species with no sex classifier is simply not sexed, which is the honest answer rather than a guess. Life stage offers a third choice per species, "Size-based", which is the box-area measurement rather than a model, and is the default because no life-stage model has been published yet. Which of the two decides a species is one decision in one place rather than a model choice plus a switch elsewhere.
    - Life stage is measured on the geo-referenced boxes where geo-referencing has run, which makes the areas metric; otherwise on the camera-frame boxes, which still work because the comparison never leaves the flight. Which was used is recorded with every verdict, so metric and pixel figures are never mixed. Animals a classifier already called still count towards the cohort statistics - excluding them would shift everyone else's score - they simply do not take a verdict from it.
    - Occlusion is optional. It selects which frames carry usable evidence, but species and sex run either way: over the frames the occlusion classifier called clear, or over occlusion values you annotated by hand, or over every frame. The run log always says which of the three it used, so voting over occluded frames is never silent.
    - Results are written onto the animals themselves - species, sex, life stage and per-frame occlusion - which is what makes them visible to the exports, the map layers, the survey analytics and the labelling tool. Optional, repeatable, and it never changes a track you annotated by hand or a species the detector itself identified.
    - Both perspective and orthorectified crops are supported, with the classifier variant following the imagery so a model always sees the kind of image it was trained on.
    - Dependency Manager: a Classification group installing transformers and huggingface-hub. The same packages serve local SAM3 segmentation, which needs transformers 5.0 or newer.
    - Export: a GeoJSON exporter for the SAM3 segmentation masks, writing their outlines as polygons in WGS84 longitude/latitude, as GeoJSON requires. Masks that were never geo-referenced are skipped and reported rather than written in pixel coordinates.
    - S1 Run SAM3 Segmentation can run on your own machine. "Run SAM3 locally" in the SAM3 Segmentation tab loads Meta's facebook/sam3 through transformers instead of sending every frame to Roboflow, so no Roboflow key is needed. The model is gated on Hugging Face exactly like DINOv3: request access once, and the token from the Classification tab downloads it into the shared model cache on the first run. A "Check access" button answers before the frames are loaded, the model field accepts a fine-tuned repository or a pinned revision, and the device follows the Classification tab. The masks are written in the same structure as the Roboflow results, so geo-referencing and the exports do not care which backend produced them.
    - Track details in the inspector: the Feature Viewer now lists what the project knows about the track it is showing - the same facts as its row in the track inventory: species, sex and age with their votes, box and occlusion counts, confidences, frames and times, positions, movement, the matched track on the other camera and the labelling-tool annotation - with the inventory's Approved checkmark beside it. A clicked detection shows the track it belongs to. Ticking Approved in the viewer or in the report is the same verdict, and an open report follows.
    - Reviewing in the inspector: a wrong result can be deleted from the project in the Feature Viewer. "Delete detection" removes the highlighted box on the current frame - the detection, its ground position, its place in the track, its classifier results and its match; the track keeps its other boxes, and a track whose last box goes is removed with it. "Delete track" removes the track with every one of its detections; a track drawn in the labelling tool takes its annotation with it. Both ask for confirmation and cannot be undone. The map follows: the box leaves the detection layers, a track that lost a box is redrawn from the ones it still has, and a deleted track's layers, group, matched-pair line and inventory point are removed - so no orphan boxes stay behind. An open track inventory report drops or refreshes the row, and the written inventory, density, distance-sampling and population results are marked stale, since their counts no longer hold. Boxes of a label track are not deleted one by one here; they are defined by its key frames and are edited in the Labelling Tool.
    Changed:
    - The GeoJSON export is now named "GeoJSON (animals)" alongside the new "GeoJSON (segmentations)". The old name described how the file was made rather than what is in it, which is no help when choosing between two of them.
    - New projects seed the occlusion vocabulary as clear/occluded, matching what the occlusion classifier reports, so predictions and hand annotations share one vocabulary. Projects created earlier keep the values they have - enum ids are append-only and are never renumbered - and a classifier is pointed at them through its class mapping. Migrating a 5.x flight appends whatever levels it carries rather than dropping them.
    - The Processing tab is grouped into Detection and Tracking, Classification, and Segmentation. They are not one sequence: classification works on animals the first section found, and segmentation neither needs that section nor feeds into it. Steps are prefixed A, C and S accordingly, so SAM3 segmentation is now S1.
    Fixed:
    - GeoJSON exports wrote the project CRS - usually UTM metres - into a format that has exactly one coordinate system, WGS84 longitude/latitude. The crs member that named the projection was dropped from the GeoJSON specification years ago and readers ignore it, so every animal landed at a longitude of several hundred thousand degrees. Both GeoJSON exports now project to WGS84 and write no crs member; an export without a known project CRS is refused rather than written wrong.
    - Camtrap DP media.csv left the timestamp column empty. The poses store each frame's capture time as an ISO-8601 string with the drone clock's timezone, which the exporter took for epoch seconds and silently failed to parse. Both forms are read now, and the timestamps are written at second precision with their offset; this also fills deploymentStart and deploymentEnd in deployments.csv and eventDate in the Darwin Core Archive, which went through the same code.

tracker=https://github.com/bambi-eco/Bambi-QGIS/issues
repository=https://github.com/bambi-eco/Bambi-QGIS
tags=wildlife,detection,tracking,drone,thermal,YOLO,geo-referencing

homepage=https://github.com/bambi-eco/Bambi-QGIS
category=Analysis
icon=icons/icon.png
experimental=True
deprecated=False
