Smart local audit, delta storage, surgical restore: travel back in time through your QGIS edits

v4.3 QGIS 3.44 - 4.x Local SQLite Automatic capture Multi-format Health monitoring Multi-writer safe
Version 4.3: 100% local journal (SQLite, schema v4). Smart delta storage (only changed attributes are recorded). Automatic schema drift detection. Datasource alias resolver (reconcile fingerprints after path moves). PID-based writer lock preventing concurrent QGIS instances from corrupting the journal. Consolidated audit report (top users, top layers, operation counts). Asynchronous writes with no UI freeze. Health monitoring with disk space alerts. Maintenance dialog with export. Automatic user identity resolution. Works with all editable formats: GeoPackage, Shapefile, PostgreSQL, SpatiaLite, MSSQL, Oracle, GeoJSON, CSV, and more.
Capture
Automatic recording

Every commit (attribute, geometry, deletion, insertion) is intercepted and recorded without any manual action.

Search
Paginated search

Filter by layer, operation type, and time window. Up to 500 events per page with full attribute reconstruction.

Restore
Selective restore

Individual or batch selection. Schema compatibility check before restore. Rollback on error.

Self-contained
No external dependency

SQLite only. No server, no network connection, no database configuration required.

Health
Live health monitoring

Journal size, event count, and disk space are monitored continuously. Alerts appear before a problem becomes critical.

Maintenance
Maintenance dialog

Retention policy, purge by age or session, async VACUUM, integrity check, and journal export from a single panel.

What RecoverLand does

  • Captures every committed edit on monitored editable layers (attributes, geometry, INSERT, UPDATE, DELETE)
  • Delta storage: only the attributes that actually changed are recorded (10–50x volume reduction versus full-row snapshots)
  • Smart edit buffer: merges multiple changes on the same feature into one net event before writing
  • Automatically filters layer audit fields (date_modif, updated_by, etc.) so metadata-only changes are not recorded
  • Paginated history search by layer, operation type, time window, and user
  • Selective restore with schema compatibility report before applying
  • Schema drift detection between the saved state and the current layer structure
  • Datasource alias resolver: reconcile fingerprints after a project path move without losing history
  • Writer lock: refuses to open the journal when another live QGIS instance is already writing to it (PID-based, Windows and POSIX)
  • Asynchronous batch writes in a dedicated thread (zero UI blocking)
  • Startup integrity check (WAL checkpoint, pending-event recovery, schema version validation)
  • Automatic user identification from: plugin config → env variable RECOVERLAND_USER → OS login → QGIS profile name
  • Live health monitoring with thresholds on journal size (50 MB / 200 MB / 500 MB) and event count
  • Disk space monitoring: auto-disable tracking at critical threshold (100 MB free), warning at 500 MB
  • Maintenance dialog: retention policy, purge by age or session, async VACUUM, integrity check, journal export
  • Orphan journal cleanup: unsaved-project journals older than 30 days are automatically removed
  • Works with all editable formats: GeoPackage, Shapefile, PostGIS, SpatiaLite, GeoJSON, CSV, FlatGeobuf, MS SQL Server, Oracle Spatial

What RecoverLand does not do

  • Does not replace a versioning system (Git, GeoGig)
  • Does not back up source files, only feature-level changes
  • Does not work on non-editable layers (WMS, rasters, virtual layers)
  • Does not capture changes made outside QGIS (direct file editing, external scripts)
  • Does not synchronize between workstations
  • Cannot guarantee restore on weak-identity formats (CSV, repacked shapefiles)
Honesty: RecoverLand is a local safety net, not a distributed versioning solution. It protects against editing mistakes in QGIS, not against loss of the source file itself.

Architecture

The plugin is organized in independent layers. Each layer has a single responsibility and communicates with others through defined interfaces.

QGIS: Editable layers (GeoPackage, Shapefile, PostGIS, ...) EditTracker (QGIS signals) WriteQueue (dedicated thread) SQLite Journal (audit_event) SearchService RestoreService RecoverDialog (user interface)
Runs inside QGIS
Stores data locally
No mandatory server
No background system service
User can export the journal
Trust boundary: In local mode, RecoverLand stays inside the QGIS process and writes to a SQLite file on the user's machine. It does not install a Windows service, does not inject code into the system, and does not require a remote database to protect local edits.

Main components

ComponentRoleFile
EditTrackerListens to QGIS signals (beforeCommitChanges, afterCommitChanges, afterRollBack) and generates audit eventscore/edit_tracker.py
EditSessionBufferIn-memory snapshot buffer. Merges multiple changes on the same feature into a single net event before commitcore/edit_buffer.py
WriteQueueThread-safe queue that writes events to SQLite in batches of 500. Zero UI blocking.core/write_queue.py
JournalManagerResolves journal path (project folder or QGIS profile), opens/creates the SQLite file, manages read/write connections and orphan cleanupcore/journal_manager.py
SearchServicePaginated search and attribute reconstruction from the journalcore/search_service.py
RestoreServiceCompatibility check, selective restore with per-batch isolation and rollback on errorcore/restore_service.py
JournalAuditConsolidated read-only introspection: operation counts, top users, top layers, time rangecore/journal_audit.py
DatasourceAliasReconciles datasource fingerprints across project moves via a transitive alias tablecore/datasource_alias.py
HealthMonitorEvaluates journal health (size, event count, disk space) and produces actionable messages for the UIcore/health_monitor.py
DiskMonitorPeriodically checks free disk space on the journal volume; disables tracking at the critical thresholdcore/disk_monitor.py
SupportPolicyDefines capture/restore support level and identity strength for each QGIS providercore/support_policy.py
AuditFieldPolicyIdentifies and filters layer audit metadata fields so they are never included in delta eventscore/audit_field_policy.py
LocalSettingsPersists per-project configuration (retention policy, user override, capture options) inside the journal's backend_settings tablecore/local_settings.py

Local trust boundary

The design goal is simple: make the recovery system visible, local, and bounded. Users can understand where data lives, who writes it, and what RecoverLand does not touch.

User workstation boundary QGIS + RecoverLand plugin Capture, search, restore, health checks All logic runs in the user session Local SQLite journal .recoverland/recoverland_audit.sqlite WAL mode, user-readable exportable file Touches only project data + journal Editable layers and one local audit file Does not install system components No Windows service, no driver, no registry dependency
What users can verify
Visible local artifact

The journal is a plain SQLite file stored next to the project or inside the QGIS profile for unsaved projects. It can be located, copied, exported, and inspected.

What remains bounded
No hidden multiplication

RecoverLand records events and lightweight restore traces, not full duplicated datasets or uncontrolled background replicas of the project.

SQLite Journal

The journal is a single SQLite file per QGIS project. It uses WAL mode (Write-Ahead Logging) to allow concurrent reads and writes without blocking.

Journal location

SituationPath
Saved project at C:/projects/site.qgzC:/projects/.recoverland/recoverland_audit.sqlite
Unsaved project[QGIS profile]/recoverland/audit/audit_.sqlite

Unsaved-project journals older than 30 days that no longer correspond to an open project are automatically removed at startup (orphan cleanup).

Schema of the audit_event table

Each row represents a single committed change on one feature.

ColumnTypeRole
event_idINTEGER PKAuto-incremented unique identifier
project_fingerprintTEXTIdentifies the source QGIS project
datasource_fingerprintTEXTUniquely identifies the source layer
feature_identity_jsonTEXTIdentifies the feature (FID + primary key when available)
entity_fingerprintTEXTCanonical restore key such as pk:id=42 or fid:7
operation_typeTEXTINSERT, UPDATE, or DELETE
attributes_jsonTEXTAttribute state before the change (delta or full snapshot depending on operation)
geometry_wkbBLOBGeometry in WKB format
new_geometry_wkbBLOBPost-edit geometry when the event schema provides forward geometry data
user_nameTEXTUser who made the change (auto-resolved)
created_atTEXTUTC timestamp of the event
restored_from_event_idINTEGERReference used by lightweight restore trace events
event_schema_versionINTEGERSchema version of the event payload written by the plugin

Simplified schema. The real table also contains: layer_id_snapshot, layer_name_snapshot, provider_type, geometry_type, crs_authid, field_schema_json, and session_id.

Local data model

The local database is intentionally compact. A few tables cover settings, datasource registry, edit sessions, and the event stream. The center of gravity is audit_event, which stores the history at feature level.

audit_event event_id, datasource_fingerprint entity_fingerprint, operation_type attributes_json, geometry_wkb created_at, restored_from_event_id audit_session One logical edit session Links to many events datasource_registry Fingerprint to source URI Used to recreate layers safely backend_settings Retention, overrides, options schema_version Migration tracking 1 session N events 1 datasource N events
TableResponsibilityWhy users should care
audit_eventImmutable event stream of edits and restore tracesMain recovery history, scoped at feature level
audit_sessionGroups edits made in the same sessionMakes large operations explainable and purgeable by session
datasource_registryRemembers how to find the original layer sourceHelps reconnect the history to the right layer later
backend_settingsStores local project settingsAvoids hidden system-wide configuration
schema_versionTracks database migrationsPrevents silent schema drift inside the journal

Automatic capture

Capture is transparent. The user edits layers normally in QGIS. RecoverLand intercepts edit signals at commit time.

User edits beforeCommitChanges Feature snapshot afterCommitChanges Write to journal

When capture triggers

User actionQGIS signalResult in the journal
Modify an attribute and savebeforeCommitChanges + afterCommitChangesUPDATE event with the previous state
Move a geometry and saveSameUPDATE event with the previous geometry
Delete a feature and saveSameDELETE event with the full state before deletion
Add a feature and saveSameINSERT event with the state after creation
Cancel edit (rollback)afterRollBackNothing (pending snapshots are discarded)
Changes outside QGIS: If a file is modified directly (text editor, external Python script, another GIS tool), RecoverLand does not detect it. Only changes made through the QGIS edit interface are captured.

Smart edit buffer

The edit buffer (EditSessionBuffer) is the working memory of RecoverLand. It captures the initial state of every modified feature at the start of the edit session and computes the real net effect at commit time.

Why a buffer?

A user may edit the same feature multiple times before saving. Without a buffer, each micro-change would generate a separate event. The buffer merges everything into one event representing the actual delta between the initial state and the final state.

Modify attribute A Modify attribute B Revert A to original Net effect: only B changed

Special cases handled by the buffer

ScenarioResult in the journal
Modify then revert to original valueNo event (no-change detected)
Insert then delete before commitNo event (net effect is null)
Modify an attribute + move geometryA single UPDATE event with attribute delta + previous geometry
Session rollbackBuffer flushed, nothing written

Audit field filtering

Layer metadata fields (date_modif, updated_by, updated_at, etc.) are automatically excluded from the delta by the AuditFieldPolicy module. If a commit only touches these fields, it is treated as a no-change and nothing is written. This filtering is applied at three levels: recording, reading, and restore.

Memory cap: The buffer has a RAM limit. Beyond 10,000 features or 200 MB, snapshots are flushed to a staging SQLite table to prevent memory overflow on large edits.

Delta storage

A naive audit would store the entire row on every change. On a 50-column table, changing one field would store 50 values. RecoverLand uses a differentiated strategy:

OperationStrategyContent stored
DELETEFull snapshotAll attributes + WKB geometry (the feature disappears; everything is needed to recreate it)
UPDATEDelta onlyPrevious values of only the changed attributes + previous geometry if it moved
INSERTFull snapshotFeature state after creation (enables "undo insert")

Storage saving

Before (PG mode)
Full snapshot

50 attributes stored for every UPDATE, even if only one changed. High volume, noisy preview.

Now (local mode)
Smart delta

Only the attributes that actually changed are stored. 10–50x volume reduction. The preview directly shows "what changed".

Delta format

The attributes_json field changes semantics depending on the operation:

DELETE: {"all_attributes": {"name": "Dupont", "age": 42, "city": "Paris"}}
UPDATE: {"changed_only": {"city": {"old": "Paris", "new": "Lyon"}}}
INSERT: {"all_attributes": {"name": "Martin", "age": 35, "city": "Lyon"}}
Robust serialization: Every QVariant type (QString, int, double, QDate, QDateTime, QByteArray, bool, lists, maps, NULL) is serialized to JSON reversibly. BLOBs are base64-encoded with a b64: prefix. NaN and Infinity are stored as null with a flag.

Identification system

The system uses three levels of identification to precisely locate each feature in each layer of each project.

LEVEL 1 project_fingerprint = project::C:/projects/site_A/my_project.qgz LEVEL 2 datasource_fingerprint = ogr::C:/data/vegetation.shp LEVEL 3 feature_identity_json = {"fid": 42, "pk_field": "gid", "pk_value": 42}

Layer fingerprint

The fingerprint is the core piece of the identification system. It distinguishes two layers that share the same name but come from different sources.

How it is computed

Format: provider::normalized_source

Layer typeFingerprint example
Shapefileogr::c:/data/plots/vegetation.shp
GeoPackageogr::c:/data/communes.gpkg|layername=buildings
PostGISpostgres::host=10.241.228.107 port=5432 dbname=MYDB schema=public table=cables
SpatiaLitespatialite::c:/data/local.sqlite|layername=roads
GeoJSONogr::c:/export/zones.geojson
Two layers named "vegetation" in different folders have different fingerprints. The layer display name is never used as an identifier: only the absolute source path matters.

Smart restore without actor multiplication

RecoverLand does not restore "by name" and does not create a new actor for every historical row. It restores against the current entity stream, grouped by entity_fingerprint, and writes only lightweight trace records for auditability.

Events of the same feature share one entity fingerprint Example: pk:id=42 DELETE evt 120 pk:id=42 UPDATE evt 154 pk:id=42 UPDATE evt 201 pk:id=42 Current live feature resolved once for restore Tracker suppressed during restore Avoids re-recording the restore as a fresh user edit Trace event is reference-only Uses restored_from_event_id, not a full duplicate payload
MechanismImplementation intentUser-facing effect
entity_fingerprintCanonical key like pk:id=42 or fid:7The same feature is followed across its event stream instead of being treated as unrelated rows
FID pre-resolution cacheBatch lookup before restoreFewer repeated scans, more predictable restore time
Tracker suppressionCapture is disabled while restore runsNo feedback loop, no accidental re-audit inflation
Trace eventReference to the original event via restored_from_event_idRestore remains auditable without cloning the full event history
Default search filtersTrace events excluded from standard totals and listingsThe user sees the business history first, not technical bookkeeping noise
Important nuance: RecoverLand keeps the original event history and may add a lightweight restore trace, but it does not multiply business entities. The feature identity stays anchored by the datasource fingerprint plus the entity fingerprint.

Identity strength

Not all formats offer the same reliability for identifying a feature. RecoverLand qualifies each format with an identity strength level defined in support_policy.py.

StrengthMeaningFormatsRestore risk
STRONGStable primary key, does not change between sessionsGeoPackage, SpatiaLite, PostgreSQL, Oracle, MS SQL Server, FlatGeobufMinimal
MEDIUMFID generally stable but can change after certain operationsShapefile, GeoJSONPossible if the file has been re-exported or repacked
WEAKNo primary key; FID = row numberCSV, delimited text, ExcelHigh: row number shifts as soon as a row is added or deleted
NONENo stable identifier; data is not persistedMemory layerRestore impossible (capture is informational only)

Schema drift

Between the time a change is recorded and the time a user tries to restore it, the layer structure may have changed. The schema_drift.py module detects these drifts before any restore.

What the module compares

Each event stores the field schema at capture time (field_schema_json). Before restore, this historical schema is compared against the current layer schema:

Drift typeImpactPlugin action
Field added since captureThe snapshot does not contain this fieldField is left at its default value
Field removed since captureThe snapshot contains a field that no longer existsValue is ignored with a warning
Field type changedType incompatibilityConversion attempted; refused if impossible
Field renamedOld name not foundTreated as deleted field + new field
Compatibility report: Before every restore, an explicit report lists compatible fields, missing fields, and incompatible fields. A restore is never silently partial.

Integrity and recovery

The integrity.py module verifies journal health at plugin startup and after any incident.

Startup checks

CheckAction on failure
PRAGMA integrity_checkUser notification if corruption detected; offer to rename the file and create a fresh journal
WAL file checkAutomatic checkpoint to consolidate pending writes
schema_version tableInterrupted migration detected; resume or alert
Recovery file (recoverland_pending.json)Re-integration of pending events lost during a previous crash
Zero silent loss: If events could not be written (crash, full disk), the plugin saves them to a recovery file and re-integrates them at the next startup. The user is always informed.

Asynchronous writes

Events are written to SQLite by a dedicated thread (WriteQueue). The UI thread never waits for the write to complete. The queue uses batches of 500 events per transaction to maximize throughput.

UI thread queue.Queue Writer thread SQLite (batch 500)

Retention and purge

The journal grows over time. The retention.py module provides four mechanisms to control its size:

By age
Age-based purge

Delete events older than a configurable threshold. The default retention policy is 365 days (up to 10 years).

By count
Max-events cap

Automatically delete the oldest events once the event count exceeds a configurable maximum (default: 1,000,000).

By session
Session purge

Delete all events from a specific edit session. Useful to clean up test sessions or mass imports.

Compaction
Async VACUUM

Reclaim disk space after a purge. VACUUM runs in a background thread so the UI stays responsive.

Typical size: A delta UPDATE event (2 changed attributes) takes 200–500 bytes. A DELETE with 20 attributes + point geometry takes 1–3 KB. 10,000 modifications represent 5–20 MB.

Health monitoring

RecoverLand continuously evaluates journal health and disk space. Alerts appear in the info bar at the top of the main dialog and can be clicked to open the maintenance panel.

Journal health thresholds

LevelSize triggerCount triggerAction
Healthy< 50 MB< 100,000 eventsNo message
Info50–200 MB100,000–500,000 eventsInformational notice
Warning200–500 MB500,000–1,000,000 eventsSuggestion to purge old events
Critical> 500 MB> 1,000,000 eventsPurge strongly recommended

Disk space thresholds

LevelFree spaceAction
Warning< 500 MB freeWarning displayed in the info bar
Critical< 100 MB freeTracking automatically disabled to prevent data loss
Auto-disable: When the disk reaches the critical threshold, RecoverLand stops capturing new events and notifies the user. Tracking must be re-enabled manually after freeing space.

Memory management

RecoverLand is designed to operate safely even on large datasets.

Edit buffer cap

The EditSessionBuffer holds feature snapshots in RAM during an edit session. Beyond 10,000 features or 200 MB of snapshot data, snapshots are flushed to a temporary SQLite staging table to avoid memory overflow. This is transparent to the user.

Paginated search

Search results are never fully loaded into memory. The SearchService uses paginated SQL queries (up to 500 events per page) to keep RAM usage flat regardless of journal size.

Write queue

The WriteQueue is bounded. If the writer thread cannot keep up (e.g., very intensive editing), back-pressure is applied to prevent unbounded memory growth.

Restore isolation

Restore operations process features in isolated batches. If one feature fails, the others are still processed. No single large transaction locks the database for a long time.

Rule of thumb: If editing a layer with > 100,000 features in one session, the buffer flush to staging will activate automatically. You may notice a brief pause; this is expected and protects memory.

Event lifecycle

QGIS edit Snapshot (RAM) Commit? Net effect WriteQueue SQLite

If the user rolls back instead of committing, the in-RAM snapshot is simply discarded. Nothing is written to the journal.

User identification

Every event carries a user_name resolved automatically by the user_identity.py module. Sources are tested in order:

Plugin config override Env var RECOVERLAND_USER OS login (os.getlogin) QGIS profile name "unknown"

The first non-empty source is used. No event is ever written with an empty user_name.

Project saving and journal relocation

If a QGIS project transitions from unsaved to saved, the journal is relocated from the QGIS profile directory to the .recoverland/ subfolder next to the .qgz file. Events captured before saving are migrated automatically.