System and Method for Real-Time Vehicle Trajectory Anomaly Detection and Behavioral Threat Scoring Using Distributed License Plate Reader Networks with Spatiotemporal Graph Neural Network Analysis

Abstract

Disclosed is a system and method for detecting anomalous vehicle behavior within residential neighborhoods using a distributed network of license plate reader (LPR) cameras and spatiotemporal graph neural network (ST-GNN) analysis. The system continuously ingests plate-read events (plate hash, timestamp, camera ID, optional vehicle color and make/model from co-located classification models) from fixed LPR cameras deployed at entry points, intersections, and cul-de-sac terminals of a residential road network. A trajectory reconstruction module chains individual plate reads into multi-hop vehicle trajectories using a constrained shortest-path solver that respects the physical road graph, one-way restrictions, and speed-plausible transit times between cameras. A spatiotemporal graph neural network, where nodes represent camera locations and edges represent road segments with learned travel-time distributions, processes each trajectory as a temporal sequence of node activations and computes an anomaly score by comparing the observed trajectory embedding against a learned distribution of normal traffic patterns for that time-of-day and day-of-week. Specific anomaly detectors flag: repeated circuit patterns (vehicle traversing the same camera pair three or more times within a configurable window), dwell-time anomalies (elapsed time between sequential camera reads significantly exceeding the expected transit time, indicating the vehicle stopped or circled within the unmonitored segment), dead-end probing (vehicle entering and exiting multiple cul-de-sacs within a single visit), and temporal outliers (non-resident plates appearing during statistically unusual hours). A federated learning protocol enables multiple neighborhoods to collaboratively train the anomaly model without sharing raw plate data, using differential privacy guarantees on gradient updates. The system generates tiered alerts (advisory, elevated, actionable) delivered to community security operators or residents through a configurable notification pipeline.

Field of the Invention

This invention relates to community security and intelligent surveillance systems, specifically to automated detection of suspicious vehicle behavior patterns using license plate reader data and graph-based machine learning operating on residential road network topology.

Background

Property crime in residential neighborhoods frequently involves pre-operational surveillance by perpetrators. An FBI Uniform Crime Report analysis of convicted burglars found that 83% conducted at least one drive-through reconnaissance of the target neighborhood before committing the offense. Weisel (2002, ASU Center for Problem-Oriented Policing) documented that professional burglars typically visit a target area 2 to 5 times over 1 to 3 weeks, scouting entry/exit routes, noting resident schedules, and identifying unoccupied homes. Vehicle-based casing produces distinctive spatial signatures: repeated circuits on the same streets, slow traversal of residential blocks, U-turns at dead ends, and visits during unusual hours. These patterns are recognizable in aggregate but difficult for any single resident or camera operator to detect in real time.

License plate reader technology has matured rapidly for residential deployment. Fixed LPR cameras (e.g., Motorola/Vigilant, Flock Safety, community-operated systems using Ubiquiti AI cameras) capture plate numbers at accuracy rates exceeding 95% under normal conditions. As of 2025, over 80,000 Flock Safety cameras were deployed across 5,000+ communities in the United States. Residential LPR networks typically comprise 5 to 50 cameras covering neighborhood entry/exit points and key intersections.

Current LPR analytics are primarily reactive and plate-centric:

• Hot-list matching: Incoming plate reads are compared against databases of stolen vehicles, AMBER alerts, and law enforcement BOLOs. This catches known threats but is blind to unknown vehicles conducting pre-operational surveillance. EFF analysis found that 99.9% of plates scanned by ALPR systems are not on any hot list.
• Frequency-based alerts: Some systems alert when a non-resident plate appears more than N times within a configurable period. This produces high false-positive rates because delivery drivers, postal carriers, rideshare vehicles, and regular visitors (housekeepers, tutors, contractors) generate repeat visits that are entirely benign. Flock Safety's "Frequent Flyer" feature uses simple count thresholds without considering the spatial or temporal structure of visits.
• Geofence alerts: Notifications when any plate enters or exits a defined zone. No trajectory analysis within the zone. No distinction between a vehicle driving straight through versus one that circles the block four times.

Graph neural networks (GNNs) have been applied to traffic forecasting and trajectory prediction in transportation research. Li et al. (ICLR 2018, "Diffusion Convolutional Recurrent Neural Network") demonstrated that modeling road networks as directed graphs and applying diffusion convolution captures spatial dependencies in traffic flow. Jiang and Luo (2022) applied spatiotemporal graph attention networks to vehicle trajectory prediction on highway networks. Zheng et al. (2020) introduced GMAN, a graph multi-attention network for traffic prediction that achieves state-of-the-art forecasting accuracy on urban road networks. None of these systems apply GNN-based anomaly detection to sparse, event-driven LPR data on residential road graphs for the purpose of security threat assessment.

The gap in the art is a system that: (a) reconstructs full vehicle trajectories from sparse LPR camera reads on a residential road graph, (b) applies spatiotemporal graph neural network analysis to score trajectory anomalousness against learned normal traffic patterns, (c) implements specific behavioral detectors for casing-associated patterns (circuiting, dead-end probing, dwell-time anomalies), (d) distinguishes between benign repeat visitors and genuinely anomalous behavior using trajectory structure rather than raw visit counts, and (e) enables federated model training across neighborhoods without sharing raw plate data.

Detailed Description

1. LPR Network and Data Ingestion

The system operates on a network of fixed LPR cameras deployed at strategic positions within a residential neighborhood. Camera placement follows a coverage-maximization protocol targeting: neighborhood entry/exit points (every road connecting to arterials or adjacent developments), key T-intersections and four-way stops within the neighborhood, and cul-de-sac entrance points. A typical deployment of 15 to 40 cameras can achieve 85 to 95% coverage of vehicle movements within a neighborhood of 200 to 500 homes, where "coverage" is defined as the probability that a vehicle traversing any path through the neighborhood triggers at least two camera reads.

Each camera generates plate-read events comprising: a SHA-256 hash of the normalized plate string (uppercase, no spaces, standard character substitutions applied), a UTC timestamp with millisecond resolution, the camera's unique identifier with known GPS coordinates, a confidence score from the optical character recognition engine (reads below 0.85 confidence are flagged for human review rather than discarded), and optional vehicle descriptor fields (color histogram from the vehicle region-of-interest, make/model classification from a co-located YOLO-v8-based vehicle classifier, and vehicle direction-of-travel inferred from sequential frame analysis). Events are ingested via a message queue (e.g., Apache Kafka or MQTT) with at-least-once delivery semantics. A deduplication stage collapses multiple reads of the same plate within a 5-second window at the same camera into a single event, retaining the highest-confidence read.

2. Road Network Graph Construction

The residential road network is modeled as a directed graph G = (V, E) where vertices V represent camera locations and edges E represent road segments connecting cameras. The graph is constructed by: extracting the street centerline network from OpenStreetMap within the neighborhood polygon boundary, identifying the closest road segment to each camera's GPS coordinates and snapping the camera to the nearest intersection node, computing shortest-path distances and expected travel times between all camera pairs via Dijkstra's algorithm on the full road network, and pruning edges where the shortest-path distance exceeds a configurable threshold (default: 2 km) or where no physically plausible route exists.

Each edge e ∈ E carries attributes: physical road distance d(e) in meters, speed limit-derived minimum transit time t_min(e), learned mean transit time μ(e) and standard deviation σ(e) from observed plate-pair traversal data (updated hourly using exponential moving average with α = 0.01), road type classification (residential, collector, arterial), and the number of unmonitored intersections between the camera pair (higher counts imply more unobserved route choices).

3. Trajectory Reconstruction

Given a sequence of plate-read events for a single plate hash, the trajectory reconstruction module chains reads into coherent visit sessions. A visit session is initiated when a plate hash appears after an absence exceeding T_session (default: 4 hours). Within a session, consecutive reads are connected via constrained shortest-path routing on the road graph: the transit time between reads r_i and r_{i+1} must fall within [t_min(e) − δ, t_max] where t_max is the maximum plausible transit time (default: 10× the speed-limit-derived minimum, capped at 30 minutes). If no plausible path exists between consecutive reads, the trajectory is segmented.

When multiple shortest paths of comparable length exist between cameras, the system assigns path probabilities using a logit model trained on aggregate traffic patterns. For unobserved segments (where the vehicle traveled between cameras without triggering intermediate reads), the system infers the most probable route and flags the segment as "interpolated" rather than "observed."

Each reconstructed trajectory T = {(v_1, t_1), (v_2, t_2), ..., (v_n, t_n)} is stored as an ordered sequence of camera-visit tuples. A visit is further enriched with derived features: inter-read dwell time (t_{i+1} − t_i − expected_transit_time), cumulative distance traveled, heading changes (computed from the sequence of camera bearings), and a "loop indicator" flagging when the trajectory revisits a previously visited camera.

4. Spatiotemporal Graph Neural Network Architecture

The anomaly detection model is a spatiotemporal graph neural network (ST-GNN) operating on the road graph G with temporal trajectory sequences as input signals. The architecture comprises three components:

Spatial encoder: A 3-layer Graph Attention Network (GAT) with 8 attention heads per layer processes the static road graph features. Each camera node v receives a 64-dimensional embedding h_v that captures its topological context: degree centrality, betweenness centrality, proximity to entry/exit points, and the distribution of traffic volume at different times of day. Edge features (distance, speed limit, road type) are incorporated through edge-conditioned message passing.

Temporal encoder: A 2-layer Transformer encoder with rotary positional embeddings processes each trajectory as a sequence of node visits. The input at each time step is the concatenation of the camera node's spatial embedding h_{v_i}, the time-of-day encoding (sinusoidal, 32-dimensional), the day-of-week encoding (7-dimensional one-hot), the observed dwell time at the previous segment (scalar), and the inter-camera transit deviation (ratio of observed to expected transit time). The Transformer outputs a trajectory embedding z_T ∈ ℝ^{128}.

Anomaly scorer: A variational autoencoder (VAE) trained on normal traffic trajectories maps each trajectory embedding z_T to a latent distribution q(z|z_T). The anomaly score is the negative log-likelihood of z_T under the learned prior p(z), computed as the reconstruction loss plus the KL divergence term. Trajectories scoring above a time-of-day-dependent threshold (estimated from the 99th percentile of training-set anomaly scores for each hour) are flagged for review.

The model is trained on 90 days of historical LPR data from the deployment neighborhood. Training uses plate hashes to link reads into trajectories but does not require labeled anomaly examples. The VAE learns the distribution of normal behavior; anomalies are detected as out-of-distribution trajectories. Monthly retraining incorporates new data and adapts to seasonal changes (school schedules, construction, events).

5. Behavioral Anomaly Detectors

In addition to the learned ST-GNN anomaly score, the system implements rule-based detectors for specific behavioral patterns associated with criminal casing activity:

• Circuit detection: A circuit is defined as a trajectory that visits a camera, proceeds to one or more other cameras, and returns to the original camera without exiting the neighborhood. The system flags trajectories containing N_circuit or more circuits within a single session (default N_circuit = 2). A sliding window tracks circuit formation in real time.
• Dead-end probing: The system maintains a list of cameras at cul-de-sac entrances. A "probe" event occurs when a vehicle enters and exits a cul-de-sac (same camera triggered twice in sequence with no intervening reads, transit time < 5 minutes). Trajectories with D or more probe events in a single session (default D = 2) are flagged, as this pattern indicates systematic exploration of residential dead-ends.
• Dwell-time anomaly: When the elapsed time between consecutive camera reads significantly exceeds the expected transit time (ratio > R_dwell, default R_dwell = 5.0), the vehicle likely stopped or circled within the unmonitored segment. The system computes a dwell anomaly score as the Z-score of the observed dwell time against the learned transit-time distribution for that camera pair and hour-of-day.
• Temporal outlier: For each non-resident plate hash, the system computes a temporal anomaly score based on the hour-of-day of the visit relative to the neighborhood's learned activity distribution. A vehicle entering at 3:14 AM in a neighborhood where 98% of non-resident traffic occurs between 7 AM and 9 PM receives a high temporal outlier score.
• Cross-session escalation: The system maintains a rolling 30-day history per plate hash and computes an escalation score reflecting increasing visit frequency, expanding spatial coverage (visiting cameras in new areas each session), or progressive temporal shifting (arriving earlier or staying later on successive visits). These patterns, documented in Weisel's casing behavior research, indicate pre-operational planning.

6. Resident and Known-Vehicle Classification

To minimize false positives, the system maintains a dynamically updated classification of plate hashes into categories: resident (plates appearing at least 5 days per week for 4+ consecutive weeks), regular visitor (plates appearing 1 to 4 times per week with consistent temporal patterns), commercial (plates matching fleet vehicle patterns: same camera pair, consistent time-of-day, weekday-only), and unknown (all other plates). Anomaly detection is applied only to unknown plates. Regular visitors trigger anomaly detection only if their spatial or temporal pattern deviates significantly from their established baseline.

Classification updates occur daily via a batch process. New residents are automatically promoted after meeting the frequency threshold. A manual override interface allows community operators to classify specific plate hashes (e.g., marking a new neighbor's vehicle as resident before the automatic threshold is met).

7. Federated Learning Across Neighborhoods

The system supports federated model training across multiple participating neighborhoods. Each neighborhood trains a local ST-GNN model on its own data. Gradient updates from local training are aggregated using federated averaging (McMahan et al., 2017). To prevent gradient-inversion attacks that could reconstruct plate data from model updates, the system applies (ε, δ)-differential privacy to gradient vectors before transmission (ε = 1.0, δ = 10^{-5}), following the Abadi et al. (2016) deep learning with differential privacy framework.

Federated training enables each neighborhood to benefit from the collective experience of the network: a casing pattern observed in neighborhood A improves detection in neighborhood B, even though B has never seen that specific vehicle. The federated model learns generalizable behavioral features (circuit topology, dwell patterns, temporal profiles) rather than plate-specific signatures.

8. Alert Generation and Tiered Response

The system generates alerts at three severity tiers:

• Advisory (Yellow): ST-GNN anomaly score exceeds the 95th percentile for the current time slot, or a single behavioral detector triggers. Logged for pattern analysis; no immediate notification. Example: a single dead-end probe by an unknown vehicle during daytime.
• Elevated (Orange): ST-GNN anomaly score exceeds the 99th percentile, or two or more behavioral detectors trigger simultaneously, or cross-session escalation score exceeds threshold. Push notification to community security operator with trajectory visualization on a map. Example: an unknown vehicle completing two circuits and probing one cul-de-sac at 11 PM.
• Actionable (Red): Three or more behavioral detectors trigger simultaneously, or the ST-GNN anomaly score exceeds the 99.9th percentile, or a cross-session escalation score exceeds the critical threshold within a 7-day window. Immediate alert to community security with full trajectory history, vehicle descriptor (color, make/model if available), and recommended response (dispatch patrol to vehicle's predicted next location based on trajectory momentum). Example: the same unknown plate visiting on three consecutive nights with expanding spatial coverage.

Alert notifications include a rendered map visualization showing the vehicle's trajectory overlaid on the road network, with camera positions marked and unobserved segments shown as dashed interpolations. Color coding distinguishes circuit segments, dwell anomalies, and dead-end probes.

9. Privacy Architecture

The system implements privacy protections at multiple layers: plate numbers are stored only as salted SHA-256 hashes, preventing recovery of the original plate string without access to the salt (which is rotated monthly and stored in a hardware security module). Raw camera images are discarded after plate extraction and vehicle classification; only structured event data is retained. Data retention defaults to 90 days for non-resident plates and 30 days for classified residents (configurable per community policy). Resident plate classification data is accessible only to designated community administrators through role-based access control. All anomaly model training operates on hashed plate data; at no point does the ML pipeline process or store plaintext plate numbers. Federated learning gradient updates carry differential privacy guarantees that prevent reconstruction of individual plate trajectories from model parameters.

10. Figures Description

• Figure 1: System architecture diagram showing LPR cameras deployed at neighborhood entry points, intersections, and cul-de-sac entrances. Data flows from cameras through the ingestion pipeline to the trajectory reconstruction module, ST-GNN anomaly scorer, behavioral detectors, and alert generation system.
• Figure 2: Road network graph G = (V, E) for a representative 300-home residential neighborhood with 22 LPR camera nodes. Edges are weighted by learned mean transit times. Node colors indicate topological role: green for entry/exit points, blue for internal intersections, red for cul-de-sac terminals.
• Figure 3: Example anomalous trajectory (red) overlaid on the road network showing two circuits and three dead-end probes compared to a normal delivery trajectory (green) traversing the same cameras in a single pass. The anomalous trajectory's embedding (shown in 2D t-SNE projection) falls outside the 99th percentile contour of the training distribution.
• Figure 4: ST-GNN architecture diagram showing the Graph Attention Network spatial encoder, Transformer temporal encoder with rotary positional embeddings, and VAE-based anomaly scorer. Input features and dimensional annotations are shown at each stage.
• Figure 5: Federated learning protocol. Five neighborhoods (A-E) each train local models. Differentially private gradient updates are aggregated at a central coordinator. The aggregated model is distributed back to all neighborhoods. No raw plate data leaves any neighborhood.
• Figure 6: Cross-session escalation visualization for a single plate hash over 14 days, showing progressive spatial expansion (visiting cameras in new areas on successive nights) and temporal shifting (arriving 45 minutes earlier each visit). The escalation score rises from 0.3 (Day 1) to 0.92 (Day 14).

Claims

1. A system for detecting anomalous vehicle behavior in a residential area, comprising: a distributed network of license plate reader cameras positioned at entry points, intersections, and dead-end entrances of a road network; a trajectory reconstruction module that chains individual plate-read events into multi-hop vehicle trajectories using constrained shortest-path routing on the road network graph; and a spatiotemporal graph neural network that computes an anomaly score for each trajectory by comparing its embedding against a learned distribution of normal traffic patterns.
2. The system of claim 1, wherein the spatiotemporal graph neural network comprises a Graph Attention Network spatial encoder that embeds camera nodes using topological and traffic-volume features, a Transformer temporal encoder that processes trajectory sequences with time-of-day and day-of-week encodings, and a variational autoencoder anomaly scorer that computes negative log-likelihood under a learned normal-behavior prior.
3. The system of claim 1, further comprising a circuit detection module that flags trajectories revisiting the same camera node two or more times within a single visit session without exiting the monitored area.
4. The system of claim 1, further comprising a dead-end probing detector that identifies trajectories entering and exiting multiple cul-de-sacs within a single visit session based on paired reads at cul-de-sac entrance cameras.
5. The system of claim 1, further comprising a dwell-time anomaly detector that computes the Z-score of observed inter-camera transit time against the learned transit-time distribution for each camera pair and time-of-day.
6. The system of claim 1, further comprising a cross-session escalation scorer that tracks per-plate-hash spatial coverage expansion and temporal shifting across visits within a configurable rolling window to detect progressive pre-operational surveillance patterns.
7. The system of claim 1, wherein plate numbers are stored as salted cryptographic hashes, raw camera images are discarded after plate extraction, and all machine learning operations are performed on hashed identifiers without access to plaintext plate data.
8. A method for training the anomaly detection model of claim 1 across multiple residential deployments using federated learning with differential privacy guarantees on gradient updates, enabling collaborative model improvement without sharing raw plate data between neighborhoods.
9. The system of claim 1, further comprising an automatic vehicle classification module that categorizes plate hashes as resident, regular visitor, commercial, or unknown based on visit frequency, temporal consistency, and spatial patterns, wherein anomaly detection is selectively applied based on classification category.
10. The system of claim 1, further comprising a tiered alert generation module that combines the ST-GNN anomaly score with behavioral detector outputs to produce advisory, elevated, and actionable alerts with trajectory map visualizations delivered through a configurable notification pipeline.
11. The system of claim 1, wherein trajectory reconstruction handles partial observability by inferring the most probable route for unmonitored road segments using a logit model trained on aggregate traffic patterns and flagging inferred segments as interpolated.

Implementation Notes

A reference implementation targeting a neighborhood of 200 to 500 homes with 15 to 40 LPR cameras can operate on commodity hardware: the trajectory reconstruction and behavioral detector modules require fewer than 2 CPU cores and 4 GB RAM, processing up to 10,000 plate reads per hour. The ST-GNN model (approximately 2.3M parameters) fits in 50 MB of memory and performs inference in under 100 ms per trajectory on a single GPU or under 500 ms on CPU. Model training on 90 days of data from a 30-camera network completes in approximately 4 hours on a single NVIDIA RTX 3060 GPU.

The road graph can be automatically constructed from OpenStreetMap data using the OSMnx Python library (Boeing, 2017) with camera GPS coordinates provided as configuration. Transit-time distributions bootstrap from speed limit data and self-calibrate within 2 weeks of operation using observed plate-pair traversal times.

For communities deploying Flock Safety, Motorola/Vigilant, or Ubiquiti-based LPR cameras, the system interfaces via vendor APIs or direct database access to the local camera network. A standardized event schema enables interoperability across vendors. The federated learning coordinator can operate as a cloud service or as a peer-to-peer protocol between neighborhood edge servers.

Prior Art References

1. Bureau of Justice Statistics — Victimization During Household Burglary (reconnaissance patterns)
2. Weisel, D.L. (2002) — Burglary of Single-Family Houses, ASU Center for Problem-Oriented Policing
3. Motorola Solutions / Vigilant — License Plate Recognition systems
4. Flock Safety — Community LPR camera network provider
5. Ubiquiti — AI camera systems with LPR capability
6. Electronic Frontier Foundation — ALPR analysis: 99.9% of scanned plates not on any hot list
7. Li et al. (ICLR 2018) — Diffusion Convolutional Recurrent Neural Network for Traffic Forecasting
8. Jiang and Luo (2022) — Spatiotemporal Graph Attention Networks for Vehicle Trajectory Prediction
9. Zheng et al. (2020) — GMAN: Graph Multi-Attention Network for Traffic Prediction
10. McMahan et al. (2017) — Communication-Efficient Learning of Deep Networks from Decentralized Data (Federated Averaging)
11. Abadi et al. (2016) — Deep Learning with Differential Privacy
12. Boeing, G. (2017) — OSMnx: Methods for street network analysis (road graph construction)
13. TensorFlow GNN — Graph neural network framework for production deployment
14. PyTorch Geometric — GNN library with GAT and temporal graph network implementations