UniFi Protect Smart Detections in Home Assistant

A smart detection is the moment a camera stops being a recorder. UniFi Protect decides that a shape is a person, a vehicle or a package, and Home Assistant can act on that decision the moment the integration's event entity fires.

The catch is that two pipelines carry those decisions, and they do not carry the same things. Home Assistant's UniFi Protect integration turns detections into event entities, while Ubiquiti's Alarm Manager sends chosen events out as webhooks.

What's more, the two pipelines read from different places, since the integration reads the public API while Alarm Manager sits inside Protect itself. Ubiquiti's public Protect API publishes the object type and whether it came from a zone, a line or a loitering zone. It keeps recognized faces, plate text on the smart detection event and line names to itself.

Accordingly, this guide stays on the detections themselves: which ones reach Home Assistant, what each event carries, and where the public API stops. The Protect integration guide explains the connection modes and the entities more generally, and the automations guide explains the webhook plumbing on the network side.

First, though, what the cameras can actually see.

Ubiquiti's AI detections article is the table to keep open. It says all of that edge AI processing runs on-premises, either on the camera or on an AI Port or AI Key, so nothing leaves the house to be classified.

Person and vehicle detection are the baseline, and the G6, G5 and G4 generations have them built in. Vehicle type and color, however, are native only on the G6 and AI series, which is why a G6 Turret reports a white truck where a G5 reports a vehicle.

Audio recognition, such as speech and glass break, needs a microphone, so it applies to microphone-equipped models only. Line cross counting, meanwhile, is native on the G6 and AI series and needs an AI Port everywhere else.

The accessories change the picture in two different ways:

• AI Port. Ubiquiti's AI Port FAQ says it adds face and plate recognition to G4 and G5 cameras, brings G3 and ONVIF cameras into smart detection entirely, and does not work with AI series cameras. Capacity depends on resolution, from one 4K ONVIF camera up to five HD Protect cameras, and a Port takes one kind or the other, not both.
• AI Key. The AI Key FAQ describes an appliance that takes a camera's existing smart detections and runs them through further models: natural language search, speech-to-text and AI-enhanced alarms. The capability table adds clothing type, color and accessories. It processes up to 1,000 detections an hour through a queue of 200, and a full queue can delay results by up to 15 minutes. Face and plate recognition for G4 and G5 cameras run through that same queue, so Ubiquiti calls them asynchronous on an AI Key and real-time on an AI Port.

Keep in mind that the AI Key only ever sees what a camera already classified. Its FAQ says any event the camera did not recognize as a smart detection is never processed.

Finally, zones decide what counts. Ubiquiti's camera zones article describes smart detection zones, exclusion zones that suppress detections inside a larger zone, and a zone sensitivity setting whose higher values make a camera more willing to log subtle motion. Every detection Home Assistant receives has already been through that filter, which is the right place to fix a false positive.

The integration, per its documentation, creates two kinds of entity for detections. Binary sensors report a state: one detected object sensor per smart detection type the camera supports, plus a combined sensor for any object at all.

Events report that something happened. Each camera exposes a motion event, a smart detection event, a sound detection event where the camera has a microphone, and a package event where the camera supports it.

The smart detection event is the workhorse, and these are its attributes:

• event_type names the object: person, vehicle, animal, package, license_plate, face, car or pet, with new types surfacing automatically as Protect adds them.
• event_source names the raw Protect source: smartDetectZone, smartDetectLine or smartDetectLoiterZone.
• smart_detect_types lists every object type seen during the event.
• event_id identifies the event, and it is the key to the thumbnail and clip.

The documentation says the event fires once for each object and source pair across the lifecycle of a detection, so a type Protect only reports partway through still surfaces. That design has a consequence covered in the guardrails below.

Package is, however, the exception to the sensor rule. Protect reports a package as a single, already-ended detection, which a sustained binary sensor cannot represent, so it exists only as an event.

Sound detection follows the same pattern, in fact, with its own vocabulary. The documented types are smoke, co, siren, baby_cry, speaking, bark, car_alarm, car_horn and glass_break.

Be aware that the connection mode decides how much of this you get. In API-key-only mode, the documentation lists motion, smart detection, sound detection and package events, while the doorbell ring, NFC, fingerprint and vehicle events need full access.

Full access also unlocks the media source and the proxy views. A notification can carry a thumbnail from /api/unifiprotect/thumbnail/ followed by the config entry ID and the event_id, and Home Assistant signs those URLs automatically when they are used in an automation.

Vehicles are the one detection the integration deliberately delays. Its documentation explains that plate data and the best thumbnail arrive after the detection itself, so firing immediately would deliver the event without them.

Here is how the documented sequence runs:

1. The camera starts sending thumbnails for a detected vehicle.
2. A three-second timer starts, and it resets each time another thumbnail arrives.
3. When the timer expires, the event fires with the best thumbnail, chosen by plate detection first, then confidence, then recency.
4. If a new vehicle appears while a timer is pending, the old event fires immediately and a new timer starts.
5. In rare cases, data that arrives after the event has fired produces a second event with the new information.

The event that fires therefore carries license_plate when recognition succeeded, plus a confidence score from 0 to 100 and the timestamp of the best frame, both of which the documentation marks optional. A nested set of Protect attributes adds the vehicle type, color and zone list.

Of course, recognition has to be possible first. On the capability table above, that means a G6 or AI series camera, or a G4 or G5 camera with an AI Port or AI Key behind it. Ubiquiti's dedicated AI LPR camera exists for exactly this job, with an optional LPR-only mode that gives plates every camera resource.

Its setup article gives the placement numbers: a mounting height of no more than 5 m, a capture distance of 12 m, and a camera angle under 25 degrees. Fast traffic tightens the angle to under 15 degrees.

Note that the 30 degrees in the camera placement guide is the figure Ubiquiti's own note allows for slow or stopped traffic. The article adds that the camera's glare-cut filter leaves it seeing little beyond plates and headlights.

Now the warning that belongs beside any plate-driven automation. Home Assistant's documentation says recognition can be triggered by images or printed material showing a plate. It tells you to add further verification or time restrictions before a plate opens a gate or a garage.

Ubiquiti has its own answer to that problem, and it keeps Home Assistant out of the loop entirely. License Plate Unlock in UniFi Access pairs an AI camera directly with an Access hub and, in its vehicle-and-plate mode, requires the vehicle to sit at least 90 percent inside the detection zone. A cooldown then suppresses repeated detections of the same plate, and the UniFi Access guide explains what Home Assistant can see of the result.

Moreover, Ubiquiti's Gate Hub article says the feature will not work unless entrapment protection devices are wired to the gate operator's UL 325 terminals.

Home Assistant is plain about the gap. Its documentation says, in fact, that the smart detection event reports which type was detected, not the richer metadata behind it. The public API, it explains, does not currently expose the recognized plate text, vehicle color and type, or confidence on that event.

Line crossing has the same shape. The event can tell a line crossing apart from a zone or loitering detection, but it carries no line name, line ID or direction.

Faces are the largest omission. A camera that recognizes a household member reports a face type and no name, so the integration cannot tell you who arrived.

Ubiquiti's Alarm Manager is where that information lives. Its Protect triggers sit in five categories, and the first below is the one the integration cannot see into:

• ID Recognition. Known and unknown faces, license plates, and persons of interest.
• AI/Object Detection. Person, vehicle, package and animal.
• Activity. Motion, sound, line crossing, NFC scan and doorbell ring.
• System and Sensors. Two further categories: device issues and setting changes, plus leak, smoke and CO from Sense devices.

An alarm pairs one trigger with a scope and an action, and the webhook action sends a GET or POST to any URL. Ubiquiti's webhook article shows the POST body: the alarm's name, its conditions, the triggering device, and a timestamp.

That body is the reason to name alarms carefully. A Home Assistant webhook trigger exposes a JSON body as trigger.json, provided the request carries a JSON content type. An alarm called "Front door known face" can then be matched on its name, or given a webhook ID of its own.

In addition, the AI Key extends the trigger list. Its FAQ describes an "AI Key Advanced" trigger that fires on a natural language query above a chosen confidence, up to five queries per key. The alarm then has the same notification and webhook actions as any other.

Remember that Alarm Manager also acts without Home Assistant. Protect alarms can turn on floodlights, play a tone through an AI Horn Speaker, or move a PTZ camera to a preset. They can also unlock a door through UniFi Access, which is Ubiquiti's own path for recognition-driven doors.

The double fire is documented as a behavior change. An automation that does not filter event_source can run more than once for the same event_id when Protect emits overlapping sources, and one that should react only to line crossings must check for smartDetectLine.

Startup is the other trap. Every event-entity example in the integration's documentation guards against an old state of unavailable, and most also reject a restored state. An entity passing through those states during a Protect restart looks like a fresh event otherwise.

A device dropping offline causes the same repeat. The documentation notes that when a device goes unavailable and returns, an event can be processed again, and it raises that specifically for doors.

Then there are the credentials that are weaker than they look:

• Plates can be spoofed with a printed image, so add a time window or a second factor before any gate opens.
• NFC cards scan by serial number with third-party cards, which the documentation calls easy to duplicate, and unknown cards trigger the scan event too.
• Webhook IDs carry no authentication beyond the ID itself. Home Assistant's own guidance is to keep destructive actions such as locks and garage doors off webhooks, leave the local-only option on, and never reuse an ID from a public source.

Ubiquiti adds its own best practices for the alarm side. Scope alarms to specific devices rather than all of them, and combine schedules with triggers to cut false positives. Enable Ignore Repeated Actions where one event would otherwise spam, and test a webhook against a request bin before pointing it at anything real.

In addition, recognition has to be switched on: Ubiquiti's License Plate Unlock troubleshooting points at Protect's Settings, Intelligence and Legal Permissions page, where smart detection and plate recognition are enabled.

Finally, mind the AI Key queue. A full queue can hold results for up to 15 minutes and discards whatever arrives beyond its 200 events. An AI Key alarm therefore suits a notification and does not suit a light that should come on as someone walks up.

These five stay inside what the two pipelines actually publish, and each names its trigger:

1. Package on the step. Trigger on the package event with the usual restart guard, wait two seconds, then send a notification whose image is the proxy thumbnail for that event_id. Full access required.
2. Person after dark. Trigger on the smart detection event with a person type and the camera's Is Dark sensor on, then turn on the Protect floodlight's light entity. Alarm Manager can do the same without Home Assistant.
3. Unknown vehicle in the driveway. Trigger on the vehicle event, and condition on the plate attribute being absent or outside your list, then notify with the plate text when there is one.
4. Side gate line crossed. Trigger on the smart detection event with event_source equal to smartDetectLine on the one camera whose line matters, since the event cannot name the line.
5. Known face at the front door. An Alarm Manager ID Recognition alarm scoped to the doorbell camera, posting to a Home Assistant webhook that announces the arrival. A notification, naturally, rather than an unlock.

Start with the package one. After all, it exercises the event, the thumbnail proxy and the startup condition in a single automation, and nothing happens to the house if it misfires.