Anomaly detection for connected-device telemetry

The scenario

Consider a fleet of connected devices reporting sensor readings and operational status. A product team wants to investigate unusual behaviour before it becomes a recurring support issue. The reference workflow would help prioritize a human review queue; it would not automatically declare a device faulty or control physical equipment.

Separate bad data from unusual behaviour

The ingestion service would validate units, schema versions, timestamps, sequence gaps, and the expected device mode. A missing measurement would remain missing. It would not become zero simply because the model expects a numeric input.

Feature windows would carry hardware revision, firmware version, and operating-state context. A device being installed or updated may behave differently from one in steady operation. Those states would need explicit treatment so normal maintenance does not become a stream of unhelpful alerts.

Compare a model with simple operating rules

The first baseline would use documented thresholds and rate-of-change checks. An Isolation Forest could then be evaluated as one candidate for finding less obvious unusual observations. Its score would not be presented as a calibrated probability that hardware has failed.

Training and evaluation periods would be separated, with attention to devices and firmware revisions that appear in both. MLflow could record the feature schema, dataset snapshot, model parameters, and reviewer labels. Evaluation would consider how many useful investigations the team can handle, rather than optimizing a score without an operational question.

Close the loop with the people investigating alerts

A review item would show the relevant time window, comparison context, data-quality flags, and model version. Reviewers could identify known maintenance, missing data, an interesting pattern, or a confirmed issue. Their findings would become an evaluation dataset rather than an automatic justification for the model.

A shadow rollout would compare candidate alerts with the rule baseline before enabling a production queue. Later checks would monitor feature availability, score distributions, and alert volume by device revision. The team would keep a way to disable model-based prioritization while retaining telemetry intake.

What to validate

• Missing or malformed readings follow a data-quality path.
• Evaluation does not use information from after the scored window.
• An alert explains which data and model version produced it.
• Model-based prioritization can be disabled independently of ingestion.

The tradeoff

Rare hardware faults provide little labelled evidence, and a changing device population can change what looks unusual. A maintained rule baseline and review process may deliver more value than a more elaborate model without usable feedback.