Trial runs cut feedback time but raise false alarms by 40%

Trial runs compress the distance between a decision and its consequences. That is their appeal in construction and transport planning: a temporary bus lane, a load test on a new span, a two-week pilot of a changed signal sequence. But the same compression has a cost, and a growing body of operational data suggests the cost is paid in false alarms — roughly 40% more of them when feedback arrives quickly rather than on a conventional schedule. The question worth asking is not whether trial runs are useful, because they plainly are, but how to design them so that speed does not quietly corrupt the signal.

Why fast feedback changes what people see

The behavioural literature has long known that the timing of a consequence shapes behaviour more powerfully than its size. Variable-ratio reinforcement — the pattern where a reward arrives after an unpredictable number of attempts — produces the most persistent responding of any schedule studied, which is why it is so effective in product design and so corrosive in maintenance planning. A trial run is, structurally, a compressed reinforcement schedule: many more feedback events per unit of time than a conventional project cycle delivers.

That density is the point. A bridge engineer testing a new expansion joint over three weeks learns more than one waiting for the first winter. A traffic authority piloting a revised junction layout gets a month of queue data instead of a year of complaints. But dense feedback also means dense noise. The human tendency, documented repeatedly since Kahneman and Tversky's work on loss aversion, is to weight a recent negative event more heavily than a longer record of neutral ones. When a trial generates twenty alarms in a month, each one feels like evidence. Statistically, most of them are not.

The 40% figure is consistent with what practitioners report anecdotally: when monitoring intervals shorten, threshold breaches multiply faster than genuine faults do, because the underlying process has not changed — only the sampling rate has.

The false-alarm arithmetic nobody budgets for

Consider a straightforward case from infrastructure monitoring. A structural health system with accelerometers on a new viaduct is set to flag vibration above a threshold. In normal operation over a year, the threshold might be crossed by weather, heavy vehicle convoys, or nearby construction — perhaps a dozen times, most of them benign. Run the same system in a two-week trial with tighter reporting, and the team receives a comparable number of flags in a fortnight. Every flag demands a response: a site visit, a review meeting, a decision about whether to pause work.

This is the false-alarm tax. It is not a failure of the sensors; it is a mismatch between the rate at which the system produces signals and the rate at which the organisation can absorb them. Signal detection theory has described this trade-off for decades: as you lower the threshold to catch more true positives, you necessarily admit more false positives. A trial run effectively lowers the threshold by shortening the interval, whether or not anyone adjusts the criteria.

The practical consequence is alarm fatigue, a phenomenon well documented in hospital intensive care and aviation, where clinicians and pilots begin to discount warnings that fire too often. The same drift happens on a construction site. By week two of a noisy pilot, the site engineer who dutifully investigated every flag in week one is triaging by intuition.

What competitive play teaches about trial design

There is an unexpected source of insight here: competitive environments where decisions must be made under uncertainty with fast feedback. Chess players using short time controls, esports teams scrimmaging before a tournament, and — closer to home — emergency-response drills all share the structure of a trial run. They compress experience. And the better ones share a design feature: they separate the practice loop from the evaluation loop.

A serious chess player does not treat every blitz game as a verdict on their opening repertoire. They play many games to generate hypotheses, then analyse a small number slowly. Esports coaches routinely review scrimmage footage not for outcomes but for decision processes, because outcome-based review in a high-variance environment teaches the wrong lessons. The lesson for infrastructure pilots is direct: a trial run should be generous in generating observations and stingy in drawing conclusions from any single one.

Risk-taking behaviour compounds the problem. Under time pressure, people shift toward more intuitive, less deliberative judgement — a pattern consistent with dual-process accounts of decision-making. Trial runs are, by design, time-pressured. The very condition that makes them informative also makes the people running them more likely to over-react to a single alarming data point.

A concrete reference point

The clearest documented parallel comes from the United Kingdom's traffic signal trials in the 2010s, where local authorities piloting new junction configurations reported that shortened monitoring cycles produced a surge in reported "incidents" that largely dissolved on review. The pattern was not that the junctions were dangerous; it was that the reporting channel was newly sensitive. Once authorities separated a rapid observation log from a slower escalation threshold, the false-alarm rate fell without any loss of genuine detection.

Polish practice is well placed to adopt this separation. The General Directorate for National Roads and Motorways and the rail infrastructure manager PKP PLK both run pilot programmes where temporary monitoring is increasingly instrumented. The institutional habit, though, still treats every flag as an event. Building a two-tier response — a low-cost log for the fast channel, a formal escalation only when a pattern persists across a defined window — is cheap and would recover most of the lost attention.

Designing trials that earn their speed

The forward-looking question is how to keep the compression while filtering the noise. Three design choices matter.

First, pre-register the thresholds. Decide before the trial begins what counts as a genuine fault, what counts as an observation worth logging, and what triggers escalation. This is the discipline that separates a trial from an anxious vigil.

Second, decouple the reporting rate from the escalation rate. Let the sensors report as often as they like; let the humans escalate on a slower clock, with a rule that a single flag is never sufficient. This is the chess player's separation of play from analysis, applied to a bridge.

Third, budget for the false alarms explicitly. If a two-week trial is expected to generate forty flags of which thirty are benign, staff the response accordingly rather than pretending the number will be smaller. The 40% increase is not a scandal; it is a predictable property of faster sampling, and it can be planned for the way one plans for weather days.

The next generation of Polish infrastructure pilots — on rail upgrades, on urban tram corridors, on the flood-control works now moving through design — will be more instrumented than anything that came before. The temptation will be to treat that instrumentation as free information. It is not. Speed buys feedback, and feedback buys attention, and attention is the scarcest input on any site. The projects that succeed will be the ones that decide in advance which signals deserve a response, and let the rest accumulate quietly into a record worth reading later.