Predictive analytics has a credibility problem, and it is self-inflicted. The field is described almost entirely in terms of its promise — foresight, competitive edge, data-driven transformation — and almost never in terms of what a working system actually requires or what it realistically returns.
So here is the unglamorous version: a prediction is only worth the decision it changes. That single constraint eliminates most proposed projects before any modelling starts, and it is the most useful filter available.
The test a prediction has to pass
Before committing to a predictive project, three questions have to have concrete answers. If any one of them doesn't, the model will be built, reported on, and quietly ignored.
- What decision does this change? Not "it gives us insight" — which specific recurring decision does someone make differently because of this output?
- Who acts on it, and can they? A churn prediction is worthless without a retention budget and someone authorised to spend it. A demand forecast is worthless if purchasing is locked into quarterly commitments. The action has to be available.
- What is being wrong worth? False positives and false negatives have different costs, and the ratio determines your threshold. A team that cannot state this ratio has not defined the problem yet.
A forecast nobody is authorised to act on is a report, not an analytics capability.
Where it reliably works
The applications with the most consistent record share a shape: a decision made repeatedly, with a measurable outcome, on a timescale that allows a response.
Demand forecasting
The most consistently valuable application in most businesses that hold stock. Better forecasting reduces both stockouts and the capital tied up in inventory, and the improvement is straightforward to value in money. The decision — how much to order — is already being made, already on a schedule, and already owned.
Predictive maintenance
Where equipment failure is expensive and there is a signal before it happens, the economics are typically strong. The pitfall is prediction windows too short to act on: knowing a machine will fail in four hours is only useful if a technician and a part can be there in four hours.
Churn and retention
Works when there is a genuine intervention available and the model identifies customers whose behaviour can actually be changed. Many churn models mostly identify customers who have already decided — accurate and useless. The useful target is not "who will leave" but "who will leave and would stay if we did something".
Credit and risk scoring
Mature, well-understood, and heavily regulated — which means explainability and fairness testing are requirements rather than refinements. Under the EU AI Act, creditworthiness assessment falls within the high-risk category, with obligations deferred to December 2027 under the Digital Omnibus but not removed.
The three ways these projects fail
The data does not support the question
Predicting a rare event requires enough examples of it to learn from. Predicting an outcome requires that the outcome was recorded reliably. A surprising number of projects discover only after months that the target variable was never captured consistently — or was captured in a way that leaks the answer.
That last one is worth naming: target leakage, where a feature encodes information that would not be available at prediction time. It produces spectacular validation accuracy and complete failure in production, and it is the single most common way a promising model turns out to be worthless.
Nobody owns the response
The model runs, the dashboard updates, and no process consumes it. This is the most common failure and the least technical. The fix is to design the response workflow before the model — who sees what, when, and what they are expected to do about it.
It is never measured against a baseline
The honest comparison is not model accuracy in isolation but the model against what you did before — which is often a simple heuristic that performs surprisingly well. "Last year's demand plus 5%" is a real baseline, and a sophisticated model that fails to beat it has told you something valuable and unwelcome.
A sensible way to start
- Pick one recurring decision with a measurable outcome and an owner who can act.
- Establish the baseline — what accuracy does the current heuristic achieve? Write it down before building anything.
- Check the data honestly, on real historical records rather than a cleaned extract, with a deliberate hunt for leakage.
- Build the simplest thing that could work. Simple models are faster to build, easier to explain and frequently competitive. Complexity should have to justify itself.
- Design the response workflow alongside the model, not after it.
- Measure in production against the baseline, on the business metric, on a schedule someone owns.
The realistic summary
Predictive analytics works, within limits that are well understood by the people doing it and rarely stated by the people selling it. The constraint is almost never algorithmic sophistication — capable methods are freely available and well documented.
The constraint is whether you have reliable historical data, a decision worth changing, someone able to act on it, and the discipline to measure whether it helped. Businesses that have those four things get real value. Businesses that have a data warehouse and an ambition tend to get a dashboard.
We build forecasting and decision-support systems that are measured against a baseline and wired into a real workflow. Tell us which decision you want to improve.
