Catch the full webinar: Personalization Without the Bullshit: What’s Actually Lifting Revenue
I argued in a recent post that enterprise personalization has a proof problem rather than a data problem, and that proof requires volume: enough clean, isolated changes, shipped fast enough that the signal outruns the noise.
That argument has an obvious follow-up. What is actually eating the time?
The answer is measurable. Most organizations have never measured it, and the ones that try measure the wrong quarter of it.
Insight-to-execution time is the elapsed time between the moment a revenue-relevant insight becomes available and the moment the resulting change is live to real traffic. Rebecca Kerper put the idea on the table during our RTM Nexus panel, and it is the metric I named in that post.
It is not sprint velocity, which measures your engineering team and starts counting far too late. It is not ticket cycle time, which starts when somebody writes the ticket, by which point you are already halfway through the story. The clock starts when your organization could have known. It stops when a customer sees the change.
Detection lag: behavior shifts, until somebody in your building knows.
Decision lag: somebody knows, until somebody with authority approves acting.
Build lag: approved, until built, reviewed and QA’d.
Release lag: built, until live to real traffic.
Most leaders assume the damage is concentrated in build lag, because build lag is the only segment with a Jira board and therefore the only one with a paper trail. In the organizations I walk into, it is rarely the biggest of the four. It is just the only one anyone can see, which is why it is the only one that ever gets a postmortem.
Map the four onto the two gaps and the picture sharpens. Detection lag is the Insight Gap. Build and release lag are the Activation Gap. Decision lag sits between them, and it is the only one of the four that is genuinely about people.
Behavior shifts in March. Somebody finds out in June, from a quarterly deck.
Mobile filter usage collapses on your three highest-traffic category pages after a template release. Nobody notices, because filter interaction is not in the standard reporting package and the conversion dip is small enough to look like weather. It surfaces eventually, when an analyst is pulling numbers for something else entirely.
That is not an analyst failure. It is what happens when behavior sits behind tagging plans, SQL and a queue you do not control. Every week in that segment is a week the insight is aging and nobody has started the clock, because nothing in the chain is instrumented and therefore nothing in it is anyone’s fault.
This is the segment most enterprises have never even attempted to measure, and it is the one where no-code analytics for ecommerce changes the arithmetic outright. A merchandiser who can interrogate behavior without filing a request removes weeks that never appeared on any board.
Approval layers. Unclear ownership. A shared template that touches four brands, so the fix needs sign-off from two brand leads and a director, and two of the three are in market visits.
Rebecca raised the version of this that nobody in the room had an answer for: if a team tests something and it loses, are they expected to stop or keep going? Most organizations have never decided, so the default is stop. The default is expensive, because a losing test is not a failure. It is one of the two out of three you were always going to get.
A leader can fix this segment by decree. Most don’t, because it is nobody’s KPI.
These two exist because acting on an idea requires a briefing document, a design file, a backlog slot, a sprint, a QA pass and a release window. Six handoffs, and the idea gets colder at every one.
Release lag is the one that surprises people. The build finishes, then the change misses a sprint boundary by two days and lands in a code freeze ahead of a promotional period. The work is done. It just isn’t live, and a change that isn’t live has earned nothing.
You can flatten an approval chain in a memo. You cannot memo your way to a merchandiser publishing a PDP variant without a developer. That is the line between the process half of this problem and the architecture half, and it is why “we need better alignment” never fixes it. Experimentation without dev support is not a cultural achievement. It is a property of the system you bought.
Even a well-run test is slow, and the insight decays the whole time it runs.
Georgi Georgiev’s meta-analysis of 1,001 A/B tests, published on Analytics-Toolkit in October 2022, found a median test duration of 30 days. That is only the time the test spends running. It excludes the weeks spent building it, and it excludes the weeks after it wins while somebody finds a release slot for the permanent version.
The same study found that about 33.5% of tests produced a statistically significant win, at confidence thresholds spread between 80% and 95%. That sample is expert CRO practitioners on a single platform, which makes the number generous rather than harsh. Even among people who do this for a living, and at thresholds as loose as 80%, two ideas in three do not work.
Now stack it. A detection lag measured in weeks, a decision lag measured in weeks, a month of runtime, and a one-in-three chance the answer was worth having. By the time the fix goes live, the seasonal assortment behind those filters has turned over and the change is shipping against a catalog it was never designed for. That is not optimization. That is archaeology.
You do not need new tooling to start. You need four timestamps and the discipline to record them.
Record all four moments for every experience-layer change: signal available, decision made, build complete, live to traffic.
Report the median, not the mean. One nightmare project drags an average into uselessness, and the average is the number people will use to argue nothing is wrong.
Segment by change type. Copy, layout, full template, personalization rule and net-new page have completely different profiles. A blended number hides which one is bleeding.
Publish it monthly, next to conversion rate. Conversion tells you how last month went. This tells you how next quarter is going to go.
The first report buys you one uncomfortable meeting. The second one buys you a roadmap.
A velocity metric without guardrails is how good ideas turn into bad quarters.
A fast clock on a weak hypothesis just gets you to the wrong answer sooner. Insight-to-execution time measures velocity, and velocity without direction is churn. Pair it with impact-ranked prioritization, not opinion-ranked, or you will ship a great deal of nothing very efficiently.
Some things should be slow. Pricing logic, checkout flows, anything touching payment or regulated data. This is a metric for the experience layer. Publish it as a company-wide target and somebody will apply it where deliberation is the point.
It is gameable. Make it a KPI without a companion metric and teams will hit it by shipping trivia, because a headline swap scores identically to a rebuilt category template. Report it beside the share of shipped changes that produced measurable lift and the incentive corrects itself.
Every enterprise I walk into has a list of things it wants to test, and nobody’s list is short. Ideas were never the constraint. The constraint is that the cost of acting on any one of them is measured in weeks of other people’s calendars, which means the list is really a ranking exercise in what you are willing to not do this year.
Cut the cost of acting and the list stops being a ranking exercise. That is the whole unlock, and it is not available through better planning.
Three of the four segments are ours to shorten. Fastr Optimize collapses detection lag with behavioral intelligence that needs no tagging plan, no SQL and no slot in the analyst queue. Fastr Frontend collapses build and release, because the people who had the idea publish the change themselves. That is what reducing dev dependency actually looks like in an ecommerce organization: not fewer developers, but developers who are no longer standing between an insight and a live page.
The fourth segment is decision lag, and that one stays yours. It always was.
Your competitors are not running better experiments than you. Some quarters they are running worse ones. They are running more of them, and finding out sooner which ones deserved to survive. Clock speed is a property of the system, not of the people working inside it.
So measure it. Then go change what is setting it.
This came out of an RTM Nexus panel with Rebecca Kerper and Tim Zawislack. The full panel is worth the forty minutes.
What your own number looks like? We’ll show you your own site, not ours. 20 minutes. No slides.