Eight samples. Every number carries how it was measured.

The challenge

Each country finance team calculated gross margin in its own spreadsheet from its own ERP extract. Group could not explain a two-point gap between the board pack and the management accounts, and the monthly consolidation consumed six days of three people’s time before anyone could discuss a decision.

The approach

A five-day metric audit defined gross margin once, reconciled it to the finance system, and named the owner. Freight allocation turned out to be the real disagreement: two markets charged inbound freight to COGS, three treated it as overhead.

How it was measured

Close duration measured over the first three live closes against the prior six months. The leakage figure is what the model surfaced when the 20 largest supplier agreements were re-tested; not all of it is recoverable.

What changed

• One margin definition, signed by the group CFO, versioned in code.
• Consolidated close runs against the warehouse: six days of hand-work became a four-hour review.
• Margin bridge per market showing price, volume, mix and freight as separate drivers.
• Three legacy extracts retired, including one nobody would admit to owning.

Why it worked

The CFO owned the definition and chaired the one argument that mattered. Freight allocation was settled in a 40-minute meeting because the decision rights were written down before the meeting.

What we would do differently

We modelled freight twice. The first version followed the contracts, the second followed what the warehouse actually did. That cost a week and should have been the first question.

The challenge

Telematics data landed in a vendor portal nobody exported. Maintenance was reactive, fuel was reported as a monthly total with no route comparison, and the only signal the depot managers trusted was the repair invoice.

The approach

A stream from the telematics API into the warehouse, joined to route plans, fuel cards and workshop records. Idling, harsh braking and route deviation became driver-level metrics with a league table the depot managers asked for.

How it was measured

Fuel is measured per kilometre on routes with unchanged plans, so a route change does not flatter the number. Uptime counts stoppages requiring roadside attendance, over four quarters, same vehicle set.

What changed

• Fuel per kilometre split by route, vehicle and driver, refreshed every 15 minutes.
• Preventive maintenance triggered on engine hours rather than the calendar.
• Six vehicles flagged for mechanical faults from fuel-efficiency outliers before the workshop found them.
• Depot level reporting replaced the vendor portal entirely.

Why it worked

The depot managers shaped the league table. Tying the metric to a driver and publishing it weekly changed behaviour faster than any dashboard we have shipped.

What we would do differently

Driver-level metrics needed a works-council conversation we had not planned for. On a second run we would open that in week one, before the first schema was written.

The challenge

Each clinic reported treatment volumes to the group in a spreadsheet of its own design. Clinical leads could not answer whether a pathway change had improved recovery, and the group could not produce a compliance pack without three weeks of manual assembly.

The approach

Regional deployment with pseudonymised identifiers and EU-only storage. A small set of clinical outcome metrics was agreed with the medical director before any pipeline was built, then enforced as code with row-level security matching site and role.

How it was measured

Reporting effort is clinic-reported hours logged over two quarters. The EU-residency claim is architectural: all storage, processing and model calls run in the Frankfurt region, and the pattern is documented for your DPO.

What changed

• One outcome set across nine sites, defined once and used by every clinical lead.
• Compliance pack generated from the warehouse instead of assembled by hand.
• Site-level dashboards that respect patient volume masking thresholds in small cohorts.
• Access audit trail retained for every query against patient-level data.

Why it worked

We agreed the metrics before the technology. The medical director signed off the outcome definitions, which removed the political argument that usually stalls clinical reporting.

What we would do differently

Two of the original outcome metrics were dropped: the data existed but was recorded inconsistently by site. Publishing that gap early would have saved a month of modelling.

The challenge

Fourteen sub-ledgers produced in different currencies and two local charts of accounts. Consolidation was an eleven-day exercise in a shared workbook, and the intercompany reconciliation was the recurring reason the group audit ran late.

The approach

A mapped chart of accounts, FX rate handling with an auditable rate source, and an intercompany matching engine that proposes pairs and routes everything else to a review queue with a reason code.

How it was measured

Cycle time measured from the last sub-ledger close to group sign-off across two live quarters. The unmatched 6 per cent is published with a reason breakdown rather than hidden in an adjustment.

What changed

• Consolidation runs nightly, so group finance reviews a two-day-old position instead of a quarter-old one.
• Every FX rate and mapping is versioned, with the effective date carried through to the report.
• Intercompany exceptions have an owner and an ageing bucket instead of living in a workbook.
• Audit pack builds itself from the same lineage the reports use.

Why it worked

The matching engine was built to refuse. Anything under a confidence threshold goes to a human queue, which is why the finance team trusts the 94 per cent it approves.

What we would do differently

We under-scoped the FX rate governance. Three sources of truth for one rate is a governance problem, not a modelling problem, and it should have been a week-one deliverable.

The challenge

Quality control recorded scrap per shift on paper. Nobody could connect a bad batch to a machine setting, and product margin was calculated at product-family level, which hid the lines losing money.

The approach

Two lines instrumented first. Sensor readings joined to work orders at batch level, so scrap could be attributed to a machine setting, a material lot and a customer order simultaneously. Margin was recalculated at the line-order grain.

How it was measured

Scrap rate is defined as rejected units over produced units on two lines, measured over six months against the prior twelve. The saving is scrap cost avoided at standard cost, not revenue.

What changed

• Scrap attributed to setting combinations, which took the trial-and-error out of changing over.
• Two product families were found to be unprofitable at current yields, which had not been visible.
• Operators see yield at the end of a shift rather than at the end of the month.
• The roll-out pattern to the remaining seven lines is documented with measured effort per line.

Why it worked

Two lines instead of nine. The plant manager asked for full scope in phase one; measuring the effort on a small scope made the second phase quote credible.

What we would do differently

Sensor clocks drifted by up to 40 seconds between lines, which broke batch joins until we caught it. Clock synchronisation is now the first item on this type of project.

The challenge

The finance team keyed supplier invoices by hand. The legacy scanner handled the clean ones, which were the ones that needed the least attention, and failed on exactly the invoices that carried the risk: multi-page, credit notes, and anything with a purchase-order reference in a table.

The approach

Extraction with a layout model plus a validation layer that runs before anything reaches the ERP: purchase-order match, approved vendor list, arithmetic, duplicate detection and a date plausibility check. Failures land in a queue with the reason and the page image, so the reviewer starts at the problem.

How it was measured

The 87 per cent is extraction accuracy against expected values on the held-out set. Cycle time is measured from batch receipt to ERP posting across the first eight weeks after cutover.

What changed

• Accuracy reported on a 1,240-invoice held-out set the model never trained on, with the sample size printed beside the number.
• Every rejection carries a reason code, so finance can see whether a supplier is generating the same defect every month.
• Cost per document and per exception are tracked as a live metric, not a one-off business case.
• One supplier reformatted their invoices after the reason codes pointed at them for three consecutive months.

Why it worked

We did not chase the last 13 per cent. The exception queue is a design feature, and reviewers who trust the queue process more exceptions per hour than they used to type invoices.

What we would do differently

We spent two weeks on supplier-specific templates before accepting that a general layout model with a strong validation layer was the cheaper architecture.

The challenge

Regulatory affairs answered the same class of question repeatedly across hundreds of documents, guidance notes and previous submissions. Generic chatbots were tried and banned within a month: they produced confident answers with no traceable source, which is the one failure mode this team cannot accept.

The approach

Hybrid retrieval over the internal corpus with access control inherited from the document store, and an answer contract: cite, or refuse. Answers quote the passage, name the document and version, and store the retrieval trail. A 340-question evaluation set runs on every change, with a regression gate before deployment.

How it was measured

Time per question measured over twelve weeks across 96 logged questions. Accuracy is the share of the 340-question set where the cited passage supports the answer, graded by the regulatory lead, not by a model.

What changed

• Every answer carries document, section and version. No answer without a source reaches the user.
• The refusal rate is published internally and treated as a health metric, not a defect.
• Evaluation runs in CI, so a retrieval change that breaks an old answer cannot ship.
• The team uses the tool to find documents, and still writes the determination themselves.

Why it worked

The evaluation set existed before the first retrieval change. Having 340 graded questions in week one is what made it safe to iterate fast later.

What we would do differently

The first two weeks were spent improving chunking, which moved accuracy by a point. Re-ranking the retrieval moved it by eleven. Baseline before you optimise.

The challenge

Demand planning ran on a spreadsheet with a growth factor applied to last year. Stockouts and overstock happened in the same quarter, and the planners had stopped using the output, which nobody had noticed.

The approach

The pilot began by scoring the existing process on held-back history, because a model that cannot beat the incumbent is a cost with no upside. Gradient boosting on order history, price changes and delivery lead time was then measured on the same weeks.

How it was measured

MAPE measured on 13 weeks of held-back history, per family, with the training window printed. The two losing families are published in the business case rather than averaged away.

What changed

• Model error below the incumbent on four families, above it on two short-lifecycle ranges where promotions dominate.
• Planners now see both forecasts and a disagreement flag instead of one number with no provenance.
• Replenishment alerts tuned to the two exception patterns that actually move service levels.
• Drift and input-quality monitors live before the first planning cycle used the output.

Why it worked

We reported the failures in the same table as the wins. The planners trusted the tool on the families where it worked because we had been honest about the ones where it did not.

What we would do differently

Two families should have been excluded in scoping. Their promotion history had two observations per year, which is not a forecasting problem, it is a data-collection problem.