Designing a KYB risk score you can defend

I keep having the same conversation with compliance leads at regulated firms. It is usually about a customer risk model they inherited, built in Excel, or are about to ask a vendor to replace, and it starts with weights and thresholds.

It gets more useful once we move to the question underneath: what the number is supposed to do once it exists.

This is written for the people who own that decision. Heads of compliance, MLROs and onboarding leads at payments firms, banks and brokers who are weighing whether to build a customer risk assessment model in-house, buy one, or live with the spreadsheet for another year.

It is not a description of any particular product, including ours. It sets out the handful of design choices that determine whether a score holds up under regulatory review, and which of them are worth settling before anyone writes a line of code.

‍

What the score is for

A risk score does not know whether a business is honest. It cannot. What it knows, if it is built properly, is how much scrutiny a file deserves before someone commits the firm's name to a decision. That is a modest claim, and it is the only one a scoring model can genuinely support.

So the useful output is a routing instruction rather than a number. Low risk goes through. Medium risk triggers a request for specific evidence. High risk goes to a named human with the reasons already assembled. A prohibited condition stops the file regardless of what everything else says.

The practical reason this matters is queue design. Most onboarding backlogs are not caused by hard cases. They are caused by straightforward cases sitting in the same queue as hard ones, waiting for the same analyst.

A score that changes nothing downstream has produced a report, and the backlog stays where it was.

For how Detelio connects scoring to downstream decisions and review paths, see KYB Risk Scoring and the Decision Engine.

‍

There is more than one shape a model can take

Firms often talk about risk scoring as though there were a single accepted method. There are several, each with a different failure mode, and choosing between them deliberately is more than half the work.

Weighted factors

The most common approach and the easiest to explain. Each factor (jurisdiction, business activity, ownership complexity, expected volumes, screening results) is assigned a value and a weight, the weighted values are summed, and the total is compared against band thresholds.

It is transparent, it tunes easily, and every contribution is visible. Its weakness is that it is compensatory by construction: enough moderate factors will absorb a severe one, and it assumes factors are independent when in practice they are not.

Matrix models

Two or three dimensions crossed against each other, with the band read from the cell rather than from a sum. Customer type against jurisdiction, or inherent risk against control effectiveness.

This captures interaction, which is where additive models are weakest: a nominee structure in a low-risk country and the same structure in a high-risk one are not the same exposure, and a weighted sum will treat the difference as arithmetic rather than as a step change.

The cost is combinatorial. Cells multiply quickly, and every one of them needs a justification that somebody is prepared to defend.

Rule sets and decision trees

Sequential conditions leading to an outcome. Highly explainable for any individual case, and useful where policy is genuinely procedural. They become brittle at scale, and they make it hard to compare two customers who took different paths through the tree.

Inherent against residual

A separate layer rather than a competing method. Score the inherent exposure, then apply the mitigating effect of controls you actually operate (enhanced monitoring, restricted product access, reduced limits) to reach a residual rating.

Worth doing where the mitigation is real and evidenced. Worth avoiding where it becomes a mechanism for talking a score down.

Most firms that get this right end up with a deliberate hybrid: prohibited conditions evaluated as gates before anything is calculated, a weighted model doing the bulk of the work, and a matrix overlay on the two or three interactions that genuinely compound.

What matters is that the shape is a recorded decision with a rationale behind it, and that the engine underneath does not permanently assume one shape. Firms change methodology far less often than they change weights, but when they do change it, rebuilding the platform is an expensive way to find out the architecture assumed otherwise.

‍

Weights are a statement of risk appetite

There is no correct weighting, and a template from a consultancy deck is a starting point rather than an answer. A payments firm onboarding domestic merchants and a prime broker onboarding offshore funds should not use the same factor weights.

Weights encode appetite. The right question is what the board considers material and whether that can be defended in writing, rather than what the industry standard weight for jurisdiction happens to be.

Two firms can score the same applicant differently and both be right, provided each can produce the policy that explains the difference and the approval behind it.

The EBA's ML/TF Risk Factors Guidelines likewise expect firms to identify and assess relevant ML/TF risk factors and apply a risk-sensitive approach to due diligence. See the EBA Risk Factors Guidelines.

This has a build consequence. If changing a country's risk band requires an engineering ticket, the model belongs to the engineering team rather than to compliance, and compliance will quietly go back to the spreadsheet where it can make the change itself.

Configuration has to be editable, versioned and approved by the people who own the policy.

‍

Some conditions should never be scored

A weighted model has one dangerous property: everything in it is negotiable. Given enough low-risk factors, a high-risk one will eventually be diluted.

Certain conditions have to sit outside that arithmetic. A confirmed sanctions match, a prohibited jurisdiction, a prohibited line of business. These work as gates, evaluated before any calculation, and they fail closed.

A model that allows a sanctions hit to be offset by a clean ownership structure and a domestic bank account is broken, and describing it as conservatively tuned does not change that.

Where a reason would never be approved on its own, configure it as a stop rather than as a heavy weight.

‍

Where AI belongs, and where it should not go

Firms get this wrong in both directions, either keeping AI away from the risk model entirely or letting it produce the score outright. The useful boundary runs between interpretation and calculation.

Interpretation is messy and genuinely difficult. An applicant writes "logistics and freight forwarding, occasional trade finance" into a free-text field, and something has to decide which row of a reference table that corresponds to. A passport image has to become a country code. Three pages of adverse media have to become a severity tier. This is language work, models are good at it, and doing it manually is a large part of why onboarding is slow.

Calculation works differently. Multiplying a factor value by a weight, summing, comparing against a threshold: this belongs in deterministic code reading an explicit configuration. Same inputs, same policy version, same answer, every time, this year and in four years when somebody asks how a particular number came about.

The reason is operational rather than philosophical. Everything useful downstream depends on repeatability. Simulating the impact of a policy change across an existing book requires it. Tying out against a historical sample requires it. Explaining a specific number to an auditor requires it. Once a model is doing the arithmetic, all three become unavailable.

A second obligation follows. Every interpreted value has to carry its provenance: what the input was, how it was resolved, whether a human or a model resolved it, and with what confidence. A low-confidence resolution then routes to review in the same way a high band does.

A short illustration, using a deliberately synthetic applicant:

The middle row is the only line a model touched, and it is the one a human should confirm. A logistics company that occasionally finances trade and a trade finance business are different customers, and the gap between them is worth several points. Surfacing that line is what lets a reviewer correct it in seconds instead of re-reading the file.

‍

The model should ask the next question while the applicant is still there

Scoring only at the end of onboarding is a wasted opportunity. By then the applicant has submitted, the case is queued, and the remedy is an email asking for one more document, which starts a cycle measured in days.

Where the factor structure is explicit, the model knows which missing or ambiguous input is moving the outcome. That question is worth asking immediately, in the form, while the applicant is still engaged: confirm your primary activity from this list, upload the ownership chart for this entity, clarify this expected volume.

One targeted request at the right moment replaces a long correspondence cycle, and it improves the quality of what arrives.

This depends on structure. An opaque process can tell you a file looks risky, while leaving you to guess which single answer would resolve it.

‍

What an auditor should be able to replay

The test is simple. Take a decision from eighteen months ago and reconstruct it end to end: the evidence, how each input was resolved and by what, the rule applied, the contribution it made, the resulting band, the action that band triggered, any human override with its stated reason, and the final outcome.

Two things make that possible.

First, versioned policy. A score is only meaningful alongside the exact configuration that produced it, so configurations are versioned and published rather than edited in place.

Second, a stored snapshot of the data the score was computed from. Re-running an old case against today's policy will produce a different number, which is the expected behaviour and the reason both are recorded.

Overrides deserve their own note. Analysts should be able to override outcomes, because models are wrong at the edges and judgement is the job. What should be impossible is overriding silently. The original score, the override, the person and the rationale all stay in the record.

A consistent pattern of overrides in one direction is the most valuable tuning signal a firm has, which is a good argument for making it visible rather than awkward.

For the evidence and decision history around these controls, see Detelio's Audit Trail.

‍

The short version

Build the score to allocate attention. Choose the methodology deliberately and write down why. Keep prohibited conditions out of the arithmetic entirely. Let models read evidence and let deterministic code do the maths. Ask the applicant the one question that matters while they are still in the form. Version the policy and store the snapshot, so any number can be explained years later by somebody who was not in the room.

None of this is exotic. It mostly comes down to deciding, before you build, which parts of the process are allowed to be probabilistic and which are not.