How we report a finding

The point of SOMAT is to link what people do at work with what happens afterwards, and that's also where it would be easiest to overclaim. A dashboard that says "inspectors who record a likely cause prevent repeat reports" would be persuasive, and it might even be right, but the data behind it usually can't show that, so we don't write findings that way. This article sets out what a finding does say, using an illustrative example from social housing.

The example

Suppose a housing provider has 387 damp and mould inspections in SOMAT, each linked to its repairs data. In 251 of them the inspector recorded a likely cause, and 27 of those homes (11%) had a repeat damp report within six months. In the other 136, no cause was recorded, and 38 of those homes (28%) had a repeat report.

What the finding includes

Every finding we report includes the same few things alongside the headline.

• The counts as well as the percentages. 11% of 251 is a different level of evidence from 11% of 25, and anyone reading the finding should be able to see which they're looking at.
• Whether it held up when the data was split. In the example, the gap appeared in each of the provider's three areas when they were analysed separately. A pattern that only shows up in one area is more likely to be about that area than about the practice.
• A plain statement of what kind of finding it is. Here, it's that the two things were found together, which does not show that one causes the other.
• An analyst's note. Someone who understands both the data and the work says what they think it supports, which in this case was making "likely cause" a required, coded field.

Cause and effect

There are several reasons why homes with a recorded cause might have fewer repeat reports that have nothing to do with the recording itself. Inspectors may find it easier to name a cause in homes where the problem is simpler, and simpler problems are less likely to come back. More experienced inspectors may both record causes more often and be better at the inspection generally. Some of these can be checked in the data, for example by comparing inspectors with themselves across different homes, and we do check them where the numbers allow, but there's always something that can't be ruled out.

The finding is still useful. Requiring a coded cause costs very little and gives the repairs team better information whatever else it does, and its effect can be tested properly by introducing it in one area before the others and comparing the results. We'd present a finding like this to a board or a regulator as a good reason to try something and measure it.

When a finding is withdrawn

We rerun findings as new sessions come in. If the gap narrows until it could easily be down to chance, or stops appearing in one of the areas, we withdraw the finding and explain why, so that anyone who has acted on it knows that it has changed.