In my November newsletter I discussed how an error that appears small at the time it occurs can have a big result down the line. Gordon Bagot replied as follows:

I too have come across this problem with a former client. A currency exchange rate was wrongly transferred from one electronic data file to another, manually, and with numbers transposed. The whole firm used the transposed number, in all their systems, until I was asked to do some consultancy work where I used my own source of exchange rates. The firm argued with me, delayed payment too, until I spent the time trying to resolve the problem with one of the client’s staff. Result, I was right, client wrong, no apology,but I did get payment then more promptly.

There is so much in the way of statistical analyses done, on which quite major investment decisions are made, that I can’t understand why time, money, resources is not allocated to ensure data is 100% correct as is possible.

I do hope items such as this are noted by your clients.

Me too!

At the recent GIRO conference, Rob Curtis from the FSA drew our attention to the recent consultation paper: CP06/16: Prudential changes for insurers. The part that made me prick up my ears was the following:

The written record of a firm’s individual capital assessment, as carried out in accordance with Sub-Principle 1 submitted by the firm to the FSA must:

1. in relation to the assessment comparable to a 99.5% probability over a one year timeframe that the value of assets exceeds the value of liabilities, document the reasoning and judgements underlying that assessment and, in particular, justify:

(a) the assumptions used;
(b) the appropriateness of the methodology used; and
(c) the results of the assessment.

2. identify the major differences between that assessment and any other assessments carried out by the firm using a different probability measure.

It’s 1 (c) that caught my attention, of course. I’ve written elsewhere about what you have to do to believe the results of your models: you have to be able to trace the results back to model specification, data and parameters. This means having good audit trails, thorough testing (and records of those tests) and effective version control.