Newsletter Old site

Newsletter Apr 2007

News update 2007-04: April 2007

1. Last issue
2. Newsletter information

1. Last issue

Some of you may have noticed that this newsletter hasn’t appeared
for a while, and somebody has even asked why (hi Steve!).

Early this year I decided that I was going to stop being an
independent consultant, and started to look for another role. The
search was successful, and at the beginning of May 2007 I start
work at the Board for Actuarial Standards, which is part of the
Financial Reporting Council.

Under the circumstances the newsletter rather faded out of the
picture. It’s not as if there’s been a dearth of things to write
about, though; most obviously, the recent RIM outage which was caused,
apparently, by a software upgrade that went wrong.

Looking back over the incidents and issues I’ve written about over
the past few years, I think there’s a simple theme that emerges:
you can’t count on things not going wrong. The fact that something
hasn’t happened in the past doesn’t mean that it’s not going to
happen in the future. The trouble is that we learn from experience.
In practice, this often means that we don’t learn from what we
don’t experience. There’s a new book just out, which I haven’t read
yet but which looks very interesting, that’s relevant here: The
“Black Swan: The Impact of the Highly Improbable” by Nassim
Nicholas Taleb. I’m looking forward to reading it.

Even when you are aware of a risk, because the bad event has
happened before, it’s often quite difficult to be as realistic
about it as you should. Software development is an excellent
example here; all software developers have written software that
they thought would work, only to be surprised when it turns out to
contain bugs. The first time this happens, the initial belief may
be reasonable. When it has happened over and over again, the
developer’s confidence may seem less reasonable. However, each time
the developer (and I speak personally here) can produce plausible
justifications: they have learned from their previous experiences,
and taken steps to avoid all the problems that have arisen in the
past. The problem is that there are always more, new, problems that
can arise. Never believe anyone who says “it could never happen”.

An excellent newsletter about risks is the Risks Digest. Although a
number of its items are fairly technical, there are many that raise
more general issues that are widely applicable. I’ve been reading
Risks on and off for nearly twenty years, and shall certainly
continue doing so.

2. Newsletter information

This is the last issue of a monthly newsletter on risk management
in financial services, operational risk and user-developed software
from Louise Pryor ( Copyright (c)
Louise Pryor 2007. All rights reserved. You may distribute it in
whole or in part as long as this notice is included.

To subscribe, email news-subscribe AT To
unsubscribe, email news-unsubscribe AT Send all
comments, feedback and other queries to news-admin AT (Change ” AT ” to “@”). All comments will be
considered as publishable unless you state otherwise. My blog is at The newsletter is archived at

Newsletter Old site

Newsletter Nov 2006

News update 2006-11: November 2006

1. Small error, big result
2. Nobody will ever…
3. Just pick it up and carry it
4. Newsletter information

1. Small error, big result

Guess what happens when you put the wrong data in? Surprise,
surprise, you get the wrong answer out at the other end. This has
just happened to HBOS, where the accumulated wrong answers are
worth £17m.,,1958434,00.html

Apparently there’s been an error in the unit pricing data for four
years. It all started about four years ago, when a decimal point
was put in the wrong place. The errors have mounted up over time.
The problem was noticed around a year ago, and since then Clerical
Medical (the arm of HBOS where this happened) have gone back
through all their policy records to work out exactly which policies
have been affected and by how much.

There are a couple of interesting issues here.

The first one is, of course, the way the mistake happened to start
with. The press stories imply that it was a simple data input
error: somebody typed the wrong number, 23.4 instead of 2.34 or
something like that. It’s easy to do. I know I’ve mistyped things
in the past, and I’m sure you have too. The question is, how can
data input errors be prevented?

Well, first off, you try to avoid manual data input. Most data
comes from a computer anyway, and you try to set up a direct feed
rather than rely on somebody reading numbers from a screen or
printout and typing them in. If it turns out that a direct feed is
impossible, maybe you try copying and pasting: there are plenty of
risks there, but they are different to those of manual input.

You also want to have some sort of data validation on the inputs:
maybe a range outside which entries are queried, or a maximum
difference between the new value and the old value, or between it
and other, related values. It’s often difficult to set this sort of
thing up so that there are no false alarms and all errors are
caught, but these data validation checks are nevertheless useful.

Another option is to have some higher level checks built in. Maybe
all manual data inputs have to be made twice, by different people.
This is time-consuming and a bore, so it may be overkill in many
cases. Or you can set up some check totals: make sure that the
entries that have just been input add up to the same as the total
on the sheet from which you are copying them. This can mean
inputting data you don’t actually need, because it’s included in
the total, but it’s sometimes worth it.

The second interesting issue is connected with overall
reasonableness checks. It sounds as if the error was small at the
beginning, so wasn’t noticed. Then, as it grew larger, it still
wasn’t noticed. This is pure speculation on my part, but people
often do reasonableness checks in terms of the difference from one
time period to the next. And in this case, as no new error was
introduced, everything would be consistent so nothing wrong would
be noticed.

A really important lesson to learn is that consistency with the
prior period, or previous version, doesn’t necessarily mean that
there’s no error. It just means that there is no new error, or that
any new error is insignificant. And errors don’t always stay

More generally, relying on the results being consistent with your
expectations, or being reasonable in the light of your experience,
can be risky if your expectations or experience are based on the
use of the system that you are assessing. If you’ve been using a
program for some time, you have probably built up some expectations
about how the results are likely to vary with the inputs, and
you’ll probably have some good explanations of why things work in
the way they do. However, this can be risky; we tend to be very
good at post hoc rationalisation. If you rely on overall
reasonableness checks, or on consistency with experience, you need
to be sure that your checks are genuinely independent, rather than
being based on your experience of the system you are trying to

2. Nobody will ever…

The space shuttle can’t cope if it’s off the ground over 31st
December/1st January. Or, at least, its software can’t cope. The
software is about 30 years old, and doesn’t have any way of moving
from day 365 of the year back to day 1. So on 1 January 2007, for
example, it will think it is day 366 of 2006. Although they’ve done
simulations of flights lasting over the year end, and have
encountered no problems, NASA isn’t keen to try it in earnest.
There are distinct echoes of Y2k here. Nobody really knows if
anything will go wrong, but they don’t want to risk it.

It just shows that the “nobody will ever want to do X” design
principle is one to be avoided. Also, of course, that software can
stay in service much longer than you expect.

And, of course, nobody would ever hack into an ATM using an MP3
player. Except that Maxwell Parsons did. He targeted free-standing
ATMs (ie, not built in to walls) in bars and bingo halls. They were
connected to the bank using an ordinary telephone line, and as they
were free-standing he could physically get at it. He used the MP3
player to record the data going down the line,and then decoded it
on a laptop and used it to counterfeit credit cards. Apparently the
banking industry have now plugged the loophole.,,29389-2453590,00.html

3. Just pick it up and carry it

We tend to think of data theft as being primarily a network
problem: nasty people hack into a computer somewhere, and syphon
off the data; or they persuade nice people to visit a nasty
website, and enter confidential information. Physical theft is
pretty common, though. Laptops can and do hold vast amounts of
data, and are very easy to pick up and carry. Just this month, in
the UK, three laptops containing payroll details of 15,000
Metropolitan Police officers were stolen from LogicaCMG, and a
laptop containing customer information was stolen from an employee
of the Nationwide building society.

Often the data theft is probably accidental–it’s the laptop that’s
the real target, rather than the data on it–but that doesn’t
really make the data any more secure. In some cases it’s against
company policy to hold confidential data on a laptop. You’d think
that this should be the general rule, with very few exceptions.
Yes, it may be convenient to be able to work at home, or while
you’re travelling, but do the benefits outweigh the risk?

Physical security isn’t an issue only where laptops are concerned.
Earlier this month thieves stole a number of router cards from a
London data centre, resulting in serious service disruptions. A
couple of weeks earlier thieves got into a different data centre,
and drove off with a van full of equipment.,1000000189,39284520,00.htm

If they could get in and walk (or drive) off with equipment, what’s
to stop them walking off with whole disk drives or servers?

4. Newsletter information

This is a monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
( Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long as
this notice is included.

To subscribe, email news-subscribe AT To
unsubscribe, email news-unsubscribe AT Send all
comments, feedback and other queries to news-admin AT (Change ” AT ” to “@”). All comments will be
considered as publishable unless you state otherwise. My blog is at The newsletter is archived at

Newsletter Old site

Newsletter Oct 2006

News update 2006-10: October 2006

1. Software versions
2. Bogus data
3. Is testing necessary?
4. Blogging and wikis
5. Newsletter information

1. Software versions

Want to lose 4.8 billion euros? It looks as if a good way to do it
is to make sure that different parts of your organisation are using
different versions of the same software package. The wiring
problems that have delayed (yet again) deliveries of the Airbus
A380 arose because incompatible versions of the CAD software were
being used. French and British engineers had upgraded to version 5,
while the German and Spanish engineers were still using version 4.
The two versions use different file formats.

How did this happen? It must have been a combination of factors.
First, the software manufacturer (Dassault in this case) changed
the file format without providing backwards compatibility. Then it
was decided that parts of Airbus should upgrade, and that parts
should not. Either those decisions were made independently, and
there is no overall software policy (which is a big problem) or
there were particular reasons for different parts of the
organisation to make different decisions, but nobody thought about
how they would then work together. Probably the truth is somewhere
between the two.

This isn’t, of course, a problem only with CAD software. It
shouldn’t surprise anyone that incompatibility problems can arise
with Excel too. There are still people using Excel 97, in which
macros written in later versions generally don’t work. And I’ve
come across macros written in Excel 2000 that don’t work in later
versions. In fact, to state the obvious, each new release of Excel
contains features that don’t work in earlier releases. More subtly,
some of the statistical functions were changed in Excel 2003, so
the results produced by a spreadsheet can depend on the version
under which it was last recalculated.

As Excel 2007 hits the streets (or rather desktops) incompatibility
problems are going to become more common. It’s actually going to
have a “compatibility mode” which will ensure “that content created
in the 2007 Office release can be converted or downgraded to a form
that can be used by previous versions of Office.” I like the use of
the word “downgraded” in that sentence. The trouble is, though,
that if you use the compatibility mode you won’t be able to take
advantage of all the new features.

IT departments are going to have to think carefully about their
upgrading strategy. However, even if individual organisations get
it right, there will still be problems when spreadsheets are sent
between organisations.

2. Bogus data

Computer models are all very well, but they are only as good as the
data that goes into them. Two Citibank traders recently pleaded
guilty to falsifying bank records and wire fraud. Among their
nefarious activities, they manipulated a computer model that was
monitoring options trading, by inputting bogus data. Apparently
they got a broker to supply them with false market quotes.

Maybe they had taken lessons from John Rusnak, the fraudster in the
AIB/Allfirst case. He manipulated the inputs into a spreadsheet
that monitored his trades, by making sure that exchange rate feeds
didn’t go in to it directly but through his own PC.

Deliberate data manipulation like this is always going to be a risk
when people’s remuneration depends on the results. Accidental
manipulation is always a risk, though, regardless of the uses to
which the model is put. That’s why it’s really important to have a
good audit trail from the source of the data right through to the
end results of the model.

A good audit trail is one that would make any discrepancy
immediately obvious, without requiring laborious manual
comparisons. There are various ways to accomplish this, depending
on the circumstances. However, there are also numerous ways to
invalidate an audit trail, and these are probably more common.
Obvious problems include documentation that doesn’t reflect the
actual procedures, lack of documentation, manual procedures such as
copying and pasting, over reliance on check totals, non-standard
items that get special treatment, and any stage in the process
where manual alterations are possible, whether deliberate or

3. Is testing necessary?

I think you can guess what my answer would be, but it appears that
not everyone has the same opinion. It’s often felt that testing is
expensive, and can be skimped (or skipped altogether) as the
benefits it provides aren’t worth it. This view assumes that the
testing process won’t uncover any significant problems. Experience
shows that this assumption is usually over-optimistic.

It’s important to remember, too, that testing isn’t just about
getting the calculations right (though that’s important). You may
also want to test performance (under both normal and abnormal
conditions) and usability. Doing the right thing covers every
aspect of a deployed system, and it’s important to get it all

A recent article reports the government as saying the following
about ID cards:

* It won’t be possible to test everything in advance
* They’ll use off-the-shelf technology for some parts; this will
have been adequately tested elsewhere
* Trials will have to be limited in order to stay within budget
* Instead of trials, they’ll use incremental roll-outs

So they will be testing, it’ll just be on live data (and hence real
people). And just because a product is off-the-shelf it doesn’t
mean it’ll work under all circumstances, especially if it’s part of
a larger system. Interfaces between different components are always
potentially dodgy.,39020654,39284263,00.htm

I can just see this whole ID card project heading in the same
direction as the NHS National programme for IT, which has become a
byword for disastrous IT projects. Not testing it properly is just
asking for trouble.

A recent survey suggests that many applications fail when they are
deployed. If you don’t plan for performance issues in advance,
during the development process, things can go pear shaped in the
production environment. Performance can be significantly affected
by network issues, for example: often, development takes place on a
LAN, but the production environment is a WAN. If you don’t test in
an environment as much like the production environment as possible,
you’re just not going to find the problems.

The report says “perhaps the most telling statistic from the survey
is that most IT departments (71 per cent) seem to rely on end users
calling the help desks to alert them that performance problems
exist. This means problems are only reported after their impact is

In other words, use your users as testers. They won’t mind, will

When it comes down to it, testing is useful. If you don’t look for
problems, you mightn’t find them until they are really

4. Blogging and wikis

I’ve started a new blog at It’s
likely that many, but not all, of the items in this newsletter will
be mentioned the blog first. There will also be things that appear in
the blog that don’t make it into the newsletter.

One of the GIRO working parties this year is on “Building an open
source ICA model”. If you’d like to join the working party, please
let me know. If you’re interested in what we’re doing, take a look
at our wiki at http:// You’ll need the
password (or a pbwiki ID) to edit it; again, let me know if you’d
like to contribute at that level, without committing to the working

5. Newsletter information

This is a monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
( Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long as
this notice is included.

To subscribe, email news-subscribe AT To unsubscribe,
email news-unsubscribe AT Send all comments, feedback
and other queries to news-admin AT (Change ” AT ” to
“@”). All comments will be considered as publishable unless you state
otherwise. The newsletter is archived at

Newsletter Old site

Newsletter Sep 2006

News update 2006-09: September 2006

1. Shock horror spreadsheet risk!
2. Are you aligned?
3. Whose risk is it again?
4. Newsletter information

1. Shock horror spreadsheet risk!

There’s been some coverage recently about how spreadsheets can pose
a security risk. Apparently spreadsheets are widely used to analyse
corporate data, and the systems and controls around them are not as
highly developed as they are around more conventional enterprise
systems. Well, who would have thunk it?

Security risk, check. Calculation risk, check. Data risk, check.
It’s entirely possible that a spreadsheet could use the wrong data
to perform the wrong calculation, the results of which would then
get into the wrong hands. Moreover, even if it doesn’t get into the
wrong hands, but just does everything else wrong, it’s most
unlikely to be clear to anybody just what exactly it _is_ doing.

At last week’s GIRO conference in Vienna, Caroline Butler-Yeats
gave us a very interesting talk on risk management issues for GI
actuaries. (GI means General Insurance in this context). She was
telling us what to do to make sure that we aren’t sued, or found
professionally negligent. She said that one of the main principles
was that we should be able to demonstrate to other people that what
we had done was reasonable, and that many of the spreadsheets she
sees in the course of her work do absolutely nothing to help her
clients (she’s a solicitor working in the area of PI). She used the
word “spaghetti” to describe some of the sets of spreadsheets she
has encountered.

It’s not going too far to say that some or even many of the
spreadsheets commonly used by actuaries are unprofessional. And all
the evidence shows that actuaries are using a lot of spreadsheets,
as two recent surveys demonstrate.

The first was conducted by the GIRO working party on software use
in general insurance, which I chaired. There were over 700
responses, from all over the world, and although there were no big
surprises there were some interesting results. Possibly the most
worrying issue to emerge was the lack of awareness of the
limitations of Excel; there are many actuarial users who don’t seem
to know that its statistical functions are unreliable, especially
in the tails of distributions.

The second survey was one that I conducted on the use of software
in ICAs. This was much smaller, with 45 respondents, and covered
both life and general insurance. It looked especially at the
systems and controls around software use. There was general
agreement that the systems and controls around spreadsheets are not
up to the standards of those around other user developed software,
such as actuarial models.

This is definitely worrying, especially in the light of the FSA’s
emphasis that the same standards should apply to spreadsheets as to
other software.

If you need help with your spreadsheets or other user developed
software, just let me know. I can advise you on systems and
controls, review current practice, and recommend development
processes for better risk management and productivity.

2. Are you aligned?

We all know that there are security holes in computer software, and
that we rely on the manufacturers to issue patches. Most obviously,
Microsoft issues patches on a regular basis. Microsoft is
especially important in this context as so many people use their
products: if something goes wrong, the number of people likely to
be affected is enormous. Of course, this also explains why so many
viruses, worms, and other nasties are aimed at Microsoft products.
It’s widely known as the “mono-culture” problem.

So, there are risks attached to using software. And it’s in the
users’ interests for software manufacturers to issue patches. But
is it in the manufacturers’ interests?

In many ways, the answer is “No.” If you issue a patch, you are
publicly admitting that there is something wrong with your product,
so it doesn’t do your reputation much good. You have to develop,
test and distribute the patch. There’s a chance that the patch
won’t work, which will annoy the users even more. On the other
hand, users are going to get annoyed if you don’t issue a patch and
they fall victim to some sort of malware attack. If you’re a
software manufacturer, it’s a tricky line to tread. That’s why
Microsoft issues patches monthly, rather than as soon as a problem
is discovered. Even if there’s a serious security hole, it’s most
unusual for it to be patched outside the regular monthly cycle.

On the other hand, if there’s a problem in a piece of software that
could lead directly to the manufacturer losing money, they are
likely to react pretty quickly. Recently, a hacker developed a way
of getting round the copy protection in Microsoft’s digital rights
management (DRM) software. This isn’t going to bother most users in
the slightest (rather the opposite, if anything), but it certainly
bothered Microsoft. A patch was issued three days after the hack
was discovered. Another hack was developed to circumvent the patch,
of course: I’m not sure what the current state of play is.,39020396,39282692,00.htm

The lesson is that if you rely on other people’s behaviour to keep
your risks under control, you’d better make sure that their
interests are aligned with yours. This applies at all levels, from
suppliers and outsourcers to other departments and colleagues.

3. Whose risk is it again?

Last month I commented on the laptop battery saga, saying that it
seemed that Sony was picking up the bill for the risk to the
reputation of Dell and Macintosh. A reader commented that “Dell and
Apple are simply enforcing the contract that they have with Sony.
Clearly Sony did not provide goods of satisfactory standard, and
must pay any remedial costs with making good their failure.” Well,
yes, but… The but in this case is in the definition of failure.
Failure is defined in such a way as to protect the reputation of
the outsourcer, which of course is absolutely the right thing to
do, from the perspective of the outsourcer.

In the last few weeks the saga has continued. There’s a recall out
on some Toshiba batteries manufactured by Sony, although Toshiba
has said that it’s not related to the Dell and Apple recalls.
Although the batteries might fail, there is no risk of fire,
apparently. The numbers involved are pretty small: 340,000
batteries are affected, compared to nearly 6 million Dell and Apple

Then just last week a recall was issued on half a million Sony
batteries used in ThinkPad laptops. One of them burst into fire at
LA airport, “causing enough smoke and sparking that a fire
extinguisher was used to put it out,” according to the US Consumer
Product Safety Commission. If this goes on it’ll only be a matter
of time before one ignites actually on an aeroplane, instead of
just on its way onto one. Sony is now starting a replacement
programme for laptop batteries that meet certain manufacturing
criteria, regardless of the laptops that they are used in.

Earlier in the month Matsushita (makers of Panasonic laptops)
recalled 6,000 batteries that might overheat if they are dropped
repeatedly. These batteries are not made by Sony.,39020351,39282681,00.htm

Over the past few years, we’ve also seen battery problems with
cameras and mobile phones. Some of these problems probably occur
because the limits of battery technology are being pushed.
Batteries have got much smaller and lighter over the last few
years, and last longer before needing to be recharged. (Of course,
much of the improvement is due to reduced power consumption by the
appliances that the batteries power.)

However, apparently some computers use Lithium-ion batteries
outside their design envelope. The recharging cycle is much faster
than usual. When the batteries are exposed to rapid charging, it is
possible for metal fragments to be formed. The fragments can then
cause major short circuits and thus over-heating. This is why it’s
not cut and dried that Sony will have to cover the whole cost of
the recall. Sony is also saying that the risk of fire “can be
affected by variations in the system configurations found in
different notebook computers.” Apparently Sony engineers haven’t
been able to reproduce the battery failures, lending credence to
the view that the way the batteries are being used may be at least
partly at fault.

All this just goes to show the importance of getting outsourcing
agreements right, from both sides. It now looks as if Sony’s
reputation may be suffering because of the actions of the laptop
manufacturers, rather than vice versa.

4. Newsletter information

This is a monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
( Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long as
this notice is included.

To subscribe, email news-subscribe AT To unsubscribe,
email news-unsubscribe AT Send all comments, feedback
and other queries to news-admin AT (Change ” AT ” to
“@”). All comments will be considered as publishable unless you state
otherwise. The newsletter is archived at

Newsletter Old site

Newsletter Aug 2006

News update 2006-08: August 2006

1. Whose risk is it anyway?
2. Replacing spreadsheets to reduce risk
3. Preaching to the converted
4. It’s everywhere
5. Newsletter information

1. Whose risk is it anyway?

There’s been a fair amount of press coverage recently about Dell laptop
batteries bursting into flames. There are even video clips on the web,
to convince us that it really is happening. Spontaneous ignition is
clearly undesirable behaviour as far as batteries are concerned, and
Dell now has a product recall in place. Apparently it’s the largest
computer product recall ever.

Like many manufacturers, Dell outsources the production of many
components, and it’s actually Sony that made the batteries in question.
Sony also supply batteries for a number of other makes of computer, but
so far only Apple have issued a recall. So, either the fault was
confined to batteries destined for Dell or Apple machines, or there are
a whole lot of other computers out there that might burst into flames.
In fact, there are going to be a whole lot of computers out there that
might burst into flames anyway, as it seems unlikely that all the owners
of machines that are affected will actually replace their batteries.

So, who’s bearing the risk? Or, possibly more importantly, who’s
suffering the consequences?

Dell, Apple and Sony are clearly suffering at least some consequences. A
product recall on this scale costs money, and there’s the damage to
reputation, too. It appears that much of the monetary cost will be borne
by Sony, while Dell and Apple will take most of the reputational hit.
However, it’s widely thought that by acting promptly, and acknowledging
that the problem exists, the reputational damage has been minimised.

It could be argued, though, that the recall is unnecessary. Nobody has
been killed, and it seems that there have been about 10 or 15 incidents,
out of nearly 2 million batteries. There are many everyday dangers
around daily activities that are much more likely. On this view, the
only rational reason for the product recall is to limit the reputational

What this means is that Sony is paying the price of protecting Dell’s
and Apple’s reputations. This is an interesting twist on what is usually
thought to be the situation with outsourcing. The standard line goes
that it’s often the outsourcer who has to carry the can when something
goes wrong at the supplier’s end. In this case, it could be argued that
the supplier is carrying the can to protect the outsourcer.

2. Replacing spreadsheets to reduce risk

“Australian Pharmaceutical Industries revealed yesterday that it had
lost $17 million – and a managing director – but was unable or unwilling
to discuss the details behind either.” That’s the start of a report in
The Australian, which explains that API found unreconcilable
discrepancies between their three computer systems. They’ve apparently
been integrating the three systems over the last three years.

The same issue of The Australian contained a feature article on how API
successfully moved from spreadsheets to a specialist tool. Previously,
there had been one system running sub-ledgers, another running the
general ledger system, and spreadsheets were used for budgeting and
performance reporting. “The spreadsheets were difficult and very
labour-intensive and slow to respond to change.” Doug Horwood, the
information management leader, went on to say, about using the new tool,
“We got the core budget information entered and were able to do
bottom-up and top-down adjustments, and the aggregation of all the
different cost centres literally turned into minutes rather than the six
to 12 hours it used to take. It was also fraught with the possibility of
manual error in the processing.”

So, the new system improved things as far as budgeting was concerned,
but was apparently not an improvement on the financial reporting front.

Horwood had some interesting things to say about their previous use of
spreadsheets. “In the past it was such a horrendous job to change all
the spreadsheets, that in effect we only ever adjusted the budget at a
very high level.” Spreadsheets are usually thought of as being extremely
flexible; one of the advantages touted for their use is that they are so
easy to change. This is certainly true of individual spreadsheets, but,
as Horwood realised, is rarely true of whole systems built out of
spreadsheets communicating with each other. The trouble is that it’s
very difficult to make nice abstract interfaces between spreadsheets,
that are independent of the details of the internal workings.
Spreadsheets were designed as personal productivity applications, and
although they are often used for enterprise level systems it’s very hard
to make them really work well in that context.

If you are using large systems built from spreadsheets, and rely on them
for mission-critical reporting or modelling, you should make sure that
you are aware of the risks. Have you thought about whether spreadsheets
are really the appropriate technology? Have you reviewed the spreadsheet
techniques that you are using, to make sure that you are managing the
risks effectively? And do you have good systems and controls around the
development and use of the spreadsheets? If you’d like to know how I
could help you answer these questions, and more like them, please get in
touch either by replying to this email, or contacting me through my web

Climate change actuary

3. Preaching to the converted

We are always being told how important it is to make sure that we
protect our computers from viruses, spyware, and other nasties. “You
must have anti-virus software!” “You must have a firewall!” It’s all
true, of course, but what happens when the software doesn’t work? Or
when it works wrong?

One of the most useful characteristics of security software is that you
should be able to trust it. In particular, you don’t want it to start
removing perfectly harmless, but very useful, programs that you have
installed on your computer. Many Church of England clergy had a nasty
shock earlier this month when Symantec’s Norton Antivirus software
wrongly identified an innocent file as a piece of spyware. The software
prompted them to remove the file, which was in fact a vital component of
Visual Liturgy, an application that is used to choose services, plan
Bible readings and create booklets.,39020375,39280391,00.htm

The vendors of Visual Liturgy advised their users to ignore the Norton
warning, so that they could continue to use Visual Liturgy. Meanwhile,
they tried to contact Symantec to get the Norton program fixed. There is
some disagreement about how effective this was, but eventually
everything was sorted out.

Of course, it’s not unusual for there to be errors in computer programs,
and however careful Symantec is to test and review the Norton program
they can never be sure that a bug won’t slip through. Moreover, Symantec
can hardly be expected to test it against every single piece of
legitimate software available on the planet.

However, there are now many users who may not trust Norton warnings in
future, and who will be very wary of deleting files just because Norton
tells them to. So the overall effect is probably to decrease computer

4. It’s everywhere

Google is starting to reach the parts that only that other software
giant, Microsoft, can reach. As well as providing web-based searching
services, and desktop tools, they are now providing web-based tools,
including a spreadsheet and word processor. Soon, you’ll be able to use
Google to do just about everything you need to do on a computer. Or, at
least, that’s the impression Google would probably like to give. The
latest offering is a service providing corporate email, instant
messaging, calendar and web page creation. “Google Apps for Your Domain
lets you offer our communication and collaboration tools to your entire
organization – customizable with your branding, color scheme and content
through the administrative control panel, and with no hardware or
software to install or maintain.”

It’ll be interesting to see whether this takes off. It’s only in beta at
the moment, is free, and isn’t available to everybody. Presumably they
won’t accept any large organisations until they’ve got the premium (ie
paying) service up and running.

One of the big issues, both with Google Apps and with other tools, is
that of privacy. Should organisations be wary of storing confidential
information on Google’s servers? Or using Google desktop tools on
machines that might have access to confidential information? This is
probably a judgement call that has to be made by each organisation for
itself. Google have privacy policies, obviously, but organisations will
have to decide whether they are adequate, whether they trust Google to
abide them, and whether they trust Google not to change them in the

As with the Google spreadsheet, the real market might be not in web
based applications, but in applications for the intranet. If that
happens with Google Apps, Google really will be going head to head with
Microsoft on its home turf.

5. Newsletter information

This is a monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
( Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long as
this notice is included.

To subscribe, email news-subscribe AT To unsubscribe,
email news-unsubscribe AT Send all comments, feedback
and other queries to news-admin AT (Change ” AT ” to
“@”). All comments will be considered as publishable unless you state
otherwise. The newsletter is archived at

Newsletter Old site

Newsletter Jul 2006

News update 2006-07: July 2006

1. Software in ICAs
2. Is it safe?
3. Don’t try this at home
4. Measuring risk
5. Newsletter information

1. Software in ICAs

Many thanks to everyone who has completed the online survey about
software use in ICAs, at

If you haven’t completed the survey this is your last chance, as the
survey closes on 1st August. It should only take about a quarter of an
hour, and the more responses there are the more useful the results will
be. All participants will get a full analysis of the results, and
responses are confidential.

2. Is it safe?

There’s an interesting piece in the New Scientist, by Henry Petroski,
about how apparent success can mask flaws that will lead to disaster. He
is writing about engineering flaws, but there are lessons for the
software world as well.

Imagine that the Titanic had not struck an iceberg on its maiden voyage:
it would have had a triumphal arrival in New York, and other ships would
have been built to similar designs. The designs would have been adapted
to use smaller engines and lighter hulls. They would have continued to
be thought of as “unsinkable” and at the leading edge of technology as
long as none of them did actually sink; and that state would probably
have lasted for some time. It was only luck that the Titanic happened to
strike an iceberg when it did.

Similarly, foam fell off nearly all space shuttles as they were launched
before 2003; because no damage was done, it was accepted as a routine
event. When foam fell off Columbia, it wasn’t immediately perceived as a
potential threat.

As Petroski says, “latent systematic weaknesses in a design are most
likely to be revealed through failure.” The fact that something has
worked well for a long time is not in itself a guarantee that it will
continue to do so: it may be that the circumstances in which it won’t
work just haven’t happened to arise.

It’s obviously a lot easier to test things in the software world: it’s
possible to run a program with many different sets of inputs, whereas
launching the space shuttle many times is not feasible. However, just
because testing is easier it doesn’t mean that it’s actually done.

In addition, there’s the “it won’t happen” syndrome. The roof of a road
tunnel in Boston collapsed recently; apparently an engineer warned of
the potential problem back in 1999. He was told that the method used to
support the ceiling panels was tried and tested, and that the work was
being done to design specifications and that the ceiling would hold.
Petroski says that it’s important to have “a healthy scepticism about
built things, and an awareness that apparent success can mask imminent

The point about unlikely events is that they don’t happen very often. A
piece of software that has worked many times before isn’t necessarily
going to work in the future. It may get a combination of inputs that has
never occurred before. The best defence is to perform thorough testing
and code review.
(requires subscription)

If you want to know how to go about testing and reviewing your
spreadsheets and other user developed applications, please get in touch.
Having good processes in place not only makes the applications more
reliable, it also reduces the time required to develop them.

3. Don’t try this at home

Or, at least, don’t do things on live data unless you’ve tested it out
first. It’s a sensible and common safety precaution, though probably not
common enough. Always make changes on a copy of your spreadsheet, so
that you can go back to the last known working version. If you’re
operating on a database, don’t try things out on the production system:
you might corrupt or destroy data as well as making the system

Even if you’re aware of the problems, things can still go wrong.
PlusNet, a UK ISP, recently deleted more than 700GB of its customers’
e-mail and prevented about half its 140,000 users from sending and
receiving new e-mail. Things went horribly wrong during the process of
bringing a new back-up storage server online. Instead of deleting the
contents of the old back-up disks, the engineer deleted the live
storage. PlusNet explained what happened: “At the time of making this
change the engineer had two management console sessions open — one to
the backup storage system and one to live storage. These both have the
same interface, and until Friday it was impossible to open more than one
connection to any part of the storage system at once. The patches we
installed on Friday evening removed this limitation, but unaware of
this, the engineer made an incorrect presumption that the window he was
working in was the back-up rather than the live server. Subsequently the
command to reconfigure the disk pack and remove all data therein was
made to the wrong server.”

PlusNet have called in data recovery engineers, but nearly 3 weeks after
the disaster the missing emails haven’t yet been restored. It took
nearly 2 weeks for email service to be restored for all customers.

With hindsight, of course, it would have been better had the management
console sessions indicated clearly which storage system they were
operating on. Before the patches, it appears that this wasn’t strictly
speaking necessary, but it’s always dangerous to rely on assumptions of
the form “this couldn’t possibly happen” (or, even worse, “no reasonable
user would…”). Of course it’s often difficult to recognise that type
of assumption, as they often aren’t explicit.

4. Measuring risk

I usually enjoy the “Gavyn Davies does the maths” column in the Guardian
on Thursdays, and this week was no exception. Apparently punters do
better, long term, by betting on favourites rather than long shots. They
still lose money, but they lose less money. As Davies points out, this
is unexpected: in a rational world, punters would refuse to back the
long shots until the odds were more realistic, removing the anomaly. He
suggests three possible explanations, favouring the one that says that
the whole market really does have a misperception about risk, failing to
appreciate that a there is a major bias in the odds. In other words, the
risk measure (the odds) isn’t an accurate reflection of the actual risk.,,1830876,00.html

The same sort of thing may be happening in the financial world.
According to the Bank of England’s Financial Stability Report, trading
profits have recently risen as a proportion of income at UK commercial
and global investment banks. however, the risk taken on by the banks, as
measured by value at risk (VaR) has risen by much less. This is
surprising because it’s commonly accepted that in order to make more
money, you have to take on more risk. The Bank presents two possible
explanations, but favours the second, that VaR just isn’t a good measure
of risk in the trading book. Unfortunately, it’s very widely used, and
is the only risk metric that banks must disclose, even though they may
well use others internally. This may mean that more weight is put on it
than is justified.

The trouble is that measuring risk is difficult, and, like many
phenomena, there is no single metric that is adequate. You can’t measure
buildings just by their height: other dimensions provide other
information, and you can only get the full picture by looking at them
all. Two towns with the same population may be very different in other
respects. VaR measures only the maximum loss likely to occur over a
given period at a certain level of probability. It doesn’t measure the
expected loss, or even the maximum loss at a different level of
probability or over a different period.

5. Newsletter information

This is a monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
( Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long as
this notice is included.

To subscribe, email news-subscribe AT To unsubscribe,
email news-unsubscribe AT Send all comments, feedback
and other queries to news-admin AT (Change ” AT ” to
“@”). All comments will be considered as publishable unless you state
otherwise. The newsletter is archived at

Newsletter Old site

Newsletter Jun 2006

News update 2006-06: June 2006

1. Software in ICAs
2. The proof of the pudding is in the testing
3. Portable insecurity
4. Sharing spreadsheets
5. Newsletter information

1. Software in ICAs

Many thanks to everyone who has completed the online survey about
software use in ICAs, at

If you are involved in the ICA process, at whatever level, I’d
appreciate it if you could fill the survey in: it should only take
about a quarter of an hour, and the more responses there are the
more useful the results will be. All participants will get a full
analysis of the results, and responses are confidential.

The survey concentrates on an overview of what types of software
are used in ICAs, and the general level of systems and controls
that are in place. Responses saying that you don’t know are equally
as valuable as those that give rankings. I hope that the results of
the survey will provide an impression of the general level of
confidence in existing systems and controls, and how broad the
range of practice is. It will also be interesting to see how much,
if at all, practice differs between life and non-life insurance

Here’s that link again:

2. The proof of the pudding is in the testing

Why should you test software? It’s time consuming, and therefore
often expensive. If you could prove that a program was correct,
surely you wouldn’t need to test it. And maybe a thorough review of
the code would mean that you wouldn’t need to run thorough tests.

Dream on. It would be lovely if it were true, but it ain’t, as two
recent episodes demonstrate.

An algorithm that has been accepted as correct for years (proved in
a book in 1986; probably proved elsewhere before that) has been
found to have a bug. It’s a widely used sorting algorithm, that’s
been used in many different places, including the implementation of
Java. It only fails in somewhat unusual circumstances, when you’re
sorting an array of more than about 2^30 elements (that’s about a
billion). When the proof was published, back in the 1980’s, the
idea of sorting such large arrays was inconceivable. The bug was
recently reported after being encountered in practice in Java, nine
years after the relevant code was released, and 20 years after the

So what went wrong? The algorithm contains the following line:

int mid =(low + high) / 2;

which takes the average of two numbers, and truncates it to an
integer. This doesn’t work when the sum of low and high is greater
than the maximum positive integer value (2^32 – 1). In this case
the sum overflows to a negative number, and all hell breaks loose
(or at least exceptions are thrown or unpredictable things occur,
depending on the programming language).

There are some simple ways of changing this line so that it always
works: for example,

int mid = low + ((high – low) / 2);

works as long as both high and low are positive integers less than
2^32 – 1, and high is greater than low (which is guaranteed
elsewhere in the algorithm).

The proof relied on implicit assumptions about the size of the
array being sorted. Those assumptions held in practice at the time
that the proof was first published, but have become unrealistic in
the years since that time. A proof always relies on assumptions,
and can only be trusted as far as the assumptions hold in
practice. As Yogi Berra once said: In theory there is no difference
between theory and practice; in practice, there is. Which is why
there is no substitute for actually testing the practical

This is a lesson that NASA engineers have learned to their
cost. The Genesis space probe crashed in 2004, as it was bringing
back solar wind particles for analysis. Its parachutes failed to
open, because the gravity switches that should have operated them
were installed backwards.

There was a test that would have uncovered the fatal flaw, but it
wasn’t performed because of time constraints. It was thought that
comparing the design drawings to those of a previous, successful,
spacecraft, would do instead.

Again, this shows up the risks of relying on theory rather than
practice. Comparing the design drawings assumes that the
implementation (in this case, the physical spacecraft) corresponds
exactly to the theoretical model (the design drawings). This is a
big assumption to make, and in this case it turned out to be

I see the same pattern occurring with spreadsheets and other user
developed software, including actuarial models. Even if the
specification is correct, the implementation may not be; and a
thorough review of the code doesn’t guarantee that it will work in
practice. Code reviews and theoretical proofs have a role to play,
of course, but can’t replace proper testing of the actual

If you’d like help in setting up a systematic testing process for
your user developed software, just let me know by replying to this
email, or contacting me through my web site at

Climate change actuary

3. Portable insecurity

There are (or should be) some very red faces at the Federal Trade
Commission, the US body responsible for protecting citizens from
identity theft and fraud. Two FTC laptops, containing the personal
details of 110 people, have been stolen.

This is just the latest in a long string of stories about stolen
laptops containing personal details: just enter “laptop personal
data” into Google to see some of them.

40,000 BP workers on an Ernst and Young laptop… 13,000 people on
an ING laptop… 243,000 hotel guests… 68,000 YMCA
members… it’s a lot of people. In many cases the laptops were
probably stolen as hardware, rather than for the information they
held, but that doesn’t really lessen the risk. In some of these
cases we are told that the information was password protected, but
that may not mean much. The password protection in Excel, for
example, is notoriously weak. There are any number of tools
available to crack Excel passwords.

All the stories we read in the press are about the loss of laptops
containing personal details, but I’m willing to bet that there are
a lot of laptops out there that contain highly confidential
business information: business plans, marketing plans, ICA
calculations, and so on. Now, it may be rational behaviour: the
chance of losing a laptop is actually quite small, and the chance
that the casual thief would be interested in such information, or
know what to do with it, must be even smaller.

The circumstances surrounding all these incidents will vary, but I
imagine that in at least some of them security guidelines were
breached. Company guidelines may limit the type of information that
can go onto laptops, but that doesn’t necessarily stop people
putting it there. Their own convenience wins out. When it comes
down to it, all the guidelines in the world aren’t going to make a
difference if they are ignored by the people on the ground. In
fact, this is just part of a wider problem: many security breaches
are not technology driven at all, but happen because people write
their passwords down, leave laptops lying around, or otherwise
lower their guard.

What looks like an interesting book, “Information Security and
Employee Behaviour,” discusses the issues involved and presents
some solutions. From the review I have read it takes a fairly
balanced view, discussing why why security people get fixated on
their field, and often over-emphasise minor problems. As with all
areas of risk management, one of the problems is that people just
aren’t very good at assessing risk.

Another problem is that the risk to the laptop owner is not the
same as the risk to the people whose personal details are on the
laptop. Identity theft is both disturbing and time consuming to
fix, for the victims, but losing a laptop, while annoying, doesn’t
really have the same effect.

4. Sharing spreadsheets

It’s all go on the spreadsheet front at the moment, with Microsoft
releasing a white paper on the next version of Excel (see last
month’s newsletter) and Google Spreadsheets making an appearance.

Here’s what Google says about Google Spreadsheets:

1. Share spreadsheets instantly & collaborate real-time.
– Pick exactly who can access your spreadsheets.
2. Edit your spreadsheets from anywhere.
– Nothing to download — your browser is all you need.
3. Store your spreadsheets securely online.
– Offsite storage plus multi-site data backup.
4. Easy to use.
– Import your current spreadsheets to get started quickly.
– Clean, uncluttered screens with a familiar, desktop.

All this sounds good, but there are some drawbacks.

First, the functionality isn’t complete. It’s a pretty basic
spreadsheet, which is fair enough for a first release. It seems to
have all the same actual functions as Excel, but has no macros, you
can’t print your spreadsheets out, there’s only limited formatting
(no cell borders, no conditional formatting), no defined Names, no
data tables, no pivot tables, no objects (charts, text boxes,
shapes, etc) …

Some of these omissions are undoubtedly because of the limitations
of browser technology. You just can’t manage the user interface
through a browser in the same way that you can with a purpose built

Second, it doesn’t work with all browsers yet. It’s fine with
Firefox and Internet Explorer, but Opera isn’t supported. I suspect
that it doesn’t work well with text to speech software, either.

Third, there’s the security problem. The spreadsheets are stored
on Google’s servers. Whatever the safeguards, it’s just not the
same as having them on your own machines.

Fourth, it’s all very well sharing a spreadsheet, and there is some
control in that the creator is the only person who can give other
people access (either read-only or editing), but once you have
several people being able to change a spreadsheet you want to be
able to track the changes. Of course, this is a drawback with
Excel, too, but the problem will (apparently) be addressed in the
next release.

For a first release, though, it’s pretty impressive, and easy to
use. No doubt they’ll come up with some ways of addressing some of
the functionality issues. The sharing is particularly good for
collaborative working, with a chat window so that you can talk
about the changes you are making.

It wouldn’t surprise me if it was eventually sold for use on
company intranets, which would deal with the security issue. And
with some sort of version control, so you could lock down a version
after a spell of collaborative work, and assuming that they find a
way to add more functionality, it could be very useful.

5. Newsletter information

This is a monthly newsletter on risk management in financial
services, operational risk and user-developed software from Louise
Pryor ( Copyright (c) Louise Pryor
2006. All rights reserved. You may distribute it in whole or in
part as long as this notice is included.

To subscribe, email news-subscribe AT To
unsubscribe, email news-unsubscribe AT Send all
comments, feedback and other queries to news-admin AT (Change ” AT ” to “@”). All comments will be
considered as publishable unless you state otherwise. The
newsletter is archived at

Newsletter Old site

Newsletter May 2006

News update 2006-05: May 2006

1. Software in ICAs
2. Spreadsheet compliance
3. Crying wolf
4. Accountability
5. Newsletter information

1. Software in ICAs

Over the last few years there have been some big changes in how
insurance companies are run, not the least of which has been the
introduction of ICAs, or Individual Capital Assessments. Insurance
companies now have to calculate the risk-based capital they believe
they need; the result of the calculation is used by the FSA in the
determination of the company’s ICG, or Individual Capital Guidance,
which is the level of capital that the FSA believes they need. The
ICA calculation is now a very important one to most insurance
companies, and is treated very seriously.

Each company approaches their ICA calculation in a slightly
different way, but the one thing that all the calculations have in
common is that they use end-user software: software such as
actuarial models and spreadsheets that is developed by the
end-user, rather than farmed out to the IT department. But what
software? And how is it being used?

That’s where you can help. If you are involved in producing an ICA
for an insurance company, whether it’s a life or non-life company,
whether you are running the whole ICA effort or are a cog (but
all cogs are vital) in a well-oiled machine, or anything in
between, please take part in my online survey. You’ll find it at, and it should
only take about 10 minutes of your time. All participants will
receive a full analysis of the results.

The survey looks at what software is being used, and, equally
importantly, the systems and controls that are in place around
it. The FSA has stressed that the systems and controls are
important: it’s no use simply producing a number and expecting it
to be believed, you’ve got to be able to give the reasons why other
people should have confidence in your results. Other people might
entertain doubts about their validity unless you can:

– demonstrate a full audit trail from data and parameters through
to the final results
– provide details of the tests and reviews that your models have
– demonstrate that the results come from the versions of your
models that have passed the tests and reviews
– provide clear documentation of your models, including the
assumptions on which they are based

I’ve written a brief paper on ‘How to believe your models’, which
is available at

The survey should provide some idea of the current levels of
systems and controls that are in place. If you take part, you’ll be
able to compare your standards with those of your peers
(anonymously, of course).

2. Spreadsheet compliance

I would not be at all surprised to learn that spreadsheets are used
in the preparation of every single ICA. In fact, I’d be interested
to hear of an ICA whose calculation didn’t involve a spreadsheet;
and if I was told of one, I’d want to trace all the calculations
through from start to finish before I believed it. There is no
doubt that spreadsheets are a vital tool in business today, and
that the most widely used spreadsheet is Microsoft’s Excel.

Over the years many people have pointed out that although Excel is
used for many mission critical applications, it is extremely
difficult to apply the same systems and controls to spreadsheets as
one would to more conventionally developed software. Of course,
Excel is by no means the only end-user software in this position.

However, it appears that the situation is going to change. There’s
a new version of Excel due out next year, and Microsoft have
recently issued a White Paper that discusses some of the new
features that it will contain. It is called “Spreadsheet Compliance
in the 2007 Microsoft Office System” and can be found at There’s a blog entry about
it at
The paper is definitely recommended reading. It’s good to see that
Microsoft are taking compliance issues seriously.

The paper, quite rightly, stresses the importance of having good
process in place. “One common misconception in organisations is
that spreadsheet compliance can be achieved through the use of
technology. While technology plays a role in any compliance
strategy, the most important component is process. Critical
spreadsheets and other enterprise IT resources require sound
development and usage practices that include controlled testing,
deployment, maintenance, and use.” Having said that, and after
describing some of the elements of a good process, it goes on to
describe new features in Excel and Office that will support a sound
process. To me, one of the most interesting is good support for
versioning. It will also be much easier to restrict the circulation
of spreadsheets, based on whether they have been approved for wider
use. There will be other new features within Excel that are
intended to make it easier to use consistent coding practices and
standards while developing and maintaining spreadsheets.

Obviously, it remains to be be seen how easy it will be to make use
of all this new functionality in practice. And however easy it is,
there will have to be a willingness to use it if working practices
are to change. Moreover, given that there is still a significant
proportion of people using Excel 97, we have to wonder what the
take-up rate of Excel 2007 will be.

Brandon Weber of Microsoft will be giving a talk on some of the
issues and features discussed in the White Paper at the EuSpRIG
conference in Cambridge at the beginning of July. His talk fits in
nicely with the theme of this year’s conference, which is ‘Managing
Spreadsheets: Improving corporate performance, compliance and
governance’. Other topics covered by papers that will be presented
at the conference include:

– Assessing current spreadsheet use
– Risk and other classification systems
– Proving effectiveness
– Available control techniques
– Planning which kind of techniques fit which risks
– Maintaining integrity and compliance
– Discovering and promoting training resources and good practice

Full details of the conference, including a registration form, are
at Early registration
is advised, as accommodation in Cambridge is scarce and expensive at
that time of year.

Although the forthcoming enhancements to Excel are very welcome,
they are not here yet, and people are using Excel now, at this very
moment. Waiting until next year in order to implement basic systems
and controls is not an option. Some organisations are using Excel
competently, but many more are either complacent or compromised.
Which are you? Take a 30 second quiz to find out, at

3. Crying wolf

Every so often there’s a furore about cash machines charging people
to withdraw cash; popular opinion is pretty firm that they should
be free. According to a recent survey a surprising number of users
don’t realise that they’ve been charged a fee even when they’ve
been warned on the screen and have had to confirm their acceptance
of the charge — up to 15% of users.

But is it surprising? Often people are multi-tasking as they take
cash out: carrying on a conversation, chatting on their mobile, or
thinking about what they are going to do with the money. Besides,
many of the machines that charge are in pubs or clubs, so their
users aren’t necessarily on tip-top intellectual form. We all know
how easy it is to click on things on the screen without really
taking them in; it happens all the time with warning screens in
desktop applications. Are you sure you want to open rather than
save? Are you sure you want to delete this? It’s all too easy to
use the default options without thinking, and find that you have
overwritten a vital file or lost important information.

There isn’t really any reliable way of making sure that the user is
doing what they intend to. Thought reading would be nice, but just
isn’t possible. We have WYSIWYG (What You See Is What You Get)
interfaces, but DWIM (Do What I Mean) interfaces are still a long
way off. Good interface design is hard.

Many of the big computer error stories that make it through to the
headlines are actually stories about the risks of poorly designed
interfaces: ones that lack data validation, reasonableness checks,
or just make it too easy to do the wrong thing. But even well
designed interfaces aren’t going to eliminate all mistakes; all
they can do is reduce the risks. Too many warnings can be as
dangerous as too few.

Just to make the obvious point, interface design is important in
end-use computing, too. If someone else is going to use your
spreadsheet, have you make it obvious what information they should
enter? Is the information checked for reasonableness? Do you make
it hard for other users to overwrite vital calculations, or miss
important parameters? If you have a sneaking suspicion that your
spreadsheets or other user-developed applications could use some
improvement, please contact me either by replying to this email or
through my web site at

4. Accountability

Australian bank ANZ suffered a major credit card processing failure
earlier this year: 200,000 credit card holders were accidentally
charged twice. They refunded A$45 million to those who were
affected. Although there was some publicity, it’s unlikely that
their reputation will suffer unduly: none of the card holders in
question were their own customers. The problem affected only
non-ANZ cardholders using ANZ eftpos terminals. Transactions for
ANZ cardholders were handled by a different database, which was
operating normally.

A normal customer, paying by credit card in a shop, isn’t aware
who supplies the eftpos machine they are using. From their point of
view, they are dealing with the shop, or with their own credit card
issuer. When problems arise, those are the people who are going to
get the blame. Indeed, one report mentioned that many cardholders
were blaming the retailers — who of course had nothing to do with

It is often thought that one of the drivers for good customer
service is the reaction of customers: bad service will drive them
away. How does that work in a case like this? I think the pressure
is still there, but at second hand, and somewhat attenuated. It is
the retailers who are ANZ’s customers; it’s the retailers who got
bad feedback from the cardholders; and it will be the retailers who
vote with their feet, by changing to another bank for their eftpos
facilities. Of course it’s not as easy as all that to change, as
eftpos is just one among many services that small businesses get
from their banks. If the affected cardholders had been ANZ
customers directly, it’s likely that some of them would have
switched to different cards, which is altogether easier than a
retailer switching to another eftpos provider.

So, to some extent, the moral of the story, if you are in the
credit card business, is to make sure that any problems don’t
affect your own cardholders. Indeed, in this situation it’s
possible that some cardholders will change card issuers as a
result of this, even though it wasn’t their issuer that made the
mistake. Who knows, some of them may even change to ANZ.

5. Newsletter information

This is a monthly newsletter on risk management in financial
services, operational risk and user-developed software from Louise
Pryor ( Copyright (c) Louise Pryor
2006. All rights reserved. You may distribute it in whole or in
part as long as this notice is included.

To subscribe, email news-subscribe AT To
unsubscribe, email news-unsubscribe AT Send all
comments, feedback and other queries to news-admin AT (Change ” AT ” to “@”). All comments will be
considered as publishable unless you state otherwise. The
newsletter is archived at

Newsletter Old site

Newsletter Apr 2006

News update 2006-04: April 2006

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor

Comments and feedback to Please tell me if
you don’t want to be quoted.

Subscribe by sending an email to news-subscribe AT
Unsubscribe by sending an email to news-unsubscribe AT (Change ” AT ” to “@”). Newsletter archived at

In this issue:
1. Oops!
2. Documentation
3. Is that what you meant to read?
4. A horseshoe nail
5. Newsletter information

1. Oops!

The emails have been flooding in… Did you really mean… Shurely
shome mishtake…

Well, yes, it was indeed a mistake, and some people noticed (half a
dozen emails, not quite a flood I admit). This is the second
newsletter that most of you have received from me this week; I am
sorry for clogging up your mailboxes unnecessarily. The first one,
a repeat of the March issue, wasn’t meant to happen. My processes
failed and have now been (I hope) fixed.

So what actually went wrong, and why? I run the mailing list using
a set of filters and templates in the email program I use, The Bat!
(the exclamation mark is part of the name). Some things, like new
subscribers and unsubscribers, are handled more or less
automatically; I don’t have to write or edit the outgoing email
message acknowledging receipt of the request at all. When
I send an actual newsletter out, there is a slightly complicated
incantation that has to go in the Bcc: field. I can never remember
exactly what the incantation is, so had set things up so that any
new message automatically contained the incantation. I then removed
it when writing a message that wasn’t an actual newsletter issue.

When somebody asked for a repeat sending of the March newsletter
yesterday, I wrote a new message, checked that I’d removed the
incantation from the Bcc: field, and fired the message off. It was
unfortunate that I didn’t double check, as evidently the check

My original reasoning had been that there would be very few
messages other than newsletter issues. This has nearly been borne
out in practice, though there are more one-off messages than I had
expected. The assumption that the reduced hassle for the newsletter
issues would outweigh the risk of sending an unwanted message out
to all recipients has become invalid as the number of people
receiving the newsletter has increased. So a couple of slightly
dodgy assumptions have gradually morphed into a couple of
definitely unsafe ones, without my processes changing accordingly.

I have now changed the default, so that I have to add the
incantation in by hand when it is needed. The risk of having to
resend an issue because it failed to reach the recipients is
clearly a better one than that of sending unwanted messages out to
all and sundry. Risk management in action, though not as effective
as it should be!

2. Documentation

It is a truth universally acknowledged, that software documentation
is a Good Thing, and spreadsheets are no exception. The FSA, in a
recent newsletter, described what they had seen in the way of good
practice for financial modelling systems: “Acceptable standards of
documentation were established, agreed by the firm, and themselves
documented.” They went on to say “The standards of control and
documentation applicable to systems developments applied equally to

But what is documentation for? It seems to be more or less assumed
that all documentation is worth while, and the more documentation
the better. However, given that most financial models and
spreadsheets are developed with limited resources, and writing
documentation takes time, it’s important to consider what forms of
documentation are most useful and productive. In order to do this
we must think about what we are trying to achieve through the
production of documentation.

Documentation can do four things: specify, record, explain, and
give instructions. These four types of documentation are of more or
less use to different types of user. Furthermore, different forms
of documentation (separate documents, as part of the user
interface, etc) are suited to convey different types of information
to the various types of user. I’ve put together a brief summary of
the options, which is available either as a single pdf document or
as a series of notes on my web site.

If you’d like to know more about how I can help you with good
documentation practices, just let me know!

3. Is that what you meant to read?

In previous issues I’ve talked about the problems of hidden data:
how you can inadvertently include information in documents that
isn’t intended to be there. There’s are other sides to the problem,
though: sometimes the reader sees things that they don’t want to,
or don’t see what they should.

For example, if you select some text in your browser, and paste it
in to another document, you expect to see the same text in the new
document as the text that you selected in your browser. Some web
pages have text in them that isn’t always displayed. There are
various reasons for this: for example, it may be displayed only in
some browsers (text-only browsers, or if javascript is disabled,
for instance). Usually, though, this text is pasted in to your new
document whether or not it is visible in the browser you are
using. This can be disconcerting, and could be worse than that; if
there is only a small difference between what you expect and what
you get you might not notice it, even if the sense is very

In some circumstances a single digit can make all the difference. A
recent article in the New York Times describes an incident in which
one partner erased the digit “1” in a Word document, giving another
partner a 5% share in a software firm that had just been sold,
rather than the 15% which was rightfully his. The article goes on
to discuss how computer forensics experts can detect that sort of
tampering and find out when it was done and by whom. It strikes me
as careless beyond belief to rely on a document in electronic form
for that sort of agreement: personally, I would want a signed hard
copy. The huge advantage of electronic documents for some purposes
is that they are so easy to change and revise; no more retyping
page after page for a single minor change. That’s also a big
disadvantage for other purposes, though. It’s a risk that is pretty
easy to foresee, and also one that should be easy to manage; don’t
rely on electronic documents to remain unchanged.

4. A horseshoe nail

Well, not quite a horseshoe nail, and it wasn’t a kingdom that was
lost, but small errors can have major consequences. The true value
(for taxation purposes) of a house in Valparaiso, Indiana, was
$121,900. Somehow some user of the County’s computer system managed
to accidentally change the value of the house to $400 million. The
error meant that the house’s tax bill rose to $8 million from
$1,500, thus radically changing the budget amounts of 18 government
taxing units.

Some interesting points arise. First, how on earth could an outside
user change the taxable value of a house? The official theory is
that “the user probably tried to access a real estate record
display by pressing R-E-D, but accidentally typed R-E-R, which
brought up an assessment program written in 1995. The program is no
longer in use, and technology officials did not know it could be
accessed.” That’s scary. Good design of the interface would check
for valid inputs; when the old program had been taken out of use,
the validation should have been changed. But far safer, when a
program is taken out of use, is to actually move it, so that any
links to it no longer work. Relying on changing or removing every
link is asking for trouble.

Second, how come nobody noticed? “The city of Valparaiso and the
Valparaiso Community School Corp. were asked to return $2.7
million.” Hadn’t they been surprised when the budget came in at a
different level from the one they were expecting? Or is $2.7
million a very small proportion of the total, in which case it
might be insignificant anyway, but this seems unlikely. Surely a
few reasonableness checks, or a simple analysis of changes, should
have rung some alarm bells somewhere.

Third, the county treasurer said his office “spotted the $400
million error after it caused an improper billing, but apparently
it wasn’t corrected elsewhere.” Sounds like a major communication
breakdown. It seems odd that an error can propagate through to 18
government taxing units, while a correction can’t.

There seems to be a general lack of sanity checks in all sorts of
systems. We hear of a publican in West Sussex who got a £600,000
telephone bill. Apparently a member of BT’s staff had “filled in
the wrong details in the wrong box”. The correct amount was £92,
which was exactly in line with the bills the publican normally
got. However, this is as nothing compared to the bill for 806
trillion Ringgit (about £126 trillion) received by a meat importer
in Malaysia. A few days later Telekom Malaysia said “It was a typo,
and it was not our mistake”–apparently it was all the fault of a
debt collection agency. Now, I get lost in the billion and trillion
range, but a quick search shows that a trillion has either 12 or 18
zeroes, depending. Why would a billing system even be able to issue
a bill for that amount of money?

It’s not just billing systems that should have data validation and
overall sanity checks. Any financial model or spreadsheet needs
them too. It’s possible to automate data validation, of course, and
some sanity checks, but it’s impossible to overestimate the utility
of a thoroughly cynical user. Instead of thinking “well, I’m pretty
sure the spreadsheet is correct so the answer must be right” you
should always be alert for anything that seems unusual, odd, or
even just unlikely.

If you’d like to find out more about adding data validation or
sanity checking into your spreadsheets or models, just get in

5. Newsletter information

This newsletter is issued approximately monthly by Louise Pryor
( Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long
as this notice is included. To subscribe, email news-subscribe AT To unsubscribe, email news-unsubscribe AT All comments, feedback and other queries to
news-admin AT (Change ” AT ” to “@”). Archives at

Newsletter Old site

Newsletter Mar 2006

News update 2006-03: March 2006

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor

Comments and feedback to Please tell me if
you don’t want to be quoted.

Subscribe by sending an email to news-subscribe AT
Unsubscribe by sending an email to news-unsubscribe AT (Change ” AT ” to “@”). Newsletter archived at

In this issue:
1. Farcical bid evaluation
2. Black box models
3. Data protection
4. Newsletter information

1. Farcical bid evaluation

“This is farcical; a spreadsheet error on a multi-billion pound
contract… ” and OCG Buying Solutions has “egg on its face”: these
are just two of the comments after the Government’s IT procurement
body admitted that the discovery of an error in a spreadsheet may
lead to some companies losing their accredited status in the new
Catalist procurement programme. Public sector managers can buy
goods and services more easily from accredited suppliers than
others, so losing accredited status can make a significant
difference to a company.

A leaked letter said “Unfortunately, [we are] not, as we had hoped,
in a position to accept your tender at this time…. This is
because an error in the original evaluation spreadsheet has been
identified, necessitating rescoring of all tenders for this
project… this error has now been corrected and this has caused a
small number of changes to the original award decision.”

This is unlike many of the spreadsheet errors that come to light,
in that it isn’t the organisation whose spreadsheet it is that is
suffering in this case. The OCG probably isn’t affected much if its
reputation takes a hit, and hasn’t lost a lot of money because of
the error. It is other stakeholders, ie the suppliers wanting
accreditation, who have the problems; and they can’t do anything
about it, except to go through a possibly lengthy appeals
process. Not surprisingly, they are a bit fed up, hence the quotes
at the start of this article.

We don’t know exactly what was wrong with the spreadsheet, but one
supplier believes that there was a formula error: too much weight
was placed on environmental factors, and not enough on pricing and
discounts. The OCG said “Every effort is made to ensure that
mistakes do not happen, and OGC Buying Solutions has a quality
assurance process that has been enacted in this instance.” Apart
from the rather mangled language in the second half of this
sentence, there are other criticisms to be made.

A successful quality assurance process would have caught the error
before suppliers were told that they had achieved accredited
status, rather than after. Making an effort to avoid mistakes in
spreadsheets is not enough: what is needed is an effective
development process, which includes appropriate testing and review
stages. If there are routine ways of performing testing and review
they are less likely to be skipped, and more likely to catch
errors. And spreadsheet errors can affect your reputation and hurt
others as well as losing you money.

2. Black box models

The latest Life Insurance newsletter from the FSA has a section on
Actuarial Systems and Controls. As it says, “Accurate output from a
life office’s actuarial area is important because any shortcomings
can potentially cost a firm dearly.” The FSA visited six firms, and
found quite a range of standards; but no firm was completely
satisfactory in every area. And the areas in which most improvement
was possible were those of documentation and spreadsheet use. The
FSA appears to have the following concerns:

– Systems and processes should have up to date documentation

– Every actuarial system should have a full audit trail

– Documentation and systems should have proper change processes and
version controls in place

– Documentation is especially important for systems supported by
just a few knowledgeable individuals

– Data and assumptions should be fully documented, consistent
across the business, and validated

– Financial models should be fully understood, and actuarial
departments should be able to demonstrate why their models should
be believed

– Spreadsheets should be taken as seriously as any other software
development environment.

Regular readers of this newsletter will recognise a number of these
themes. If you’ve read my short article on “Believing your Results”
it should all be very familiar. I find it interesting that the FSA
are now paying more attention to how the modelling results are
achieved, in the life insurance area at least. Models will
undoubtedly increase in both number and complexity as the ICAS
process becomes more mature in both life and non-life insurance,
and their care and feeding will become ever more important.

Just to state the obvious, I can help you to set up development
processes for financial models and spreadsheets, advise on suitable
methods of documentation, change processes and version control,
help you implement appropriate forms of testing and review for
models and spreadsheets, and provide training on these and related
issues. There’s information on my web site, and please get in touch
if you’d like further details.

Climate change actuary

3. Data protection

There’s an awful lot of data about each and every one of us hanging
around on various computer systems, and not all of it is very
secure. This matters both to the people whose data is held, and
those doing the holding (or, in some cases, letting go). The risks
are many in the areas of both fraud and privacy.

There seems to have been a minor epidemic of laptops going missing
recently. Both Fidelity and Ernst & Young have lost laptops
containing employee data from their clients.

E&Y appear to have lost at least five laptops, one of which held
data on BP, Sun, Cisco and IBM employees. It’s not clear how many
people are affected, but one source has estimated that it’s over
100,000 IBM employees alone. HP staff were told that the laptop
lost by Fidelity exposed 196,000 current and former HP
employees. The data apparently includes names, addresses, US Social
Security numbers, and tax identification numbers, providing an
absolute bonanza for identity thieves.

Laptops are inherently much less secure than desktop machines, as
it is much easier for unauthorised people to gain physical access
to them. Organisations should both have and enforce strong policies
on what data should be allowable on laptops, and should use
appropriate security measures, including strong encryption. Simple
password protection is unlikely to be effective. The password
protection in Microsoft Office products is particularly
ineffective, with password cracking programs readily available on
the internet. It’s not only personal data that is vulnerable, but
also commercially sensitive information: would you like your
business or marketing plans, or details of a planned takeover, to
be available to all and sundry?

Even personal data held on desktops or mainframes isn’t safe if the
wrong people get hold of it. Because of the physical access
problems, it’s usually an insider job. A call centre employee in
Leeds apparently stole confidential customer information from two
banks, enabling his gang to defraud the banks of over
£400,000. They targeted customers who didn’t use their credit cards
regularly, obtained their names and security codes, and then
impersonated them in order to change the account address and get a
new card and PIN.

Call centre employees are regularly targeted by identity thieves,
according to a story in The Guardian. They work the bars and pubs,
looking for low-paid call centre workers who might be willing to
make a bit on the side.,,1744116,00.html

Having good policies in place around the protection of data on
laptops isn’t enough to avoid problems; you’ve got to enforce the
policies too. A recent survey (by a company that sells web content
filtering) found that 70% of companies recognised that an
acceptable internet use policy was crucial to the security of their
IT systems, but 38% of employees governed by a policy are unaware
of its contents. 40% of respondents said that a policy was in place
but was not enforced.

The existence of a policy isn’t going to make any difference to how
things actually happen unless people know about the policy,
understand the point of it, and find it easier to comply with the
policy than to breach it. This applies just as much to software
development and use as to security issues.

4. Newsletter information

This newsletter is issued approximately monthly by Louise Pryor
( Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long
as this notice is included. To subscribe, email news-subscribe AT To unsubscribe, email news-unsubscribe AT All comments, feedback and other queries to
news-admin AT (Change ” AT ” to “@”). Archives at