Author: Louise Pryor

News update 2006-09: September 2006
===================

Contents:
1. Shock horror spreadsheet risk!
2. Are you aligned?
3. Whose risk is it again?
4. Newsletter information

===============
1. Shock horror spreadsheet risk!

There’s been some coverage recently about how spreadsheets can pose
a security risk. Apparently spreadsheets are widely used to analyse
corporate data, and the systems and controls around them are not as
highly developed as they are around more conventional enterprise
systems. Well, who would have thunk it?

http://naplogis.notlong.com

Security risk, check. Calculation risk, check. Data risk, check.
It’s entirely possible that a spreadsheet could use the wrong data
to perform the wrong calculation, the results of which would then
get into the wrong hands. Moreover, even if it doesn’t get into the
wrong hands, but just does everything else wrong, it’s most
unlikely to be clear to anybody just what exactly it _is_ doing.

At last week’s GIRO conference in Vienna, Caroline Butler-Yeats
gave us a very interesting talk on risk management issues for GI
actuaries. (GI means General Insurance in this context). She was
telling us what to do to make sure that we aren’t sued, or found
professionally negligent. She said that one of the main principles
was that we should be able to demonstrate to other people that what
we had done was reasonable, and that many of the spreadsheets she
sees in the course of her work do absolutely nothing to help her
clients (she’s a solicitor working in the area of PI). She used the
word “spaghetti” to describe some of the sets of spreadsheets she
has encountered.

It’s not going too far to say that some or even many of the
spreadsheets commonly used by actuaries are unprofessional. And all
the evidence shows that actuaries are using a lot of spreadsheets,
as two recent surveys demonstrate.

The first was conducted by the GIRO working party on software use
in general insurance, which I chaired. There were over 700
responses, from all over the world, and although there were no big
surprises there were some interesting results. Possibly the most
worrying issue to emerge was the lack of awareness of the
limitations of Excel; there are many actuarial users who don’t seem
to know that its statistical functions are unreliable, especially
in the tails of distributions.

http://www.louisepryor.com/papers/ActuariesExcel.pdf

The second survey was one that I conducted on the use of software
in ICAs. This was much smaller, with 45 respondents, and covered
both life and general insurance. It looked especially at the
systems and controls around software use. There was general
agreement that the systems and controls around spreadsheets are not
up to the standards of those around other user developed software,
such as actuarial models.

http://www.louisepryor.com/papers/SurveyResults.pdf

This is definitely worrying, especially in the light of the FSA’s
emphasis that the same standards should apply to spreadsheets as to
other software.

If you need help with your spreadsheets or other user developed
software, just let me know. I can advise you on systems and
controls, review current practice, and recommend development
processes for better risk management and productivity.

===============
2. Are you aligned?

We all know that there are security holes in computer software, and
that we rely on the manufacturers to issue patches. Most obviously,
Microsoft issues patches on a regular basis. Microsoft is
especially important in this context as so many people use their
products: if something goes wrong, the number of people likely to
be affected is enormous. Of course, this also explains why so many
viruses, worms, and other nasties are aimed at Microsoft products.
It’s widely known as the “mono-culture” problem.

So, there are risks attached to using software. And it’s in the
users’ interests for software manufacturers to issue patches. But
is it in the manufacturers’ interests?

In many ways, the answer is “No.” If you issue a patch, you are
publicly admitting that there is something wrong with your product,
so it doesn’t do your reputation much good. You have to develop,
test and distribute the patch. There’s a chance that the patch
won’t work, which will annoy the users even more. On the other
hand, users are going to get annoyed if you don’t issue a patch and
they fall victim to some sort of malware attack. If you’re a
software manufacturer, it’s a tricky line to tread. That’s why
Microsoft issues patches monthly, rather than as soon as a problem
is discovered. Even if there’s a serious security hole, it’s most
unusual for it to be patched outside the regular monthly cycle.

On the other hand, if there’s a problem in a piece of software that
could lead directly to the manufacturer losing money, they are
likely to react pretty quickly. Recently, a hacker developed a way
of getting round the copy protection in Microsoft’s digital rights
management (DRM) software. This isn’t going to bother most users in
the slightest (rather the opposite, if anything), but it certainly
bothered Microsoft. A patch was issued three days after the hack
was discovered. Another hack was developed to circumvent the patch,
of course: I’m not sure what the current state of play is.

Microsoft and FairUse4WM

http://news.zdnet.co.uk/software/windows/0,39020396,39282692,00.htm

The lesson is that if you rely on other people’s behaviour to keep
your risks under control, you’d better make sure that their
interests are aligned with yours. This applies at all levels, from
suppliers and outsourcers to other departments and colleagues.

===============
3. Whose risk is it again?

Last month I commented on the laptop battery saga, saying that it
seemed that Sony was picking up the bill for the risk to the
reputation of Dell and Macintosh. A reader commented that “Dell and
Apple are simply enforcing the contract that they have with Sony.
Clearly Sony did not provide goods of satisfactory standard, and
must pay any remedial costs with making good their failure.” Well,
yes, but… The but in this case is in the definition of failure.
Failure is defined in such a way as to protect the reputation of
the outsourcer, which of course is absolutely the right thing to
do, from the perspective of the outsourcer.

In the last few weeks the saga has continued. There’s a recall out
on some Toshiba batteries manufactured by Sony, although Toshiba
has said that it’s not related to the Dell and Apple recalls.
Although the batteries might fail, there is no risk of fire,
apparently. The numbers involved are pretty small: 340,000
batteries are affected, compared to nearly 6 million Dell and Apple
batteries.

http://www.reghardware.co.uk/2006/09/19/toshiba_notebook_battery_recall/
http://news.zdnet.co.uk/hardware/mobile/0,39020360,39283432,00.htm

Then just last week a recall was issued on half a million Sony
batteries used in ThinkPad laptops. One of them burst into fire at
LA airport, “causing enough smoke and sparking that a fire
extinguisher was used to put it out,” according to the US Consumer
Product Safety Commission. If this goes on it’ll only be a matter
of time before one ignites actually on an aeroplane, instead of
just on its way onto one. Sony is now starting a replacement
programme for laptop batteries that meet certain manufacturing
criteria, regardless of the laptops that they are used in.

http://cohyotea.notlong.com

Earlier in the month Matsushita (makers of Panasonic laptops)
recalled 6,000 batteries that might overheat if they are dropped
repeatedly. These batteries are not made by Sony.

http://news.zdnet.co.uk/hardware/0,39020351,39282681,00.htm

Over the past few years, we’ve also seen battery problems with
cameras and mobile phones. Some of these problems probably occur
because the limits of battery technology are being pushed.
Batteries have got much smaller and lighter over the last few
years, and last longer before needing to be recharged. (Of course,
much of the improvement is due to reduced power consumption by the
appliances that the batteries power.)

However, apparently some computers use Lithium-ion batteries
outside their design envelope. The recharging cycle is much faster
than usual. When the batteries are exposed to rapid charging, it is
possible for metal fragments to be formed. The fragments can then
cause major short circuits and thus over-heating. This is why it’s
not cut and dried that Sony will have to cover the whole cost of
the recall. Sony is also saying that the risk of fire “can be
affected by variations in the system configurations found in
different notebook computers.” Apparently Sony engineers haven’t
been able to reproduce the battery failures, lending credence to
the view that the way the batteries are being used may be at least
partly at fault.

http://catless.ncl.ac.uk/Risks/24.41.html#subj10.1
http://cohyotea.notlong.com

All this just goes to show the importance of getting outsourcing
agreements right, from both sides. It now looks as if Sony’s
reputation may be suffering because of the actions of the laptop
manufacturers, rather than vice versa.

===============
4. Newsletter information

This is a monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long as
this notice is included.

To subscribe, email news-subscribe AT louisepryor.com. To unsubscribe,
email news-unsubscribe AT louisepryor.com. Send all comments, feedback
and other queries to news-admin AT louisepryor.com. (Change ” AT ” to
“@”). All comments will be considered as publishable unless you state
otherwise. The newsletter is archived at
http://www.louisepryor.com/newsArchive.do.

News update 2006-08: August 2006
===================

Contents:
1. Whose risk is it anyway?
2. Replacing spreadsheets to reduce risk
3. Preaching to the converted
4. It’s everywhere
5. Newsletter information

===============
1. Whose risk is it anyway?

There’s been a fair amount of press coverage recently about Dell laptop
batteries bursting into flames. There are even video clips on the web,
to convince us that it really is happening. Spontaneous ignition is
clearly undesirable behaviour as far as batteries are concerned, and
Dell now has a product recall in place. Apparently it’s the largest
computer product recall ever.

http://www.dellbatteryprogram.com/
http://www.gizmodo.com/gadgets/dell/dell-laptop-explodes-in-flames-182257.php

Like many manufacturers, Dell outsources the production of many
components, and it’s actually Sony that made the batteries in question.
Sony also supply batteries for a number of other makes of computer, but
so far only Apple have issued a recall. So, either the fault was
confined to batteries destined for Dell or Apple machines, or there are
a whole lot of other computers out there that might burst into flames.
In fact, there are going to be a whole lot of computers out there that
might burst into flames anyway, as it seems unlikely that all the owners
of machines that are affected will actually replace their batteries.

So, who’s bearing the risk? Or, possibly more importantly, who’s
suffering the consequences?

Dell, Apple and Sony are clearly suffering at least some consequences. A
product recall on this scale costs money, and there’s the damage to
reputation, too. It appears that much of the monetary cost will be borne
by Sony, while Dell and Apple will take most of the reputational hit.
However, it’s widely thought that by acting promptly, and acknowledging
that the problem exists, the reputational damage has been minimised.

It could be argued, though, that the recall is unnecessary. Nobody has
been killed, and it seems that there have been about 10 or 15 incidents,
out of nearly 2 million batteries. There are many everyday dangers
around daily activities that are much more likely. On this view, the
only rational reason for the product recall is to limit the reputational
risk.

http://catless.ncl.ac.uk/Risks/24.40.html#subj8.1

What this means is that Sony is paying the price of protecting Dell’s
and Apple’s reputations. This is an interesting twist on what is usually
thought to be the situation with outsourcing. The standard line goes
that it’s often the outsourcer who has to carry the can when something
goes wrong at the supplier’s end. In this case, it could be argued that
the supplier is carrying the can to protect the outsourcer.

===============
2. Replacing spreadsheets to reduce risk

“Australian Pharmaceutical Industries revealed yesterday that it had
lost $17 million – and a managing director – but was unable or unwilling
to discuss the details behind either.” That’s the start of a report in
The Australian, which explains that API found unreconcilable
discrepancies between their three computer systems. They’ve apparently
been integrating the three systems over the last three years.

http://neafilah.notlong.com

The same issue of The Australian contained a feature article on how API
successfully moved from spreadsheets to a specialist tool. Previously,
there had been one system running sub-ledgers, another running the
general ledger system, and spreadsheets were used for budgeting and
performance reporting. “The spreadsheets were difficult and very
labour-intensive and slow to respond to change.” Doug Horwood, the
information management leader, went on to say, about using the new tool,
“We got the core budget information entered and were able to do
bottom-up and top-down adjustments, and the aggregation of all the
different cost centres literally turned into minutes rather than the six
to 12 hours it used to take. It was also fraught with the possibility of
manual error in the processing.”

http://jinrotea.notlong.com

So, the new system improved things as far as budgeting was concerned,
but was apparently not an improvement on the financial reporting front.

Horwood had some interesting things to say about their previous use of
spreadsheets. “In the past it was such a horrendous job to change all
the spreadsheets, that in effect we only ever adjusted the budget at a
very high level.” Spreadsheets are usually thought of as being extremely
flexible; one of the advantages touted for their use is that they are so
easy to change. This is certainly true of individual spreadsheets, but,
as Horwood realised, is rarely true of whole systems built out of
spreadsheets communicating with each other. The trouble is that it’s
very difficult to make nice abstract interfaces between spreadsheets,
that are independent of the details of the internal workings.
Spreadsheets were designed as personal productivity applications, and
although they are often used for enterprise level systems it’s very hard
to make them really work well in that context.

If you are using large systems built from spreadsheets, and rely on them
for mission-critical reporting or modelling, you should make sure that
you are aware of the risks. Have you thought about whether spreadsheets
are really the appropriate technology? Have you reviewed the spreadsheet
techniques that you are using, to make sure that you are managing the
risks effectively? And do you have good systems and controls around the
development and use of the spreadsheets? If you’d like to know how I
could help you answer these questions, and more like them, please get in
touch either by replying to this email, or contacting me through my web
site.

Climate change actuary

===============
3. Preaching to the converted

We are always being told how important it is to make sure that we
protect our computers from viruses, spyware, and other nasties. “You
must have anti-virus software!” “You must have a firewall!” It’s all
true, of course, but what happens when the software doesn’t work? Or
when it works wrong?

One of the most useful characteristics of security software is that you
should be able to trust it. In particular, you don’t want it to start
removing perfectly harmless, but very useful, programs that you have
installed on your computer. Many Church of England clergy had a nasty
shock earlier this month when Symantec’s Norton Antivirus software
wrongly identified an innocent file as a piece of spyware. The software
prompted them to remove the file, which was in fact a vital component of
Visual Liturgy, an application that is used to choose services, plan
Bible readings and create booklets.

http://news.zdnet.co.uk/internet/security/0,39020375,39280391,00.htm
http://vislit.com/articles/060804norton.html

The vendors of Visual Liturgy advised their users to ignore the Norton
warning, so that they could continue to use Visual Liturgy. Meanwhile,
they tried to contact Symantec to get the Norton program fixed. There is
some disagreement about how effective this was, but eventually
everything was sorted out.

Of course, it’s not unusual for there to be errors in computer programs,
and however careful Symantec is to test and review the Norton program
they can never be sure that a bug won’t slip through. Moreover, Symantec
can hardly be expected to test it against every single piece of
legitimate software available on the planet.

However, there are now many users who may not trust Norton warnings in
future, and who will be very wary of deleting files just because Norton
tells them to. So the overall effect is probably to decrease computer
security.

===============
4. It’s everywhere

Google is starting to reach the parts that only that other software
giant, Microsoft, can reach. As well as providing web-based searching
services, and desktop tools, they are now providing web-based tools,
including a spreadsheet and word processor. Soon, you’ll be able to use
Google to do just about everything you need to do on a computer. Or, at
least, that’s the impression Google would probably like to give. The
latest offering is a service providing corporate email, instant
messaging, calendar and web page creation. “Google Apps for Your Domain
lets you offer our communication and collaboration tools to your entire
organization – customizable with your branding, color scheme and content
through the administrative control panel, and with no hardware or
software to install or maintain.”

https://www.google.com/a/

It’ll be interesting to see whether this takes off. It’s only in beta at
the moment, is free, and isn’t available to everybody. Presumably they
won’t accept any large organisations until they’ve got the premium (ie
paying) service up and running.

One of the big issues, both with Google Apps and with other tools, is
that of privacy. Should organisations be wary of storing confidential
information on Google’s servers? Or using Google desktop tools on
machines that might have access to confidential information? This is
probably a judgement call that has to be made by each organisation for
itself. Google have privacy policies, obviously, but organisations will
have to decide whether they are adequate, whether they trust Google to
abide them, and whether they trust Google not to change them in the
future.

As with the Google spreadsheet, the real market might be not in web
based applications, but in applications for the intranet. If that
happens with Google Apps, Google really will be going head to head with
Microsoft on its home turf.

===============
5. Newsletter information

This is a monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long as
this notice is included.

To subscribe, email news-subscribe AT louisepryor.com. To unsubscribe,
email news-unsubscribe AT louisepryor.com. Send all comments, feedback
and other queries to news-admin AT louisepryor.com. (Change ” AT ” to
“@”). All comments will be considered as publishable unless you state
otherwise. The newsletter is archived at
http://www.louisepryor.com/newsArchive.do.

News update 2006-07: July 2006
===================

Contents:
1. Software in ICAs
2. Is it safe?
3. Don’t try this at home
4. Measuring risk
5. Newsletter information

===============
1. Software in ICAs

Many thanks to everyone who has completed the online survey about
software use in ICAs, at http://www.louisepryor.com/survey.html

If you haven’t completed the survey this is your last chance, as the
survey closes on 1st August. It should only take about a quarter of an
hour, and the more responses there are the more useful the results will
be. All participants will get a full analysis of the results, and
responses are confidential.

===============
2. Is it safe?

There’s an interesting piece in the New Scientist, by Henry Petroski,
about how apparent success can mask flaws that will lead to disaster. He
is writing about engineering flaws, but there are lessons for the
software world as well.

Imagine that the Titanic had not struck an iceberg on its maiden voyage:
it would have had a triumphal arrival in New York, and other ships would
have been built to similar designs. The designs would have been adapted
to use smaller engines and lighter hulls. They would have continued to
be thought of as “unsinkable” and at the leading edge of technology as
long as none of them did actually sink; and that state would probably
have lasted for some time. It was only luck that the Titanic happened to
strike an iceberg when it did.

Similarly, foam fell off nearly all space shuttles as they were launched
before 2003; because no damage was done, it was accepted as a routine
event. When foam fell off Columbia, it wasn’t immediately perceived as a
potential threat.

As Petroski says, “latent systematic weaknesses in a design are most
likely to be revealed through failure.” The fact that something has
worked well for a long time is not in itself a guarantee that it will
continue to do so: it may be that the circumstances in which it won’t
work just haven’t happened to arise.

It’s obviously a lot easier to test things in the software world: it’s
possible to run a program with many different sets of inputs, whereas
launching the space shuttle many times is not feasible. However, just
because testing is easier it doesn’t mean that it’s actually done.

In addition, there’s the “it won’t happen” syndrome. The roof of a road
tunnel in Boston collapsed recently; apparently an engineer warned of
the potential problem back in 1999. He was told that the method used to
support the ceiling panels was tried and tested, and that the work was
being done to design specifications and that the ceiling would hold.
Petroski says that it’s important to have “a healthy scepticism about
built things, and an awareness that apparent success can mask imminent
failure”.

The point about unlikely events is that they don’t happen very often. A
piece of software that has worked many times before isn’t necessarily
going to work in the future. It may get a combination of inputs that has
never occurred before. The best defence is to perform thorough testing
and code review.

http://www.newscientist.com/article/mg19125625.600.html
(requires subscription)
http://athricea.notlong.com

If you want to know how to go about testing and reviewing your
spreadsheets and other user developed applications, please get in touch.
Having good processes in place not only makes the applications more
reliable, it also reduces the time required to develop them.

===============
3. Don’t try this at home

Or, at least, don’t do things on live data unless you’ve tested it out
first. It’s a sensible and common safety precaution, though probably not
common enough. Always make changes on a copy of your spreadsheet, so
that you can go back to the last known working version. If you’re
operating on a database, don’t try things out on the production system:
you might corrupt or destroy data as well as making the system
unavailable.

Even if you’re aware of the problems, things can still go wrong.
PlusNet, a UK ISP, recently deleted more than 700GB of its customers’
e-mail and prevented about half its 140,000 users from sending and
receiving new e-mail. Things went horribly wrong during the process of
bringing a new back-up storage server online. Instead of deleting the
contents of the old back-up disks, the engineer deleted the live
storage. PlusNet explained what happened: “At the time of making this
change the engineer had two management console sessions open — one to
the backup storage system and one to live storage. These both have the
same interface, and until Friday it was impossible to open more than one
connection to any part of the storage system at once. The patches we
installed on Friday evening removed this limitation, but unaware of
this, the engineer made an incorrect presumption that the window he was
working in was the back-up rather than the live server. Subsequently the
command to reconfigure the disk pack and remove all data therein was
made to the wrong server.”

PlusNet have called in data recovery engineers, but nearly 3 weeks after
the disaster the missing emails haven’t yet been restored. It took
nearly 2 weeks for email service to be restored for all customers.

With hindsight, of course, it would have been better had the management
console sessions indicated clearly which storage system they were
operating on. Before the patches, it appears that this wasn’t strictly
speaking necessary, but it’s always dangerous to rely on assumptions of
the form “this couldn’t possibly happen” (or, even worse, “no reasonable
user would…”). Of course it’s often difficult to recognise that type
of assumption, as they often aren’t explicit.

http://www.theregister.co.uk/2006/07/11/plusnet_email_fiasco/

===============
4. Measuring risk

I usually enjoy the “Gavyn Davies does the maths” column in the Guardian
on Thursdays, and this week was no exception. Apparently punters do
better, long term, by betting on favourites rather than long shots. They
still lose money, but they lose less money. As Davies points out, this
is unexpected: in a rational world, punters would refuse to back the
long shots until the odds were more realistic, removing the anomaly. He
suggests three possible explanations, favouring the one that says that
the whole market really does have a misperception about risk, failing to
appreciate that a there is a major bias in the odds. In other words, the
risk measure (the odds) isn’t an accurate reflection of the actual risk.

http://www.guardian.co.uk/g2/story/0,,1830876,00.html

The same sort of thing may be happening in the financial world.
According to the Bank of England’s Financial Stability Report, trading
profits have recently risen as a proportion of income at UK commercial
and global investment banks. however, the risk taken on by the banks, as
measured by value at risk (VaR) has risen by much less. This is
surprising because it’s commonly accepted that in order to make more
money, you have to take on more risk. The Bank presents two possible
explanations, but favours the second, that VaR just isn’t a good measure
of risk in the trading book. Unfortunately, it’s very widely used, and
is the only risk metric that banks must disclose, even though they may
well use others internally. This may mean that more weight is put on it
than is justified.

http://www.bankofengland.co.uk/publications/fsr/2006/index.htm

The trouble is that measuring risk is difficult, and, like many
phenomena, there is no single metric that is adequate. You can’t measure
buildings just by their height: other dimensions provide other
information, and you can only get the full picture by looking at them
all. Two towns with the same population may be very different in other
respects. VaR measures only the maximum loss likely to occur over a
given period at a certain level of probability. It doesn’t measure the
expected loss, or even the maximum loss at a different level of
probability or over a different period.

===============
5. Newsletter information

This is a monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long as
this notice is included.

To subscribe, email news-subscribe AT louisepryor.com. To unsubscribe,
email news-unsubscribe AT louisepryor.com. Send all comments, feedback
and other queries to news-admin AT louisepryor.com. (Change ” AT ” to
“@”). All comments will be considered as publishable unless you state
otherwise. The newsletter is archived at
http://www.louisepryor.com/newsArchive.do.