Category: Newsletter

The newsletters I sent out roughly monthly from December 2002 to April 2007.

Newsletter Feb 2006

Post author By Louise Pryor
Post date 28 February 2006

News update 2006-02: February 2006
===================

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).

Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.

Subscribe by sending an email to news-subscribe AT louisepryor.com.
Unsubscribe by sending an email to news-unsubscribe AT
louisepryor.com. (Change ” AT ” to “@”). Newsletter archived at
http://www.louisepryor.com/newsArchive.do.

In this issue:
1. The tools for the job
2. Email: is it safe?
3. Russian viruses?
4. Attachment
5. Newsletter information

===============
1. The tools for the job

How do you decide what tools to use? Yes, it’s possible to beat an
egg white with a knife, or loosen a screw with a nail file, but an
egg-beater or screwdriver would be a better choice. Not
surprisingly, the same goes for software. Excel is a bit like a
Swiss Army knife: you can do most things with it if necessary, but
specialist tools often work better.

This was recently demonstrated at Roll-Royce, where engineers were
asked to calculate how many times a simple pendulum hit a target
pin. Those who used Excel came up with three as the answer, while
those using specialist mathematical software came up with four
(which was the correct answer). Even using a much smaller time step
in Excel made no difference.

http://tinyurl.com/kpb9f

It’s well known that Excel’s statistical functions are less than
perfect. For a start, its functionality is limited; there are many
analyses that you just can’t do. Then there’s the issue of
accuracy: Excel doesn’t use the best available algorithms. The
values differ in the third digit and beyond. Some of the problems
are more evident in the tails of the distributions. A number of the
statistical functions use non-standard definitions. The random
number generator is not really random enough (and in one version of
Excel it actually produced negative numbers, which are definitely
outside the [0,1] range).

http://www.practicalstats.com/Pages/excelstats.html

Microsoft has made improvements, but this means that current
versions of Excel are not consistent with earlier versions. The
same workbook will give different results depending on which Excel
version is used. Also, it is claimed that some of the fixes have
made the problem worse: “Microsoft attempted to fix errors in the
standard normal random number generator and the inverse normal
function, and in the former case actually made the problem worse.”

http://support.microsoft.com/default.aspx?kbid=828888&product=xl2003

Some improvement can be gained by using specialist statistical
add-ins for Excel, but if you are doing any serious statistical
work you should probably consider using a specialist tool. The nail
file might do for one screw, but if you are putting a shelf up a
screwdriver is more or less indispensable. A Swiss Army knife,
although handy, can’t replace a whole toolbox.

On a somewhat related note, I’m currently chairing the GIRO working
party on software use, which is looking into what software GI
actuaries (and others) use, and how they use it. As our first step,
finding out what software is actually used, we are conducting an
online survey. So, if you work in General Insurance, as an actuary
or otherwise, we’d really appreciate if you’d help us out by
completing the survey, which is at
http://www.surveymonkey.com/s.asp?u=891841542723. There’s a prize
of a bottle of champagne for one lucky participant!

===============
2. E-mail: is it safe?

Fifteen years ago e-mail was new and shiny and more or less unknown
in the business world. Nowadays it’s hard to imagine how businesses
could operate without it. In some ways, though, we haven’t really
got to grips with it. Those physical carbon copies, one into the
file and one into the day book, certainly meant that records of
outgoing letters were kept. Morgan Stanley must wish that life was
that easy. The firm has just made an offer of $15 million to settle
an investigation by the SEC into an alleged failure to produce
email evidence during a legal dispute. Morgan Stanley was involved
in a court case, and failed to produce e-mails and documents sought
by the opposing side’s lawyers. The regulators were concerned that
this indicated a breach of federal regulations, and began their own
investigations. Meanwhile, the judge was so annoyed that she
reversed the burden of proof in the court case, so Morgan Stanley
had to prove its innocence.

http://www.out-law.com/page-6656

Of course we all know that the Cc: designation on an e-mail means
“Carbon copy”, and Bcc: means “Blind carbon copy”. Was it easier to
make sure that copies went to right people when actual carbon paper
was used? I suspect that mistakes happened back then, just as they
happen now. Every business day, the US Defense Department is meant
to announce contracts awarded valued at $5 million. A new employee
at the Air Force dropped the Defense Department from the e-mail
distribution list for new contracts in December. Nearly $4 billion
in contracts slipped through the net before the error was
discovered. “It’s awfully embarrassing,” said an Air Force
spokesperson.

http://cwflyris.computerworld.com/t/296929/664274/9136/0/

It turns out that not expecting an e-mail is no excuse for not
reading it. Bernuth Lines Ltd listed an e-mail address in the
Lloyd’s Maritime Directory, and on their web site. When they were
involved in an arbitration case, both the arbitrator and the other
party to the dispute sent e-mail to the listed address, even though
it had not been specifically notified to them. It appears that the
e-mails weren’t read, and may even have been deleted as spam. This
cut no ice with the judge who said that receiving an e-mail is no
different to the receipt at a company’s office of a letter or
telex. He ruled that e-mail counts as a “recognised means of
communication”. The moral of this story is pretty obvious: if you
publicise an e-mail address, for goodness sake make sure that you
read the e-mail that is delivered to it.

http://tinyurl.com/ftff6

And, finally, having made sure that you keep copies of your
e-mails, that they go to the right people, and that you read the
ones that you receive, how effective is e-mail as a communication
medium anyway? A recent article in the Journal of Personality and
Social Psychology argues that people tend to believe that they are
much better e-mail communicators than they really are. The authors
of the article conducted five separate studies to investigate the
communication of humour, sarcasm, and other emotions. The study
found that the people tend to look to their own perspectives when
attempting to understanding messages from others. The problem is
made worse when the other parties to the communication are coming
from rather a different perspective.

E-mails are often less formal than written letters, and we may be
tempted to introduce humour or sarcasm in order to lighten the
tone. This is probably a mistake, if we want to make sure that the
intended message is getting across.

http://arstechnica.com/news.ars/post/20060214-6176.html

===============
3. Russian viruses?

Stock exchanges and viruses just don’t mix. Early in February the
Russian Trading System (RTS) was infected with a virus, and trading
was suspended for an hour. The virus generated a large amount of
outgoing e-mail traffic, and disrupted legitimate e-mail, both
outgoing and incoming.

“The virus got into a computer connected to a test trading system
from the Internet,” RTS vice president Dmitry Shatsky said in a
statement. “The infected computer started generating huge volumes
of parasitic traffic, which overloaded the RTS’s support
routers. The result was that normal traffic – data going into and
out of the trading system – was not processed.”

Note that it was only a test trading system that was infected; you
have to be careful with any machine connected to the internet,
regardless of what function it is performing.

http://www.finextra.com/fullstory.asp?id=14867
http://news.zdnet.co.uk/internet/security/0,39020375,39250774,00.htm

That was a virus in Russia; it was a virus from Russia, or at least
operated by Russians, that was used to steal more than 1m euros
from bank accounts in France. The virus, or “sleeper bug”, is
downloaded from a web site, or received in an email, but doesn’t do
anything until the user starts using online banking. The bug then
records passwords and security codes and forwards them to the
thieves. Yet another reason, if one were needed, to have up to date
virus and malware protection.

http://www.guardian.co.uk/france/story/0,,1703777,00.html

It appears that those Russian thieves were using old
technology. The newest malware programs on the block are Trojans
that don’t bother with your password and security details, they
just wait until you’ve done that bit for them and then access your
bank account directly. These new Trojans are programmed to work
with specific online banking web sites. Remember, just don’t click
on that unsolicited and unexpected e-mail attachment.

http://news.zdnet.co.uk/internet/security/0,39020375,39253433,00.htm

===============
4. Attachment

The BlackBerry addicts among us can sleep a bit easier for the
moment, as the possible shutdowns of service in the UK and US
look less likely to happen. RIM, the makers of the BlackBerry, had
in any case announced that there would be a work around that would
let the service continue in the event of the threatened US
injunction. It’s not over until the fat lady sings, though, and
things could still go horribly wrong for RIM and BlackBerry users.

http://news.ft.com/cms/s/130ae8b0-9410-11da-82ea-0000779e2340.html
http://techrepublic.com.com/2100-1035_11-6037169.html?tag=nl.e019
http://www.finextra.com/fullstory.asp?id=14964

One survey showed that 81% of asset managers questioned consider
the Blackberry service as “vital” and “critical” to their
business. Let’s hope they, and everybody else, were making suitable
contingency plans. To me, this whole saga underlines the problems
associated with relying on a proprietary technology for which there
is a single supplier. Personally, I don’t expect to have to take a
view on the likely validity or otherwise of a set of patents before
I invest in a new piece of equipment.

As so many people appear to be addicted to their BlackBerry, maybe
somebody will start making clothes with special BlackBerry
pockets. Don’t laugh too much; apparently you can now get a tie
(yes, one of those things that goes round your neck) that will hold
your iPod.

http://www.reghardware.co.uk/2006/02/23/pink_ipod_tie/

===============
5. Newsletter information

This newsletter is issued approximately monthly by Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long
as this notice is included. To subscribe, email news-subscribe AT
louisepryor.com. To unsubscribe, email news-unsubscribe AT
louisepryor.com. All comments, feedback and other queries to
news-admin AT louisepryor.com. (Change ” AT ” to “@”). Archives at
http://www.louisepryor.com/newsArchive.do.

Newsletter Old site

Newsletter Jan 2006

Post author By Louise Pryor
Post date 31 January 2006

News update 2006-01: January 2006
===================

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).

Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.

In this issue:
1. Control freakery
2. Doing without
3. Safe documents
4. Reports and surveys
5. Newsletter information

===============
1. Control freakery

I admit it. I’m a control freak. Not necessarily in all aspects of
life, but as far as my computer is concerned, there is no
question. I don’t let anything (Microsoft included) update my
software automatically, and if I change a setup in any way I want
it to stay changed. I don’t like it when my computer tries to be
helpful, by guessing what I want to do. It’s just not that clever,
and certainly isn’t a mind reader, and often guesses wrong.

In July 2004 I discussed how Excel tries to be clever when
importing data by recognising dates and converting them to date
values. It also automatically converts strings that it believes are
floating point numbers. For example, the string “2310009E13” is
converted to the number 2.31E+13. In both cases the original data
is irretrievably lost.

It is possible, although not easy, to avoid these automatic
conversions, but you have to remain vigilant. You can’t turn them
off, but have to take special steps each time you import data. Some
solutions are described by Microsoft in
http://support.microsoft.com/default.aspx?scid=kb;EN-US;Q214233

The auto-completion feature is another source of problems. When you
type data in, Excel tries to guess what you mean by looking at
other values in the same column. If you are entering credit
ratings, for example, and the first one you enter is AAA, if the
next one you enter also starts with an A, Excel will suggest AAA as
an auto-completion. If you hit enter, it thinks you are accepting
the suggestion. So if you are actually trying to enter A, you type
A, enter, and Excel helpfully puts AAA in the cell. This one is
easier to fix: simply turn off auto-completion. On the Tools menu,
click Options, and then click the Edit tab. Once there, select or
clear the “Enable AutoComplete for cell values” check box.

Dean Buckner has just found another example of what might be called
(if you are feeling charitable) over-helpful behaviour. He writes:

I’ve always tried to discourage our users from MS ‘shortcuts’,
i.e. little files that can be placed on the desktop and which
point to applications. The problem is that they sometimes point
to the wrong thing – we had a case last year of an emailed short
cut pointing to a correspondence log which was an old version.
Thus some people were using a correct version, some were
updating an old one in parallel. The business of merging the
two streams was a nightmare.

Anyway I forbade this practice, and restricted the use of
shortcuts to those which are centrally maintained and have a
strict change control. (For they can be useful in that they can
control the way an application is called, for example with a
security file).

Or so I thought. I took our main system down over the break in
order to do maintenance work. I did this by deleting the main
production file, and working on a copy, as one would. However,
someone tried to access it over the break, and it turned out
that when a shortcut fails, it will “search” around until it
finds an application that looks similar. In this case it found
one of the backup files which we save down every week. Once
this happens, it then changes not only the user’s own desktop
version of the shortcut, but also the central file itself. I
only spotted this when, working on the application, I noticed
some clearly out of date information.

Ouch. As Dean goes on to say, it’s difficult to find out what is
going on and work out how to turn it off (the help system is
decidedly unhelpful on this). The answer is in the following
Microsoft document:
http://support.microsoft.com/default.aspx?scid=kb;en-us;299780&sd=tech
The document says: “If you disable a shortcut, the NTFS File System
in Windows XP and Windows 2000 automatically attempts to locate the
shortcut destination by searching all paths that are associated
with the shortcut.” One of the solutions involves editing the
Registry, which is not for the faint-hearted. Also, if you make
backups or copies of files in a nearby directory, it’s probably
best to change the name so that Windows will not identify the copy
with the original.

===============
2. Doing without

I moved house about 3 weeks ago, and have had problems getting
broadband connected. It’s more accurate to say that I am having
problems, as I haven’t actually succeeded yet (though progress is
being made). Apparently my order “got stuck in the system”. I
suggested WD40, but apparently that wouldn’t work. Anyway, I have
had to fall back on good old dial-up, and the effects have been
much wider than I anticipated. I hadn’t quite realised how much I
had come to rely on the always-on, high bandwidth access. Even when
doing things that I thought were definitely off-line in nature,
I’ve found it frustrating not to be online. There’s a lot of
information out there that I have never bothered to download,
because it’s so readily available on the web… but when you have
to connect explicitly (tying up your phone line) and download over
what now feels like a piece of wet string, the term “readily
available” takes on a whole new meaning.

What’s interesting about this is that it shows how ingrained habits
can become. After all, there are many people in this country (and
elsewhere) who don’t have broadband, and who manage very happily
with dial-up. But in the three and a bit years that I have had
broadband, I appear to have changed my working patterns quite
radically, and it’s proving difficult to revert to the old ones.

One thing I don’t have is a BlackBerry. When travelling, I manage
quite happily with my Palm hooked up to my mobile via
bluetooth. It’s slightly clunky, but surprisingly effective. There
are, however, large numbers of people who do have BlackBerries, and
who rely on them quite heavily. Are they right to do so? How would
they manage without them?

We know that some people can cope, for short periods at least,
because in October we heard how the BBC had suspended its
Blackberry service because of email problems. But many more may
have to cope for long periods if not for ever, and there may be
more short outages.

As BlackBerries become more popular and hence more mainstream, they
become more attractive targets to the bad guys. Early in January
there was a news story about security vulnerabilities that provided
loopholes through which hackers could launch denial of service
attacks. No actual problems were reported, but it’s probably only a
matter of time.

Meanwhile, Research In Motion (RIM, the makers of the BlackBerry)
are involved in a US court action over a patent infringement. A US
federal appeals court wanted to use US patent law, which bars
unauthorised use of a patented invention within the US, to block
the BlackBerry service. The Supreme Court refused to consider RIM’s
argument that the law should not be used to block the service,
which is run from Canada. The end result could be a lot of unhappy
US BlackBerry addicts, though they would doubtless soon find usable
alternatives. FT readers certainly think they would, according to
an online poll.

http://news.zdnet.com/2100-1035_22-6029671.html
http://makeashorterlink.com/?P6392309C

This story also emphasises one difficulty caused by the
globalisation of services: just what jurisdiction do they come
under?

===============
3. Safe documents

Last month I mentioned how Westpac had been forced to halt trading
on its shares and deliver its annual profit briefing a day early
after it accidentally sent its results by email to research
analysts. It had sent out a spreadsheet containing the results for
previous years; it also contained the latest results, obscured by
blacking out the relevant cells. The cells may have looked black on
the screen, but it didn’t need advanced technical skills to find
out what they contained.

The US National Security Agency (NSA) has now issued a document
that describes how to issue an electronic document that has no
sensitive information hidden in it.

http://www.fas.org/sgp/othergov/dod/nsa-redact.pdf

The document is 14 pages long, and covers some of the ways in which
problems can arise, as well as how to avoid them. In brief, the
recommended procedure is:

1. Make a copy of the original document, and work on the copy.
2. Turn off change tracking etc.
3. Delete sensitive text and diagrams.
4. Change the document name to make it clear that it’s an edited
version.
5. Open a new blank document.
6. Copy the contents of the edited document into the new one.
7. Convert the new document to pdf.

The NSA gives detailed instructions for doing all this in a Word
document, but the same principles apply to spreadsheets and other
formats.

===============
4. Reports and surveys

If you work in General Insurance, you’ll probably know about GIRO,
the general insurance conference held each year by the Actuarial
Profession. I’m currently chairing the GIRO working party on
software use, which is looking into what software GI actuaries (and
others) use, and how they use it. As our first step, finding out
what software is actually used, we are conducting an online
survey. So, if you work in GI, as an actuary or otherwise, we’d
really appreciate if you’d help us out by completing the survey,
which is at http://www.surveymonkey.com/s.asp?u=891841542723.
There’s a prize of a bottle of champagne for one lucky participant!

The FSA has just issued its annual report on the Financial Risk
Outlook. One of the main points that is made is the need for stress
testing: how firms would respond to extreme risk scenarios. The
scenarios imagined by the FSA include natural disasters (possibly
driven by climate change), global pandemic, political instability
in a major economy, a large terrorist attack, or a major corporate
bankruptcy.

http://www.fsa.gov.uk/Pages/Library/corporate/Outlook/fro_2006.shtml

Although these scenarios are important, there are undoubtedly
others, possibly as yet barely imagined, that could cause similar
levels of disruption. The FSA emphasise that it is important to use
historical experience to inform hypothetical scenarios, rather than
simply re-running past events. Another important aspect of
effective stress testing is the effect of unsuspected risk
correlations across the different parts of large, complex
businesses.

===============
5. Newsletter information

Newsletter Old site

Newsletter Dec 2005

Post author By Louise Pryor
Post date 21 December 2005

News update 2005-12: December 2005
===================

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).

Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.

In this issue:
1. You get what you wish for
2. Supplying risks
3. Mistakes will happen
4. Seasonal greetings
5. Newsletter information

===============
1. You get what you wish for

Every child knows that you have to be careful when you make a
wish. It’s something that Midas learned the hard way.
Unfortunately, it’s something that a number of other people are
learning the hard way, too, even though they weren’t the ones to
make the wish.

Imagine you are designing a mobile phone, in the early days of
their use. You realise that you should have a way of locking the
keypad, so that calls aren’t made by accident when the phone is
knocking around in someone’s pocket or bag. Then you introduce a
further safety measure: it will always be possible to dial the
emergency numbers (999 or 122) when the keypad is locked; in an
emergency, nobody is going to want to fiddle around unlocking the
keypad.

The unintended consequence is that the centres handling emergency
calls get around one call every four seconds that is from a mobile,
and on which there is no communication at all, just silence. Most
of these calls are from phones that have accidently dialled 999 or
112. After all, if the 9 key is jogged by accident, it’s really
quite likely that it’s going to be jogged 3 times. Worse, if it’s
jogged 3 times by accident, it’s often jogged three more times, and
three more times, so there are many mistaken calls from the same
phone.

The operators answering the emergency calls have procedures to deal
with this problem. If there is nobody speaking into the phone, they
ask the caller to tap on the screen; if they get no response, they
put the call through to an automated system, which asks the caller
to press one of the keys. If there is any response at all, the call
is put through to the police.

Now, imagine you are badly injured, alone except for you
mobile phone. You can just manage to dial 999, but can’t get the
phone near your head, so can’t speak into it or hear what the
operator is saying to you. You hope that by making several calls in
quick succession the operators will realise that something is up,
trace the calls, and send help.

Sadly, no help will arrive. You are just one of the 900 silent
calls per hour coming in from mobiles. The operators can’t tell the
difference between you and a jogged keypad in someone’s bag.

There are no easy answers, given that mind reading is impossible.
Make keypad locking lock out the emergency numbers, and there will
be some people who can’t dial the emergency number in an
emergency. There just aren’t the resources to send help to all the
silent, repeated calls, just on the off-chance that they are
genuine. And as things are, some people won’t get help when they
need it.

Could all this have been foreseen? With hindsight, obviously, but
at the time? I don’t know. But the good news is that the number of
silent calls is decreasing as the proportion of folding mobiles
increases.

There was a programme on BBC Radio 4 that covered this and related
issues. The transcript is available from
http://news.bbc.co.uk/1/hi/programmes/file_on_4/3708232.stm

===============
2. Supplying risks

There has been some comment in the press recently about how
companies are getting more worried, and hence more clued up about,
risks to their supply chains. This worry is easy to understand in
the case of manufacturing companies, who often have long supply
chains with very little slack. Others should also be worried.

I heard an interesting story recently about a transport company in
the USA. Just as Hurricane Katrina struck New Orleans, they had
finished a major risk identification exercise. It had been an
extremely thorough affair, they thought, and they were feeling
pretty satisfied with a job well done. Of course they missed a
biggy. They hadn’t realised that all the steel they needed for
their fleet replacement programme passed through New Orleans. Ouch!

Closer to home (to my home, at any rate) the explosion at the
Buncefield oil depot raised many health and safety fears; it was
very lucky that nobody was killed, and the fallout from the smoke
doesn’t appear to have been as bad as it could have been. But some
of the knock-on effects were unexpected.

One of the businesses affected was Northgate Information Solutions,
whose head office is right next door to the depot. The building was
seriously damaged, and the on-site backup systems were, as they
say, “rendered inoperable”. Not surprising, really. Northgate had a
business continuity plan which they were able to put into action,
but there was a break in service. A number of their customers were
quite seriously affected. These included Richer Sounds, the hifi
retailer, whose web site and email systems were affected, the
Labour party, whose web site was down, and Addenbrooke’s hospital,
in Cambridge. Their internal information system of patient data was
in the destroyed Northgate building. They had to revert to manual
records for a period.

Who would have thought that an explosion in Hemel Hempstead would
affect the running of a hospital in Cambridge? (For those who don’t
know, the two are about 50 miles apart.) But of course, with modern
technology there is no particular reason why the data centre should
be anywhere near the hospital. And it’s not just large
organisations who outsource some of their IT functionality; my own
web site is hosted in Canada. Nowadays there are long supply chains
even in the services sector, subject to just the same risks as any
other supply chain. There are around 50 other oil depots like
Buncefield around the country; I wonder how many other data centres
are near them.

http://www.theregister.co.uk/2005/12/14/oil_blast_prangs_newlabour/
http://news.yahoo.com/s/ap/20051212/ap_on_bi_ge/britain_explosion_business

===============
3. Mistakes will happen

To death and taxes I would add mistakes, as being something you can
be sure of. However hard you try, mistakes will happen. You may be
able to reduce the number of mistakes, or limit the damage they
cause, but you can’t totally eliminate them.

Trying to reduce their incidence and severity are both valid risk
management strategies, but in most cases it’s a mistake (another
one) to rely totally on the former. It’s always a good idea to have
good ways of picking up the pieces when something does go wrong.

This is a lesson that Takuo Tsurushima, chief executive of the
Tokyo Stock Exchange (TSE), and two of his colleagues, have learned
to their cost. The nightmare started when a broker mis-keyed a
deal, placing an order to sell 610,000 shares in J-Com at 1 yen
apiece instead of 1 share at 610,000 yen. The company in question
had only around 15,000 shares outstanding, so selling 610,000 of
them was always going to be problematic. Commentators have said
that more or less anywhere else in the world such an aberrant order
would have been spotted and could have been de-keyed. In fact, it
appears that the mistake was spotted quite quickly, but the TSE
systems had no method of cancelling the order.

Oddly, there was a similar case almost exactly four years earlier,
when a trader intended to enter an order to sell 16 Dentsu shares
at 610,000 yen but actually keyed in 610,000 shares at 16 yen. The
recent press coverage doesn’t appear to have picked up on this
coincidence, though it does seem odd: is 610,000 a special number
in some way?

There are some obvious things that could be done to try to prevent
errors of this type, such as some sanity checks on the number of
shares in a deal, and the price, asking for confirmation if they
are outside certain limits. Of course, these things always turn out
not to be as simple as they seem to an outsider, and we have all
encountered situations where you just click through vital warning
boxes, because too many of the wretched things appear.

http://www.economist.com/displaystory.cfm?story_id=5310558
http://catless.ncl.ac.uk/Risks/24.12.html#subj8.1

===============
4. Seasonal greetings

I do like it when I can say “I told you so.” Back in June I wrote a
piece in my newsletter about hidden data in documents, and pointed
out that the problem existed in Excel as well as Word; early in
November Westpac was forced to halt trading on its shares and
deliver its annual profit briefing a day early after it
accidentally sent its results by email to research analysts. It had
sent out a spreadsheet containing the results for previous years;
it also contained the latest results, obscured by blacking out the
relevant cells. The cells may have looked black on the screen, but
it didn’t need advanced technical skills to find out what they
contained.

http://makeashorterlink.com/?I23D21A5C

May all your communications be intended, all your spreadsheets
reliable, and all your risks well managed in 2006!

===============
5. Newsletter information

This newsletter is issued approximately monthly by Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2005. All
rights reserved. You may distribute it in whole or in part as long
as this notice is included. To subscribe, email news-subscribe AT
louisepryor.com. To unsubscribe, email news-unsubscribe AT
louisepryor.com. All comments, feedback and other queries to
news-admin AT louisepryor.com. (Change ” AT ” to “@”). Archives at
http://www.louisepryor.com/newsArchive.do.

Newsletter Old site

Newsletter Oct 2005

Post author By Louise Pryor
Post date 28 October 2005

News update 2005-10: October 2005
===================

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).

Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.

In this issue:
1. Reputation risk
2. Blackberry jam
3. Where does the buck stop?
4. If it can go wrong…
5. Newsletter information

===============
1. Reputation risk

It’s early days yet, but it looks as if Southern Water are handling
a big problem well. The story so far: Southern Water were
implementing a new billing system. Among the checks that they
performed as they did so, they reconciled the results that it
produced for various aspects of customer service to the known
results. They found that the new system showed much larger numbers
of customer complaints than they were expecting. They called
auditors in, to check that there was indeed something wrong, and
then reported the problem to Ofwat (the water industry regulator)
and the Serious Fraud Office. Southern Water have asked external
lawyers and accountants to conduct an investigation to examine
“certain inconsistencies” in internal procedures.

The number of customer complaints, and how they are handled, is an
important issue because as part of the regulatory framework water
companies are required to handle complaints in a timely manner, pay
compensation to customers when they fail to do so, and turn in a
report to Ofwat each year with details of numbers of complaints,
how quickly they were handled, and so on. Last year’s report showed
that all 2,500 written complaints received by Southern Water were
replied to within 10 days (the customer gets compensation if it
takes longer). In fact, it now appears that there were closer to
8,000 written complaints, not all of which received timely
replies. In addition, there were other forms of compensation that
had not been paid to customers, for things such as failed
supplies. The current estimate of the outstanding compensation bill
is £2.5m.

In the radio interview that I heard, Les Dawson, the new MD of
Southern Water (this all happened less than a week after he was
appointed) was pressed hard to say what had gone wrong, and who was
responsible. He very reasonably pointed out that the point of
having an investigation was to find these things out. If Southern
Water had stayed quiet about this until after the investigation,
they would have been accused of hiding it.

This episode does, I think, have wider implications. First, the
report the water companies have to submit to Ofwat is externally
audited. So as well as the investigation into internal procedures,
questions will have to be asked about how the auditors were
misled. Second, there are other non-financial reports made to
regulators in other industries. I should imagine (and hope) that
some hasty checking is going on all over the land. Third, it shows
the value of performing proper checks and tests when implementing
new systems. It was obviously some kind of regression testing that
exposed the problem. Also, when you perform regression testing, you
shouldn’t necessarily assume that a difference shows that there is
a problem in the new system. Finally, you have to wait for the
results of an investigation before being able to say what its
conclusions are. We just do not know yet how the problem arose:
incompetence, fraud, poor systems design, or something else.

http://tinyurl.com/96v3x
http://news.independent.co.uk/business/news/article322320.ece

===============
2. Blackberry jam

Are you one of those people who is almost physically addicted to
your Blackberry? Never go anywhere without it, and check for new
emails every five minutes? Then be thankful that you don’t work for
the BBC, which has suspended its Blackberry service. Apparently
they were getting the equivalent of crossed lines: people reported
receiving parts of messages not intended for them. When the problem
became known, users started frantically checking that they had not
sent any confidential or potentially embarrassing messages that
could have gone astray. Apparently the service has now been down
for a week, and it is likely to stay down for at least another
fortnight.

I haven’t been able to find anything out about what the problem is,
except that there was one reference to an “obscure bug.” Well,
that’s all right then. If it’s obscure, it can’t matter too
much. Actually, no. In this case, obscure probably just means “it’s
not something we thought of checking,” or “it can only happen in
really weird circumstances, which turn out to have actually
occurred.”

I also don’t know where the problem is: in the Blackberry software
itself, in the BBC’s email system (run by Siemens), or somewhere in
Vodaphone’s systems (who supply the wireless bit). If it’s in the
Blackberry software itself (or in Vodaphone’s systems), a lot of
users should be very worried indeed. If it’s in the BBC’s email
system, which from some hints in some of the press coverage seems
the most likely, the problem is less wide ranging, but could still
affect other organisations using the same type of email system.

This episode shows that you can never be sure that complex systems
will work properly. Presumably the BBC folk had been happily using
their Blackberries for some time, with no obvious signs that there
was anything wrong. Then, probably, some small change was made to
the system, that wasn’t expected to have any very serious effects,
and hey presto! Little fragments of stray emails all over the
place.

Meanwhile, people at the BBC are having to slum it with other,
older, technology: phones and computers.

http://www.guardian.co.uk/uk_news/story/0,3604,1601142,00.html

===============
3. Where does the buck stop?

Whose fault is it when there is a security hole in a piece of
software, or an implementation bug in a financial model? A former
White House cybersecurity adviser recently said that software
developers should be personally responsible for security holes in
the software they write. Howard Schmidt said “In software
development, we need to have personal quality assurances from
developers that the code they write is secure.”

Although this idea may seem attractive at first, it shows a
misunderstanding of how software is written. It is only rarely that
a single person is responsible for all aspects of code design and
development.

On a large project, there might be the following people involved:
business analyst, software architect, lead developer, several other
developers, database specialist, usability specialist. For some
projects there may be more. In addition, there may be inputs from
the marketing and quality assurance (testing) teams. Even during
code development (leaving aside the specification and design
stages) there may be collaboration between developers, and formal
code reviews. How can you possibly assign the blame for any
problems to a single person?

Even pinning it down to a small group is difficult, as there are so
many different influences affecting the final form of the
code. Whatever the size of the project (a single spreadsheet
through to an enterprise level software solution) there are always
trade-offs between the functionality that is included, the
usability, the delivery time, and the level of testing and
review. The developers often feel that the marketing folk are the
most responsible for errors: if they hadn’t promised the
impossible, the developers might have been able to deliver a
quality version of the possible. It is unreasonable to give the
responsibility for errors to a group of people if you don’t also
make it possible for them to deliver an error-free product, for
example by giving them enough time and budget to do so.

Even when there is just one person developing a simple spreadsheet
there would often be a good argument that the developer should not
be held responsible for all the mistakes. If there is intense
pressure to produce an answer to a deadline, there may not be time
to perform all the necessary testing and reviewing. The corporate
culture may not encourage (or even discourage) a disciplined
approach to spreadsheet development. The developer may have had no
training in software development techniques (after all, anyone can
write a spreadsheet, can’t they?)

Although spreadsheets are often used as part of routine business
processes, they have often not been through the type of quality
control that other software has. Again, it may not be the fault of
the individual developer, but the fault of the managers who don’t
understand the risks involved.

If you would like to know how I can help you have more confidence
in the spreadsheets and other user-developed software that your
organisation uses, send me an email at the address below.

http://news.zdnet.co.uk/0,39020330,39228663,00.htm

===============
4. If it can go wrong…

You may have noticed that there has been a slight hitch in the
monotonous regularity with which these newsletters appear. One
thing I have learned over the past few months is not to take things
for granted. For example, don’t assume that getting your boiler
serviced just after major house redecoration is a mere
formality. You might find that you have a blocked flue and have to
resite the boiler. A house sale can fall through up to about three
weeks before the moving date (and possibly later, though I don’t
have an actual example of that). Even if you are generally healthy,
don’t assume that you won’t break your leg just before you move
house. Just because both BT and your ISP say that your line is
fine, it doesn’t mean that either ADSL or dial-up will actually
work without a visit from the BT engineer. However, normal service
has now been resumed, and we apologise for any inconvenience
caused.

As ever, if you have any comments on any items in this newsletter,
or any suggestions for future issues, just drop me an email at the
address below.

===============
5. Newsletter information

Newsletter Old site

Newsletter Jun 2005

Post author By Louise Pryor
Post date 22 June 2005

News update 2005-06: June 2005
===================

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).

Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.

In this issue:
1. Hidden text and meta-data
2. What is a spreadsheet?
3. Forthcoming events
4. Phishing
5. Newsletter information

===============
1. Hidden text and meta-data

Every so often there is an article in the press about hidden text
being left in a file that is released to the public: it usually
involves a Word file, but it’s possible for the same sort of thing
to happen in Excel, or even PowerPoint, too. For instance, in April
the Pru managed to send out a version of their first-quarter new
business figures in which recipients could see many of the changes
that had been made during the preparation of the document. It’s not
only text from previous drafts that may be present for all to see,
but also other information, such as the original author of the
document.

I am always slightly surprised that important documents are sent
out as Word or Excel files in the first place: surely a format that
is intended to be edited just isn’t suitable, unless you want the
recipients to edit it? Just think of the risks that may arise if
the recipient can either edit the file or see changes that were
made in the past. You may be misrepresented, fraudulent use made of
the modified documents, confidential information may leak, and so
on. Although there has as yet been no big scandal from this sort of
problem, I am sure that there are many more occurrences and many
more problems, some of them resulting in monetary losses, than we
hear about in the press.

There are some simple steps that can be taken to minimise the
risks. They are not foolproof, but add little extra burden on to
the user (so are at least practical to apply). Useful techniques
include:

– Password protect the file to make it read-only. This doesn’t
provide a lot of protection (Microsoft Office passwords are
generally easy to break, and it’s possible simply to cut and
paste the contents into another document) but does at least stop
inadvertent changes.

– Use the options that Word and Excel have to “Remove personal
information from file properties on save”.

– Use the Word option to “Warn before printing, saving or sending a
file that contains tracked changes or comments”. To remove
tracked changes, accept them all and turn tracking off before
saving.

– Don’t allow fast saves in Word: if these are enabled, old text
may be saved even if you don’t track changes.

– In Excel, protect sheets and workbook structure so that users
can’t make unwanted changes.

– Use Microsoft’s “Remove Hidden Data” add-in, which lets you
remove hidden data from Word, Excel and PowerPoint files.

– Send out pdf files instead. Although it is possible to edit these
if you have the right software, most people can only read
them. There are various ways of producing pdf files, including
some commercial software packages, but the easiest is to install
a free package like pdf995, PrimoPDF, or CutePDF Writer. These
add an option to your list of printers, so that instead of
printing to paper you print to the pdf creator which makes a pdf
file.

http://makeashorterlink.com/?C1CF23A4B (Pru’s new business figures)
http://makeashorterlink.com/?A27814267 (Microsoft’s add-in)
http://www.pdf995.com/
http://www.primopdf.com/
http://www.cutepdf.com/

===============
2. What is a spreadsheet?

How many spreadsheets do you have in your organisation? In your
department? On your machine? It’s a tricky question to answer: it
depends on what you mean by a spreadsheet.

You could simply count the files that have an extension of “.xls”
(assuming you use Excel). But then you may well be double counting,
especially if you are a careful person and save interim versions as
you develop a complex spreadsheet. It’s also possible (though
unlikely) that some other application creates files with the same
extension as .xls files. You may be under-counting, if some files
are zipped up to save disk space.

Then there are those half-worked out spreadsheets that were never
actually used for anything, and the others that simply served
instead of the back of an envelope.

And what about all those spreadsheets that are nearly but not quite
the same? Ones that do essentially the same calculations, but on
different data–perhaps for monthly reporting, or for pricing
potential contracts. In many cases these are derived from a central
template, or simply from last month’s version.

Another complication is raised by files that have the same name but
different contents. A naive way of counting might assume that they
are in fact the same spreadsheet, when in fact they might come into
the previous group (same calculations on different data) or might
simply have been given the same name by accident, and in fact
contain totally different calculations.

Of course, the actual count probably doesn’t matter too much in the
end. What does matter is that you know what spreadsheets there are,
what they are used for, and how they are related. It’s important to
have good processes in place both for developing them and keeping
track of them. For instance, if you use a template for pricing, you
should know for any given spreadsheet whether it was derived from
that template, and if so what version of the template, and if there
were any changes from the template. All this is important if you
want to be sure of the results coming out of the spreadsheets. You
must to be able to trace back from the results to the data and
assumptions, and to be able to trust the calculations that are
performed. If you can’t do this, you may run foul of Sarbanes-Oxley
(if it applies to you) or of the FSA, who consistently stress the
importance of systems and controls, especially in ICAS work.

There are a number of techniques you can use to do this, including
documentation inside the spreadsheets themselves, using
systematic naming conventions for both files and folders, and
having good backup and archiving processes. Poor naming conventions
can lead to big problems: natural gas prices in the USA were
significantly affected when a company submitted the wrong week’s
gas storage figures in November 2004. “One explanation for the
error was that the company had used the same computer file name for
each week’s storage balance report, making it easy for the wrong
one to be sent.” Not only easy, but positively likely, I should
have thought. Apparently the company in question has now changed
its naming conventions. (Thanks to Patrick O’Beirne for noticing
this report, and posting it on the EuSpRIG spreadsheet errors
page).

http://makeashorterlink.com/?W65C21A4B
http://www.eusprig.org/stories.htm

===============
3. Forthcoming events

The 2005 EuSpRIG conference is being held at the University of
Greenwich on 7th and 8th July. Details are at
http://www.eusprig.org/ . The theme this year is “Managing
Spreadsheets in the light of Sarbanes-Oxley”, so it promises to be
both interesting and timely.

The FSA is holding a summer school from 21st-24th August at St
John’s College, Cambridge. Its aim is “To provide firms with a
unique insight into strategic thinking of senior levels within the
FSA.” There will be presentations from senior FSA staff, and some
case study work in small groups. Details are at
http://www.fsa.gov.uk/Pages/Doing/Events/events/summer.shtml .

There is a half day seminar on “The strategic management of risks”
at the Institute of Actuaries in London on 23rd November. Details
are at
http://www.actuaries.org.uk/files/pdf/cpd/risks_seminar_20051123.pdf

The Actuarial profession will be holding a seminar on Operational
Risk in December; no details have yet been announced.

===============
4. Phishing

There is so much phishing spam at the moment that sensible people
won’t reply to, or click on any link in, any email from a financial
institution. Many of them rely on people reading their email in
html: I read mine in plain text, and the links in the emails are
often perfectly genuine. It’s only in the html, in which the
highlighted text need have nothing to do with the actual link that
will be used, that the danger applies. Even then, some simple
observation will often indicate that something is wrong. When I
view email in html, and hover the cursor over a link (without
clicking) the link destination is displayed. If it is not the same
as the text of the link, smell fish!

Phishing isn’t limited to email, either. I recently received a
telephone call purporting to be from a bank with whom I have a
credit card, saying that they had noticed possible fraudulent
activity on my account. Would I give them the day and month of my
birthday in order that they could be sure they were speaking to the
correct person? Well, no, I wouldn’t. But it wasn’t my full birth
date, only the day and month. I still refused, and asked if there
was a number I could call them back on so that I could be sure I
really was talking to the bank. They gave me an 0845 number, but it
just got through to a recorded message. I later checked this out
with the real bank, and they confirmed that it was a scam, and that
the number was not one of theirs.

Apparently techniques similar to those used in phishing attacks are
now being used to gather commercially or economical valuable
information from companies. The National Infrastructure Security
Co-ordination Centre (NISCC) has recently issued an alert
describing these attacks. The alert says:

“The emails employ social engineering, including use of a spoofed
sender address and information relevant to the recipient’s job or
interests to entice them into opening the documents.

“Once installed on a user machine, trojans may be used to obtain
passwords, scan networks, exfiltrate information and launch further
attacks.”

Apparently the attacks, which appear to originate in the Far East,
normally focus on people who work with commercially or economically
sensitive data.

Just like phishing, these attacks rely on the unwitting cooperation
of the recipient of the email. As a reminder, NISCC confirms that
the following precautions are sensible:

– Don’t open attachments unless the email is consistent with
previous communications from the sender, and the attachment has
been scanned for viruses

– Don’t use the preview pane in your email program

– View emails in plain text, not html

These are all precautions that the home user should take, too. In
addition, you should make sure that you use a good virus and
malware detection program on your home machine, and have applied
all security updates to your operating system and other software.

As an aside, the phishing problem is the reason that I now use
www.makeashorterlink.com to abbreviate URLs. When you click on one
of their links, you will be shown the actual URL before you are
taken to the page.

http://www.niscc.gov.uk/niscc/docs/ttea.pdf

===============
5. Newsletter information

Newsletter Old site

Newsletter Mar 2005

Post author By Louise Pryor
Post date 31 March 2005

News update 2005-03: March 2005
===================

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).

Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.

Subscribe by sending an email to news-subscribe@louisepryor.com.
Unsubscribe by sending an email to news-unsubscribe@louisepryor.com.
Newsletter archived at http://www.louisepryor.com/newsArchive.do.

In this issue:
1. There’s a little man inside the box…
2. Somebody else’s problem
3. FSA update
4. Count on failure
5. Newsletter information

===============
1. There’s a little man inside the box…

The clocks changed in the UK at the weekend, as they do twice a
year. So you’d think that computer systems would be able to cope,
and that there would be no major disruption. And, on the whole,
you’d be right, though you wouldn’t necessarily know it from the
press coverage.

About 1,500 Barclays ATMs (out of a total of about 4,000) were out
of action for over 12 hours on Sunday. We were told that a manager
put the clocks back rather than forward, and that this mistake had
caused the problems. The Daily Telegraph carried a leader opining
on the lessons that Barclays could learn from its employee’s
blunder.

But hang on a minute: A real live person, changing the clocks in
the data centre at 01:00 on Sunday morning? It just doesn’t make
sense. Why on earth wouldn’t the time change be automated? After
all, it is in just about every other computer in the world. Did you
have to change the time on your PC this weekend?

And in fact, Barclays say that it was a hardware fault, and not
related to the time change at all. This is much more plausible, and
is what I heard a Barclays person say on the radio. But if it’s
true, where did the story of the error-prone manager come from? The
Telegraph said that they had it from customer services staff.

I imagine it happened something like this: The ATMs go down. (And,
it appears, the online banking too). Calls pile into the call
centre. Nobody at the call centre knows what the problem is. (And
why should they know? They are not omniscient, and these things
often take time to track down.) They are talking to each other
about what is going on. Someone says that it must be something to
do with the clocks changing, as that’s something that doesn’t
happen every day. And someone else says “Yeah, I bet that’s
it. Some stupid person changed them in the wrong direction!” And
before you know where you are, an off the cuff remark (probably
made in jest) has spread around the call centre and becomes the
official version.

People are very unwilling to believe in coincidences. They also
have mental models of how things work. And surprisingly often,
those mental models boil down to a little man in the box (or, in
this case, in the data centre). So when they were told that the
problem arose because a person made a mistake, they didn’t stop to
think about whether the story really made sense.

http://news.zdnet.co.uk/hardware/0,39020351,39193138,00.htm
http://makeashorterlink.com/?M170229CA
http://www.forbes.com/facesinthenews/2005/03/28/0328autofacescan05.html
http://edition.cnn.com/2005/BUSINESS/03/28/barclays.machines/

===============
2. Somebody else’s problem

There have been a number of stories about outsourcing and its
problems recently, though they are rarely expressed as such. To
mammothly over-simplify, the trouble with outsourcing is that you
lose control, the benefit is that you offload the problems onto
someone else. The risk is that there are gaps: the outsourcer
doesn’t deal with the problems, and you no longer can.

Computer security is a prime candidate for outsourcing. Specialists
can do a much better job of keeping up with all the latest threats
and how to deal with them. But a number of organisations recently
lost whole tranches of email messages because there was a bug in a
system that an outsourcer used for email scanning. When the update
mechanism tried to install the updates on the customer networks,
the system started to delete all emails by default. Oops! At least
one customer claimed that someone at the outsourcer said that the
update hadn’t been tested, but this was denied.

http://news.zdnet.co.uk/internet/security/0,39020375,39189933,00.htm

Internet hosting is also outsourced. There are comparatively few
organisations that have the expertise or funds to run a full data
centre themselves; it’s a field in which there are very definitely
economies of scale. Interestingly, there are often two or three
layers of outsourcing: the end customer uses an ISP, who in turn
uses one of the big data centres (or may bulk buy from another ISP,
who uses…). So if anything goes wrong in one of the big data
centres, the effects are widely felt.

Which is just what happened recently. A routine test was being
carried out when a fault developed in a switchgear panel (whatever
that is). This caused a short circuit in the UPS (uninterruptible
power supply) modules, so everything moved to battery power. The
fire alarms also went off (I can’t make out whether this was
connected, or just a coincidence), the building was evacuated and
everyone stood around outside while the batteries ran down.
Customers suffered hours of downtime, and a number of them had
equipment destroyed by a power surge that occurred at some point
during the episode.

One of the problems here for the end customer is that they may not
even know where the chain ends for them, and so have next to no
chance of really being able to manage the risks. I believe that the
physical bits and bytes that make up my web sites currently live in
Calgary, for example, but when I chose my hosting company I was at
least as interested in the software they supported as the historic
uptimes. And I didn’t do any work on finding out whether I expected
future performance to reflect historic, or whether there were
special factors that should have caused me to be wary. (In fact, I
haven’t had any trouble since my last move 18 months ago).

http://makeashorterlink.com/?G3B0219CA
http://news.zdnet.co.uk/business/0,39020645,39190518,00.htm

A recent Gartner survey has pointed out another problem with
outsourcing: it can raise costs. Apparently outsourced customer
service operations can cost almost a third more than those retained
in-house.

http://makeashorterlink.com/?H221249CA

According to Jamie Oliver, this applies to school meals, too.

===============
3. FSA update

The FSA’s new web site appears to have outgrown some of its
teething problems. Many of the old links now work again, which
makes life easier.

New issues of both the General Insurance and Life Insurances
newsletters have appeared. Both of them contain information on the
FSA’s current thinking on various aspects of the ICAS process,
including confidence levels and time horizons.

http://www.fsa.gov.uk/pubs/other/gi_newsletter5.pdf
http://www.fsa.gov.uk/pubs/other/li_newsletter3.pdf

New consultation and discussion papers out this month:
—————————————————–

CP05/4 FSMA 2 Year Review: Financial Ombudsman Service

DP05/1 Integrated Regulatory Reporting (IRR) for: Deposit
takers, principal position takers, and other investment
firms subject to the Capital Requirements Directive

Feedback published this month:
—————————–

PS05/3 Implementation of the Market Abuse Directive

A list of current consultations is available at
http://www.fsa.gov.uk/Pages/Library/Policy/CP/current/index.shtml

===============
4. Count on failure

One of the reasons for Google’s success is that the folk there
count on bad things happening. It’s well known that they use
large numbers of cheap machines for the heavy computations that are
involved in indexing so many web pages, instead of buying expensive
supercomputers. A normal PC might fail once in three years (that
seems a bit optimistic to me), so if you have thousands of them you
can expect on the order of one failure a day. So they assume that
failures will happen, and develop systems to handle them.

http://makeashorterlink.com/?V2A1149CA

It’s obvious when you put it like that: clearly you should allow
for anything that happens as often as once a day. But where do you
draw the line? And how can you tell how often failure is likely to
occur? Consider spreadsheets, for example (I had to get there
eventually…) How often are you likely to get something going
wrong in a spreadsheet? An optimistic estimate is that about 1% of
unique formulae will have errors in them. A spreadsheet only has to
have about 69 unique formulae to be more likely than not to contain
an error. And that’s not a particularly large spreadsheet. So what
do you do about it? Testing, reviewing, good development
processes… If you want to know more, do get in touch!

I discussed banks, robberies and phishing in the last issue. This
month it came to light that key logging software was used in an
attempt to steal £220 million from a Japanese bank in the
city. Arrests have been made. The bank’s security worked well, in
that it was internal security officers who first spotted the
attempt.

http://www.timesonline.co.uk/article/0,,2-1529429,00.html

===============
5. Newsletter information

This newsletter is issued approximately monthly by Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2005. All
rights reserved. You may distribute it in whole or in part as long
as this notice is included. To subscribe, email
news-subscribe@louisepryor.com. To unsubscribe, email
news-unsubscribe@louisepryor.com. All comments, feedback and other
queries to news-admin@louisepryor.com. Archives at
http://www.louisepryor.com/newsArchive.do.

Newsletter Old site

Newsletter Feb 2005

Post author By Louise Pryor
Post date 28 February 2005

News update 2005-02: February 2005
===================

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).

Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.

Subscribe by sending an email to news-subscribe@louisepryor.com.
Unsubscribe by sending an email to news-unsubscribe@louisepryor.com.
Newsletter archived at http://www.louisepryor.com/newsArchive.do.

In this issue:
1. Where the money is
2. Is it risky?
3. FSA update
4. Is testing worth it?
5. Newsletter information

===============
1. Where the money is

Apparently Willie Sutton, the bank robber, said when he was asked
why he robbed banks, “because that’s where the money is”. It’s a
statement of the blooming obvious, and holds true even today, when
the mechanics of robbing banks have moved on from the tried and
tested methods used 50 or a hundred years ago (although the old
methods are still used: the Northern Bank robbery in Belfast in
December involved huge amounts of cash–see
http://news.bbc.co.uk/2/hi/uk_news/northern_ireland/4117219.stm).

Nowadays, you don’t actually have to physically break in to a bank
vault to get your hands on money that isn’t yours. Instead, you can
commit some sort of cybercrime. The problem (if you are a
fastidious bank robber) is that in most cases you are no longer
targeting a large anonymous institution, but are stealing directly
from the bank’s customers. However, few bank robbers have many
scruples, and there are now several more options open to such
people: interfering with ATMs, so as to get card details which you
can then use fraudulently; phishing, or trying to get people to
give you their personal details such as account numbers and
passwords (ditto); and using some kind of spy-ware to log their key
strokes, thus letting you listen in on the personal details which
you can then use in the usual way.

All these methods take advantages of weaknesses in the overall
security systems of the banks. ATM fraud is based on physical
weakness: it is possible to interfere with the actual, physical ATM
machines. The other two are based on targeting the weakest link in
the whole system, the bank’s customers and their computer system. I
call this the weakest link because it is an area over which the
bank has no control. It can try to improve security in this area by
educating its customers, and by introducing security procedures and
authentication mechanisms that make it harder for fraudsters to
impersonate genuine customers, but there are huge difficulties in
making these measures effective.

There may, however, be new incentives for the banks to take these
issues even more seriously than they do at present (and although I
am using banks as an example, the issues are the same for any
online service provider). A Miami business man is suing his bank,
because he claims the bank is liable for the losses he suffered
through fraud. Apparently his computer was infected with a virus
that logged his keystrokes, enabling someone to steal $90,000 from
his account. He says that the bank should have told its customers
about the virus and the dangers it posed.

http://outjacay.notlong.com

This whole area raises a number of tricky questions:

– To what extent should banks (or other online service providers)
be responsible for educating their customers in computer security
matters? It seems to be generally accepted that they should warn
customers about not disclosing passwords and PINs, being wary of
unsolicited emails that could be phishing scams, and so on, but
what about specific viruses? Should they encourage their
customers to use spy-ware detectors, virus protection software and
firewalls? What about using public access computers, in libraries
or internet cafes? Or security measures for their home wifi
networks?

– If online service providers try to improve their security
measures, especially as far as user authentication is concerned,
their sites are often perceived as being more difficult to use,
so they lose customers. Where should the balance lie? Is it again
a question of educating the user to accept more complex
procedures?

– Would biometric authentication mechanisms help? Or could packet
sniffing software be used to mimic fingerprints or iris patterns?
And in any case would users be prepared to invest in the
necessary hardware?

As a footnote, it appears that Willie Sutton never actually made
the smart alec remark that is attributed to him. Also, he was
usually known as Bill, not Willie. He actually said “Why did I rob
banks? Because I enjoyed it. I loved it. I was more alive when I
was inside a bank, robbing it, than at any other time in my life. I
enjoyed everything about it so much that one or two weeks later I’d
be out looking for the next job. But to me the money was the chips,
that’s all.”

http://www.banking.com/aba/profile_0397.htm

===============
2. Is it risky?

The latest “Banana Skins” survey has identified regulatory overkill
as the biggest risk facing banks today. The survey covered 440
respondents from 56 countries.

http://www.csfi.org.uk/Banana%20Skins%20Press%20Release.pdf
http://munrippi.notlong.com

When I first read about this, I was amazed. Regulation the biggest
risk? You mean *regulation* is going to make banks go bust? And
surely the major thrust of most current regulation is to reduce
risk–have the regulators really got things so horribly wrong? Then
I realised that we weren’t being given the full context. Risk is
not a concrete entity, sitting out there in glorious isolation
waiting to be identified. There are only risks in relation to
goals, or other desirable outcomes.

The respondents to the survey were probably thinking of the risks
to their profits. Heavier regulatory requirements undoubtedly
result in heavier costs to those who are regulated. However, they
may reduce the risks of customers losing their money, and the risks
of the regulatory bodies failing to achieve their objectives. For
instance, the FSA make it vary clear that the risks they are
worried about are the risks to their statutory objectives;
inasmuch as these risks coincide with risks to the profits or
shareholder value of the firms that they regulate, all well and
good, but that’s really a side effect.

When you are discussing the riskiness or otherwise of various
courses of action it’s important to be very clear about the
context. The risk to what? Often, it is assumed that all parties to
the conversation share the same context, which is never made
explicit. This can lead to quite violent disagreements, which turn
out to be based on problems of definition rather than on
fundamental differences.

I’ve been at a couple of gatherings of actuaries recently at which
the discussion turned to the topic of whether cash is more or less
risky as an investment than equities. This is a question to which
there is no correct answer. Whether cash is more or less risky
depends on what you are trying to do. If you are investing in order
to meet future outgoings as they fall due, it obviously depends on
what the outgoings are. For fixed monetary amounts in the short
term, cash is beautifully risk free. For real amounts in the longer
term, especially if the inflationary outlook is uncertain, cash
starts to look much more dodgy.

In the investment world volatility is often used as a synonym for
risk. And indeed, a highly volatile investment is risky if you are
interested in getting a pay-out defined in nominal terms on a
specific date. Over a longer term, especially when you can choose
when to disinvest based on circumstances at the time, volatility
becomes less of an issue. Again, the context is all-important.

===============
3. FSA update

The FSA have a new web site. I haven’t used it enough yet to be
able to tell whether it’s an improvement or not (I had no
particular gripes about the old site), but I do find it annoying
that a number of my bookmarks no longer work. For example, there
used to be a handy page that listed current consultations, with
dates by which responses should be received by the FSA. I haven’t
been able to track down the new version of this, if it exists. In
addition, the URLs for all existing documents have changed. The
listing of publications by date is less comprehensive than before,
as it no longer includes press releases, Dear CEO letters or
speeches.

I found the Dear CEO letter on credit derivatives somewhat
interesting (I never thought I’d hear myself say those words, or
maybe I mean see myself write them…) It turns out that insurance
companies are not years behind the banks on all risk management
issues. It’s been known for some time in the insurance world that
the habit of not finalising contracts is a risky one, and there has
been a bit of a push on to try to improve practice in this
area. Well, apparently traders in credit derivatives have the same
problem. Some transactions remain unconfirmed for months. The risks
are obvious.

http://www.fsa.gov.uk/pubs/ceo/derivatives_22feb05.pdf

New consultation and discussion papers out this month:
—————————————————–

None

Feedback published this month:
—————————–

PS05/02 Insurance regulatory reporting: changes to the publicly
available annual return for insurers – Feedback on CP202
and CP04/1 and made text

===============
4. Is testing worth it?

In my last newsletter I rather glibly said that, with hindsight,
the Huygens team got their cost benefit analysis completely wrong
when deciding whether to subject every system to a simulation of
the exact signals and conditions it would experience during
flight. Alan Chaplin quite rightly pointed out that this was an
over simplistic comment. As he says, it’s likely that this wasn’t
the only test that was omitted. He also gave a very succinct
summary of how the analysis should be performed:

The cost benefit analysis on the test runs something like:

Cost of test = m
Probability of finding a problem = x
Cost of fixing problem = n
Value of fixing a problem found by test = b

So cost = m + (n * x)
Benefit = (b * x)

Carry out test if benefit > cost

So if x is very small the cost benefit analysis says don’t do
it. The fact that the error did turn out to exist does not
necessarily mean that the probability a priori was wrong.

This is absolutely right. The difficulty is estimating x, n and
b. It’s difficult to know what information the Huygens team had
available at the time, and so whether they made the right decision
or not.

In my view x, the probability of finding a problem, is especially
difficult. It depends on so many things, including the coverage of
the tests you’ve done so far – and on the whole I think we tend to
overestimate the efficacy of our tests and underestimate the a
priori likelihood of mistakes.

n, the cost of fixing a problem, is not easy either, as it varies
so much depending on what the problem is. In this case they were
pretty lucky that it turned out to be possible to fix it at all, it
seems. And if you have no idea of what sort of problems, if any,
may be uncovered by your tests, how can you possibly tell what it
will cost to fix them?

And of course b, the value of fixing a problem, depends on what the
problem is. In some cases this is very high indeed: if the problem
means that the whole mission fails, for example. This is subject to
many of the same issues as estimating the cost of fixing a problem.

I have no really good answers to this. Through experience on
numbers of similar projects one gets a reasonable feel for the
centres of the distributions — ie, typical numbers and sizes of
problems found. But on a one-off project, or out in the tail of the
distributions, it’s not easy at all.

===============
5. Newsletter information

This newsletter is issued approximately monthly by Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2005. All
rights reserved. You may distribute it in whole or in part as long
as this notice is included. To subscribe, email
news-subscribe@louisepryor.com. To unsubscribe, email
news-unsubscribe@louisepryor.com. All comments, feedback and other
queries to news-admin@louisepryor.com. Archives at
http://www.louisepryor.com/newsArchive.do.

——————————————————————–
The Edinburgh Bach Choir will be performing Bach’s St Matthew Passion
at St Cuthbert’s Church, Lothian Road on Saturday March 12th. See
http://www.edinburghbachchoir.org.uk for more details. Tickets from
the Usher Hall or members of the Choir.

Newsletter Old site

Newsletter Jan 2005

Post author By Louise Pryor
Post date 31 January 2005

News update 2005-01: January 2005
===================

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).

Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.

Subscribe by sending an email to news-subscribe@louisepryor.com.
Unsubscribe by sending an email to news-unsubscribe@louisepryor.com.
Newsletter archived at http://www.louisepryor.com/newsArchive.do.

In this issue:
1. Safety nets can be dangerous
2. Rocket science
3. FSA update
4. Disastrous IT projects
5. Newsletter information

===============
1. Safety nets can be dangerous

A good way to prevent a bad thing happening is to add more ways of
stopping it, right? Wrong. Adding more redundancy often increases
the risk. The following four factors may come into play:

– Redundancy only works if the different methods (systems,
procedures, or whatever) are truly independent. If they are not,
and sometimes the failure of one can make the others fail too,
the redundancy is more apparent than real. For example, you
might think that adding more engines to an air-plane would make
crashes from engine failures less likely. In fact, this is only
true up to a certain limit, depending on the plane and the
engines concerned. If you have more engines it is more likely
that one of them will fail; and if the failure is likely to be
catastrophic, such as by starting a fire that destroys all the
other engines, the risks of having more engines may outweigh the
possible benefits.

– There is a well know psychological phenomenon known as bystander
apathy. The more witnesses there are to an accident, the less
likely each individual witness is to call an ambulance.
Similarly, the more people responsible for performing a check,
the less thorough each will be. Each thinks “there are n other
people doing this – if there is anything wrong, one of them is
bound to spot it”. If everybody is responsible then nobody takes
the responsibility.

– If a system is believed to be safer, people are likely to take
more risks with it. They drive faster if they are wearing seat
belts. Add two levels of warning to a system and people will
ignore the first level.

– Dedicated workers will bypass safety systems if they interfere
with what they see as their primary responsibilities. If it’s
important to get a report out on time, people will not bother to
password protect it or update the security systems on their
computer. Software developers will code all night, but not bother
with backups, source control or systematic testing. Fire doors
are propped open.

The basic arguments are set out in an excellent paper by Scott
Sagan. He is discussing the problems of making nuclear
installations more secure against terrorism, but of course the
principles apply to all types of risk management systems and
processes. Further discussion and some great examples were provided
by Don Norman and Geoffrey Newbury in the Risks forum.

As Norman points out, three of these four factors are psychological
rather than technical. They should be taken seriously when devising
any method of risk control.

http://cisac.stanford.edu/publications/20274/
http://catless.ncl.ac.uk/Risks/23.63.html#subj11.1
http://catless.ncl.ac.uk/Risks/23.64.html#subj10.1

===============
2. Rocket science

Wasn’t the Huygens landing a great way to start the year! The
pictures were stunning, even though there were fewer of them than
expected.

As you probably recall, Huygens was dropped onto Titan from the
Cassini interplanetary probe. Signals from Huygens were received by
Cassini and then transmitted back to earth; direct transmission was
not possible because of the limited power available to Huygens.
There was redundancy built into the design, with two radio channels
being used for the communications, so that if one failed the data
would still get through on the other. In the event, one of the
channels did fail (a software error: Cassini’s receiver was never
told to turn on). But guess what? The two channels weren’t fully
redundant. Channel A, the one that failed, was the only one
carrying data that would help measure wind speeds. And half the
pictures taken during the descent were sent only over channel A,
and the other half only over channel B.

Redundancy only works if it’s genuine (see above), and it can
sometimes be subverted by people trying to do their job (in this
case, trying to get as much information back from Titan as
possible).

However, it turns out we were lucky to get any pictures at all.

When Cassini was launched in 1997, the team were pretty confident
that things would go well. Cassini and Huygens had been thoroughly
tested on the ground, both separately and together. But there was
one test that had been omitted: they decided not to subject every
system to a simulation of the exact signals and conditions it would
experience during flight, because this would have meant
disassembling some of the communications components. It would have
been time consuming and expensive to do this, then reassemble,
retest and recertify them. It was a simple cost-benefit analysis,
which in hindsight was completely wrong.

A couple of the team in Darmstadt were worried that this test had
not been performed, and eventually persuaded mission control that
it would be possible to perform a similar test during Cassini’s
long trip out to Saturn. They devised a test to send a signal from
Earth to Cassini to simulate the signals that Cassini would receive
from Huygens during the landing. Cassini could then echo the
information back to Earth, where the team could tell whether it had
been received and deciphered correctly.

The test failed. It turned out that the reception mechanism on
Cassini had not been designed to account correctly for the Doppler
shift in the signal caused by the high relative acceleration
between Cassini and Huygens. Rather annoyingly, the problem could
have been fixed by some trivial parameter changes in the firmware,
but once Cassini had left Earth these changes could no longer be
made.

Eventually they worked out a way of changing the trajectory of
Huygens so that the Doppler shift would be reduced. This is why the
landing wasn’t until this January, instead of late 2004 as
originally planned.

There are some obvious morals to this story. However much testing
you do, it may always be the next test you do that uncovers the
problem. And the problem may be a big one. Just because you’ve done
99% of the testing it doesn’t mean that the system is 99% likely to
work.

Moreover, thorough review is no substitute for testing. The problem
was spotted by none of the design reviews of the communications
link. An issue that was overlooked by the design team was also
overlooked by the reviewing team. There are some hints in some of
the published accounts that the reviewing team didn’t imagine that
the design team could possibly overlook the effects of the Doppler
shift: so don’t make unwarranted assumptions, and do check up on
any assumptions you make.

The full IEEE Spectrum article on this episode contains all the
details.
http://www.spectrum.ieee.org/WEBONLY/publicfeature/oct04/1004titan.html

Wikipedia has a useful summary, while the ESA site is the horse’s
mouth.
http://en.wikipedia.org/wiki/Huygens_probe
http://www.esa.int/SPECIALS/Cassini-Huygens/index.html

===============
3. FSA update

The FSA have just issued their annual Financial Risk Outlook and
Annual Plan. If you want to know what is on their mind, these two
documents are vital reading.

http://www.fsa.gov.uk/pubs/plan/financial_risk_outlook_2005.pdf
http://www.fsa.gov.uk/pubs/plan/pb2005_06.pdf

New consultation and discussion papers out this month:
—————————————————–

CP05/01 Quarterly Consultation (No 3)
CP05/02 Regulatory fees and levies 2005/06
CP05/03 Strengthening capital standards. Including feedback on
CP189

Feedback published this month:
—————————–

PS04/28 Lloyd’s: Integrated prudential requirements and changes to
actuarial and auditing requirements – Including feedback
on CP04/7, CP04/13 (part) and CP04/15 (part) and ‘made
text’
PS05/01 Treating with-profits policyholders fairly – Feedback on
CP04/14 and made text

Current consultations, with dates by which responses should be
received by the FSA, are listed at
http://www.fsa.gov.uk/pubs/2_consultations.html

===============
4. Disastrous IT projects

We often hear about disastrous IT projects. Stories recently have
included:

– A new system for keeping track of the MoT status of cars is
running two and a half years late. The budget (in 2003) was
£230m. I haven’t been able to find out what the budget overrun
is.

– The computer systems at the Child Support agency don’t work and
are about £29m over a £456m budget.

– An FBI system intended to be an important tool in fighting
terrorism may be scrapped because it doesn’t work and may never
work. So far it has cost $170m.

http://www.theregister.co.uk/2004/12/30/pcs_slams_mot_system_delays/
http://www.pcw.co.uk/news/1160762
http://news.bbc.co.uk/1/hi/uk_politics/4205221.stm
http://www.cnn.com/2005/US/01/13/fbi.software/

The amounts of money involved in spreadsheet errors are sometimes
comparable. For example, the state of New Hampshire has recently
had to find an extra $70m in its budget.

http://www.theunionleader.com/articles_showfast.html?article=50185

It seems to me that the press coverage of both types of problem
gives a misleading impression. Although a number of IT projects are
spectacular disasters, there are many others that are extremely
successful. And the disasters are rarely caused entirely by IT
factors; they often go hand in hand with more general management
problems. On the other hand, there are probably far more problems
arising from spreadsheets than we ever hear about. But spreadsheets
don’t count as IT. They probably should.

===============
5. Newsletter information

This newsletter is issued approximately monthly by Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2005. All
rights reserved. You may distribute it in whole or in part as long
as this notice is included. To subscribe, email
news-subscribe@louisepryor.com. To unsubscribe, email
news-unsubscribe@louisepryor.com. All comments, feedback and other
queries to news-admin@louisepryor.com. Archives at
http://www.louisepryor.com/newsArchive.do.

——————————————————————–
The Edinburgh Bach Choir will be performing Bach’s St Matthew Passion
at St Cuthberts Church, Lothian Road on Saturday March 12th. See
http://www.edinburghbachchoir.org.uk for more details. Tickets from
the Usher Hall or members of the Choir.

Newsletter Old site

Newsletter Dec 2004

Post author By Louise Pryor
Post date 22 December 2004

News update 2004-12: December 2004
===================

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).

Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.

Subscribe by sending an email to news-subscribe@louisepryor.com.
Unsubscribe by sending an email to news-unsubscribe@louisepryor.com.
Newsletter archived at http://www.louisepryor.com/newsArchive.do.

In this issue:
1. Year end
2. Certified models
3. FSA update
4. Seasonal greetings
5. Newsletter information

===============
1. Year end

The end of another year, and what an end it is for some! The first
ICAS for UK insurance companies as well as the first year end for
reporting under Sarbanes-Oxley for many organisations. All the new
requirements are creating huge amounts of extra work for the
auditors and consultants. There are occasional reports of how tough
it all is, and how some large(ish) number of companies are not
going to comply with the Sarbanes-Oxley section 404 deadline (75
days after their year end date, for the the auditor to identify
‘any material internal control weakness’ or ‘significant
deficiency’, in verifying that management has sufficient
operational command to produce reliable and compliant financial
reports). I saw one report that said that 300 companies have
already warned the SEC of their likely non-compliance.

So what’s going on? Why is it so difficult for companies to satisfy
the auditors that they can produce reliable and compliant financial
reports? The obvious answer is that a number of companies really
do lack the requisite internal controls, and that they are not
necessarily producing reliable results. In many cases the results
probably are reasonably OK, but it’s more a matter of luck than
judgement. And we’ve seen some cases in which the results weren’t
OK (SunTrust Banks last month, for example).

http://www.computing.co.uk/news/1159434
http://ryanjors.notlong.com

In a company of any size the process of producing financial reports
is incredibly complex, and is only as reliable as its weakest
link. Add in the effects of changes to requirements, which often
lead to people patching the gaps using manual procedures or ad hoc
spreadsheets, and the risks are obvious. Regular readers may
recognise that I am getting on to a familiar hobby horse here. One
aspect of the problem is that you have large, complex software
systems being built by people with no software engineering
expertise. Systems fail and have bugs in even when they are built
by professionals; when built by people who don’t even realise what
they are doing, they are even less likely to be foolproof.

===============
2. Certified models

Steve Rowley sent an interesting response to my piece last month
about the case where the parties concerned had to accept the
result of a spreadsheet error. Steve is a bio-informatics
researcher in the USA.

He says:

There’s an interesting twist on this in the pharma industry.

Particularly in the regulated part of the business (preparing
FDA/EMEA submissions), it’s important that the results you
derive from your data are (a) correct and (b) reproducible.
That leads to 2 tactics: extreme conservatism in study design,
and standardization of tools. Both are along the lines of “use
only what’s known to work”.

The latter is what bears on your case: good laboratory practices
(GLP) or good manufacturing practices (GMP). These are
mind-numbingly complex, rather bureaucratic specifications of
How Things Are Done. They specify every lab technique, every
software package, every bit of computer hardware, documentation
for anybody who wants to repeat the experiment step-by-step, and
so on. If you use ANYTHING else in the process, then you’re not
compliant. (There’s federal regulation in the US, known by the
charming sobriquet “21 CFR Part 11”.)

On the good side, once software has been GLP certified,
everybody trusts it. So the loan calculation story couldn’t
happen, because they’d be using certified software on certified
data. (Well, other things could happen, but not that particular
blunder.)

On the bad side, GLP procedures are completely stultifying.
It’s impossible to do research under such conditions. So often
companies split into 2 parts, only one of which is GLP/GMP
compliant (the government submission part), and one which is not
(the research part).

In fact, there are entire companies whose business is to repeat
your experiments, but this time guaranteeing and documenting
that it’s all GLP/GMP throughout. That is, it’s better to pay
for the experiment TWICE.

Most days, that even makes sense to me. You have one group of
weird, lawless researchers (hey, that’s me!) who figure out new
stuff and another group of button-down careful folk who nail
down every aspect of reproducibility for regulatory submission.

I guess we can bear that expense in pharma, when people’s lives
will depend on the result. It’s slightly unclear how that would
apply to financial services, except that I’m surprised there
aren’t standard, well-trusted, certified packages of software to
calculate things like interest.

There are some interesting comparisons that can be made with
financial services regulation in the UK.

First, the FSA is explicitly not going down the route of approving
specific models.

Also, even if a model has been certified it can still produce the
wrong results. Take the example of calculating loan interest. The
model would presumably make various assumptions about timings of
payments and the interest rate frequency (an annual rate; daily
rate annualised; or whatever). If the assumptions don’t agree with
the actual loan agreement (either because they are hard coded in
the model or someone gets the inputs wrong) then the model will not
produce the correct results. It’s not just the model that matters,
it’s the correspondence of the model to the real world as well.

Second, the division into creative but lawless scientists doing the
actual work and conforming bureaucrats producing the certified
recorded results isn’t an obvious one for financial
reporting. Though it’s a nice image… the actuary with wild hair
and staring eyes stooping over a test tube… the evil laugh… and
the mild mannered accountant clearing up the mess… No, it would
never work.

For a start, while developing a drug and getting regulatory
approval for it is essentially a large, one-off project which may
take several years, financial reporting is not. Financial reporting
has hard deadlines, which come round with horrifying frequency. The
strategy of doing everything twice, once to see what the answer
should be and the second time to prove it, is unrealistic.

It’s clear, though, that financial modelling for reporting purposes
(and probably for other purposes too) has to move more down the
tried-and-trusted, totally-reproducible route, with all processes
documented and all decisions recorded. In practice, this means more
planning ahead and much less last minute “we need these extra
results by yesterday”. It’s all going to have to be less reactive,
and so will seem less flexible.

===============
3. FSA update

At least three quarterly newsletters are issued by the FSA. The
latest issues are as follows:

Financial Crime, December 2004
http://www.fsa.gov.uk/pubs/other/fc_newsletter1.pdf

Life Insurance, December 2004
http://www.fsa.gov.uk/pubs/other/li_newsletter2.pdf

General Insurance, September 2004
http://www.fsa.gov.uk/pubs/other/gi_newsletter3.pdf

All of them include information on how to receive them regularly
(although past history, of which there is admittedly little,
indicates that the “quarterly” is perhaps more of a goal than an
achievement).

Two reports on cost benefit analysis (CBA) have been issued as part
of the N2+2 review (I can’t work out whether my predominant feeling
is “goodness, only two years,” or “goodness, two years
already?”). One is on CBA methodologies
(http://www.fsa.gov.uk/pubs/other/nera_cba_report.pdf) and the
other is on embedding CBA more deeply in the FSA
(http://www.fsa.gov.uk/pubs/other/howell_report.pdf). Before I
looked at them I was hoping to be able to say something along the
lines of “even if you don’t care about how the FSA do things, read
these reports for their more generally applicable comments.” I
haven’t read them very thoroughly, I admit, but so far haven’t come
across many particularly juicy nuggets.

However, I did like the FSA’s comment that “we recognise that
cumulative CBA, in the pure sense of the term, is in practical
terms impossible. This reflects the extreme difficulty of
modelling, with any reasonable certainty, what the net cost-benefit
effect of regulation on UK’s financial services markets would now
be like, compared with a position prior to regulation.”

New consultation and discussion papers out this month:
—————————————————–

None

Feedback published this month:
—————————–

PS04/28 Lloyd’s: Integrated prudential requirements and changes to
actuarial and auditing requirements – Including feedback
on CP04/7, CP04/13 (part) and CP04/15 (part) and ‘made
text’

Current consultations, with dates by which responses should be
received by the FSA, are listed at
http://www.fsa.gov.uk/pubs/2_consultations.html

===============
4. Seasonal greetings

No feeble jokes about the risks of the holiday season – just my
wishes for an enjoyable and relaxing holiday and a peaceful 2005.

===============
5. Newsletter information

This newsletter is issued approximately monthly by Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2004. All
rights reserved. You may distribute it in whole or in part as long
as this notice is included. To subscribe, email
news-subscribe@louisepryor.com. To unsubscribe, email
news-unsubscribe@louisepryor.com. All comments, feedback and other
queries to news-admin@louisepryor.com. Archives at
http://www.louisepryor.com/newsArchive.do.

Newsletter Old site

Newsletter Nov 2004

Post author By Louise Pryor
Post date 29 November 2004

News update 2004-11: November 2004
===================

A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).

Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.

Subscribe by sending an email to news-subscribe@louisepryor.com.
Unsubscribe by sending an email to news-unsubscribe@louisepryor.com.
Newsletter archived at http://www.louisepryor.com/newsArchive.do.

In this issue:
1. Model risk
2. More spreadsheet woes
3. FSA update
4. Time constraints
5. Newsletter information

===============
1. Model risk

A couple of weeks ago one of the big US banks, SunTrust Banks,
announced a restatement of earnings for the first two
quarters. This followed problems they had found with their loan
loss allowances, or more specifically, with the model they were
using for their loan loss allowances. Part of their press release
says: “There were numerous errors in the loan loss allowance
calculations for the first and second quarters, including data,
model and formulaic errors.” In other words, they are saying that
the data that went into the model was wrong, the model itself was
not a good fit to reality, and on top of that they hadn’t even
implemented this faulty model properly. That covers pretty much
everything that can go wrong with a model, if you count data as
including parameters (see my article and slides on how to believe
your models,
http://www.louisepryor.com/papers/actuary_04_08_08.pdf
http://www.louisepryor.com/papers/believeModelsGIRO2004.pdf).

The fall out from the modelling problem was definitely non
trivial. Q1 earnings were restated by 1%, Q2 earnings by 6%. In Q2,
the loan loss allowances changed by 90%. The loan loss allowances
had been overestimated, so this means that in the second quarter
they were out by an order of magnitude. Three people lost their
jobs as a result of the problem, including the Chief Credit
Officer. The Financial Controller was reassigned to a position
“with responsibilities that involve areas other than accounting or
financial reporting” – ie, neither finance nor control.

Moreover, SunTrust’s directors will be unable to sign off under
section 404 of Sarbanes-Oxley at the coming year end. They say that
they will likely “not be able to conclude that the Company’s
internal control over financial reporting was effective at such
date.”

So why did all this happen? Well, apparently they were bringing in
new processes and a new model in order to comply with
Sarbanes-Oxley. This evidently proved more difficult than they
anticipated. They say “The Company’s implementation of a new
allowance framework in the first quarter was deficient. The
deficiencies included inadequate internal control procedures,
insufficient validation and testing of the new framework,
inadequate documentation and a failure to detect errors in the
allowance calculation.” They also point to deficiencies in
spotting the problem, and then in doing something about it. In
particular, “certain members of the Company’s management did not
treat certain matters raised by the Company’s independent auditor
with an appropriate level of seriousness.”

The morals are fairly obvious. First, models matter, and mistakes
in models can be significant. Second, change is risky. It can be
very risky. (On the other hand, not changing also has its
risks). Thirdly, take problems seriously.

http://www.suntrust.com/common/AboutST/NewsRoom/news/20041110.asp

As a footnote to this episode, I note that many of the phishing
spams I’m getting at the moment purport to come from SunTrust. I
hope this email gets through your spam filters.

===============
2. More spreadsheet woes

While I’m on the topic of model errors, let’s get a bit more
specific about spreadsheet errors. There have been good ones
recently – Patrick O’Beirne maintains a good list at
http://www.eusprig.org/stories.htm. Just read it if you don’t
believe that there are large numbers of spreadsheet errors, and
remember these are only the ones that have both come to light and
been made public.

The one that really struck home to me was the one where the judge
said that you can’t undo mistakes. In other words, getting it right
does matter. Two parties had a disagreement about the interest on a
loan (one party failed to pay it, and the other wanted it). They
reached a settlement based on a spreadsheet prepared by one of the
parties. The settlement went through the legal process (something
called a Tomlin Order) and that should have been that.

However, some hours later a mistake was discovered in the
spreadsheet – so the parties had agreed the settlement on the basis
of information that turned out to be incorrect. However, they had
both accepted the spreadsheet and its results earlier, and the
judge ruled that the settlement was for a specific sum, regardless
of how that sum had been reached.

There are now a couple more people in the world who will always
check their spreadsheets.

http://tinyurl.com/6lzct

===============
3. FSA update

Those FSA folk continue to make speeches, some of which are well
worth reading. They are all available on the FSA web site, at
http://www.fsa.gov.uk/pubs. Last month I highlighted a speech that
had been made on financial fraud; this month a report has come out
on Countering financial crime risks in information security
(http://www.fsa.gov.uk/pubs/other/fcrime_sector.pdf). It’s worth
reading on at least two counts. First, it’s pretty interesting;
second, it contains a lot of useful information on resources you
can use to deal with financial crime. Apparently a big problem
among smaller firms is that they don’t know who to report problems
to: this report contains names and addresses.

New consultation and discussion papers out this month:
—————————————————–

CP04/18 Implementation of the Simplified Prospectus requirements
in the UCITS Management Company Directive

Feedback published this month:
—————————–

DP25 Feedback Statement on DP25 – Development of transaction
monitoring systems
CP176 PS04/23: Bundled brokerage and soft commission
arrangements
CP204 PS04/24: Insurance groups – Supplementary feedback on
CP204 and made text
CP04/3 PS04/27: Reforming Polarisation: Implementation – Feedback
on CP04/3 (A menu for being open with consumers) and made
text
CP04/10 PS04/26: Child Trust Funds – Feedback on CP04/10 and made
text
CP04/11 PS04/22: A basic advice regime on the sale of stakeholder
products – Feedback on CP04/11 and near-final text
CP04/13 PS04/25: Amendments to switch on the Integrated Prudential
sourcebook as it applies to insurers – Feedback to CP04/13

Current consultations, with dates by which responses should be
received by the FSA, are listed at
http://www.fsa.gov.uk/pubs/2_consultations.html

===============
4. Time constraints

There never seems to be enough time. The observant amongst you will
have noticed that this newsletter only just qualifies as the
November issue, and that it is shorter than usual. The two are not
unrelated: any longer and it definitely wouldn’t be November any
more. I only hope that not too many unintended errors have crept
in as I rush to write it (the intended errors are, of course, meant
to be there).

Looking at the two stories I wrote about above, SunTrust and the
wrong spreadsheet that the judge said had to stand, you have to
wonder about the role that time pressures played.

SunTrust obviously had a hard deadline; they had to comply with
Sarbanes-Oxley, and that means conforming to the time limits that
it sets. I can certainly imagine that there was a lot of pressure
to get things done quickly, and that possibly testing and review
were skimped as a result. Of course in the end the deadline won’t
be met at all.

In the second case, time pressures probably played a part, but I
bet that over-confidence did too. I can imagine someone thinking
“It’s only a simple little spreadsheet… calculating loan interest
isn’t that hard…” But mistakes can be and are made in simple
spreadsheets too, especially if you are not expecting to make them.

I always think that it’s a pity we don’t learn more about these
mistakes that come to light. Of course we can’t, because it’s all
commercially sensitive information, but it would be really useful
to know what really went wrong and why.

===============
5. Newsletter information