News update 2006-04: April 2006
===================
A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).
Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.
Subscribe by sending an email to news-subscribe AT louisepryor.com.
Unsubscribe by sending an email to news-unsubscribe AT
louisepryor.com. (Change ” AT ” to “@”). Newsletter archived at
http://www.louisepryor.com/newsArchive.do.
In this issue:
1. Oops!
2. Documentation
3. Is that what you meant to read?
4. A horseshoe nail
5. Newsletter information
===============
1. Oops!
The emails have been flooding in… Did you really mean… Shurely
shome mishtake…
Well, yes, it was indeed a mistake, and some people noticed (half a
dozen emails, not quite a flood I admit). This is the second
newsletter that most of you have received from me this week; I am
sorry for clogging up your mailboxes unnecessarily. The first one,
a repeat of the March issue, wasn’t meant to happen. My processes
failed and have now been (I hope) fixed.
So what actually went wrong, and why? I run the mailing list using
a set of filters and templates in the email program I use, The Bat!
(the exclamation mark is part of the name). Some things, like new
subscribers and unsubscribers, are handled more or less
automatically; I don’t have to write or edit the outgoing email
message acknowledging receipt of the request at all. When
I send an actual newsletter out, there is a slightly complicated
incantation that has to go in the Bcc: field. I can never remember
exactly what the incantation is, so had set things up so that any
new message automatically contained the incantation. I then removed
it when writing a message that wasn’t an actual newsletter issue.
When somebody asked for a repeat sending of the March newsletter
yesterday, I wrote a new message, checked that I’d removed the
incantation from the Bcc: field, and fired the message off. It was
unfortunate that I didn’t double check, as evidently the check
failed.
My original reasoning had been that there would be very few
messages other than newsletter issues. This has nearly been borne
out in practice, though there are more one-off messages than I had
expected. The assumption that the reduced hassle for the newsletter
issues would outweigh the risk of sending an unwanted message out
to all recipients has become invalid as the number of people
receiving the newsletter has increased. So a couple of slightly
dodgy assumptions have gradually morphed into a couple of
definitely unsafe ones, without my processes changing accordingly.
I have now changed the default, so that I have to add the
incantation in by hand when it is needed. The risk of having to
resend an issue because it failed to reach the recipients is
clearly a better one than that of sending unwanted messages out to
all and sundry. Risk management in action, though not as effective
as it should be!
===============
2. Documentation
It is a truth universally acknowledged, that software documentation
is a Good Thing, and spreadsheets are no exception. The FSA, in a
recent newsletter, described what they had seen in the way of good
practice for financial modelling systems: “Acceptable standards of
documentation were established, agreed by the firm, and themselves
documented.” They went on to say “The standards of control and
documentation applicable to systems developments applied equally to
spreadsheets.”
But what is documentation for? It seems to be more or less assumed
that all documentation is worth while, and the more documentation
the better. However, given that most financial models and
spreadsheets are developed with limited resources, and writing
documentation takes time, it’s important to consider what forms of
documentation are most useful and productive. In order to do this
we must think about what we are trying to achieve through the
production of documentation.
Documentation can do four things: specify, record, explain, and
give instructions. These four types of documentation are of more or
less use to different types of user. Furthermore, different forms
of documentation (separate documents, as part of the user
interface, etc) are suited to convey different types of information
to the various types of user. I’ve put together a brief summary of
the options, which is available either as a single pdf document or
as a series of notes on my web site.
http://www.louisepryor.com/papers/documentation.pdf
http://www.louisepryor.com/showTheme.do?theme=17
If you’d like to know more about how I can help you with good
documentation practices, just let me know!
===============
3. Is that what you meant to read?
In previous issues I’ve talked about the problems of hidden data:
how you can inadvertently include information in documents that
isn’t intended to be there. There’s are other sides to the problem,
though: sometimes the reader sees things that they don’t want to,
or don’t see what they should.
For example, if you select some text in your browser, and paste it
in to another document, you expect to see the same text in the new
document as the text that you selected in your browser. Some web
pages have text in them that isn’t always displayed. There are
various reasons for this: for example, it may be displayed only in
some browsers (text-only browsers, or if javascript is disabled,
for instance). Usually, though, this text is pasted in to your new
document whether or not it is visible in the browser you are
using. This can be disconcerting, and could be worse than that; if
there is only a small difference between what you expect and what
you get you might not notice it, even if the sense is very
different.
In some circumstances a single digit can make all the difference. A
recent article in the New York Times describes an incident in which
one partner erased the digit “1” in a Word document, giving another
partner a 5% share in a software firm that had just been sold,
rather than the 15% which was rightfully his. The article goes on
to discuss how computer forensics experts can detect that sort of
tampering and find out when it was done and by whom. It strikes me
as careless beyond belief to rely on a document in electronic form
for that sort of agreement: personally, I would want a signed hard
copy. The huge advantage of electronic documents for some purposes
is that they are so easy to change and revise; no more retyping
page after page for a single minor change. That’s also a big
disadvantage for other purposes, though. It’s a risk that is pretty
easy to foresee, and also one that should be easy to manage; don’t
rely on electronic documents to remain unchanged.
http://catless.ncl.ac.uk/Risks/24.24.html#subj9.1
http://makeashorterlink.com/?F1F85250D
===============
4. A horseshoe nail
Well, not quite a horseshoe nail, and it wasn’t a kingdom that was
lost, but small errors can have major consequences. The true value
(for taxation purposes) of a house in Valparaiso, Indiana, was
$121,900. Somehow some user of the County’s computer system managed
to accidentally change the value of the house to $400 million. The
error meant that the house’s tax bill rose to $8 million from
$1,500, thus radically changing the budget amounts of 18 government
taxing units.
Some interesting points arise. First, how on earth could an outside
user change the taxable value of a house? The official theory is
that “the user probably tried to access a real estate record
display by pressing R-E-D, but accidentally typed R-E-R, which
brought up an assessment program written in 1995. The program is no
longer in use, and technology officials did not know it could be
accessed.” That’s scary. Good design of the interface would check
for valid inputs; when the old program had been taken out of use,
the validation should have been changed. But far safer, when a
program is taken out of use, is to actually move it, so that any
links to it no longer work. Relying on changing or removing every
link is asking for trouble.
Second, how come nobody noticed? “The city of Valparaiso and the
Valparaiso Community School Corp. were asked to return $2.7
million.” Hadn’t they been surprised when the budget came in at a
different level from the one they were expecting? Or is $2.7
million a very small proportion of the total, in which case it
might be insignificant anyway, but this seems unlikely. Surely a
few reasonableness checks, or a simple analysis of changes, should
have rung some alarm bells somewhere.
Third, the county treasurer said his office “spotted the $400
million error after it caused an improper billing, but apparently
it wasn’t corrected elsewhere.” Sounds like a major communication
breakdown. It seems odd that an error can propagate through to 18
government taxing units, while a correction can’t.
http://www.wjxx.com/news/strange/news-article.aspx?storyid=51489
There seems to be a general lack of sanity checks in all sorts of
systems. We hear of a publican in West Sussex who got a £600,000
telephone bill. Apparently a member of BT’s staff had “filled in
the wrong details in the wrong box”. The correct amount was £92,
which was exactly in line with the bills the publican normally
got. However, this is as nothing compared to the bill for 806
trillion Ringgit (about £126 trillion) received by a meat importer
in Malaysia. A few days later Telekom Malaysia said “It was a typo,
and it was not our mistake”–apparently it was all the fault of a
debt collection agency. Now, I get lost in the billion and trillion
range, but a quick search shows that a trillion has either 12 or 18
zeroes, depending. Why would a billing system even be able to issue
a bill for that amount of money?
http://www.theregister.co.uk/2006/04/10/bt_bill/
http://uniquoad.notlong.com
It’s not just billing systems that should have data validation and
overall sanity checks. Any financial model or spreadsheet needs
them too. It’s possible to automate data validation, of course, and
some sanity checks, but it’s impossible to overestimate the utility
of a thoroughly cynical user. Instead of thinking “well, I’m pretty
sure the spreadsheet is correct so the answer must be right” you
should always be alert for anything that seems unusual, odd, or
even just unlikely.
If you’d like to find out more about adding data validation or
sanity checking into your spreadsheets or models, just get in
touch.
===============
5. Newsletter information
This newsletter is issued approximately monthly by Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long
as this notice is included. To subscribe, email news-subscribe AT
louisepryor.com. To unsubscribe, email news-unsubscribe AT
louisepryor.com. All comments, feedback and other queries to
news-admin AT louisepryor.com. (Change ” AT ” to “@”). Archives at
http://www.louisepryor.com/newsArchive.do.