News update 2005-10: October 2005
===================
A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).
Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.
Subscribe by sending an email to news-subscribe AT louisepryor.com.
Unsubscribe by sending an email to news-unsubscribe AT
louisepryor.com. (Change ” AT ” to “@”). Newsletter archived at
http://www.louisepryor.com/newsArchive.do.
In this issue:
1. Reputation risk
2. Blackberry jam
3. Where does the buck stop?
4. If it can go wrong…
5. Newsletter information
===============
1. Reputation risk
It’s early days yet, but it looks as if Southern Water are handling
a big problem well. The story so far: Southern Water were
implementing a new billing system. Among the checks that they
performed as they did so, they reconciled the results that it
produced for various aspects of customer service to the known
results. They found that the new system showed much larger numbers
of customer complaints than they were expecting. They called
auditors in, to check that there was indeed something wrong, and
then reported the problem to Ofwat (the water industry regulator)
and the Serious Fraud Office. Southern Water have asked external
lawyers and accountants to conduct an investigation to examine
“certain inconsistencies” in internal procedures.
The number of customer complaints, and how they are handled, is an
important issue because as part of the regulatory framework water
companies are required to handle complaints in a timely manner, pay
compensation to customers when they fail to do so, and turn in a
report to Ofwat each year with details of numbers of complaints,
how quickly they were handled, and so on. Last year’s report showed
that all 2,500 written complaints received by Southern Water were
replied to within 10 days (the customer gets compensation if it
takes longer). In fact, it now appears that there were closer to
8,000 written complaints, not all of which received timely
replies. In addition, there were other forms of compensation that
had not been paid to customers, for things such as failed
supplies. The current estimate of the outstanding compensation bill
is £2.5m.
In the radio interview that I heard, Les Dawson, the new MD of
Southern Water (this all happened less than a week after he was
appointed) was pressed hard to say what had gone wrong, and who was
responsible. He very reasonably pointed out that the point of
having an investigation was to find these things out. If Southern
Water had stayed quiet about this until after the investigation,
they would have been accused of hiding it.
This episode does, I think, have wider implications. First, the
report the water companies have to submit to Ofwat is externally
audited. So as well as the investigation into internal procedures,
questions will have to be asked about how the auditors were
misled. Second, there are other non-financial reports made to
regulators in other industries. I should imagine (and hope) that
some hasty checking is going on all over the land. Third, it shows
the value of performing proper checks and tests when implementing
new systems. It was obviously some kind of regression testing that
exposed the problem. Also, when you perform regression testing, you
shouldn’t necessarily assume that a difference shows that there is
a problem in the new system. Finally, you have to wait for the
results of an investigation before being able to say what its
conclusions are. We just do not know yet how the problem arose:
incompetence, fraud, poor systems design, or something else.
http://tinyurl.com/96v3x
http://news.independent.co.uk/business/news/article322320.ece
===============
2. Blackberry jam
Are you one of those people who is almost physically addicted to
your Blackberry? Never go anywhere without it, and check for new
emails every five minutes? Then be thankful that you don’t work for
the BBC, which has suspended its Blackberry service. Apparently
they were getting the equivalent of crossed lines: people reported
receiving parts of messages not intended for them. When the problem
became known, users started frantically checking that they had not
sent any confidential or potentially embarrassing messages that
could have gone astray. Apparently the service has now been down
for a week, and it is likely to stay down for at least another
fortnight.
I haven’t been able to find anything out about what the problem is,
except that there was one reference to an “obscure bug.” Well,
that’s all right then. If it’s obscure, it can’t matter too
much. Actually, no. In this case, obscure probably just means “it’s
not something we thought of checking,” or “it can only happen in
really weird circumstances, which turn out to have actually
occurred.”
I also don’t know where the problem is: in the Blackberry software
itself, in the BBC’s email system (run by Siemens), or somewhere in
Vodaphone’s systems (who supply the wireless bit). If it’s in the
Blackberry software itself (or in Vodaphone’s systems), a lot of
users should be very worried indeed. If it’s in the BBC’s email
system, which from some hints in some of the press coverage seems
the most likely, the problem is less wide ranging, but could still
affect other organisations using the same type of email system.
This episode shows that you can never be sure that complex systems
will work properly. Presumably the BBC folk had been happily using
their Blackberries for some time, with no obvious signs that there
was anything wrong. Then, probably, some small change was made to
the system, that wasn’t expected to have any very serious effects,
and hey presto! Little fragments of stray emails all over the
place.
Meanwhile, people at the BBC are having to slum it with other,
older, technology: phones and computers.
http://www.guardian.co.uk/uk_news/story/0,3604,1601142,00.html
===============
3. Where does the buck stop?
Whose fault is it when there is a security hole in a piece of
software, or an implementation bug in a financial model? A former
White House cybersecurity adviser recently said that software
developers should be personally responsible for security holes in
the software they write. Howard Schmidt said “In software
development, we need to have personal quality assurances from
developers that the code they write is secure.”
Although this idea may seem attractive at first, it shows a
misunderstanding of how software is written. It is only rarely that
a single person is responsible for all aspects of code design and
development.
On a large project, there might be the following people involved:
business analyst, software architect, lead developer, several other
developers, database specialist, usability specialist. For some
projects there may be more. In addition, there may be inputs from
the marketing and quality assurance (testing) teams. Even during
code development (leaving aside the specification and design
stages) there may be collaboration between developers, and formal
code reviews. How can you possibly assign the blame for any
problems to a single person?
Even pinning it down to a small group is difficult, as there are so
many different influences affecting the final form of the
code. Whatever the size of the project (a single spreadsheet
through to an enterprise level software solution) there are always
trade-offs between the functionality that is included, the
usability, the delivery time, and the level of testing and
review. The developers often feel that the marketing folk are the
most responsible for errors: if they hadn’t promised the
impossible, the developers might have been able to deliver a
quality version of the possible. It is unreasonable to give the
responsibility for errors to a group of people if you don’t also
make it possible for them to deliver an error-free product, for
example by giving them enough time and budget to do so.
Even when there is just one person developing a simple spreadsheet
there would often be a good argument that the developer should not
be held responsible for all the mistakes. If there is intense
pressure to produce an answer to a deadline, there may not be time
to perform all the necessary testing and reviewing. The corporate
culture may not encourage (or even discourage) a disciplined
approach to spreadsheet development. The developer may have had no
training in software development techniques (after all, anyone can
write a spreadsheet, can’t they?)
Although spreadsheets are often used as part of routine business
processes, they have often not been through the type of quality
control that other software has. Again, it may not be the fault of
the individual developer, but the fault of the managers who don’t
understand the risks involved.
If you would like to know how I can help you have more confidence
in the spreadsheets and other user-developed software that your
organisation uses, send me an email at the address below.
http://news.zdnet.co.uk/0,39020330,39228663,00.htm
===============
4. If it can go wrong…
You may have noticed that there has been a slight hitch in the
monotonous regularity with which these newsletters appear. One
thing I have learned over the past few months is not to take things
for granted. For example, don’t assume that getting your boiler
serviced just after major house redecoration is a mere
formality. You might find that you have a blocked flue and have to
resite the boiler. A house sale can fall through up to about three
weeks before the moving date (and possibly later, though I don’t
have an actual example of that). Even if you are generally healthy,
don’t assume that you won’t break your leg just before you move
house. Just because both BT and your ISP say that your line is
fine, it doesn’t mean that either ADSL or dial-up will actually
work without a visit from the BT engineer. However, normal service
has now been resumed, and we apologise for any inconvenience
caused.
As ever, if you have any comments on any items in this newsletter,
or any suggestions for future issues, just drop me an email at the
address below.
===============
5. Newsletter information
This newsletter is issued approximately monthly by Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2005. All
rights reserved. You may distribute it in whole or in part as long
as this notice is included. To subscribe, email news-subscribe AT
louisepryor.com. To unsubscribe, email news-unsubscribe AT
louisepryor.com. All comments, feedback and other queries to
news-admin AT louisepryor.com. (Change ” AT ” to “@”). Archives at
http://www.louisepryor.com/newsArchive.do.