News update 2006-03: March 2006
===================
A monthly newsletter on risk management in financial services,
operational risk and user-developed software from Louise Pryor
(http://www.louisepryor.com).
Comments and feedback to news-admin@louisepryor.com. Please tell me if
you don’t want to be quoted.
Subscribe by sending an email to news-subscribe AT louisepryor.com.
Unsubscribe by sending an email to news-unsubscribe AT
louisepryor.com. (Change ” AT ” to “@”). Newsletter archived at
http://www.louisepryor.com/newsArchive.do.
In this issue:
1. Farcical bid evaluation
2. Black box models
3. Data protection
4. Newsletter information
===============
1. Farcical bid evaluation
“This is farcical; a spreadsheet error on a multi-billion pound
contract… ” and OCG Buying Solutions has “egg on its face”: these
are just two of the comments after the Government’s IT procurement
body admitted that the discovery of an error in a spreadsheet may
lead to some companies losing their accredited status in the new
Catalist procurement programme. Public sector managers can buy
goods and services more easily from accredited suppliers than
others, so losing accredited status can make a significant
difference to a company.
A leaked letter said “Unfortunately, [we are] not, as we had hoped,
in a position to accept your tender at this time…. This is
because an error in the original evaluation spreadsheet has been
identified, necessitating rescoring of all tenders for this
project… this error has now been corrected and this has caused a
small number of changes to the original award decision.”
This is unlike many of the spreadsheet errors that come to light,
in that it isn’t the organisation whose spreadsheet it is that is
suffering in this case. The OCG probably isn’t affected much if its
reputation takes a hit, and hasn’t lost a lot of money because of
the error. It is other stakeholders, ie the suppliers wanting
accreditation, who have the problems; and they can’t do anything
about it, except to go through a possibly lengthy appeals
process. Not surprisingly, they are a bit fed up, hence the quotes
at the start of this article.
We don’t know exactly what was wrong with the spreadsheet, but one
supplier believes that there was a formula error: too much weight
was placed on environmental factors, and not enough on pricing and
discounts. The OCG said “Every effort is made to ensure that
mistakes do not happen, and OGC Buying Solutions has a quality
assurance process that has been enacted in this instance.” Apart
from the rather mangled language in the second half of this
sentence, there are other criticisms to be made.
A successful quality assurance process would have caught the error
before suppliers were told that they had achieved accredited
status, rather than after. Making an effort to avoid mistakes in
spreadsheets is not enough: what is needed is an effective
development process, which includes appropriate testing and review
stages. If there are routine ways of performing testing and review
they are less likely to be skipped, and more likely to catch
errors. And spreadsheet errors can affect your reputation and hurt
others as well as losing you money.
http://www.channelregister.co.uk/2006/03/10/ogc_spreadsheet_snafu/
http://www.vnunet.com/crn/news/2151405/ocg-revises-catalist-line-third
===============
2. Black box models
The latest Life Insurance newsletter from the FSA has a section on
Actuarial Systems and Controls. As it says, “Accurate output from a
life office’s actuarial area is important because any shortcomings
can potentially cost a firm dearly.” The FSA visited six firms, and
found quite a range of standards; but no firm was completely
satisfactory in every area. And the areas in which most improvement
was possible were those of documentation and spreadsheet use. The
FSA appears to have the following concerns:
– Systems and processes should have up to date documentation
– Every actuarial system should have a full audit trail
– Documentation and systems should have proper change processes and
version controls in place
– Documentation is especially important for systems supported by
just a few knowledgeable individuals
– Data and assumptions should be fully documented, consistent
across the business, and validated
– Financial models should be fully understood, and actuarial
departments should be able to demonstrate why their models should
be believed
– Spreadsheets should be taken as seriously as any other software
development environment.
Regular readers of this newsletter will recognise a number of these
themes. If you’ve read my short article on “Believing your Results”
it should all be very familiar. I find it interesting that the FSA
are now paying more attention to how the modelling results are
achieved, in the life insurance area at least. Models will
undoubtedly increase in both number and complexity as the ICAS
process becomes more mature in both life and non-life insurance,
and their care and feeding will become ever more important.
Just to state the obvious, I can help you to set up development
processes for financial models and spreadsheets, advise on suitable
methods of documentation, change processes and version control,
help you implement appropriate forms of testing and review for
models and spreadsheets, and provide training on these and related
issues. There’s information on my web site, and please get in touch
if you’d like further details.
http://www.fsa.gov.uk/pubs/newsletters/li_newsletter6.pdf
http://www.louisepryor.com/papers/actuary_04_08_08.pdf
===============
3. Data protection
There’s an awful lot of data about each and every one of us hanging
around on various computer systems, and not all of it is very
secure. This matters both to the people whose data is held, and
those doing the holding (or, in some cases, letting go). The risks
are many in the areas of both fraud and privacy.
There seems to have been a minor epidemic of laptops going missing
recently. Both Fidelity and Ernst & Young have lost laptops
containing employee data from their clients.
E&Y appear to have lost at least five laptops, one of which held
data on BP, Sun, Cisco and IBM employees. It’s not clear how many
people are affected, but one source has estimated that it’s over
100,000 IBM employees alone. HP staff were told that the laptop
lost by Fidelity exposed 196,000 current and former HP
employees. The data apparently includes names, addresses, US Social
Security numbers, and tax identification numbers, providing an
absolute bonanza for identity thieves.
http://www.theregister.co.uk/2006/03/30/ey_nokia_lapop/
http://www.theregister.co.uk/2006/03/24/hp_fidelity_laptop/
Laptops are inherently much less secure than desktop machines, as
it is much easier for unauthorised people to gain physical access
to them. Organisations should both have and enforce strong policies
on what data should be allowable on laptops, and should use
appropriate security measures, including strong encryption. Simple
password protection is unlikely to be effective. The password
protection in Microsoft Office products is particularly
ineffective, with password cracking programs readily available on
the internet. It’s not only personal data that is vulnerable, but
also commercially sensitive information: would you like your
business or marketing plans, or details of a planned takeover, to
be available to all and sundry?
Even personal data held on desktops or mainframes isn’t safe if the
wrong people get hold of it. Because of the physical access
problems, it’s usually an insider job. A call centre employee in
Leeds apparently stole confidential customer information from two
banks, enabling his gang to defraud the banks of over
£400,000. They targeted customers who didn’t use their credit cards
regularly, obtained their names and security codes, and then
impersonated them in order to change the account address and get a
new card and PIN.
Call centre employees are regularly targeted by identity thieves,
according to a story in The Guardian. They work the bars and pubs,
looking for low-paid call centre workers who might be willing to
make a bit on the side.
http://www.finextra.com/fullstory.asp?id=15025
http://money.guardian.co.uk/saving/banks/story/0,,1744116,00.html
Having good policies in place around the protection of data on
laptops isn’t enough to avoid problems; you’ve got to enforce the
policies too. A recent survey (by a company that sells web content
filtering) found that 70% of companies recognised that an
acceptable internet use policy was crucial to the security of their
IT systems, but 38% of employees governed by a policy are unaware
of its contents. 40% of respondents said that a policy was in place
but was not enforced.
http://www.smoothwall.net/information/news/newsitem.php?id=971
The existence of a policy isn’t going to make any difference to how
things actually happen unless people know about the policy,
understand the point of it, and find it easier to comply with the
policy than to breach it. This applies just as much to software
development and use as to security issues.
===============
4. Newsletter information
This newsletter is issued approximately monthly by Louise Pryor
(http://www.louisepryor.com). Copyright (c) Louise Pryor 2006. All
rights reserved. You may distribute it in whole or in part as long
as this notice is included. To subscribe, email news-subscribe AT
louisepryor.com. To unsubscribe, email news-unsubscribe AT
louisepryor.com. All comments, feedback and other queries to
news-admin AT louisepryor.com. (Change ” AT ” to “@”). Archives at
http://www.louisepryor.com/newsArchive.do.