News update 2006-06: June 2006
===================
Contents:
1. Software in ICAs
2. The proof of the pudding is in the testing
3. Portable insecurity
4. Sharing spreadsheets
5. Newsletter information
===============
1. Software in ICAs
Many thanks to everyone who has completed the online survey about
software use in ICAs, at http://www.louisepryor.com/survey.html
If you are involved in the ICA process, at whatever level, I’d
appreciate it if you could fill the survey in: it should only take
about a quarter of an hour, and the more responses there are the
more useful the results will be. All participants will get a full
analysis of the results, and responses are confidential.
The survey concentrates on an overview of what types of software
are used in ICAs, and the general level of systems and controls
that are in place. Responses saying that you don’t know are equally
as valuable as those that give rankings. I hope that the results of
the survey will provide an impression of the general level of
confidence in existing systems and controls, and how broad the
range of practice is. It will also be interesting to see how much,
if at all, practice differs between life and non-life insurance
companies.
Here’s that link again: http://www.louisepryor.com/survey.html
===============
2. The proof of the pudding is in the testing
Why should you test software? It’s time consuming, and therefore
often expensive. If you could prove that a program was correct,
surely you wouldn’t need to test it. And maybe a thorough review of
the code would mean that you wouldn’t need to run thorough tests.
Dream on. It would be lovely if it were true, but it ain’t, as two
recent episodes demonstrate.
An algorithm that has been accepted as correct for years (proved in
a book in 1986; probably proved elsewhere before that) has been
found to have a bug. It’s a widely used sorting algorithm, that’s
been used in many different places, including the implementation of
Java. It only fails in somewhat unusual circumstances, when you’re
sorting an array of more than about 2^30 elements (that’s about a
billion). When the proof was published, back in the 1980’s, the
idea of sorting such large arrays was inconceivable. The bug was
recently reported after being encountered in practice in Java, nine
years after the relevant code was released, and 20 years after the
proof.
So what went wrong? The algorithm contains the following line:
int mid =(low + high) / 2;
which takes the average of two numbers, and truncates it to an
integer. This doesn’t work when the sum of low and high is greater
than the maximum positive integer value (2^32 – 1). In this case
the sum overflows to a negative number, and all hell breaks loose
(or at least exceptions are thrown or unpredictable things occur,
depending on the programming language).
There are some simple ways of changing this line so that it always
works: for example,
int mid = low + ((high – low) / 2);
works as long as both high and low are positive integers less than
2^32 – 1, and high is greater than low (which is guaranteed
elsewhere in the algorithm).
http://lawyclic.notlong.com
The proof relied on implicit assumptions about the size of the
array being sorted. Those assumptions held in practice at the time
that the proof was first published, but have become unrealistic in
the years since that time. A proof always relies on assumptions,
and can only be trusted as far as the assumptions hold in
practice. As Yogi Berra once said: In theory there is no difference
between theory and practice; in practice, there is. Which is why
there is no substitute for actually testing the practical
implementation.
http://www.brainyquote.com/quotes/authors/y/yogi_berra.html
This is a lesson that NASA engineers have learned to their
cost. The Genesis space probe crashed in 2004, as it was bringing
back solar wind particles for analysis. Its parachutes failed to
open, because the gravity switches that should have operated them
were installed backwards.
There was a test that would have uncovered the fatal flaw, but it
wasn’t performed because of time constraints. It was thought that
comparing the design drawings to those of a previous, successful,
spacecraft, would do instead.
http://edition.cnn.com/2006/TECH/space/06/14/genesis.crash.ap/index.html
Again, this shows up the risks of relying on theory rather than
practice. Comparing the design drawings assumes that the
implementation (in this case, the physical spacecraft) corresponds
exactly to the theoretical model (the design drawings). This is a
big assumption to make, and in this case it turned out to be
unrealistic.
I see the same pattern occurring with spreadsheets and other user
developed software, including actuarial models. Even if the
specification is correct, the implementation may not be; and a
thorough review of the code doesn’t guarantee that it will work in
practice. Code reviews and theoretical proofs have a role to play,
of course, but can’t replace proper testing of the actual
implementation.
If you’d like help in setting up a systematic testing process for
your user developed software, just let me know by replying to this
email, or contacting me through my web site at
Climate change actuary
===============
3. Portable insecurity
There are (or should be) some very red faces at the Federal Trade
Commission, the US body responsible for protecting citizens from
identity theft and fraud. Two FTC laptops, containing the personal
details of 110 people, have been stolen.
http://www.out-law.com/page-7032
This is just the latest in a long string of stories about stolen
laptops containing personal details: just enter “laptop personal
data” into Google to see some of them.
http://www.google.co.uk/search?q=laptop+personal+data
40,000 BP workers on an Ernst and Young laptop… 13,000 people on
an ING laptop… 243,000 hotel guests… 68,000 YMCA
members… it’s a lot of people. In many cases the laptops were
probably stolen as hardware, rather than for the information they
held, but that doesn’t really lessen the risk. In some of these
cases we are told that the information was password protected, but
that may not mean much. The password protection in Excel, for
example, is notoriously weak. There are any number of tools
available to crack Excel passwords.
http://www.google.co.uk/search?q=excel+password+cracking
All the stories we read in the press are about the loss of laptops
containing personal details, but I’m willing to bet that there are
a lot of laptops out there that contain highly confidential
business information: business plans, marketing plans, ICA
calculations, and so on. Now, it may be rational behaviour: the
chance of losing a laptop is actually quite small, and the chance
that the casual thief would be interested in such information, or
know what to do with it, must be even smaller.
The circumstances surrounding all these incidents will vary, but I
imagine that in at least some of them security guidelines were
breached. Company guidelines may limit the type of information that
can go onto laptops, but that doesn’t necessarily stop people
putting it there. Their own convenience wins out. When it comes
down to it, all the guidelines in the world aren’t going to make a
difference if they are ignored by the people on the ground. In
fact, this is just part of a wider problem: many security breaches
are not technology driven at all, but happen because people write
their passwords down, leave laptops lying around, or otherwise
lower their guard.
What looks like an interesting book, “Information Security and
Employee Behaviour,” discusses the issues involved and presents
some solutions. From the review I have read it takes a fairly
balanced view, discussing why why security people get fixated on
their field, and often over-emphasise minor problems. As with all
areas of risk management, one of the problems is that people just
aren’t very good at assessing risk.
http://catless.ncl.ac.uk/Risks/24.33.html#subj21.1
Another problem is that the risk to the laptop owner is not the
same as the risk to the people whose personal details are on the
laptop. Identity theft is both disturbing and time consuming to
fix, for the victims, but losing a laptop, while annoying, doesn’t
really have the same effect.
Aligning Interest with Capability
===============
4. Sharing spreadsheets
It’s all go on the spreadsheet front at the moment, with Microsoft
releasing a white paper on the next version of Excel (see last
month’s newsletter) and Google Spreadsheets making an appearance.
http://spreadsheets.google.com/
Here’s what Google says about Google Spreadsheets:
1. Share spreadsheets instantly & collaborate real-time.
– Pick exactly who can access your spreadsheets.
2. Edit your spreadsheets from anywhere.
– Nothing to download — your browser is all you need.
3. Store your spreadsheets securely online.
– Offsite storage plus multi-site data backup.
4. Easy to use.
– Import your current spreadsheets to get started quickly.
– Clean, uncluttered screens with a familiar, desktop.
All this sounds good, but there are some drawbacks.
First, the functionality isn’t complete. It’s a pretty basic
spreadsheet, which is fair enough for a first release. It seems to
have all the same actual functions as Excel, but has no macros, you
can’t print your spreadsheets out, there’s only limited formatting
(no cell borders, no conditional formatting), no defined Names, no
data tables, no pivot tables, no objects (charts, text boxes,
shapes, etc) …
Some of these omissions are undoubtedly because of the limitations
of browser technology. You just can’t manage the user interface
through a browser in the same way that you can with a purpose built
application.
Second, it doesn’t work with all browsers yet. It’s fine with
Firefox and Internet Explorer, but Opera isn’t supported. I suspect
that it doesn’t work well with text to speech software, either.
Third, there’s the security problem. The spreadsheets are stored
on Google’s servers. Whatever the safeguards, it’s just not the
same as having them on your own machines.
Fourth, it’s all very well sharing a spreadsheet, and there is some
control in that the creator is the only person who can give other
people access (either read-only or editing), but once you have
several people being able to change a spreadsheet you want to be
able to track the changes. Of course, this is a drawback with
Excel, too, but the problem will (apparently) be addressed in the
next release.
For a first release, though, it’s pretty impressive, and easy to
use. No doubt they’ll come up with some ways of addressing some of
the functionality issues. The sharing is particularly good for
collaborative working, with a chat window so that you can talk
about the changes you are making.
It wouldn’t surprise me if it was eventually sold for use on
company intranets, which would deal with the security issue. And
with some sort of version control, so you could lock down a version
after a spell of collaborative work, and assuming that they find a
way to add more functionality, it could be very useful.
===============
5. Newsletter information
This is a monthly newsletter on risk management in financial
services, operational risk and user-developed software from Louise
Pryor (http://www.louisepryor.com). Copyright (c) Louise Pryor
2006. All rights reserved. You may distribute it in whole or in
part as long as this notice is included.
To subscribe, email news-subscribe AT louisepryor.com. To
unsubscribe, email news-unsubscribe AT louisepryor.com. Send all
comments, feedback and other queries to news-admin AT
louisepryor.com. (Change ” AT ” to “@”). All comments will be
considered as publishable unless you state otherwise. The
newsletter is archived at http://www.louisepryor.com/newsArchive.do.