Important to start this debate. In any presumptions I would distinguish not only between hardware and software but also process control software and business/personal software. The first has a greater proportion of possible pathways tested (and the developer is potentially personally liable - see IEEE.) The second has 70% pathways not tested or used and liability tends to be limited.
As always you have given us a really good beginning to understanding so many of the complexities of these cases. Thank you for giving us so generously of your time. We appreciate your views.
I'm not sure where you will be heading, though I will read it all. My thinking is that based on negligence cases, the fact that the law does not accept the concept of uncertainty it demands exclusive and separate states, is part of the problem. This leads to injustice in both the civil and criminal courts. As the recent rape quashing demonstrated, as the law was certain the conviction was sound, it required absolute evidence that another was guilty to accept it might have made a mistake. The old saw about erring is part of being human.
The presumption is taken to be absolute, though it can't be in criminal trials as if there was no uncertainty about this presumption there could never be any conviction however much evidence to the contrary was provided. If your prior belief is 100% innocent then there is zero possibility of being guilty, and 0* any number remains 0 so beyond reasonable doubt is clearly impossible. However it happens, thus we get miscarriages of justice. But the science that explains this has been explicitly excluded from courts, I believe.
The Law Commission and section 69 of the Police and Criminal Evidence Act 1984 by Jamie Christie published in the Digital Evidence and Electronic Signature Law Review, Vol 20 (2023)
Thanks for clarifying this important point. As a computer programmer, I think everyone should be terrified that their life could be ruined by bad software, and it's horrifying that the law has been changed to make that more likely.
Interesting information regarding presumption. A key cause of the injustice is that most people are fools and will believe anything they are told by authority. Anyone who has written code would laugh at the idea that a program could presumed to be working as intended but the average persons critical thinking skills are pathetic.
A similar problem occurs in banks. In the US, there is apparently precedent (Judd vs Citibank) whereby it was ruled that the customer had no chance to rebut, as you say, so the onus had to be on the bank to prove the system was working. (see sec 10.4.3, "Incentives and Injustices" of "Security Engineering" by Ross Anderson (2nd edition online: https://www.cl.cam.ac.uk/~rja14/Papers/SEv2-c10.pdf might be a different place in the 3rd edition)
One thing that is extraordinary to me, is that the post office were claiming simultaneously:
a) That their software was fully robust
b) That (initially),when asked for records of the detection, analysis and resolution of bugs, that ‘no such document’ existed [paras 579 & 580 ]
These two claims are completely inconsistent with each other.
Any competent software company (and indeed, even most incompetent ones) has a ‘bug database’, ‘issue list’, ‘problem tracker’ or similar. To make software robust,you need to do two things:
1) The design needs to be such that all classes for which this is possible, cannot occur
2) Remaining errors are found, root caused, and removed
It is very unusual for 1) to be sufficient - it requires the complexity to be very limited. Horizon clearly failed on this count too, as its audit trail did not contain enough information to reconstruct what happened in the case of errors (which resulted in them accusing the SPMs of criminality). But you need a bug database to execute 2).
In summary, for the benefit of future litigation: If you’re going to claim, as a main part of your case, that your software is robust, then also saying you have no bug database is sufficient to refute this.
(The suggestion at 589 from counsel that it was merely a document on when you need to kick the printer is also laughable)
From my time in local government I recall 1999 was not only when the government acquired a new-found zeal for computers and “e-commerce” (with accompanying funding) but also when Tony Blair started having personal meetings with Bill Gates. This snippet from a BBC report of the time seems apposite given the change in presumption of computer evidence the same year: “a spokesman was forced to admit Mr Blair had not yet started the computer course he promised to attend when he confessed his technological illiteracy a month ago.”
Politicians having non-technology backgrounds or experience yet passing highly complex technology-based legislation continues to the present day, with the important, but dangerous, Online Safety Bill.
I remember my dad made extensive use of the evidence issues with computer records during the poll tax rebellions - which I wonder if it was s69 as these were not criminal proceedings but rather the magistrates sitting in their regulatory capacity.
Anyway it clogged the cases because with the volume of cases it was impractical for councils to provide evidence other than in the form of computer records.
The Local Government Finance Act 1992 therefore made express provision that computer records are admissible in council tax proceedings provided they are accompanied by a certificate that the computer is operating correctly.
As an electronic engineer I can assure you that computers are never operating correctly - just that their errors are usually insufficient to undermine the substantive accuracy of their results. But I don't think that point would get anyone very far in court.
Important to start this debate. In any presumptions I would distinguish not only between hardware and software but also process control software and business/personal software. The first has a greater proportion of possible pathways tested (and the developer is potentially personally liable - see IEEE.) The second has 70% pathways not tested or used and liability tends to be limited.
As always you have given us a really good beginning to understanding so many of the complexities of these cases. Thank you for giving us so generously of your time. We appreciate your views.
I'm not sure where you will be heading, though I will read it all. My thinking is that based on negligence cases, the fact that the law does not accept the concept of uncertainty it demands exclusive and separate states, is part of the problem. This leads to injustice in both the civil and criminal courts. As the recent rape quashing demonstrated, as the law was certain the conviction was sound, it required absolute evidence that another was guilty to accept it might have made a mistake. The old saw about erring is part of being human.
The presumption is taken to be absolute, though it can't be in criminal trials as if there was no uncertainty about this presumption there could never be any conviction however much evidence to the contrary was provided. If your prior belief is 100% innocent then there is zero possibility of being guilty, and 0* any number remains 0 so beyond reasonable doubt is clearly impossible. However it happens, thus we get miscarriages of justice. But the science that explains this has been explicitly excluded from courts, I believe.
See https://journals.sas.ac.uk/deeslr/article/view/5642/5310
The Law Commission and section 69 of the Police and Criminal Evidence Act 1984 by Jamie Christie published in the Digital Evidence and Electronic Signature Law Review, Vol 20 (2023)
Thanks for clarifying this important point. As a computer programmer, I think everyone should be terrified that their life could be ruined by bad software, and it's horrifying that the law has been changed to make that more likely.
Interesting information regarding presumption. A key cause of the injustice is that most people are fools and will believe anything they are told by authority. Anyone who has written code would laugh at the idea that a program could presumed to be working as intended but the average persons critical thinking skills are pathetic.
A similar problem occurs in banks. In the US, there is apparently precedent (Judd vs Citibank) whereby it was ruled that the customer had no chance to rebut, as you say, so the onus had to be on the bank to prove the system was working. (see sec 10.4.3, "Incentives and Injustices" of "Security Engineering" by Ross Anderson (2nd edition online: https://www.cl.cam.ac.uk/~rja14/Papers/SEv2-c10.pdf might be a different place in the 3rd edition)
Just read some of the 2019 judgment
One thing that is extraordinary to me, is that the post office were claiming simultaneously:
a) That their software was fully robust
b) That (initially),when asked for records of the detection, analysis and resolution of bugs, that ‘no such document’ existed [paras 579 & 580 ]
These two claims are completely inconsistent with each other.
Any competent software company (and indeed, even most incompetent ones) has a ‘bug database’, ‘issue list’, ‘problem tracker’ or similar. To make software robust,you need to do two things:
1) The design needs to be such that all classes for which this is possible, cannot occur
2) Remaining errors are found, root caused, and removed
It is very unusual for 1) to be sufficient - it requires the complexity to be very limited. Horizon clearly failed on this count too, as its audit trail did not contain enough information to reconstruct what happened in the case of errors (which resulted in them accusing the SPMs of criminality). But you need a bug database to execute 2).
In summary, for the benefit of future litigation: If you’re going to claim, as a main part of your case, that your software is robust, then also saying you have no bug database is sufficient to refute this.
(The suggestion at 589 from counsel that it was merely a document on when you need to kick the printer is also laughable)
(typo : all classes [of error])
From my time in local government I recall 1999 was not only when the government acquired a new-found zeal for computers and “e-commerce” (with accompanying funding) but also when Tony Blair started having personal meetings with Bill Gates. This snippet from a BBC report of the time seems apposite given the change in presumption of computer evidence the same year: “a spokesman was forced to admit Mr Blair had not yet started the computer course he promised to attend when he confessed his technological illiteracy a month ago.”
Indeed.
Politicians having non-technology backgrounds or experience yet passing highly complex technology-based legislation continues to the present day, with the important, but dangerous, Online Safety Bill.
And somewhat ironically, 1999 was when the scramble to update software to rectify the Y2K bug amply demonstrated the fallibility of computers.
I remember my dad made extensive use of the evidence issues with computer records during the poll tax rebellions - which I wonder if it was s69 as these were not criminal proceedings but rather the magistrates sitting in their regulatory capacity.
Anyway it clogged the cases because with the volume of cases it was impractical for councils to provide evidence other than in the form of computer records.
The Local Government Finance Act 1992 therefore made express provision that computer records are admissible in council tax proceedings provided they are accompanied by a certificate that the computer is operating correctly.
As an electronic engineer I can assure you that computers are never operating correctly - just that their errors are usually insufficient to undermine the substantive accuracy of their results. But I don't think that point would get anyone very far in court.
Stephen Mason has just written a very good Opinion piece on this topic:
https://www.computerweekly.com/opinion/The-cause-of-the-Post-Office-Horizon-scandal-The-Law-Commission-Judges-Lawyers
James Christie has today written an Opinion piece of this topic for Computer Weekly:
https://www.computerweekly.com/opinion/Law-Commission-misrepresented-experts-when-it-changed-rule-on-computer-evidence