Upcoming event

FOIMan to chair FOI and information rights event for Understanding Modern Government.

UMG bannerIn September I will be chairing a one-day event for Understanding Modern Government. Speaking at Understanding Freedom of Information Requests and Rights will be: Lynn Wyeth (@lynnfoi) of Leicester City Council; Dr Ben Worthy, Birkbeck College academic and expert on FOI and open data; Lisa Adshead from the Information Commissioner’s Office; and Heather Savory, Chair of the Open Data User Group. I’ll finish the day with a workshop looking at how to deal with some of the trickier aspects of request handling.

This event proved popular when it was run in February, so don’t miss out.

How do we stop Home Office files from going missing?

FOIMan considers what can be done to improve records management in government (and beyond).

recordsThe Home Office lost 114 files (well, actually 30,000, but let’s not go there). A couple of weeks ago I wrote about this in the context of a Home Affairs Select Committee hearing on this. This morning the Public Administration Select Committee heard from National Archives and Parliamentary Archives officials about records management and disposal across government.

As always, the hearing was somewhat dispiriting, with MPs veering wildly from giddy excitement at the thought that an archive holds 500 years of records (and perhaps moreso at the thought that they too will one day have their words of wisdom captured there) to being unreasonably rude to people trying their best to answer their badly construed questions. One day I’ll stop being surprised by their antics.

It led me to consider once again though – what is the answer?

Well, I can’t claim to be able to resolve all the problems of record keeping with one blog post, but here are some suggestions that would help prevent problems in the future. I do think fundamental change is necessary.

Senior management support. PASC was told that senior management support was crucial for improved records management – but most chief executives and other senior officers want little to do with records management – it is a menial task in their view. I was often told at the start of my career that good records management should be invisible. I always thought that was nonsense and I still do. Good records management requires active attention from everyone, and especially those at the top.

Records management must be properly resourced. As a records manager I was constantly told that we didn’t need “Rolls Royce (or gold-plated) records management” – in other words, the organisation concerned wanted “quick and dirty”. Think about the fact that most IT departments have significant budgets and staffing, can purchase solutions costing thousands on a whim (it often seems), and often their head is a senior director. All those systems will be used to create records, yet if there even is a records manager, they will be fairly junior, and probably alone, and in my experience will have no budget (the only budget I ever had was £2000 to buy boxes). How on earth do they change the culture of their organisation from that position? They don’t is the answer.

Records management needs to be built in. IT companies provide lots of solutions to public bodies and companies, but rarely do those solutions help organisations to retain the records they need and comply with organisational policies. Your organisation probably has a retention schedule specifying how long records should be kept for. But I bet if you asked your HR department whether their IT system deleted records in line with that schedule, they would look sheepish. They’re routinely breaching the Data Protection Act because IT companies don’t provide functionality that everyone needs if they’re going to comply. Similarly, we routinely use email to discuss our work, and important decisions are taken via that medium. Yet any attempt to select and preserve email remains cumbersome and a barrier to compliance with corporate policies. Why? Because of the way that the email system is built.

Improving the quality of staff leading on records management. (Sorry fellow records managers). I’m not convinced that records managers – even those who have actually completed a qualification – are suitably prepared for the realities of the workplace. Certainly I wasn’t. Perhaps it is less important for someone to be a professional records manager (whatever that means now) than to be someone who can get things done, to be a leader. And for that person to have sufficient support and credibility within the organisation.

A statutory basis for records management. Bernard Jenkin asked if destroying records inappropriately was an offence. It isn’t (unless someone has asked for it under FOI, but nobody has ever been prosecuted under that provision). I’m not sure a change in the law in that way would have the effect he hopes for, but certainly if we want records management to improve, the law needs to change. This has happened in Scotland, and whilst I don’t think it is a panacea, I do think it could help. And, as was also highlighted in the PASC hearing, it would be helpful to have record-keeping laws that extend beyond central government. At the moment, the Public Records Act, passed in 1958, only covers Whitehall and Quangos. Records management needs teeth if it is to be effective.

Realistic expectations. Information is being created constantly. It is impossible to keep on top of all of it. Organisations need to identify what is important and focus their attention on those areas. Good – no excellent – records management is possible but only if resources are sufficient and are focussed on key processes. Too many records management professionals consider themselves failures because their organisation doesn’t look like the picture painted in the international standard on records management (ISO15489 – the value of which I am personally sceptical of).

Society’s attitudes need to change. No, really. “Records management is everyone’s job” is a mantra of records managers the world over, but it really is – and not just at work. It’s not just there that we’re being bombarded with information, we get it at home too. It’s time that records management professionals stopped talking amongst themselves about the challenges and started talking to everyone else. The problems we’ve got in the Home Office and in our workplaces start at home. My tongue isn’t entirely in my cheek when I say that we need something akin to Who do you think you are? for records management. How full is your dresser drawer? perhaps.

What do you think?

FOIMan News to 18 July 2014

FOIMan reports on the latest FOI and information rights news stories.

The Information Commissioner's Annual Report was published this week.

The Information Commissioner’s Annual Report was published this week.

Information Commissioner uses launch of Annual Report to call for more resources and powers


The Information Commissioner, Christopher Graham, launched his office’s Annual Report for 2013/14 on Tuesday 15 July. Following on from his reported comments at an internal meeting earlier this year, Mr Graham highlighted the state of funding of his office, saying:

…to do our job properly, to represent people properly, we need stronger powers, more sustainable funding and a clearer guarantee of independence.

Law Commission advises Government to set up a comprehensive review of data sharing law

Last year the Law Commission opened a consultation on data sharing with a view to identifying perceived hurdles to data sharing in the UK. Last week (11 July) the Commission published its report on the consultation. It recommends a “full law reform project”:

…to create a principled and clear legal structure for data sharing, which will meet the needs of society.

It suggests that the project should map, clarify and modernise statutory provisions around data sharing, as well as looking at “soft law” such as guidance, Codes of Practice and sharing of best practice. The report was made to the Secretary of State for Justice, Chris Grayling.

Schools Trust holds information says Tribunal – eventually

In a week when the Education Secretary was removed from office, scrutiny fell on the complicated arrangements behind many academies, and the implications for FOI. Geraldine Hackett, a journalist, wanted to see the employment arrangements of the Chief Executive of the United Learning Trust, a Trust behind a number of academy schools. The Trust argued that they did not hold the information as the Chief Executive was employed by the Trust’s parent body, the United Church Schools Foundation Ltd. This argument had been upheld by the Information Commissioner and the First-Tier Information Tribunal. The Upper Tribunal overturned the FTT decision on a technicality and asked for a new First-Tier Tribunal to reconsider the case.

This time, the FTT found that because the information was held in filing cabinets that the Trust’s staff had access to, the information was held. The decision is in line with the important University of Newcastle Upper Tribunal decision which stated that “[h]old…is an ordinary English word and is not used in some technical sense…”. It also confirms the overall trend of decisions around the definition of “held” since then which suggests that if there is any doubt, the public authority probably holds the information in question. Robin Hopkins has analysed the case in more depth on 11KBW’s Panopticon Blog.

Data retention emergency legislation DRIPs through Parliament

The controversial emergency Bill requiring communications providers to retain data on telephone and internet use received Royal Assent in the same week that it was introduced to Parliament. It followed the European Court of Justice’s decision in April that the existing European Directive was unlawful as it represented a disproportionate intrusion into individuals’ privacy. At the time, the court said that the Directive:

entails an interference with the fundamental rights of practically the entire European population.

DPA without the Lawyer

DPA without the Lawyer

Publication of the Week

In a slight departure, this week I’m going to highlight a new book from The Centre for Investigative Journalism called DPA without the Lawyer. This is the latest in the series of books (including FOIA without the Lawyer and EIRs without the Lawyer) put together by the team at Request Initiative. DPA without the Lawyer is written by Jenna Corderoy and Brendan Montague and explains how journalists can take advantage of the subject access requirements of the Data Protection Act in their investigations.


FOIMan can deliver training in Freedom of Information, Data Protection and other information rights issues in your own premises. Get in touch for further details.

The mystery of the missing Home Office files

FOIMan examines claims that 114 “missing” files are a sign of sinister goings-on at the Home Office.

“Most of these files were probably destroyed because the kinds of topics that they covered would have been subject to the normal file destruction procedures that were in place at that time.” Mark Sedwill, Home Office Permanent Secretary

Are there any files missing from these boxes?

Are there any files missing from these boxes?

Having been a records manager for nearly 20 years (12 September 1994 is the date that particular file was opened), I felt obliged to watch the Home Affairs Committee’s session with the Home Office Permanent Secretary this week, which was prompted by admissions from his department that 114 files which may have related to allegations of child abuse are “missing”.

The media and many politicians have been quick to suggest that this is evidence of a conspiracy. The MPs on the Home Affairs Committee were keen to demonstrate that their minds were superior to those of Mr Sedwill and of the individual who carried out the inquiry that had identified the problem in the first place. Michael Ellis MP in particular wore his derision proudly, and why not? After all, who else would have thought to check the registry index to find out where the files had gone? His contempt for officials who fall below his and Sherlock Holmes’s intellectual level did not appear to be assuaged by Mr Sedwill’s calm assurance that, yes, that was how they’d identified the problem in the first place.

So how odd is it that these files have disappeared? Well, first, let’s look at the statistics. By the Home Office’s own admission, out of 750,000 registered files, around 30,000 files are missing. This appears to suggest that the 114 missing files are not that unusual.

The Permanent Secretary also shed some light on the history of record-keeping at the Home Office. He claimed that between 1982 and the late 90s, the Home Office adopted a government-wide filing system, and for much of the 80s this was administered by registrars (he actually said that the department had adopted the Grigg filing system in 1982 – but I think he is mistaken; the Grigg system governs the review and disposal of files, rather than how information is filed). Crucially, he explained that filing was devolved to service areas, so sometimes files were destroyed in those areas without central registry being informed. In the mid-90s I worked for a central government Quango and part of my responsibilities would be to register files created by staff, often in other buildings. The system was creaking at the seams, and I would have been unsurprised to hear that colleagues were bypassing it – keeping information outside the system and disposing of files without telling us. Mr Sedwill said that the system before that was even less controlled – a civil service Wild West if you like.

Information professionals often hark back to a golden age of file registries, when every file was registered centrally. This account casts doubt on its qualities. The Home Office Head feels more comfortable with our present age with emails that leave “digital footprints”. It is an interesting perspective, and there is some truth in it.

The truth is that – as I’ve written before (and again here) and will almost certainly do so again – records management is rarely maintained to a standard that supports the kind of forensic investigation that these MPs expect. Where there is a strong driver for good record-keeping – say, the commercial imperative for pharmaceutical companies whose products could be removed from the market without comprehensive records that can be produced at a moment’s notice – it can be highly effective. But it requires investment and commitment from the top. For most organisations, this imperative is not as strong, and the result is that organisations don’t really think about filing in anything other than a derogatory manner. Filing is something interns do. Even in those organisations that do need to take it seriously, it is usually only possible to maintain high standards in specialist areas.

As for Mr Sedwill’s suggestion that these files were almost certainly destroyed because of the subject matter, this too has the ring of truth. (And now we rightly reference the Grigg system). The Grigg system is the approach taken to review and disposal of files across central government since the 1950s. Files are closed when they are a couple of inches thick or 5 years old, then reviewed after 5 years to see if they’re still needed. At that point a destruction date is given to files unless it is decided that they have long-term value. Those files are reviewed again after 25 years, and this is when files are selected for permanent retention at the National Archives for the benefit of future historians. Guidance given to departments indicates the sort of subjects that ought to be retained – and to my mind the sort of files that appear to have been “lost” don’t fit those criteria.

In an ideal world the Home Office would have reliable records of what they had destroyed over the last 30 years. But few of us live in an ideal world. Of course, that helps those trying to hide misdeeds, because they can hide in the forest of doubt. But I’d still agree with the finding of Mr Sedwill’s inquiry that there is nothing in the loss of these 114 files that can be taken as evidence of corruption at the Home Office.

The Exemption Index: Section 43 – the exemption for commercial interests

Exemption Index logoFOIMan examines the commonly used exemption protecting information which if disclosed would prejudice commercial interests.


You’re a public authority and you want lots of companies to bid for your contracts so that you can ensure that you pay as little as possible for a good service. In order to ensure that the company you hire knows what it’s talking about, you need them to give you lots of information about their methods, and what they in particular can bring to the contract. But you’re worried that FOI will scare them off because anything they give you could become fair game for their competitors.

Then imagine yourself on the other side of a commercial venture. Your company has a great opportunity to pick up a valuable public sector contract. But dare you entrust your methods – which have been developed at great expense and through years of experience – to this public body? If your competitors got hold of your methodology and pricing structure, your advantage over them will be wiped out overnight. You’d be back to square one.

These concerns are what section 43 of the Act is designed to allay. The exemption provides a way to block disclosure of information that will harm the commercial interests of both public bodies and other organisations that they do business with. But it is a qualified exemption – subject to a public interest test – so there are no absolute guarantees…

This exemption is closely related to section 41 – the exemption for information provided in confidence, and may well apply where the specific requirements of that exemption mean that it can’t be used to protect documentation.

Information affected

Information that would (or would be likely to) affect the commercial interests of any party – including the public body itself. Could include documentation relating to procurement activities – contracts, tender submissions, sales figures, an authority’s purchasing position, but also things like a public body’s own marketing activities or products it has developed in a competitive market-place (eg course documentation in the higher education arena – though see comments below).

Things that FOI Officers need to know

  • the first subsection provides protection for “trade secrets”. This isn’t defined in the Act, but one Tribunal gave the example of the secret recipe for Coca-Cola (para 52). We might also refer to the “Colonel’s secret recipe” or Tony the Tiger’s special formula (if you’re of a certain age). Basically, it should be information used in business that is highly confidential, which gives its owner a competitive edge. Importantly, it shouldn’t be possible for someone to discover or reproduce it (para 53). There isn’t a great deal of case law on this subsection, perhaps because in most cases such information would be encompassed by the absolute exemption at section 41.
  • Some people think that the “trade secrets” exemption is an absolute exemption – this isn’t true. It is necessary to conduct a public interest test, even if you are convinced that the information concerned is a “trade secret”.
  • The more commonly used subsection at section 43 is the one for where disclosure “would, or would be likely to, prejudice the commercial interests of any person.”
  • The first stage in applying this subsection of the exemption is to establish whether the information is commercial in nature. “Commercial interests” can be interpreted broadly (para 42) (though see the discussion of financial interests v commercial interests under Things that requesters need to know below). Activities relating to the purchase or sale of goods or services (p4) can be described as commercial; but also anything affecting an organisation’s ability to be competitive in a market place. It is perfectly feasible for a public authority, even one that operates as a charity, to have commercial interests (para 31).
  • Given that both companies selling goods or services to public authorities, and the authorities themselves, can have commercial interests, it is important to state whose commercial interests you think will be prejudiced by disclosure. It may, of course, be both in some circumstances.
  • The next step is to set out what prejudice you envisage (p6) – in other words, what is the harm in disclosure? The widely accepted approach to this is set out in Hogan v IC, and summarised in the Commissioner’s guidance. You must remember to state how likely you consider the prejudice to be – choose between “would” and “would be likely to”.
  • If you are arguing that a third party – for example, a contractor – will be prejudiced by disclosure, make sure you consult them. The Commissioner and Tribunals have made clear (para 24) that they want to see evidence of the third party’s views.
  • As always, you must set out which information the exemption applies to. In relation to procurement activities, guidance issued by the Office of Government Commerce (OGC) has been recognised by Tribunals as being a useful guide as to what should and shouldn’t be withheld. One Tribunal indicated (para 87) that authorities should have to justify departing from this guidance (especially if withholding information that the OGC guidance indicates can be disclosed).
  • This subsection of s.43 is also subject to a public interest test. The OGC guidance referenced above states that: “[t]here is a strong public interest in showing who public money is being spent with, how much public money is being spent…and how the supplier arrived at the price…”.  Against that, many of the arguments an authority sets out as to the prejudice that disclosure would cause may well also serve as arguments as to why withholding the information would be in the public interest. For example, there is a public interest in public bodies being able to contract services effectively, so anything which prevents that (such as companies being fearful that commercially sensitive information will be disclosed to their competitors) will not be in the public interest. Generally speaking, it is not in the public interest for the market for particular goods or services to be distorted, through, for example, a company’s ability to compete being damaged. In the Student Loans Company case (para 55), the Tribunal took the view that damage to the ability of the company to pursue debtors was a valid argument in support of the exemption.
  • One area that has proved controversial is course materials. Whilst generally recognising that the organisation producing them has commercial interests (para 31), the Tribunals have proved reluctant to accept that disclosure would be likely to have a prejudicial effect. Universities have struggled to make this argument, but training materials developed by other public bodies are also difficult to withhold. The Commissioner does, however, appear to accept that in some circumstances, the exemption will apply to course materials – particularly where the materials are unique and the producer of the materials is operating in a competitive commercial environment.
  • As with many exemptions, there is the option to “neither confirm nor deny” whether information is held if it would or would be likely to prejudice the commercial interests of any party.

Things that requesters need to know

  • First off, has the public authority applied the exemption correctly? One common mistake is the conflation of “commercial” with “financial” interests. In Department for Work & Pensions v IC & Zola, the Tribunal agreed with the Commissioner in stating that possible increased expenditure on benefits was a financial interest, not a commercial one.
  • It’s not enough that there are commercial interests affected. The public authority has to demonstrate that they would, or would be likely to be, prejudiced. This was the problem that University of Central Lancashire had.
  • Public authorities should not apply “blanket” exemptions to whole contracts. It is necessary for them to analyse which sections need to be withheld (see the analysis at para 90), and to disclose the rest.

Essential case law

Recommended reading

FOI Man says…