Tag Archive for records management

How do we stop Home Office files from going missing?

FOIMan considers what can be done to improve records management in government (and beyond).

recordsThe Home Office lost 114 files (well, actually 30,000, but let’s not go there). A couple of weeks ago I wrote about this in the context of a Home Affairs Select Committee hearing on this. This morning the Public Administration Select Committee heard from National Archives and Parliamentary Archives officials about records management and disposal across government.

As always, the hearing was somewhat dispiriting, with MPs veering wildly from giddy excitement at the thought that an archive holds 500 years of records (and perhaps moreso at the thought that they too will one day have their words of wisdom captured there) to being unreasonably rude to people trying their best to answer their badly construed questions. One day I’ll stop being surprised by their antics.

It led me to consider once again though – what is the answer?

Well, I can’t claim to be able to resolve all the problems of record keeping with one blog post, but here are some suggestions that would help prevent problems in the future. I do think fundamental change is necessary.

Senior management support. PASC was told that senior management support was crucial for improved records management – but most chief executives and other senior officers want little to do with records management – it is a menial task in their view. I was often told at the start of my career that good records management should be invisible. I always thought that was nonsense and I still do. Good records management requires active attention from everyone, and especially those at the top.

Records management must be properly resourced. As a records manager I was constantly told that we didn’t need “Rolls Royce (or gold-plated) records management” – in other words, the organisation concerned wanted “quick and dirty”. Think about the fact that most IT departments have significant budgets and staffing, can purchase solutions costing thousands on a whim (it often seems), and often their head is a senior director. All those systems will be used to create records, yet if there even is a records manager, they will be fairly junior, and probably alone, and in my experience will have no budget (the only budget I ever had was £2000 to buy boxes). How on earth do they change the culture of their organisation from that position? They don’t is the answer.

Records management needs to be built in. IT companies provide lots of solutions to public bodies and companies, but rarely do those solutions help organisations to retain the records they need and comply with organisational policies. Your organisation probably has a retention schedule specifying how long records should be kept for. But I bet if you asked your HR department whether their IT system deleted records in line with that schedule, they would look sheepish. They’re routinely breaching the Data Protection Act because IT companies don’t provide functionality that everyone needs if they’re going to comply. Similarly, we routinely use email to discuss our work, and important decisions are taken via that medium. Yet any attempt to select and preserve email remains cumbersome and a barrier to compliance with corporate policies. Why? Because of the way that the email system is built.

Improving the quality of staff leading on records management. (Sorry fellow records managers). I’m not convinced that records managers – even those who have actually completed a qualification – are suitably prepared for the realities of the workplace. Certainly I wasn’t. Perhaps it is less important for someone to be a professional records manager (whatever that means now) than to be someone who can get things done, to be a leader. And for that person to have sufficient support and credibility within the organisation.

A statutory basis for records management. Bernard Jenkin asked if destroying records inappropriately was an offence. It isn’t (unless someone has asked for it under FOI, but nobody has ever been prosecuted under that provision). I’m not sure a change in the law in that way would have the effect he hopes for, but certainly if we want records management to improve, the law needs to change. This has happened in Scotland, and whilst I don’t think it is a panacea, I do think it could help. And, as was also highlighted in the PASC hearing, it would be helpful to have record-keeping laws that extend beyond central government. At the moment, the Public Records Act, passed in 1958, only covers Whitehall and Quangos. Records management needs teeth if it is to be effective.

Realistic expectations. Information is being created constantly. It is impossible to keep on top of all of it. Organisations need to identify what is important and focus their attention on those areas. Good – no excellent – records management is possible but only if resources are sufficient and are focussed on key processes. Too many records management professionals consider themselves failures because their organisation doesn’t look like the picture painted in the international standard on records management (ISO15489 – the value of which I am personally sceptical of).

Society’s attitudes need to change. No, really. “Records management is everyone’s job” is a mantra of records managers the world over, but it really is – and not just at work. It’s not just there that we’re being bombarded with information, we get it at home too. It’s time that records management professionals stopped talking amongst themselves about the challenges and started talking to everyone else. The problems we’ve got in the Home Office and in our workplaces start at home. My tongue isn’t entirely in my cheek when I say that we need something akin to Who do you think you are? for records management. How full is your dresser drawer? perhaps.

What do you think?

The mystery of the missing Home Office files

FOIMan examines claims that 114 “missing” files are a sign of sinister goings-on at the Home Office.

“Most of these files were probably destroyed because the kinds of topics that they covered would have been subject to the normal file destruction procedures that were in place at that time.” Mark Sedwill, Home Office Permanent Secretary

Are there any files missing from these boxes?

Are there any files missing from these boxes?

Having been a records manager for nearly 20 years (12 September 1994 is the date that particular file was opened), I felt obliged to watch the Home Affairs Committee’s session with the Home Office Permanent Secretary this week, which was prompted by admissions from his department that 114 files which may have related to allegations of child abuse are “missing”.

The media and many politicians have been quick to suggest that this is evidence of a conspiracy. The MPs on the Home Affairs Committee were keen to demonstrate that their minds were superior to those of Mr Sedwill and of the individual who carried out the inquiry that had identified the problem in the first place. Michael Ellis MP in particular wore his derision proudly, and why not? After all, who else would have thought to check the registry index to find out where the files had gone? His contempt for officials who fall below his and Sherlock Holmes’s intellectual level did not appear to be assuaged by Mr Sedwill’s calm assurance that, yes, that was how they’d identified the problem in the first place.

So how odd is it that these files have disappeared? Well, first, let’s look at the statistics. By the Home Office’s own admission, out of 750,000 registered files, around 30,000 files are missing. This appears to suggest that the 114 missing files are not that unusual.

The Permanent Secretary also shed some light on the history of record-keeping at the Home Office. He claimed that between 1982 and the late 90s, the Home Office adopted a government-wide filing system, and for much of the 80s this was administered by registrars (he actually said that the department had adopted the Grigg filing system in 1982 – but I think he is mistaken; the Grigg system governs the review and disposal of files, rather than how information is filed). Crucially, he explained that filing was devolved to service areas, so sometimes files were destroyed in those areas without central registry being informed. In the mid-90s I worked for a central government Quango and part of my responsibilities would be to register files created by staff, often in other buildings. The system was creaking at the seams, and I would have been unsurprised to hear that colleagues were bypassing it – keeping information outside the system and disposing of files without telling us. Mr Sedwill said that the system before that was even less controlled – a civil service Wild West if you like.

Information professionals often hark back to a golden age of file registries, when every file was registered centrally. This account casts doubt on its qualities. The Home Office Head feels more comfortable with our present age with emails that leave “digital footprints”. It is an interesting perspective, and there is some truth in it.

The truth is that – as I’ve written before (and again here) and will almost certainly do so again – records management is rarely maintained to a standard that supports the kind of forensic investigation that these MPs expect. Where there is a strong driver for good record-keeping – say, the commercial imperative for pharmaceutical companies whose products could be removed from the market without comprehensive records that can be produced at a moment’s notice – it can be highly effective. But it requires investment and commitment from the top. For most organisations, this imperative is not as strong, and the result is that organisations don’t really think about filing in anything other than a derogatory manner. Filing is something interns do. Even in those organisations that do need to take it seriously, it is usually only possible to maintain high standards in specialist areas.

As for Mr Sedwill’s suggestion that these files were almost certainly destroyed because of the subject matter, this too has the ring of truth. (And now we rightly reference the Grigg system). The Grigg system is the approach taken to review and disposal of files across central government since the 1950s. Files are closed when they are a couple of inches thick or 5 years old, then reviewed after 5 years to see if they’re still needed. At that point a destruction date is given to files unless it is decided that they have long-term value. Those files are reviewed again after 25 years, and this is when files are selected for permanent retention at the National Archives for the benefit of future historians. Guidance given to departments indicates the sort of subjects that ought to be retained – and to my mind the sort of files that appear to have been “lost” don’t fit those criteria.

In an ideal world the Home Office would have reliable records of what they had destroyed over the last 30 years. But few of us live in an ideal world. Of course, that helps those trying to hide misdeeds, because they can hide in the forest of doubt. But I’d still agree with the finding of Mr Sedwill’s inquiry that there is nothing in the loss of these 114 files that can be taken as evidence of corruption at the Home Office.

Sherlock’s shame

FOIMan suggests that limited assurance for records management at Barts Health NHS Trust means we can have limited faith in records management across the country.

What Moriarty thought of Barts' records management is not recorded. Or is it?

What Moriarty thought of Barts’ records management is not recorded. Or is it?

One of the Information Commissioner’s powers under the Data Protection Act is to carry out audits of organisations that process personal data. So far these have all been done on a consensual basis – the ICO does have the power to carry out compulsory audits on government departments (brought in after the infamous HMRC data loss), but has not yet used this. Executive reports of these audits are published on the ICO website, unless the body concerned asks for it not to be. In which case, you can draw your own conclusions.

I noted yesterday (thanks to a tweet from Jon Baines) that the latest recipient of an ICO audit is Barts NHS Trust. Now I’ve been concerned about Barts’ security arrangements since a private detective and his psychotic arch-enemy were allowed to access Barts’ hospital roof apparently without difficulty a couple of years ago, but leaving that aside, I was curious about the outcome of their audit.

Now, data protection is a large and complex subject, as anyone who’s ever studied for a BCS (ISEB) or other qualification in the subject will know.  So whenever the ICO is planning an audit, it agrees with the subject organisation what areas it should look at. Barts chose information security and records management.

Despite stray detectives and criminal masterminds, you’ll be relieved to hear that Barts performed relatively well on the information security aspect of the audit, receiving “reasonable assurance”. This will have looked at access to IT systems, encryption of portable media, and other measures designed to prevent accidental loss of, or damage to, personal data.

But it was records management that attracted limited assurance (the second lowest grading in the ICO’s assessment scheme). This is concerning. Not just for Barts (though I’m sure they do take this outcome seriously), but for everyone else. Barts has a pretty good reputation for records management. I’m not sure about the situation now, but for many years it actually had records managers (widely respected ones, even outside the health sector), which is certainly not the case universally in hospitals and NHS Trusts across the UK. So if Barts’ records management is considered not up to scratch then I doubt very much that most other NHS Trusts (or indeed local authorities, government agencies, schools, private companies for that matter) would fare any better under ICO scrutiny.

I’m personally sceptical of the desirability, let alone feasibility, of “perfect” records management. What’s important is that organisations can function and protect their most valuable and sensitive information. But it is fair to say that few in the public or private sector give this the priority it deserves. So unless you actually want some ammunition to make the case for more investment in this area, it may be a good idea to avoid asking the Information Commissioner to look at your records management.

After the flood

FOIMan recalls the impact a flood can have on an organisation’s information – especially if it chooses to store it in a basement.

It’s a terrible sight seeing people wading through their own homes. Those who live along the Thames, in Somerset, and elsewhere are having a terrible time and most of us can’t imagine what that must be like.

Flooded towns are nothing new, so organisations should take appropriate precautions

Flooded towns are nothing new, so organisations should take appropriate precautions

Floods can cause significant problems for information managers and their employers. Back in 2000, the south of England was suffering from a similar surfeit of wet weather. At the time, I was records manager for a council on the south coast. Many if not most organisations use basements as storage for their physical records (and often servers containing their digital records too).

And why not? These spaces are convenient and often very large. They’re otherwise wasted space – too dark and gloomy to accommodate human beings (unless they’re records managers or trogladytes).

Well, I’ll tell you why not. They’re also often damp. Many boxes were starting to show signs of mould. The plaster was peeling away from the wall. After 18 months of buttering up the Facilities Manager I finally got our office replastered and painted. We even got a plush(ish – it was local government after all) new carpet to replace the bare threads and occasional tufts of fabric which I had been assured were once a carpet about 15 years before. Which brings me to the other reason why it’s often a bad idea to store records in the basement.

Because when, like this year, there is so much rain that streets start to look like tributaries of a major river system, the sewers overflow. And the sewers in the area that I worked ran just beneath…you guessed it, the basement of the council offices. So in early 2000, you could find me wading ankle deep around a vast storeroom trying to move boxes of records above an ever-rising flood. It was like a not very exciting Indiana Jones film.

Eventually the waters subsided, and the dehumidifiers begged from the Museums Service chugged away at their heroic (and probably futile) task. A few building plans and housing files that had sunk beneath the waves were collected by a company, Harwell Document Restoration Services, that specialises in rescuing waterlogged papers. A few weeks later they would return, looking better than they had before (which wasn’t saying much). The tiles in the storeroom had lifted under the pressure of water gushing from beneath, and needed replacing. Records were sent off-site temporarily to a commercial storage company in rotation so that the floor could be repaired and retiled. And we got a (slightly less plush) carpet fitted in the office. And things returned to normal. Well, not quite. I’m sure it wasn’t entirely unconnected, but later that year I decided to climb out of the basement and found a job in London as Parliamentary Records Manager. Being based in the Victoria Tower, I figured there was much less chance that I would face another flooded records store.

Shortly after that, in early 2001, I received an email from my successor. The basement had flooded again.

If you are a facilities person, or a chief executive, and you think that you can save money by putting your records (and information managers, come to that) in the basement, you might want to think again. Aside from the fact that waterlogged records tend to make a mess of your lovely leather-embossed desk when you need to access them, there is a very good chance that your organisation will fail to meet its legal obligations. Aside from the many statutory requirements to retain information – auditors are probably going to be less than impressed if they have to don waders to check your receipts – failure to properly protect records will most likely be a breach of information law these days.

Section 46 of the Freedom of Information Act requires the Lord Chancellor to issue a Code of Practice on the management of records. The Code is written by the experts of The National Archives, and one of the requirements of the Code is that:

“Authorities should know what records they hold and where they are, and should ensure that they remain usable for as long as they are required.”

That Code is not statutory (though the Information Commissioner can take it into account in working out whether to take action against a particular authority), so perhaps more important to note is that retaining records – especially those relating to living individuals – in a basement that is prone to flooding is likely to constitute a breach of principle 7 of the Data Protection Act.

It is common knowledge that losing personal data on an unencrypted memory stick or a social worker leaving a file relating to a vulnerable child on a train can land an organisation with a fine of up to half a million pounds. But principle 7 also requires data controllers to take “appropriate technical and organisational measures” against “accidental loss or destruction of, or damage to, personal data”.

Businesses and public authorities in flood-hit parts of the country will have other – and perhaps more urgent – things on their minds at present. But all organisations should think very carefully about whether bargain basement storage really is the opportunity it appears.

Image by Keith Moseley (Drybridge Street Flooded, Monmouth) [CC-BY-SA-2.0 (http://creativecommons.org/licenses/by-sa/2.0)], via Wikimedia Commons. 

An unpopular answer

FOI Man argues that there’s an easy answer to improving information and records management. The problem is that nobody likes it.

Jimmy Savile appears to have got away with, well, not quite murder, but rather a lot of very serious crimes in his lifetime. This morning BBC Radio 4’s Today programme looked at the failure of police forces around the country to recognise what was going on, despite many of them receiving complaints during his lifetime. Drusilla Sharpling of Her Majesty’s Inspectorate of Constabulary (HMIC) stated that forces have differing standards for recording information on the police national database. The problem, she argued, was one of information management.

This is but the latest example illustrating how fundamental information management is to the provision of just about every service in the country. And also how until something goes wrong (and usually within a few weeks after something goes wrong), it’s perfectly acceptable to put it to the bottom of the pecking order. We see plenty of evidence of personal data breaches that receive the attention of the Information Commissioner, most of which come down to poor information management. Last year London Metropolitan University lost its “trusted” status in relation to international students, partly because it failed to keep attendance records up-to-date.

We’re all inclined to leave the filing on the backburner. It’s a chore to us. Even I tend to find lots of jobs that need doing to prevent me spending time on records management. Recently I ran a workshop and the most senior manager present complained that email and shared drives were a problem as they weren’t being managed. So I asked him if he was going to set up a programme to be monitored by his management team and encourage his staff to spend time filing and deleting documents. He shrugged and said they don’t have the resources for that. And we moved on. That’s what we always do. We move on.

As a result, things don’t improve. FOI requests get refused on cost grounds or take forever to answer. Data breaches continue to happen. Children get abused and the abusers get away with it because a police force somewhere doesn’t employ enough people to keep records up-to-date. (And no doubt those police forces are under political pressure to cut “back office staff” – but that’s a whole different blog post).

I first got involved with FOI because I saw it as the answer to my prayers. At the time I was stuck in a regularly flooding basement in a local authority managing to the best of my ability a file store – a big room with lots of boxing on shelves. Some of the records at some point had been put in a location under the town hall steps. When I went to see them, I had to wear a mask because of the mould spores on the old ledgers that had been allowed to become damp (actually, sodden). Records management wasn’t sexy, but worse than that, the council didn’t HAVE to do it. There were all sorts of legal requirements on the council, and those were the things that got prioritised. So a piece of legislation that introduced a Code of Practice for managing records was music to my ears. The Lord Chancellor, no less, was calling for improved records management. Colleagues would have to listen to me. Bosses would have to provide resources. Beyond the wellington boots they provided when the basement flooded.

Well, sort of. Here we are 12 years later. Lots of public authorities appointed records managers. But that was it. Unless everyone in the authority spends time managing the information they deal with, and the technology and other infrastructure is put in place, all you’ve really got is…lots more records managers in basements, usually now with lots of other responsibilities that mean they have no time to work on records management. There’s plenty of evidence that actual improvements have been fairly limited in many, probably most public authorities.

So what’s the answer? Well, firstly, we could all try to remember that managing the information and records we work with is part of the job, not something that will wait until a quiet moment that never comes. But if that fails? I’m beginning to wonder if more legislation is the answer. There is precedent.

In Scotland, an abuse scandal in children’s homes led to the Scottish Government introducing The Public Records (Scotland) Act 2011. Under this Act, public authorities are obliged to prepare and implement a records management plan. The plan itself has to be approved by the Keeper of the Records of Scotland. In England and Wales, The National Archives has stringent rules already that apply to central Government and also inspect “places of deposit” – local Record Offices that are authorised to keep national public records relating to their local area.

But by and large most public authorities are left to their own devices. They can adopt technology without worrying about management of the information that the technology creates. They can treat records management as a luxury to be given up in tough times. They can leave their staff to their own devices, however important it might be to have a record of their work. Maybe it is time to look at the way we prioritise records and information management at a national level. Because whatever we’re doing at the moment isn’t quite working.

With thanks to Pete Wadley of the National Records of Scotland for information about The Public Records (Scotland) Act 2011.