Tag Archive for Open Data

FOI v Open Data?

FOIMan questions Cabinet Minister Francis Maude’s suggestion that FOI requests will be made redundant by the government’s transparency and open data initiatives.

Francis Maude made a speech earlier today about government transparency and open data. It caught my attention partly because of a section in which he talks about the Freedom of Information Act:

Ten years ago the Freedom of Information Act came into force. Tony Blair called it his biggest mistake. But it was a historic piece of legislation, it wasn’t perfect. My aim if I’m honest with you is to make Freedom of Information redundant. My view is that we should be proactively making public everything that is appropriate. You should make redundant the need for people to ask for access to information.

Rt Hon Francis Maude MP, Minister for the Cabinet Office (reproduced under the Open Government Licence v3)

Rt Hon Francis Maude MP, Minister for the Cabinet Office (reproduced under the Open Government Licence v3)

I don’t think anybody would question the laudable aim of making information available to the public proactively. The government has made lots of data available and, as I reported in my last post, has forced other parts of the public sector to be more transparent. There are practical challenges caused by these requirements, such as how to make the vast amount of data useful and accessible to the public, and how to avoid making public authority websites unnavigable and cumbersome. But in principle it is undoubtedly a welcome development for central government to be talking so positively about transparency.

However, I do question Mr Maude’s aim of making “Freedom of Information redundant”. The key here is his phrase “we should be proactively making public everything that is appropriate.” Who decides what is “appropriate”? How do people challenge that decision? What if people have further questions about the information that has been disclosed?

Statistics also out today show that of the requests that were considered “resolvable” by the Cabinet Office between July and September this year, only 29% were granted in full. So nearly three-quarters of the time, the Cabinet Office considers that it is not appropriate to disclose the information people actually are interested in. Of course, that doesn’t mean that the Cabinet Office is wrong to withhold that information in every case, or even most cases, but it does place a whopping big question mark over Mr Maude’s ambition to make FOI redundant. No matter how much information is published, people will want to know more than government is willing to put out there.

Mr Maude’s comment echoes the Prime Minister’s statement that FOI requests are “furring up the arteries” of government. Of course, if you want to run government efficiently, FOI is not the best way to be transparent. Answering requests can be time-consuming, and it is difficult to allocate and plan resources. This is the argument of many in the public sector who criticise it. But it is the main reason I think it is so valuable. If you were running a business, nobody in their right mind would choose to obligate themselves to answer requests for information in this way. That’s the point. Delivering public services is not about running a business. It involves spending people’s money to make the country and communities work in a way that benefits as many as possible, whilst giving them as much say as possible in the way that happens. That a government recognised that people should have a right to question public bodies about the way they are delivering services, despite the inconveniences that it may involve, is something that gives me a little faith in politics – and God knows, we need more of that.

Mr Maude and the government’s ongoing efforts to publish more public sector information should be welcomed. But they will never make FOI redundant – true transparency requires both.

Practically Speaking, Part II – Game, Dataset & Match

FOIMan brings you the second in his series of articles for PDP’s Freedom of Information Journal.

A couple of weeks ago I wrote here about the series of articles I’ve been writing for PDP’s Freedom of Information Journal. The first article was on deadlines, and this week I’m making available my piece called Game, Dataset and Match, which, as the more perceptive amongst you will have gathered, is about the new dataset requirements that came in on 1 September. You should note that this article was published much earlier in the year before the Information Commissioner and Ministry of Justice had published their guidance/finalised Code of Practice, so is a little out of date already. (The general trend this year has been that if I’ve prepared a talk or an article, a major development supercedes it just before publication or presentation. All I need now is to find a way to exploit this extraordinary gift.)

However…in September I was also asked to write on this subject for Privacy Laws & Business, another information rights publication, and as a special bonus, I have their permission to reproduce that piece here as well. Bang(ish) up to date. So there you go, two articles on datasets and FOI for the price of one right here at foiman.com.

Next week I’ll be bringing you a piece on handling vexatious requests. And there may be one or two other posts in the next week or so (including another guest post from academic researcher Joe Reddington), so keep your eyes peeled.

Game, Dataset and Match

FOI Man highlights forthcoming changes to FOI and provides some hints and tips for public authorities on how to deal with them.

Last year, the Protection of Freedoms Act was passed. Amongst the changes it brought in, were a small number of amendments to the Freedom of Information Act.

But we’re still waiting for most, if not all, of those changes to come into force. To bring them into force, the Government has to lay a commencement order before Parliament…and this is yet to happen. It was expected that the commencement order would be laid last month, bringing the changes into force on 1 April. But this has now been delayed, as reported by the Information Commissioner’s Office earlier this month.

The most significant change is the requirement on public authorities to release datasets in a reusable format, and to publish disclosed datasets in their publication schemes. In my latest article for PDP’s Freedom of Information Journal, I’ve written about these requirements and how to comply with them. (And don’t forget also my report on open data work at Southampton University, which contains further tips on managing and publishing open data).

Personally, I don’t think public authorities should worry too much about these changes. There are a few reasons for this. Firstly, as I commented when the Bill was first published, the effect of these changes will be very limited in my view – they change very little. Public authorities already have to provide information in the format requested “so far as reasonably practicable”; I’ve never been convinced by Francis Maude’s claims that public authorities routinely (and deliberately) choose to disclose data in pdf just to frustrate entrepreneurs.

There may be a mad rush of requests for datasets later this summer (if indeed the Government sticks to its latest timetable), and no doubt there will be more impact for some than others. But I don’t anticipate that this is going to cause significant issues overall.

What can public authorities do to prepare? Well, I suggest the following:

  • identify your key datasets – if you regularly get requests for particular data, then you know what is likely to be asked for in future
  • work out what kind of licence you want to apply to these datasets if you disclose them; the easiest thing will be to use the Open Government Licence for information your authority owns the copyright for, but it is likely you will also be able to offer a non-commercial licence (limiting re-use to non-commercial use) or a charged licence (allowing re-use in exchange for a fee)
  • set up a section in your publication scheme for datasets and if you are happy to disclose datasets and make them available for re-use, get them up there on your website for people to use – don’t wait for the requests
  • once you’ve released a dataset and have licensed re-use, you are obliged to make it available in your publication scheme and to keep it up to date.

That last point may sound worryingly like a potentially unmanageable task for some public authorities, but the relevant amendment goes on to say “unless the authority is satisfied that it is not appropriate for the dataset to be published”. “Not appropriate” isn’t defined (as ever), but if it would be expensive to keep the dataset up to date, for instance, that might well be a justifiable reason not to do so.

So the usual advice applies to these changes – don’t panic! But we’ll have to wait and see what the actual impact will be. And indeed when that impact will be felt. At the moment we only have a draft Code of Practice to go on, so hopefully these few thoughts will be useful.



Are universities transparent enough?

FOI Man talks to Times Higher Education about universities and openness.

Times Higher Education magazine this week features an article about…higher education, and how open and transparent it is. I was interviewed for this feature a few weeks ago – wonder at my high rhetoric – “[FOI is seen] as a pain in the backside”. Seriously, it’s a comprehensive survey of all aspects of transparency in the UK university sector, including everything from FOI to open data to MOOCs (that’s massive open online courses for those of you not in the know).

Opening up Open Data

FOI Man reports on a visit to meet Open Data experts at Southampton University.

Chris Gutteridge is a techie – in the best possible sense of the word. When I first arrive, he’s eager to show me “the cool stuff”. And it really is cool.

Despite my pseudonym, I don’t have superpowers, yet Chris is able to fly me over and into the university campus, bringing us to the ground with a bump outside his building. Of course, I’m talking about a visual representation of the campus achieved through the neat trick of linking building data collected from the university estates department to Google Earth. Chris is experimenting with making the maps 3D by collecting (and in some cases creating) data on the height of buildings on campus.

And the cool stuff like this is how he sells the open data initiative to colleagues across the university. Central and local government have already made great strides in the open data arena, but Southampton are pioneers – and indeed, award winners – in the higher education sector. After all, it is the home of the Government’s open data tsars (can you have two?) Professors Nigel Shadbolt and Tim Berners-Lee (also famous these days for sitting behind a desk entertainingly at the Olympic opening ceremony – no mean feat). But even with this top level support, it can be tricky to get busy departments to cooperate, as many FOI Officers will sympathise with.

Chris starts listing the objections colleagues raised when asked to provide datasets to make available for re-use through their Open Data Service. He looks puzzled when I start laughing, but some of you will recognise “what about data protection?”, “what if terrorists exploit it?”, and “isn’t it commercially sensitive?”. And the question that often lies behind these objections in reality, “what if someone realises our data is unreliable?” (or “shit” as Chris rather more prosaically puts it). Chris has blogged a whole list of these concerns and they all sound rather familiar.

But by demonstrating that once the data has been collected centrally it can be made useful to the departments that originally provided it, Chris and his team have started winning them over. One of their biggest supporters is the Catering department, who already maintained spreadsheets with details such as the items available in the cafes, bars and restaurants across campus and their unit cost. With a little adjustment, the spreadsheets were made into reusable data. Now the Catering department no longer have to manually update their web pages as the availability and cost of food and drinks in individual outlets is now pumped live from regularly refreshed open data straight to the pages. They’re now doing similarly useful things with events calendar data.

Chris thinks this repurposing of the open data is key to making open data a success. His mantra, repeated to me several times, is that the main return on investment in open data for the university is the availability of “a huge pile of data that can be used internally without fear”. And if outside users (or indeed their own students) find the data useful to create new apps, then all the better.

Chris gives me some tips for anyone wanting to establish an open data repository. Given that recent FOI amendments mean that this will soon be a requirement, that’s pretty much all of us in the public sector.

Avoid controversy. You want to get buy-in from colleagues, so don’t startle the horses. Pick simple but useful datasets that nobody will challenge. At Southampton they started out with building data. It’s data that isn’t sensitive – people can see most of it by walking around the campus – but as we saw at the start of this piece it can be put to very effective use by developers.

Think carefully about what datasets you think should exist. Then speak to the relevant departments and see if they do have those datasets in a useful format. Chris suggests that often suppliers can be very helpful, and can give advice on how datasets can be extracted from their systems. They may even be prepared to look at how their systems are specified so that datasets can be more easily exported in future.

Encourage colleagues to give you their data even if it isn’t complete. It is better to have some data than no data at all in most circumstances.

Be ready to challenge concerns. Some colleagues will be concerned about giving data away for free, rather than making money from it. But as Chris points out, “if a Chinese takeaway gave away its food, it would soon go out of business. But if a Chinese takeaway didn’t give away its menus, it would go bust even faster.”

Look at what you’re already making available, and how. Chris demonstrates that my university is already making available data in a reusable format because we have an online repository called EPrints. He tells me that if you have an RSS feed on your website, you are “already more or less doing open data”.

Make information available in a reusable format. Open data enthusiasts grimace at the mention of portable document format (pdf). At the very least, try to make data available in a spreadsheet format.

Adopt an open licence. Open data is about more than publishing information in a reusable format. It is also about licensing. It’s really only open data if you state on your website/open data repository that data can be reused without charge. The best way to do this is to adopt the Open Government Licence.

Keep datasets up-to-date. One of the things that public bodies will be expected to do once the FOI amendments on datasets come into force is keep published datasets up-to-date. I ask Chris how Southampton maintain their datasets. It depends, of course, on where it comes from.

  • Hearts will sink to be told that a lot are maintained manually by his team – not many of us will have resource to spare. So to make this work we need to ensure that it’s as easy as possible to get things corrected. Chris suggests encouraging feedback from users of the data so that they can flag up when data needs refreshing. At Southampton they also use student volunteers to maintain the datasets, which might be something to consider for universities in particular.
  • Some datasets automatically update. Depending on your set up, some will be fed live data from systems maintained by the supplying department. Some suppliers who provide systems across the public sector are already thinking about how to build open data publication functionality into their databases.

Chris is keen to encourage the growth of an information ecology, with others across higher education publishing more and more open data. He encourages universities to consider creating ‘profile’ documents on their websites, describing where their key datasets can be found and in which formats. This will help with auto-discovery by the new open data hub for higher education, which will eventually provide potential users of datasets with a single portal to locate useful data.

So the higher education sector isn’t moving into this new era of reusable open data entirely unprepared. And if you’re thinking of taking your first steps into this brave new world, hopefully you, like me, feel slightly less daunted thanks to Chris’s enthusiasm.

I’d like to thank Chris, Ash and Patrick for letting me disturb them for a whole afternoon last Friday. Any errors here reflect my ignorance and not their skill or willingness to share!