Visit here to try out our Open Access online tool that, for the first time in one accessible place, allows the public to search meetings between UK Ministers and lobbyists. 

An aspiring Prime Minister once called lobbying ‘the next big scandal waiting to happen’. Remember that? His comments were prophetical. A month later a string of stings captured former ministers offering their services like ‘cabs for hire’. Following the subsequent general election, the new incoming coalition government sought to take action.

Their solution was more transparency. As you’d imagine, we’d find that proposition hard to disagree with – sunlight is often the best disinfectant for cleaning-out many of the misdemeanours committed in high office. However, ‘transparency’ is a much misused term and what ended-up in statute was far from what many, including Parliament’s select committee on constitutional affairs, had thought necessary to tackle the issue at hand.

The Government’s light-touch approach only requires disclosure of some minimal information from those attempting to influence ministers or senior civil servants on behalf of clients (‘consultant lobbyists’). I won’t dwell too much on the deficiencies of this regime because they have been comprehensively covered here, here and here. However, one of the key arguments deployed by ministers to explain why they didn’t deliver a lobbying register that meets international standards was as follows (see page 10 of its White Paper):

“Because Government departments already publish lists of their external meetings [i.e. with lobbyists] there is no need to provide further information about specific contacts with Government.”

This was a fudge. The two systems – voluntary disclosure by departments under the Ministerial Code and statutory reporting by a handful of lobbyists for hire – provide only a sketchy picture of what might be going on in Whitehall.

On the official register, all you can see is that consultant lobbying firm X has had some form of contact with an unknown minister(s) and/or civil servant(s) at some point during the previous three months on behalf of one of their many listed clients. Until fairly recently, the departmental lists of ministerial meetings were even worse. They were regularly published late (see Figure 7.9 here for more details), were often coy about the purpose of the discussions, and were scattered across 20+ different locations in over 1,000 files across the gov.uk website. Compared to the Scottish Register of Lobbyists, which also covers face-to-face interactions with lobbyists, albeit with a wider range of public officials and elected representatives, it looked amateurish and deliberately opaque.

In response, we have published Open Access, a searchable online tool that allows anyone to easily find any meeting between ministers and lobbyists that has been declared by Government departments. So if, for example, so wanted to find out what official meetings the Chancellor had with, say, a large investment firm who then went on to offer him job on leaving office, you could do so pretty quickly and without much effort.

This is the result of a three pronged strategy to wrestle this elusive data into something more accessible and linkable with other initiatives, like those analysing corporates’ positions on climate change and the UK arms lobby.

Firstly, we teamed-up with some academics at the University of Oxford’s Department of Sociology to pull together an initial dataset of meetings that had been published in a variety of formats (Word docs, PDFs, Excel files, CSV etc.) and locations into one place. This was fiddly business but saved weeks of intensive labour. The code for this can be found here on GitHub and was funded by a small ESRC Impact Acceleration Account grant.

Secondly, we sought to improve the quality and timeliness of disclosures through constructive engagement with officials. It also helped that the Prime Minister decided to remind colleagues of their departments’ responsibilities to publish this information around the same time. Bar the interference of the EU referendum and two general elections, this seems to have born some fruits. The promptness of publication has improved (see Figure 7.9 here for more details), the format of the files has become more regimented (and machine-readable) and overall the task of compiling the data into a single repository has become much less painful.

And thirdly, due to the 1,001 different issues with the source data, including the omission of ministers’ job titles, we’ve had to do a lot of manual correction and checking to pull together something that is more fit for public consumption. Given what you’ll see in Open Access has been finished with human hands there is bound to be some issues, so give me a shout if you spot anything odd and we’ll check to see whether there needs to be a correction.

There have also been limits to what we’ve been able to do with the data published by government. For example, we haven’t had the resources to standardise the names of lobbyists, so the Confederation of British Industry still has a wonderful array of spellings (‘CBI’, ‘Confederation of British Industry’, ‘Federation of British Industry’ etc.). However, you can download the data into CSV with the click of a button and do this yourself if you’re analysing a particular organisation, individual or industry. If the detail you find in the official record isn’t that enlightening, we’ve added a link to mySociety’s WhatDoTheyKnow tool so you can make and track a Freedom of Information request for more info.

As this is a live project, we’re keen to constantly improve the quality of the data. If you have some time or money to help us do this then please do get in-touch! We’re also particularly keen to hear when ministers don’t report engagements, like here, here, here, here, and here.

I’m not going to pretend this tool is a silver bullet. As much as I wish it’d completely clean-up politics, it won’t solve issues like MPs holding conflicts of interest they really shouldn’t, Parliamentarians taking all-expenses trips they really shouldn’t, or political parties soliciting cash for access, which they absolutely shouldn’t. However, it should help bring lobbying one step further out of the shadows.