Fishsubsidy.org: New online database of €1.1 billion in EU fisheries subsidies; concerns about declining data standards

Fishsubsidy.org, the transparency project which in 2009 launched an online database of EU fisheries subsidies from 1994 to 2006, has launched a new database of payments under the European Fisheries Fund, from 2007 to 2010.

The fisheries subsidy database is online at http://www.fishsubsidy.org/eff

A critical report on the availability of the data: http://eutransparency.org/wp-uploads/2011/11/eyes-wide-shut.pdf

The project’s co-founders have sounded a grave warning about the deteriorating quality of data released to the public, and the implications of this the waste, fraud and abuse of EU funds.

Nils Mulvad, the Danish data journalist and fishsubsidy.org co-founder who led the collection of data from the twenty seven EU member states said:

“It is really, really bad. Many governments don’t comply with basic EU laws on transparency. Some governments publish no data at all, others are publishing incomplete data in bad formats like PDF files running to thousands of pages. This is money from the EU budget, paid for by European citizens who have a right to know who gets what. The European Commission must get a grip.”

Fishsubsidy.org co-founder Jack Thurston said:

“There is a new European Transparency Initiative. But today we have less information on EU fish subsidy payments than we used to have in past years. It’s a real step backwards in transparency and at a time when we desperately need to know how this money is being spent. Are EU funds are being used to fish for over-exploited fish stocks, or perhaps worse, for criminal fishing operations. We just don’t know. What is most startling is that neither does the Commission because we know that they have not themselves asked for this data from national governments.”

A transparency index evaluates the data published by member states for completeness, details and accessibility. It shows which countries are doing better and which doing badly. The ranking is topped by Sweden. Belgium, Czech Republic, Estonia and the UK score relatively highly, though with significant deficiencies. The worst performers were Greece and Portugal, which appear to have published no data at all, despite spending a significant share of EU fisheries funds. Spain, which accounts for some 40 per cent of fisheries subsidies spending, scored just 48% in the transparency ranking.

Eyes Wide Shut: EU rules on transparency in fisheries subsidies are failing citizens – and the European Commission couldn’t care less is a report describing fishsubsidy.org’s quest for data on the European Fisheries Fund. It is available at: http://eutransparency.org/wp-uploads/2011/11/eyes-wide-shut.pdf

Fishsubsidy.org is a project coordinated by EU Transparency, a nonprofit organisation in the UK, and the Pew Environment Group. The aim is to obtain detailed data related to payments and recipients of fisheries subsidies in every EU member state and make this data available in a way that is useful to European citizens. Subsidies paid to owners of fishing vessels and others working in the fishing industry under the European Fisheries Fund total about €1 billion a year (2007-2013).

Detailed analysis of EU fisheries subsidies from 2000 to 2006 is available in “FIFG 2000-2006 Shadow Evaluation” (Cappell, R., T. Huntingdon and G. Macfadyen) at http://www.pewenvironment.eu/resources/view/id/115178?download=true

A list of vessels in the tuna fleet that receive EU subsidies is available at www.fishsubsidy.org/EU/tuna-fleet, and a list of vessels convicted of serious infringements (illegal fishing) is available at www.fishsubsidy.org/infringements. A list of vessels that received EU grants for modernisation and shortly afterwards grants for scrapping is available at: www.fishsubsidy.org/news/features/modernised-then-scrapped/

Under the European Transparency Initiative, details of all end beneficiaries of EU funds should be published, to improve accountability, legitimacy and as a way of combating fraud and abuse. See: www.ec.europa.eu/transparency/eti/index_en.htm

The Pew Environment Group is the conservation arm of The Pew Charitable Trusts, a nongovernmental organisation that applies a rigorous, analytical approach to improve public policy, inform the public and stimulate civic life.

Program for the conference

Hands-on training

Sarah Cohen on timelines

Internet

Albrecht Ude: Surfing anonymously and unfiltered

Albrecht Ude: Searching people

Yahoo Pipes

Paul Bradshaw on Yahoo Pipes

http://onlinejournalismblog.com/2008/07/16/how-to-create-basic-mashups-with-yahoo-pipes/

http://onlinejournalismblog.com/2009/12/18/how-to-create-a-custom-meta-search-in-yahoo-pipes/

Excel-training:

Luuk Sengers and John Bones: Excel 1
Luuk Sengers and John Bones: Excel 2
Extra material from Jennifer LaFleur

Google Fusion:

Jennifer LaFleurs material

Hand-out on polygons – Exercise.

Google Refine:

Combining data with Refine

Cleaning data with Refine
http://onlinejournalismblog.com/2011/07/05/cleaning-data-using-google-refine-a-quick-guide/

http://onlinejournalismblog.com/2011/04/14/data-for-journalists-json-for-beginners/

http://onlinejournalismblog.com/2011/03/18/getting-full-addresses-for-school-data-in-an-foi-response/

http://onlinejournalismblog.com/2011/09/09/how-to-use-the-cablesearch-api-to-quickly-reference-names-against-wikileaks-cables/

http://onlinejournalismblog.com/2011/07/22/how-to-grab-useful-political-data-with-the-they-work-for-you-api/

http://blog.ouseful.info/2011/05/06/merging-datesets-with-common-columns-in-google-refine/

Chase Davis on Google Refine API

Site on Google Refine with tutorials

Panels:

Crowdsourcing
Crowdsourcing

Sebastian Mondial: Advices for crowdsourcing

Nils Mulvad: New methods for covering events and investigative journalism melt together
Nils Mulvad: Crowdsourcing in investigative journalism.

Apps for investigations
Nils Mulvad: Some Apps for investigations
Story-based inquiry
English: Story-based inquiry

Russian: Story-based inquiry

Presentation of Story-based inquiry in Kiev

Access to data

DATA FLOOF IN INDIA AFTER RTI
David Smallmans handout 1
David Smallmans handout 2

Wikileaks

Andy Lehrens presentation on Wikileaks
Jan Gunnar Furelys presentation on Wikileaks
Musikilu Mojeeds presentation on Wikileaks
David Leighs presentation on Wikileaks
Jan Michael Ihls presentation: How can journalists and leakers trust each other again

Some other material

Giannina Segnini on CAR-methods

Sarah Cohen on textmining

Sarah Cohen and Aron Pilhofer on vizualisation

Andy Lehren: Social Media Tools

James Grimaldi: Multimedia

Bo Elkjærs presentation

Frédéric Zalac, Serena Tinari and Susanne Reber on Investigating Big Pharma

Arms smuggling – TRILOGY In the Name of the State – Matej Surc, Blaz Zgaga – Slovenia
(Maps in this presentation are copyrighted by Sanje Publishing House, Ljubljana, Slovenia. www.sanje.si)

Tracking the billions – with O’Murchu and Shleynov – presentation 1
Tracking the billions – with O’Murchu and Shleynov – presentation 2

Chief Bisong Etahoben: THE KILLING SOCCER STORY

Ana Aranas presentation

Result of group thinking – lots of good tips

Advice from Albrecht Ude on New Tools for Today’s Investigative Journalist.

It conflicts with human rights to publish names and amounts on recipients of EU farm subsidies. This has the European Court of Justice decided.

Germany and Ireland have now shut down their websites, but the information can be found on farmsubsidy.org.

Today it’s hard to tell the consequenses of the decission.

See Jack Thurstons, Brigitte Alfters and mine initial reaction to the decision on behalf of Farmsubsidy.org.

Reaction from the Commissionen
“Subject: Judgement of the Court of 9/11/2010 in cases C-92/09 and C-93/09 Volker und Markus Scheke GbR and Hartmut Eifert

Member States are informed that the Court issued its ruling on 09.11.2010 in the cases cited in the subject. For individuals receiving aid from EAGF and EAFRD, this ruling in an appeal partially invalidates Council Regulation (EC) No 1290 and in particular Article 44a which introduces an obligation for Member States to publish the data of beneficiaries of agricultural fund. It invalidates Commission Regulation (EC) No. 259/2008 in its entirety.

Consequently, with immediate effect, all Member States are requested to suspend the publication of data on individual beneficiaries affected by the invalidated provisions.

As regards legal persons, provided this is technically and administratively feasible, Member States may continue to publish the data. It is requested from Member States to kindly inform the Commission during the Funds Committee meeting of November 16 2010, if they are technically and administratively able to distinguish between natural and legal persons.

The Commission will embark on the drafting of a new proposal for a Council and Parliament Regulation (amendment of Regulation (EC) N° 1290/2005) which will take account of the objections raised by the Court.”

Farmsubsidy.org has released information on farm subsidies for all 27 EU memberstates for free and made the database searchable in a simple way.

Danish payments of EU farm subsidies got journalist Kjeld Hansen and I access to in 2004. Several stories followed in Danish medias. It was a breakthrough, because it was the first country in EU to release the information.

The year after together with the UK-based analyst Jack Thurston and others I founded Farmsubsidy.org. The goal was to get the data out in all EU countries. Slowly this developed. We got the data in some countries, we run court cases and other cases to press the authorities.

Farmsubsidy.org has got more data and a new design.

First time with all payments
EU decided then memberstates should release their data. In May 2009 it was the first time all countries was forced to publish data, but several countries did it in a way, where we couldn’t get it all.

In May this year the memberstates then published the data for 2009. Despite we still face a lot of problems with data quality, we managed to get a more and less complete data set for all countries. For the first time we then see a total set of data of all the payments of €56 billion in 2009.

The data has been used for several stories. Look at the overview from journalist Brigitte Alfter.

2009-data searchable
Now is the website then opened for conducting your own search. It is still possible for analysts and others to get the full dataset. Look here.

The work in Farmsubsidy.org has been financed by Hewlett Foundation, but they will stop. We seach for other sources, so we can continue to gather data and help media and others to analyse. If you have ideas please help.

Farmsubsidy.org

In practice we can divide crowdsourcing according to the part of the journalistic process where the crowd is helping, and which tools are used to organize this help. This gives us a matrix to use for analyzing projects. You can also define projects on the basis of other categories, but here I use the journalistic process and the organizing tools.

The article is published in full lenght here on the website for The Global Investigative Journalism Nettwork.

In the article I place different crowdsourcing projects in the different cells in the matrix, and use it for the purpose of analyzing how to get success out of these projects. Still it’s also obvious we’re looking at this whole process from the angle of the journalist. How can he/she use the crowd to get better stories? In a long perspective this angle might be too narrow, because also the journalist is only one party in the process of uncovering a specific story, where you’ll see a lot of other actors, so it might be better to look at the process as a whole. But for now we will focus only on the crowdsourcing – looking at it from the journalist point of view.

Timing

You might also divide the crowdsourcing projects according to other categories. This could, for instance, be timing:

1. The crowd gather information while things are happening
2. The crowd send you information after the things have occurred

Kind of media

You might also divide crowdsourcing projects according to the kind of media:

1. Big, well-known media with a long tradition of crowdsourcing
2. Well-known media experimenting with crowdsourcing
3. Small media – not so known – with a long tradition of crowdsourcing
4. New, small media starting their first project.

Kind of story

You might also divide according to the kind of story:

1. Stories with a broad appeal to the crowd where it’s obvious that the crowd has the potential to help with a lot of cases (parking fines, policy response time, doctors response time, disasters).
2. Stories where you identify an existing community and get it to help. It can be a rather specific area, but the good thing for you is that others already have gathered the crowd.
3. Stories with no community and no potential knowledge in the crowd (tax paying by multinational companies, defense contracts).
4. Stories where you have the documentation, but need help for analysis (it could also be defense contracts, state budget, and calendar of a prime minister).

Tools to raise involvement

And you might divide according to which tools you use to raise involvement and get the crowd to help:

1. It’s using more campaign-like tools like planning stories from a rap-contest to get young people into the community. And there can be a lot of other tools here.

These four other ways to categorize crowdsourcing projects might also be useful for analysis.

Getting started with crowdsourcing

I put up these tips if you have a small media or this is the first project in crowdsourcing:

1. Make Integration between comments on Facebook and the project. This is not crowdsourcing, but it’s building a community round your project, interacting with the users, and crossing the line to crowdsourcing.
2. Try to make comments from Facebook cases in the project. But be precise. Don’t use everything.
3. Check Facebook for existing communities, when you start a new project. Look also other places. If there’s a good community or a NGO, consider how you can be part of that.
4. Document and publish your data, webtv, pictures under creative commons. A good type is “Attribution Non-Commercial Share Alike”.
5. Put up a system where it’s very, very easy to contribute with whatever kind of stuff you need.
6. If it’s analyzing documents or data, make some awards and other good stuff to make it attractive to do as much as possible. Credit the contributors for their work.
7. On the other hand, make it also possible for people to contribute without becoming public. Be sure to have a system to protect sources.
8. Try to figure out some more ways to make it a good idea for people to contribute. What is in it for them.

Millionaires went up from 1040 to 1212 – expected to grow to more than 1400, when we get the full list from Portugal and UK. Press coverage of this years data harvesting festival for farmsubsidy.org was really good. Several medias round Europe used the material prepared by the festival in Brussels 3-4 of May 2010.

Adrian Mogos, Gavin Sheridan and Sebastian Mondial doing the last preparing before release in Brussels.

Both Tommy Kaas and Nils Mulvad joined the festival – Nils as responsible for the overall harvest of data. During the 2 days we also celebrated the fifth anniversery of Farmsubsidy.org – formed  3rd of May 2005. At the festival 17 people from 11 countries worked on getting the data out and harvesting the first stories in the data.

At the festival we also formed the network of CAR-journalists in Europe – datajournalism.eu.

The press release on the results of the festival: harvest-festival-press-release

The list of millionaires: http://tinyurl.com/2009-millionaires

Analyzis of all countries openness on farm subsidies:

http://tinyurl.com/transparency-rating

Brigitte Alfters blog on EU-Observer with links to all the press coverage of the release.

Jack Thurstons overview of all press coverage.

Use of data-possibilities in investigative journalism explodes in these years. That was one of my statements at the 7th Global Conference for Investigative Journalism and Computer-Assisted Reporting. With more than 500 participants from over 80 countries the conference in Geneva from 22th to 25th of April 2010 was a big revitalization of the network.

The threat from the Icelandic volcanic ash couldn’t stop people to find new ways to get to the conference from all over the world. Only few didn’t make it.

Keynote Seymour Hersh talking to the audience

On Friday the 22th of April I moderated the meeting of the Global Investigative Journalism Network. The conclusions were very promising with a lot of organizations participating the meeting. The network will continue the loose structure, having the meetings at the conferences as the highest authority. The network will accept new members when they are non-profit, non-partisan, has journalism as the main area and supporting the idea of investigative and computer-assisted reporting as a goal.

When my good friend Brant Houston and I back in 2003 in Copenhagen took the initiative to form the network we had the same statement for becoming member of the network. At that time I was in the steering committee in the network representing Danish Internationally Center for Analytical Reporting, Dicar. Dicar closed in 2006, and today I represent the Danish School of Media and Journalism in the network.

During the meeting we decided to have a volunteer group to do the practical work. I joined this group. We also heard the plans for the next conference in Kiev in Ukraine in September 2011, and there were some first ideas for places for the conference in 2013.

Look here for the website of the network and for the Geneva-conference. I took pictures during the conference. Look here for all of them (free to use).

CAR-training

On the special track on computer-assisted reporting the two main trainers, David Donald from US and Helena Bengtsson from Sweden, couldn’t come.

I then accepted to jump in Thursday morning to do the overall introduction on the status of CAR and restructure the many sessions missing trainers, relaying on other trainers. Sebastian Mondial from Germany, Luuk Sengers from Netherland, Mark Horvit and Brant Houston from US helped me by jumping in taking extra sessions on international data, beyond Google, and the use of spreadsheets and database managers in journalism.

Everybody were very enthusiastic, the audience very patient and I think all in the end went from these sessions with a lot of inspiration and useful information.
It also turned out to be really overcrowded in the training room for several of the sessions – totally changing the situation with minor interests some years ago.

After the presentations I found time to write handouts on the main content of my presentations:
Overview of CAR
International data
Beyond Google
Hands-on training in Excel and Access

An interview on the status of CAR

Moderation of two panels

Part of my job on the conference was also to moderate the presentations of Morten Frich, Berlingske Tidende in Denmark on the crowdsourcing project “the crime” and Blaz Zgaga, freelance from Slovenia, on the case of Patria – the paying of bribe from a Finnish state owned company selling arms to Slovenia. The bribes went to the former prime minister of Slovenia. Both were very impressing projects.

TV-documentary on Patria-case

And then I enjoyed the chat with some many old and new friends. It is the spirit of good journalism meeting at these conferences.

Panel at the global investigative journalism conference in Geneva 22th of April 2010
By nils.mulvad@kaasogmulvad.dk

Computer-Assisted Reporting has spread from US to the rest of the World. But still it’s only rarely spread.

Computer-Assisted Reporting spreading

Main tendencies today
1.    Journalists and medias are behind due to the fact that we hate numbers, data and information technology.
2.    We have the old school CAR being around for 20 years and connected to the PC: Excel-spreadsheet, Access-database managers, mapping, statistics as the main resources.
3.    We have a tradition of fighting for data and documents, which now are spreading to a lot of countries.
4.    Data is coming in floods now – and also new tools for working with data
5.    The main target now move to presentation on the web, meaning we see the environment of CAR-journalists melting together with the online media production. So it’s now gathering, analyzing and presentation by setting up automatic  tools. It is also melting this together with crowd sourcing and social media.

Training cover
The training at the global investigative journalism conference then cover:
1.    Find web-content
2.    Basic CAR-tools
3.    Present and share on the web – and other advanced stuff.
Then today these tools are also part of most of the other presentations at the conference. There’s often an element of the use of these tools in every good project today.

Some links to examples of the status
Using interactive graphics on the web:

European song contest from Norwegian paper VG

Word train from New York Times

Inaugural Words from US-Presidents (New York Times)

Using interactive graphics on the web and combine it with automatic generating of text:

High death rates for unskilled workers

Gathering data by robots, structure it and present it as part of an online media:

Texas Tribune