Triga, Vasiliki, Uwe Serdült, and Theodore Chadjipadelis (2012) Voting Advice Applications and State of the Art: Theory, Practice, and Comparative Insights. International Journal of Electronic Governance, 5 (3/4).

Special Journal Issue on VAAs

In this double special journal issue of the International Journal of Electronic Governance researchers from Switzerland working at the Centre for Democracy Studies Aarau (ZDA) at the University of Zurich and the idheap in Lausanne figure prominently presenting their research on so called voting aid (or advice) applications (VAAs). The best known of such online election tools in Switzerland is certainly smartvote. More experimental, less well known and mostly operating abroad is ZDA’s Preference Matcher which has genereted considerable data sets that are analyzed in this special journal issue for the first time in more detail.

What is a VAA?

“The idea behind VAAs is to allow citizens to better define their own subjective, political preferences and to match these with the stated (or coded) preferences of candidates or political parties that are stored on the online application. Around 30 policy items are typically included in a VAA although in some cases such as the Swiss smarvote it can be up to around 60. The core output of most VAAs is usually a concordance/similarity score between the user and the parties/candidates across the policy statements.” (Mendez 2012: 265)

50:50 ?

Fernando Mendez  addresses a core methodological aspect of VAA design: how voters’ policy preferences are aggregated to produce measures of concordance with parties or candidates. To this end, the paper analyses the performance of four VAA models that are based on competing algorithms. The data for this test was drawn from four experiments conducted during electoral races in the years 2010 (Brazil) and 2011 (Peru, Scotland, Cyprus). As a general, not so surprising observation we can state that VAAs are better suited for issue voters whose decision for choosing between candidates or parties is based on the policy positions of the latter. However, the more similar parties’ or candidates’ positions are on the range of policy issues posed in a VAA, the more difficult it will be for an algorithm to distinguish between them and offer a vote match. Here the choice of algorithm will matter, the point being that this is not a neutral choice but rather one decided by VAA designers. Mendez concludes that there is no single best way to aggregate policy preferences to produce a voting recommendation and that “to the extent that a voting recommendation can be produced it must be treated with some degree of scepticism” (Mendez 2012: 276).

My comment as one of the co-editors of this special journal issue: “To put it more bluntly, the results of Fernando Mendez show that on average there is only a fifty percent chance for the first-ranked party or candidate to match the users’ vote intention. Whether that is a manifest flaw of VAAs or of clueless users not being aware of their ‘real’ political preferences is open for discussion.”

Big Data

Working with the same set of countries, namely with Brazil, Peru, Scotland and Cyprus, Jonathan Wheatley demonstrates that VAA generated data can also be used for research into how policy preferences of voters can be conceptualised in terms of a multi-dimensional policy space. Probably the best known political discrimination among voters is the Left-Right dimension typically referring to economic ideologies. A more recent dimension of party competition is often labelled as GAL-TAN, “with GAL representing green/alternative/libertarian values and TAN representing traditionalism/authority /nationalism” (Wheatley 2012: 319). Both of these dimensions could be detected in all four countries. A third dimension related to the question of independence and sovereignty was only detected for the Scottish case.

In my view, a couple of points merit to be highlighted here: a) Compared to ordinary opinion polls with usually  only a bit more than 1’000 respondents, VAAs are capable of producing large n data sets at relatively low cost. The VAAs under study in this article have generated datasets with 5’000 (Cyprus) to 40’000 (Peru) respondents – after performing a thorough and transparent procedure of data cleaning; b) in political contexts as diverse as in the four selected election cases it was always possible to detect the two most well known policy dimensions mentioned above. This speaks for the robustness of the applied methods and the quality of the data; c) even though representativeness seems to be an obvious concern I would argue that on one hand the online world tends to capture an ever bigger share of politically active citizens and on the other hand representativeness as such is not a necessary condition if theory development is the goal.

Middle Category Conundrums

Former c2d researcher Vicky Triga (now lecturer at Cyprus University of Technology) and her  colleagues from the Aristotle University of Thessaloniki raise another often overlooked and seemingly unimportant issue regarding the design of a VAA: the question whether users are allowed to respond on a scale with or without a middle category. A response scale without a middle category forces to the users to one side of the argument. In case a middle category such as ”neither agree nor disagree’ is allowed the obvious question is what meaning one should attributed to it. To say it in the words of the authors:  “In our effort to explore the reasons why the respondents choose the middle point or the ‘no opinion’ alternative we identified three main categories of meaning attributed to these answers. The first category refers to those cases in which the respondents account for their choice in terms of (some sort of) lack of knowledge or indifference. The second category includes those answers that justify mid-point choice through ambivalence or indecisiveness, while the third comprises answers that argue against the main assumptions and/or formulation of the posed questions” (Baka/Figgou/Triga 2012 : 250).

As the authors point out, VAAs cannot escape the inherent problems of the use of a 5-point Likert scale with a middle category. However, what the study brings out is the fact that mid-point interpretation seems to vary according to the type of question: for technical questions they are chosen to represent a lack of knowledge, for trade-off questions involving a dilemma, picking the middle stands for disagreeing on the way the question is formulated.

My 2 cents: 1) More careful questionnaire design and testing before doing a VAA is very much advised. The ‘world of users’ out there is a messy onea and even item statements that seem to make perfectly sense might turn out to be rather useless. 2) More experiments are needed (users randomly exposed to mid-point or not). Whether this is feasible from an ethical point of view is another question. Getting consent from the users is only possible in hindsight.

Average Annual Turnout for Swiss Referendum Votes, 1879-2013

Turnout rates for Swiss referendum votes went down after the Second World War to low annual averages of fifty and then even forty percent during the 1970ies and 1980ies. During that time many cantons changed institutional incentives. With the exception of the canton of Schaffhausen they all dropped compulsory voting. Over time three further overlapping trends can help to explain turnout variations: 1) short-term referendum votes on topics such as the relation of Switzerland to Europe or the EU as well as ballots related to immigration and foreigners can boost participation; 2) mid-term a lack of competition between political parties (during the politically very stable phase of the ‘magic formula’ for the distribution of parties in government) can have a demobilizing effect on voters; 3) long-term women are tending to participate at equal rates as men or even more often. Older women who did not grow up with full political rights and thus tend to participate much less than men are slowly disappearing from the electorate.

National level referendum frequencies worldwide by continent, with Switzerland singled out, 1792-2010

On the national level direct democracy votes are on the rise worldwide. At c2d we document referendum votes since 1793 (see: http://www.c2d.ch). However, during the last two decades, for the first time in history, Latin America as well as Europe have outnumbered Switzerland regarding the frequency of national referendums. For Europe we attribute this development to the fact that most East European countries have incorporated mechanisms of direct democracy into their new Constitutions after 1991. In Latin America we observe a ‘third wave’ of democratization leading to an increase of referendum votes.

Serdült, Uwe and Welp, Yanina (2012) Direct Democracy Upside Down, Taiwan Journal of Democracy, 8 (1), 69-92.

In this TJD-article we document all national referendum votes preceded by a collection of signatures from citizens (bottom up referendums). For each case we note who initiated the signature collection: the government or president, political parties in the opposition, civil society organizations.

Direct democracy can thus fulfill three functions in a political system: a) concentration or maintenance of political power, b) party competition, c) citizen empowerment.

In a nutshell, we present four empirical patterns:

1) Worldwide, the number of bottom up referendums is increasing, however, they were applied in only twenty countries (at least once).

2) Historically speaking, this particular mechanism of direct democracy has been used mainly by political opposition parties, however, over time the institution starts to shift into the hands of civil society.

3) Bottom up referendums are oftentimes considered to be the ‘good’ ones that are in a way ‘healthy’ for a political system. However, roughly ten percent of them fulfill the function of a power concentration instrument and are organized rather from top-down, thus perverting the instrument (hence the title of the paper). Especially in Switzerland with a governmental coalition consisting of four political parties it is necessary to organize a citizens’ initiative or a referendum from time to time  in order to ascertain power and to be kept in the government.

4) On a worldwide scale referendum votes initiated by civil society organizations seem to have the highest success rates. Bottom up referendums initiated by civil society therefore seem to cover needs neither political parties in government nor in opposition seem to satisfy.