Triga, Vasiliki, Uwe Serdült, and Theodore Chadjipadelis (2012) Voting Advice Applications and State of the Art: Theory, Practice, and Comparative Insights. International Journal of Electronic Governance, 5 (3/4).

22 01 2013

Special Journal Issue on VAAs

In this double special journal issue of the International Journal of Electronic Governance researchers from Switzerland working at the Centre for Democracy Studies Aarau (ZDA) at the University of Zurich and the idheap in Lausanne figure prominently presenting their research on so called voting aid (or advice) applications (VAAs). The best known of such online election tools in Switzerland is certainly smartvote. More experimental, less well known and mostly operating abroad is ZDA’s Preference Matcher which has genereted considerable data sets that are analyzed in this special journal issue for the first time in more detail.

What is a VAA?

“The idea behind VAAs is to allow citizens to better define their own subjective, political preferences and to match these with the stated (or coded) preferences of candidates or political parties that are stored on the online application. Around 30 policy items are typically included in a VAA although in some cases such as the Swiss smarvote it can be up to around 60. The core output of most VAAs is usually a concordance/similarity score between the user and the parties/candidates across the policy statements.” (Mendez 2012: 265)

50:50 ?

Fernando Mendez  addresses a core methodological aspect of VAA design: how voters’ policy preferences are aggregated to produce measures of concordance with parties or candidates. To this end, the paper analyses the performance of four VAA models that are based on competing algorithms. The data for this test was drawn from four experiments conducted during electoral races in the years 2010 (Brazil) and 2011 (Peru, Scotland, Cyprus). As a general, not so surprising observation we can state that VAAs are better suited for issue voters whose decision for choosing between candidates or parties is based on the policy positions of the latter. However, the more similar parties’ or candidates’ positions are on the range of policy issues posed in a VAA, the more difficult it will be for an algorithm to distinguish between them and offer a vote match. Here the choice of algorithm will matter, the point being that this is not a neutral choice but rather one decided by VAA designers. Mendez concludes that there is no single best way to aggregate policy preferences to produce a voting recommendation and that “to the extent that a voting recommendation can be produced it must be treated with some degree of scepticism” (Mendez 2012: 276).

My comment as one of the co-editors of this special journal issue: “To put it more bluntly, the results of Fernando Mendez show that on average there is only a fifty percent chance for the first-ranked party or candidate to match the users’ vote intention. Whether that is a manifest flaw of VAAs or of clueless users not being aware of their ‘real’ political preferences is open for discussion.”

Big Data

Working with the same set of countries, namely with Brazil, Peru, Scotland and Cyprus, Jonathan Wheatley demonstrates that VAA generated data can also be used for research into how policy preferences of voters can be conceptualised in terms of a multi-dimensional policy space. Probably the best known political discrimination among voters is the Left-Right dimension typically referring to economic ideologies. A more recent dimension of party competition is often labelled as GAL-TAN, “with GAL representing green/alternative/libertarian values and TAN representing traditionalism/authority /nationalism” (Wheatley 2012: 319). Both of these dimensions could be detected in all four countries. A third dimension related to the question of independence and sovereignty was only detected for the Scottish case.

In my view, a couple of points merit to be highlighted here: a) Compared to ordinary opinion polls with usually  only a bit more than 1’000 respondents, VAAs are capable of producing large n data sets at relatively low cost. The VAAs under study in this article have generated datasets with 5’000 (Cyprus) to 40’000 (Peru) respondents – after performing a thorough and transparent procedure of data cleaning; b) in political contexts as diverse as in the four selected election cases it was always possible to detect the two most well known policy dimensions mentioned above. This speaks for the robustness of the applied methods and the quality of the data; c) even though representativeness seems to be an obvious concern I would argue that on one hand the online world tends to capture an ever bigger share of politically active citizens and on the other hand representativeness as such is not a necessary condition if theory development is the goal.

Middle Category Conundrums

Former c2d researcher Vicky Triga (now lecturer at Cyprus University of Technology) and her  colleagues from the Aristotle University of Thessaloniki raise another often overlooked and seemingly unimportant issue regarding the design of a VAA: the question whether users are allowed to respond on a scale with or without a middle category. A response scale without a middle category forces to the users to one side of the argument. In case a middle category such as ”neither agree nor disagree’ is allowed the obvious question is what meaning one should attributed to it. To say it in the words of the authors:  “In our effort to explore the reasons why the respondents choose the middle point or the ‘no opinion’ alternative we identified three main categories of meaning attributed to these answers. The first category refers to those cases in which the respondents account for their choice in terms of (some sort of) lack of knowledge or indifference. The second category includes those answers that justify mid-point choice through ambivalence or indecisiveness, while the third comprises answers that argue against the main assumptions and/or formulation of the posed questions” (Baka/Figgou/Triga 2012 : 250).

As the authors point out, VAAs cannot escape the inherent problems of the use of a 5-point Likert scale with a middle category. However, what the study brings out is the fact that mid-point interpretation seems to vary according to the type of question: for technical questions they are chosen to represent a lack of knowledge, for trade-off questions involving a dilemma, picking the middle stands for disagreeing on the way the question is formulated.

My 2 cents: 1) More careful questionnaire design and testing before doing a VAA is very much advised. The ‘world of users’ out there is a messy onea and even item statements that seem to make perfectly sense might turn out to be rather useless. 2) More experiments are needed (users randomly exposed to mid-point or not). Whether this is feasible from an ethical point of view is another question. Getting consent from the users is only possible in hindsight.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: