Nachspiel at Polemarchus' » methods

Does lack of rhetorical skills make you less deliberative?

sverrebm — Fri, 20 Nov 2009 11:20:25 +0000

I’m working on a master thesis about political deliberation, and I’m interested in finding a good method for measuring and evaluating the level of deliberation in a discussion. Two current methods I’ve been looking into, the Discourse Quality Index and pragma-dialectics, both seem to share the same validity problem: They register low rhetorical or logical skills for a lack of deliberative attitude.

Put simply, deliberation is a term that is used about discussion when people engage in a rational dialogue about something in which they are dedicated to the “forceless force of the better argument”. According to Jürgen Habermas, its most famous theoretician, it should be characterized by sincerity, inclusiveness, equality, reasoned critique, reflexivity, respect and be free from the influence of money and coercive power¹ . In the more realistic conceptions of the term, we are willing to consider a discussion as more or less deliberative, realizing that a few real world discussions are likely to be perfect.

As I mentioned, I’ve been looking for methods to analyse discussion in order to determine how deliberative it really is. I’ve been trying to find a method that satisfies the criteria of:

Significance – Must be a method we can expect a large portion of empirically oriented political science to accept.
Usefulness – Must be a method that is suitable for comparative study of cases, preferably on a large scale and with a multitude of institutional arrangements.
Completeness – Must be a method that is theoretically consistent with established theory of deliberation, for example Habermas’ discourse ethics.

Lately I’ve been looking at two such methods: Pragma-dialectics² and the Discourse Quality Index³ . They both have their strength and weaknesses, but they share one interesting problem with validity.

In order to determine whether or not a debate is really deliberative, they both focus on the correctness of arguments presented. According to the DQI method, they evaluate the level of justification, classifying each speech act as either containing no justification (0), inferior justification (1), qualified justification (2) or sophisticated justification (3). The pragma-dialectical method does something similar when it examines entire debates for logical consistency, identifying so-called fallacies that constitute bad arguments which should not be part of reasoned discussion.

My claim is that both these methods measure something separate from the level of deliberation, they measure the level of sophistication of the debatants. They measure how good their rhetorical and logical skills are, not their dedication to reasoned debate and fair discussion. This is not a major problem when you compare debates where you can expect people to be at a similar skill level, as when comparing two parliaments. The problem arises when you want to compare the debates in parliament to debates in other areas of society. If you compare two groups with different levels of rhetorical skill, the less sophisticated group will register as less deliberative even if they really were more dedicated to the ideals of deliberation. The core principles of deliberation dictate that the best argument should prevail no matter who presents it, but if we use rhetorical sophistication as a measurement criterion, we are already violating it by derogating the arguments of the less educated.

This is still a work in progress, but it seems likely that this specific find will play a major part in the conclusions to my thesis.

My rewrite of the rules presented in his book discourse ethics
described most elegantly in a conference paper by Nicole Curato
Outlined in: Steiner, Jürg, André Bächtiger, Markus Spörndli and Marco R. Steenbergen (2004). Deliberative Politics in Action: Analyzing Parliamentary Discourse, Cambridge: Cambridge University Press.

Experiment on election prediction markets

sverrebm — Wed, 05 Aug 2009 14:36:05 +0000

I’ve recently become involved (as a participant) in an interesting experiment performed by PhD student Sveinung Arnesen at the University of Bergen in which we are asked to predict the election result through a market model, buying and selling fictive “shares” in the outcome based on our own evaluations. This is based on prior experiments like Iowa Electronic Markets experiments by the University of Iowa in connection with American Presidential Elections, and the work of Robin Hanson.

Participants have been recruited through the political party organizations (at least I was), and appear to only have the option of buying or selling “stock” in our own party and/or government coalition. I assume part of the reason why we are restricted to our own party is the need for keeping the results secret to avoid incentives for strategic attempts at driving up the predicted value.

Unlike the Iowa experiments, this experiment is not conducted with “real money”. Each participant is given a number of virtual credits, and as far as I know there is no winning prize. This does of course give weak incentives towards predicting results accurately. If the result wasn’t being kept secret until election day, we might even have a serious adverse incentive to manipulate the predicted result to give our party “wind in the sails” towards the election. Such secrecy seems to be maintained, though, also to the extent of not letting us know the predictions made on other parties.

It seems highly likely to me, though, that the weak incentives for “winning” will cause party member participants to systematically overvalue their own party’s election result, especially when unable to also “invest” in other parties. The only alternative to holding “stock” in your own party is not holding stock…. i.e. having inert credits.

Of course it would possibly be likely that such a bias is so systematically distributed that it can be predicted and controlled for trhough discounting by some factor. Still I can’t help but think that there would be a chance for better predictions if we were either allowed to “invest” in all parties or had actual economic incentives to “win”.

In any case it’s interesting to see new methods of prediction getting tested. The Iowa results look rather good.

No Hawthorne effect at Hawthorne?

sverrebm — Tue, 09 Jun 2009 21:13:55 +0000

Is nothing holy? Must I throw out my introduction textbook in methods now? The guys over at the Freakonomics blog have dug up and reexamined the original Hawthorne data and concluded that there actually was no Hawthorne effect in the original Hawthorne study! I’m disappointed right to the very core of my post-graduate student soul. What should i believe in now?

Picking apart a quantitative analysis

sverrebm — Fri, 03 Oct 2008 12:10:53 +0000

Kai Arzheimer posted on his blog a not yet published paper he wrote along with Elizabeth Carter of Keele University. The paper picks apart another paper on effects that contribute and don’t contribute to the electoral support of Le Pen’s Front Nationale in France, especially the volume of immigration.

My interest in the paper is not so much the substantial content as the nice reminder it was to me as an aspiring political scientist to still keep a critical eye towards papers that at first eyesight appear to have a thorough empirical basis. I don’t think I’d have noticed the flaws these two scientists did.

The paper is well worth a read for anyone interested in quantitative methods.

Read Kai Arzheimer’s post and both papers here.