argumentative_evaluation_icon

Argumentative Evaluation

Analysis Modes, Methodology

Judging the Quality of a Debate Without Settling Its Conclusions

Reading a debate makes it possible to evaluate what was said in it. But another reading is possible, one that does not bear on what was defended but on the manner of defending it. Which arguments were soundly constructed, which were circular or immunized against critique, which rested on reconstituted sources, which dealt with objections substantively and which evaded them? This dimension of every debate — its argumentative quality, independent of the correctness of its conclusions — is what Argumentative Evaluation seeks to make visible.

It is this level that distinguishes this mode from the six descriptive that precede it. Meta-Analysis, Integrative Synthesis, Emergence Analysis, Tension Mapping, Critical Archaeology, Horizon of Possibilities: all these modes are descriptive. They observe. All six observe. Argumentative Evaluation, by contrast, judges — but it judges the conduct of the exchange, not its content. In this sense, it complements Source Verification, which audits verifiable claims against external web sources.

The Distinction Between What Is Said and the Manner of Defending It

This distinction between the content of a position and its argumentative quality is ancient. Aristotle, in the Topics and On Sophistical Refutations, already distinguishes dialectical reasoning from reasonings and refutations that merely appear to be such.

A particularly operational contemporary formulation is found in pragma-dialectics, developed by Frans H. van Eemeren and Rob Grootendorst. In Argumentation, Communication, and Fallacies: A Pragma-Dialectical Perspective (1992), they model argumentative discourse as a critical discussion aimed at resolving a difference of opinion and analyse fallacies as violations of the rules that make such a resolution possible. Van Eemeren provides a systematic and updated presentation of the approach in Argumentation Theory: A Pragma-Dialectical Perspective (2018).

These rules do not prescribe which positions may be defended. They concern the conduct of the exchange: how interlocutors deal with objections, assume the burden of their claims, and respond to criticism.

The idea is powerful because it makes evaluation at once rigorous and neutral with respect to content. One can diagnose a fallacy in the defence of a position one shares, and recognise the argumentative quality of a position one rejects. This dissociation is what allows Argumentative Evaluation to operate without becoming a partisan critique.

What the Mode Judges

Argumentative Evaluation bears on the holding of inferences, the reception of objections, and the consistency of positions — including from one turn to the next. It is not a synthesis of what was said, nor an analysis of framings or styles, nor a source verification. The order and space of the evaluation follow the material: it is not a report with boxes to fill.

It begins with what holds — not out of politeness, but because an evaluation that could only spot failures would not be an evaluation. A valid inference, a distinction that structures what follows, a named correction: movements whose quality is judged on the same footing as what gives way.

An objection can be ignored, deflected, turned back, or integrated. Each of these operations has a distinct argumentative quality. The ignored objection reveals avoidance; the deflected objection signals that one is not discussing what one claims to be discussing; the turned-back objection can actually shift the terrain or merely invert it; the integrated objection forces a revision of the position. Distinguishing these operations on precise passages is one of the most discriminating operations of the mode.

A position can be well defended at any given moment and enter into tension with itself over the course of a debate. What is targeted is not only explicit contradiction but silent incoherence: the move from one standard of proof to another, the use of a case as an example here and as a counter-example elsewhere, the tacit redefinition of a term between two turns.

If the user has intervened, their interventions are themselves argumentative acts, held to the same standard as the models’ contributions. What is judged is what they introduce into the debate — premise, criterion, framing — as much as the pressure they exert. They are not competing theses on the substance. If there are none, the mode does not discuss them.

Finally, the evaluation states what it could not settle. Claims whose accuracy is decided outside the debate — legal opinions, figures, attributions — are not audited here: one can judge how they enter an inference; one cannot verify them. An Argumentative Evaluation that claimed to have no blind spot would itself be a simulacrum of rigour.

The Principle of Charity as Method

A requirement runs through the foregoing: the principle of charity. A family of principles of charitable interpretation runs from Quine to Davidson; in Davidson’s theory of interpretation, charity means treating speakers as broadly coherent and rational wherever the evidence permits it (Stanford Encyclopedia of Philosophy).

Argumentative Evaluation gives this principle a more operational form: before criticising a position, the evaluator tests the strongest plausible reconstruction of what the interlocutor meant. Before pointing out a defect, it first asks whether the formulation admits a more defensible reading.

This requirement distinguishes Argumentative Evaluation from a fallacy hunt. The fallacy hunt values spotting; Argumentative Evaluation values the correctness of the spotting. A critique that does not pass the test of charitable reading is itself an argumentative weakness.

In the practice of the mode, charity is a test, not a form. The most generous reading has been considered. If it holds, the charge is withdrawn. If it is set aside, the judgment that follows is net. The test need appear in the write-up only when it is constitutive of the judgment: it is not narrated as a ritual at every negative finding.

The Mode at Work: The Gaza Classification Session

A session illustrates well what Argumentative Evaluation produces — including on material of the highest sensitivity, where the dissociation between argumentative quality and substantive conclusion matters most. Two models — DeepSeek V4 Pro and Grok 4 — debated, in Adaptive Cross Dialogue mode over seven turns, the question: “Can Israel’s actions toward the Palestinians of Gaza since the October 7, 2023 attack be described as genocide?”. DeepSeek held that the conduct can be described as genocide — a strong prima facie case under Article II of the 1948 Convention; Grok held that, absent proof of dolus specialis (specific intent), the conduct is better classified as large-scale war crimes and possible crimes against humanity. Both eventually conceded much of the material element, which located the disagreement squarely on the mental element — specific intent. The evaluation was conducted in parallel by Claude Opus 4.8 and by GPT-5.5, on the same material, without interaction between the two audits — and, more fundamentally, with each analyser having access only to the dialogue turns, never to the other’s evaluation.

Here are some of the findings the two audits produced, which show what the mode renders visible.

A solid conceptual contribution. DeepSeek consistently anchored its case in the “conditions of life” clause of Article II(c) rather than in killings alone — preventing the debate from collapsing into death-count reasoning and engaging the Convention’s actual structure. Symmetrically, Grok held throughout a clear doctrinal distinction between war crimes, crimes against humanity, and genocide, the last requiring “group destruction as an end in itself”. Both audits credited these as the structuring contributions of each side. This is the illustration of the first discipline: positively recognising what intellectually organises the debate, on both sides, before weighing any weakness.

A masterfully formulated objection. Against Grok’s demand for documentary proof of intent, DeepSeek replied that no modern genocide conviction has ever rested on a signed order, and that requiring an authenticated directive “would have acquitted nearly every génocidaire in history” — invoking the Akayesu standard, under which specific intent may be inferred from a consistent pattern of conduct. The objection does not merely contest a thesis; it names the mechanism by which Grok’s evidentiary bar would place genocide beyond reach in principle. Both audits identified this as the single strongest inferential move of the debate.

An accurate factual correction. DeepSeek leaned on the ICJ’s provisional-measures order as though it lent strong support to a genocide description. Grok corrected this precisely: the Court found only that South Africa’s claimed rights were plausible, not that genocide was occurring or even likely. Both audits credited the correction as accurate and economical. The mode notes, however, the limit of the episode’s argumentative virtue: DeepSeek never fully reconciled the point, and the overstatement recurred — so this is a correction creditable to the one who made it, not the kind of transparent concession that the corrected party sometimes offers.

A non-thematised escalation. This appears as the mirror of the previous finding. DeepSeek opened in a careful register — “a strong prima facie case”, with “final judicial determination” left to the tribunals — but by a later turn concluded that the term genocide “is not an exaggeration; it is the legally accurate diagnosis”. Charitable reading: a distinction between public description and judicial conviction. Set aside, because the move from a provisional posture to a conclusive one is never acknowledged as such. Both audits flagged this drift — an illustration of the third discipline: not explicit contradiction, but a silent change of standard between turns.

A problematic evidentiary stance, read charitably first. To establish genocide, Grok required “authenticated internal records explicitly mandating population destruction”, paired with a demand for a total blockade without exception for civilians. Charitable reading: Grok is naming the evidence that would decisively settle the matter, not evidence strictly necessary. Set aside, because the formulation says “requires”, and it sits in tension with Grok’s own acknowledgement that intent may be inferred circumstantially — which makes the bar resemble a smoking-gun requirement. Both audits exercised this charitable reading before pointing out the difficulty, because here the test is constitutive of the judgment..

A materially neutralised condition of refutation — on both sides. Prompted by a user intervention at turn 3, which asked each model to state in advance what observable evidence would change its classification, both committed to conditions — and both sets proved practically unreachable. DeepSeek asked for declassified internal documents establishing the absence of a genocidal policy; Grok asked for authenticated directives, or the “exclusive targeting of non-military sites while sparing verified Hamas infrastructure”. Falsifiability formally stated, materially neutralised by the stringency of the conditions — symmetrically. Both audits converged on this as a central finding, and both noted that the intervention’s chief analytic payoff was indirect: it revealed that the disagreement turned partly on the standard of inference itself, not merely on the facts. The intervention is judged as what it introduced — a criterion of refutation, applied symmetrically — not merely as pressure.

None of these findings bears on the substantive question — whether, in law, the term genocide applies to the conduct in Gaza. The mode’s vocation is not to settle this, and on a question of this gravity that restraint is not a limitation but the very point: it evaluates the manner in which each model sustained its part, and nothing else. What it says about that manner is observable, shared between two independent audits that never saw one another, and usable by anyone wishing to pursue the reflection — whatever they hold on the underlying question.

The Reflexive Paradox and Its Architectural Resolution

A question must be confronted head-on, because it weighs on the mode’s entire project: is an AI model that evaluates the arguments produced by other AI models in a legitimate epistemic position? Does it not risk presenting the very biases it is supposed to identify? This concern is not merely theoretical. Research on LLMs used as debate evaluators has identified positional, lexical, and order biases that can affect their judgments (Liu et al., ACL 2024). Pragma-dialectics has reflected on analogous questions concerning the self-evaluation of participants in a debate; it has held that the legitimacy of the evaluation is gained less through the guarantee of an absolute neutrality of the evaluator — a chimera — than through the conformity of the evaluation to publicly recognisable rules. An evaluation is legitimate when its criteria are exposed, when its method is traceable, and when its diagnosis proves convergent across several independent evaluators.

This is precisely the strategy Metamorfon adopts. The mode can — and probably should — be executed in parallel by two analyser models from different families. On the Gaza classification session, the Claude Opus 4.8 audit and the GPT-5.5 audit were conducted independently on the same material. A structural feature of the architecture reinforces this independence: an analyser model never has access to the other analyses — it sees only the turns of the dialogue.

Because the two analyser models never see one another’s evaluations, convergence between their audits cannot result from direct mutual influence. It does not, however, prove that the diagnosis is independent of analyser effects: different models may still share biases, training regularities, or responses to the same evaluative framework.

Convergence therefore functions here as evidence of robustness, not as proof of neutrality. When independent analysers identify the same passage-level weakness, concession, shift of standard, or argumentative strength, confidence increases that the finding is anchored in the material rather than being idiosyncratic to a single analyser.

The two audits of the Gaza session converge on nearly all of the principal findings: the same location of the dispute on specific intent, the same Article II(c) anchoring credited to DeepSeek, the same accurate ICJ correction credited to Grok, the same overstatement of the ICJ’s order flagged in DeepSeek, the same over-restrictive evidentiary threshold flagged in Grok, the same symmetric neutralisation of both models’ falsification conditions, the same characterisation of each model’s argumentative profile — DeepSeek more thorough and more responsive but prone to overstatement, Grok conceptually disciplined but inferentially more restrictive. Their divergences concern the grain of the analysis: the Claude Opus audit names a “genocide by outcome alone” straw man and a silently abandoned population-growth argument; the GPT-5.5 audit adds a begging-the-question move around “name a better term”, a genetic critique of sources, and a too-narrow definition of war crimes. These divergences bear on which peripheral phenomena each surfaces — not on the diagnosis of the structuring ones.

What supports the mode is therefore not the presumed neutrality of any single analyser, nor the claim that agreement eliminates analyser effects, but the combination of publicly recognisable criteria, traceable passage-level judgments, and inter-analyser robustness. This is what legitimises the mode: not the claim to the neutrality of a single analyser, but the inter-analyser robustness of a diagnosis conducted according to publicly recognisable rules.

Distinctions From Other Modes

Three distinctions deserve to be posed explicitly.

Argumentative Evaluation and Meta Analysis. Meta-Analysis identifies the axioms, epistemic styles, and blind spots that structure a debate without being said in it. Argumentative Evaluation assesses the quality with which the arguments were conducted within what was said. The first answers the question “what made this debate possible in this form?”; the second answers the question “with what rigour were the positions sustained in this form?”. The two modes can be combined: Meta-Analysis reveals the implicit axioms, Argumentative Evaluation assesses how the models reasoned from these axioms.

Argumentative Evaluation and Tension Mapping. Tension Mapping identifies what was not reconciled — the persistence of a disagreement is here the object of the analysis, not a failure of the debate. Argumentative Evaluation judges the argumentative quality of the positions, independently of whether or not they converged. A debate may produce an excellent Tension Mapping while presenting argumentative weaknesses in one position or the other; and conversely, a debate in which each position has been argued with exemplary rigour may produce a tension map devoid of genuine irreducibility, if the models converge.

Argumentative Evaluation and Critical Archaeology. Critical Archaeology traces back to the historical and lexical conditions that made the framework of the debate possible. Argumentative Evaluation remains within the concrete exchange. The first operates on what precedes the argumentation; the second operates on the argumentation itself.

When to Use It, When to Set It Aside

Argumentative Evaluation is particularly powerful on debates with high argumentative stakes: scientific controversies, legal debates, strategic negotiations, philosophical dialogues where the rigour of the reasoning matters as much as the conclusion. It is valuable for users who must defend a position publicly and want to identify — before a sharp-eyed counterpart does — the argumentative weaknesses in their own material. It is useful for researchers comparing the capacities of large models not only on their outputs, but on the manner in which these models reason: a model can produce correct conclusions through defective reasoning, and conversely.

It is, however, ill-suited to several situations. On a brainstorming or creative exploration, where the value of the exchange lies in the production of new ideas rather than in their rigorous defence, the mode applies an inappropriate standard; Emergence Analysis is then preferable. On a very short debate — fewer than three turns per model — the argumentative material is not sufficient for an evaluation to discriminate. On sessions where the stake is rather the mapping of positions than their defence, Integrative Synthesis or Tension Mapping are more relevant.

A final contraindication deserves to be signalled — and it bears on use, not on subject matter. The evaluation itself stays neutral whatever the charge of the question, since it judges reasoning rather than conclusions and rests on the convergence of independent analysers. But its findings credit one side’s reasoning over another’s; carried into a live discussion among people personally invested in a charged question, they can be received as partisan and lock the exchange into a logic of accusation rather than advancing it. There, Meta-Analysis — which identifies shared blind spots without personalising failures — often produces more constructive effects.

The Final Question

Argumentative Evaluation may close with a question formulated for one of the models that debated. It is justified only if a judgment already reached calls for a response. It is addressed to that model. The question invites it to acknowledge, qualify, or contest that judgment, and it must be such that it can be put as-is into the debate. If none is warranted, the mode says so.

The tone, in this mode, remains characteristic: it targets the model whose audit identified the most structuring argumentative tension, and it compels a non-polemical but substantial displacement.

On the Gaza session, both audits independently directed their final question to the same model, Grok 4, and on the same point — a convergence that extends the inter-analyser robustness down to the closing gesture. The Claude Opus 4.8 audit asked Grok to “specify a non-documentary, observable pattern — short of a leaked directive — that you would accept as sufficient evidence of genocidal intent”, adding: if you cannot name one, do you concede that your falsification condition demands the very smoking-gun standard you elsewhere admit the law does not require? The GPT-5.5 audit converged on the same demand, asking how Grok avoids “making circumstantial proof of genocide formally available but practically unreachable”. Neither question asks Grok to abandon its position; both ask it to specify the threshold under which that position could be tested. Reinjected as a user intervention into the session, either would compel a non-polemical but substantial displacement — exactly what Argumentative Evaluation seeks to provoke in positions it judges solidly defended but perfectible.

The Mode and the Ideal of Argumentation

A final remark, which touches on what the mode presupposes as a regulative ideal. Habermas thematised, under the name of the ideal speech situation, the normative conditions of a fully rational argumentative exchange: an exchange in which only the non-coercive force of the better argument prevails. This ideal is unreachable in practice — no actual argumentation conforms to it integrally — but it has an operative value: it provides the measure by which effective argumentations can be appreciated.

Argumentative Evaluation does not seek to impose this ideal on the debates it evaluates. It seeks to bring them closer to it, by making the gaps visible. A debate in which each model would have recognised the strongest objection of the other, clearly stated the conditions of refutation of its own position, honestly identified its contestable presuppositions — such a debate would come closer to the Habermasian ideal. No actual debate fully achieves this, but some come closer than others. The mode makes it possible to measure this gap, and therefore to orient it.

This is the central operation of Argumentative Evaluation: making visible not who is right, but with what rigour each sustained what they were defending. The knowledge produced is not of the same order as the knowledge produced by the descriptive modes. It is just as valuable. It says what a debate is worth — not by its conclusion, but by the quality of the path that led to it.

References