|A meta-level argumentation framework
for representing and reasoning about
The University of Edinburgh
“I know what you’re thinking about,” said Tweedledum :
“but it isn’t so, nohow.”
“Contrariwise,” continued Tweedledee,
“if it was so, it might be; and if it were so, it would be : but as it isn’t, it ain’t.
Through the Looking Glass. Lewis Carroll. 1887.
The contribution of this thesis is to the field of Artificial Intelligence (AI), specifically to the sub-field called knowledge engineering. Knowledge engineering involves the computer representation and use of the knowledge and opinions of human experts.
In real world controversies, disagreements can be treated as opportunities for exploring the beliefs and reasoning of experts via a process called argumentation. The central claim of this thesis is that a formal computer-based framework for argumentation is a useful solution to the problem of representing and reasoning with multiple conflicting viewpoints.
The problem which this thesis addresses is how to represent arguments in domains in which there is controversy and disagreement between many relevant points of view. The reason that this is a problem is that most knowledge based systems are founded in logics, such as first order predicate logic, in which inconsistencies must be eliminated from a theory in order for meaningful inference to be possible from it.
I argue that it is possible to devise an argumentation framework by describing one (FORA : Framework for Opposition and Reasoning about Arguments). FORA contains a language for representing the views of multiple experts who disagree or have differing opinions. FORA also contains a suite of software tools which can facilitate debate, exploration of multiple viewpoints, and construction and revision of knowledge bases which are challenged by opposing opinions or evidence.
A fundamental part of this thesis is the claim that arguments are meta-level structures which describe the relationships between statements contained in knowledge bases. It is important to make a clear distinction between representations in knowledge bases (the object-level) and representations of the arguments implicit in knowledge bases (the meta-level). FORA has been developed to make this distinction clear and its main benefit is that the argument representations are independent of the object-level representation language. This is useful because it facilitates integration of arguments from multiple sources using different representation languages, and because it enables knowledge engineering decisions to be made about how to structure arguments and chains of reasoning, independently of object-level representation decisions.
I argue that abstract argument representations are useful because they can facilitate a variety of knowledge engineering tasks. These include knowledge acquisition; automatic abstraction from existing formal knowledge bases; and construction, re-representation, evaluation and criticism of object-level knowledge bases. Examples of software tools contained within FORA are used to illustrate these uses of argumentation structures. The utility of a meta-level framework for argumentation, and FORA in particular, is demonstrated in terms of an important real world controversy concerning the health risks of a group of toxic compounds called aflatoxins.
I would like to thank my two supervisors, Dave Robertson and Geraint Wiggins, who have helped me through the four and a half years of research leading up to this thesis. Dave I blame for getting me into research in the first place. Geraint has made me glad I’m in it. Both have almost always had time for me when I’ve needed help, for which I’m very grateful, and have helped me to find a focus amongst my rather eclectic interests. Thanks also to John Fox and John Lee, my examiners, for being the first and most crucial readers of this document.
Amongst the many other people who have contributed to the ideas in this thesis, I’d particularly like to thank Helen Pain, Robert Muetzelfeldt, Simos Retalis, Susan Bull, Mike Uschold, John Fraser, Tony Clayton, Robert Inder, Paul Gill, Nick Fiddes, Bill Ritchie, Rob Cannell, Matt Taylor, Donny Peterson, Paul Krause, Bashar Nuseibeh, Antony Finkelstein, Hyacinth Nwana, Cindy Mason and Terry Dartnall for interesting discussions and encouragement over the years. Thanks also to all my colleagues in the department of Artificial Intelligence, too numerous to mention by name, for camaderie and for creating an inter-disciplinary academic environment. Particular thanks to Jim Howe, our head of department, for his leadership.
Finally, thanks to Ritchie Hope for believing in me and keeping me well fed, my father for continually daring me to give up, and my mother for being living proof that it’s possible not to, Zazzy, Dave and Tina for being there and all my mates for putting up with my rants.
I dedicate this thesis to the memory of my grandparents, Arthur and Kate, who taught me that I had a right to argue about religion, and a duty to argue with computers.
Chapter 1. Introduction 1
1.1 Motivation, context and background 1
1.2 Thesis message and contribution 3
1.3 Overview 3
Chapter 2. Literature Survey 5
2.1 Disagreement 5
2.1.1 Object-level logical approaches to disagreement 5
22.214.171.124 Classical logic 5
126.96.36.199 Many-valued and intuitionistic logic 8
188.8.131.52 A logic of inconsistency 10
184.108.40.206 Paraconsistent logic 13
220.127.116.11 Restricted access logics 15
2.1.2 Meta-level approaches to disagreement 16
2.1.3 AI Approaches to disagreement 19
2.2 Argumentation 25
2.2.1 Argumentation in philosophy 25
18.104.22.168 Traditions of argumentation 25
22.214.171.124 Toulmin’s practical reasoning 27
126.96.36.199 Quine’s rationalism 31
2.2.2 Argumentation in AI 32
188.8.131.52 Cohen’s Model of Endorsements 33
184.108.40.206 Argumentation at the Imperial Cancer Research Fund 35
220.127.116.11 AI implementations of Toulmin’s practical reasoning 38
18.104.22.168 AI applied to legal argumentation 39
22.214.171.124 AI applied to environmental argumentation 40
2.2.3 Less formal approaches to argumentation 40
126.96.36.199 Linguistic perspectives on argumentation 40
188.8.131.52 Issue-Based Information Systems 42
2.3 Summary 42
Chapter 3. Overview of FORA 43
3.1 Motivation 43
3.2 FORA system overview 45
3.2.1 The FORA knowledge representation language 47
3.2.2 Using knowledge bases in FORA 47
3.2.3 Automating Mark-up 48
3.2.4 Supporting Mark-down 48
3.2.5 Summary of FORA 50
3.3 Introduction to the running example 50
3.3.1 The aflatoxin debate 50
3.3.2 Why choose this example? 52
3.3.3 The Imperial Cancer Research Fund’s approach to
formalising the aflatoxin debate 53
3.4 Summary 56
Chapter 4. Knowledge Representation in FORA 57
4.1 The FORA language 57
4.1.1 Meta-level objects 57
4.1.2 Term constructor definitions 58
4.1.3 Formula constructor definitions 58
4.1.4 Term destructor definitions 60
4.1.5 Types of argument 61
4.1.6 Meta-level rules 62
4.2 Example 63
4.3 Knowledge acquisition from multiple conflicting viewpoints 67
4.3.1 The ‘Fly on the Wall’ experiment 68
4.3.2 The ‘Blue Peter’ experiment 69
4.3.3 The ‘Transcript’ experiment 74
4.3.4 Conclusions 77
4.4 Mark-up using hypertext 78
4.5 Example : Representing the aflatoxin debate in FORA 85
4.6 Summary 87
Chapter 5. Using FORA 90
5.1 FORA users and their requirements 90
5.1.1 Users 90
5.1.2 Functionality requirements 91
5.2 Implementation of FORA 92
5.2.1 Implementation of the language 93
5.2.2 User interface 93
5.2.3 Exploration of FORA knowledge bases 93
5.3 Argumentation structures 95
5.3.1 Arguments for and from a proposition 97
5.3.2 Opposition about a proposition 100
5.3.3 Summaries of support and opposition 107
5.3.4 Some comments about terminology 108
5.3.5 Summary 108
5.4 Tools for using argumentation structures 109
5.4.1. Synthesis of argumentation structures 109
5.4.2 Extending the knowledge base by arguing 112
5.4.3 The Devil’s Advocate 114
5.5 Summary 115
Chapter 6. Automating Mark-up to FORA 118
6.1 Introduction 118
6.2 Abstracting arguments from knowledge bases 120
6.2.1 Why mark-up must involve inference 121
6.2.2 When should the inference happen? 125
6.2.3 The theorem prover 126
6.2.4 Dynamic mark-up 128
6.2.5 Problems with dynamic mark-up 129
6.3 Mark-up by Parsing Proofs 130
6.3.1. Parsing Proofs 131
6.3.2. Merging relations 138
6.4 Illustration of proof-parsing in terms of the aflatoxin debate 142
6.4.1 A first order predicate logic knowledge base about
6.4.2 Proving the FDA policy conjecture 143
6.4.3 Parsing the proof 145
6.4.4. Further abstraction in FORA, merging argument steps
and proof-outlining 156
6.5 Summary 159
Chapter 7. Supporting Mark-down from FORA 161
7.1 Introduction 161
7.2 Markdown 162
7.2.1 Markdown of a relation 164
7.2.2 Supporting a range of alternative representations 167
7.2.3 Variables and quantification at the object-level 170
7.2.4 Discussion 173
7.3 Object-level inference verification. 174
7.4 Criticising the object-level knowledge base 176
7.5 Summary 178
Chapter 8. Conclusions and Future Work 179
8.1 Conclusions 179
8.2 Further Work 181
Chapter 3. Overview of FORA
Figure 3.1 The FORA System 46
Figure 3.2 Mark-up and Mark-down as inverse translations 49
Figure 3.3 The aflatoxin debate as structured by Fox 54
Chapter 4. Knowledge Representation in FORA
Figure 4.1 The alaskan oil production lease controversy 63
Figure 4.2 Mark-up stack 81
Figure 4.3 Propositions stack 82
Figure 4.4 Sets stack 83
Figure 4.5 Mark-up of the aflatoxin debate 84
Chapter 5. Using FORA
Figure 5.1 Corroboration 97
Figure 5.2 Enlargement 98
Figure 5.3 Consequence 99
Figure 5.4 Rebuttal 100
Figure 5.5 Undermining 101
Figure 5.6 Undercutting 102
Figure 5.7 Counter-argument 103
Figure 5.8 Target 104
Figure 5.9 Counter-consequence 105
Chapter 6. Automating Mark-up to FORA
Figure 6.1 Three approaches to automating mark-up 119
Figure 6.2 The theorem prover 128
Figure 6.3 Automated mark-up by parsing proofs 131
Figure 6.4 Further meta-level analysis of a proof by merging and outlining 139
Figure 6.5 Application of inference rule : all-elimination 147
Figure 6.6 Application of inference rule : implication-elimination 149
Figure 6.7 Application of inference rule : direct 151
Figure 6.8 Application of inference rule : and-introduction 153
Figure 6.9 Application of inference rule : exists-introduction 155
The contribution of this thesis is to the field of Artificial Intelligence (AI), specifically to the sub-field called knowledge engineering. Knowledge engineering involves the computer representation and use of the knowledge and opinions of human experts. This document presents a framework called FORA (Framework for Opposition and Reasoning about Arguments) for use in situations when we wish to represent the views of many people who disagree or have differing opinions, and when we wish to use a computer to facilitate debate, exploration of multiple viewpoints, or revision of knowledge bases which are challenged by opposing opinions or evidence.
The conclusion of this thesis is that arguments are meta-level structures, and I argue that it is useful to represent them as such in a computer-based framework. I show that this is possible by presenting an argumentation framework, called FORA, and that it is useful by demonstrating that FORA can facilitate a variety of knowledge engineering tasks.
1.1 Motivation, context, and background
Many knowledge bases in knowledge based systems only include representations of knowledge from one person, usually an expert in a domain such as medicine, law or chemistry. When knowledge bases need to represent multiple experts' knowledge, the knowledge is almost always constrained to be consistent. Techniques such as the Delphi Method ([Delbecq et al 75], [Ng 90]) are used during knowledge acquisition, to iron-out inconsistencies and disagreements between the experts, but only the resulting consensus viewpoint is explicitly represented in the knowledge base to be used for inference. The over-riding tendency in most AI and knowledge engineering methodologies is to see conflict as a problem and try to remove it.
The aim of this thesis is to explore whether we can instead view conflict as an opportunity for interesting reasoning. In the real world, absolute consensus or consistency is very rare. People have their own opinions and differing viewpoints, and as a result people commonly disagree with one another. A great deal of intelligent activity - discussions and arguments, learning and teaching, democratic decision making - could not occur if the agents involved did not have differing viewpoints.
Some good examples of the importance of multiple viewpoints can be found in natural resource management. When decisions have to be made which involve changes to natural resources such as oceans, forests or the atmosphere, the interests of various parties need to be weighed up. These parties include land-owners, environmental pressure groups, wildlife biologists, governments and industries. Industries are currently under increasing pressure to carry out environmental impact assessments of their proposed development plans, and to integrate their results into their decision making processes. However, ecosystems are highly complex phenomena and often a range of relevant specialists will produce conflicting predictions of the effects of environmental changes (the controversy surrounding the greenhouse theory of global warming is a particularly interesting example, see section 4.3 and [Kellogg 91]). In addition, conflicts can arise due to differing interests or goals, for example, conservation of particular wildlife habitats versus economic development (see section 4.2, for an example of a conflict of interests surrounding oil industry development in Alaska). Conflicts can also result from the use of different terminologies by people from different disciplines - such conflicts are sometimes hard to untangle as both agreement and disagreement can be obscured by differing language use (see for example [Shaw & Gaines 91].
Similarly complex examples of multiple viewpoints exist in the domain of health risk assessment, in which the views of policy makers, food and drug administrations, food and drug producers and suppliers, epidemiologists, medical experts, and the general public, all need to be taken into account in making risk assessments. I will use an example of such a risk assessment to illustrate many of the ideas described in this document (see section 3.3).
The need to handle multiple viewpoints presents a problem to the developers of knowledge based decision support systems. Single, consistent knowledge bases are incapable of meeting the needs of situations such as these. Instead we require systems containing multiple knowledge bases, each of which can represent a particular viewpoint on a problem. We then need mechanisms for identifying, exploring and evaluating conflicts within and between knowledge bases. The overall aim of the research described in this document is to support the construction of such knowledge based systems.
The users of such systems should be treated less like a novice asking a question of an expert who knows the answer, and more like a responsible decision-maker who calls, and chairs, a meeting of various relevant experts or interested parties who each argue for various positions. A debate ensues in which relevant counter-arguments need to be presented at appropriate times. By interacting with the system in this way, I believe that users would become better informed and more aware of the nature of the choices they have to make in order to make a wise decision. This model of system-user interaction accords with the ethical need for a human locus of responsibility [Whitby 88], that is, a guarantee that computers or expert systems are not in a direct line of responsibility for making decisions which may turn out to be safety-critical or life threatening. In order to construct such systems we need better representational models for arguments, and this document presents one such model.
1.2 Thesis message and contribution
The central claim of this thesis is that it is possible, and useful, to represent knowledge in the form of arguments for points of view. Doing so facilitates the handling of disagreements between experts without requiring that the conflicts be resolved.
This document describes a computational framework called FORA for articulating arguments at a high level of abstraction and exploring the resulting structures. Arguments and debate involve reasoning about the relationships between the statements and chains of inference we use, and so they involve meta-level knowledge. Methods are provided for constructing such meta-level knowledge bases both from scratch and by automatic abstraction (mark-up) from formal object-level theories or legacy knowledge bases. The usefulness of the approach is also illustrated by describing how instantiation (mark-down) of the meta-level argument structures can support construction and criticism of object-level knowledge bases or specifications.
FORA, and the theory behind it, is a contribution to the field of argumentation. I argue that it provides a more robust and rigorous set of structures for representing and reasoning about arguments than the argumentation techniques in current use (such as IBIS [Kunz & Rittel 70] [Conklin & Begeman 88], or the logic of argumentation, LA [Krause et al 95a]). The suite of tools described in chapters 6 and 7 provide automated and semi-automated transformations between FORA and other formal languages thus suggesting that the framework is general purpose and can integrate into existing knowledge representation practices. These tools also provide support for maintenance and adaptation of knowledge bases as their contents change or are disputed. This thesis is thus also a contribution to the area of knowledge and requirements engineering.
In chapter 2, I survey the related work on argumentation and disagreement and explain the usefulness of a meta-level approach to inconsistency handling.
In chapter 3, the FORA framework is introduced, along with an example of a debate in the domain of health risk assessment which is used to illustrate the techniques presented in the remainder of the document.
In chapter 4, the FORA representation language is defined. It enables arguments from knowledge bases to be reasoned about independently of their object-level representation. Four basic meta-level relations are introduced : disagreement, equivalence, justification and elaboration, and arguments are also defined. Chapter 4 also addresses the question of how to construct knowledge bases in FORA. Acquisition of knowledge from multiple viewpoints is discussed and two software tools are described which help users to express arguments (mark them up) using the formal structures of FORA.
Chapter 5 discusses how representations of arguments in FORA can be used to guide a user in exploring the range of opinion in a collection of knowledge bases. The notion of a conflict set is introduced to provide a focussed roving 'window' on the particular disagreements a user is interested in. The four primitive relations are used as the basis of definitions of more complex argument constructs, such as those encountered in the literature on argumentation. These argument constructs provide ways to automate comparison and evaluation of knowledge bases.
In order to assess the usefulness of the framework it is necessary to consider how it relates to other knowledge representation languages in which arguments can be expressed. Chapter 6 addresses how the framework can be used for reasoning about arguments in existing (legacy) knowledge bases, and describes how abstraction or mark-up to FORA can be automated from knowledge bases expressed in a formal logic. Two techniques for this are discussed and illustrated.
Chapter 7 tackles the converse issue, which is how to use representations of arguments as a basis for construction of knowledge bases in an object-level language. This transformation cannot be automated. However, the meta-level framework can provide guidance to a user who wishes to construct a formal object-level representation. Software tools are described which can ensure that a chain of reasoning is carried out by an object-level inference engine, by helping users to formalise appropriately the reasoning steps represented by an argument in FORA. An inference checker tool helps to verify that this has been successful and a knowledge base critic uses FORA representations to suggest elements at the object-level which are vulnerable to dispute.
Chapter 8 concludes by summarising the contribution of the research described in this thesis, and discussing some promising future directions for research.
This chapter provides a survey of literature in the fields of logic, philosophy and Artificial Intelligence (AI) which addresses the question of handling disagreements. It is divided into two themes. First there is the issue of disagreement itself, and section 2.1 surveys the literature about conflict, inconsistency and multiple viewpoints. This section characterises the ‘problem’. Secondly, there is the field of argumentation. Section 2.2 describes approaches to argument in philosophy, linguistics and AI. This section describes the background to the ‘solution’ adopted here.