Inductive Logic Programming: Techniques and Applications (Ellis Horwood Series in Artificial Intelli

Inductive Logic Programming: Techniques and Applications (Ellis Horwood Series in Artificial Intelligence)

By Nada Lavrac, Saso Dzeroski,

Publisher: Prentice Hall
Number Of Pages: 400
Publication Date: 1994-01
Sales Rank: 5156202
ISBN / ASIN: 0134578708
EAN: 9780134578705
Binding: Hardcover
Manufacturer: Prentice Hall
Studio: Prentice Hall
Average Rating: 4

Review:

Out of date, but still can be useful

Interest in inductive logic programming has waxed and waned over the last decade, but never fallen to zero. This book is a summary of what was known in the field in 1994, and much has changed since then. It can however still serve as an introduction to the field of inductive logic programming, in spite of its publication date. Most of the current research and applications of inductive logic programming has concentrated on introducing stochasticity into logic programming and on how to incorporate reasoning with numerics into the framework.

The authors emphasize the empirical aspects of inductive logic programming and its applications, but do spend the first few chapters detailing the theoretical foundations of the subject. The characterize machine learning paradigms as inductive, deductive, learning with genetic algorithms, and learning with neural nets. They rule out neural net learning as being a true learning system since it does not pass the "Michie strong criterion", i.e. learning must acquire new knowledge which must be understandable by humans. They do not elaborate on why neural nets fail to meet this criterion. Inductive learning of course is what they consider exclusively in the book, with inductive concept learning essentially consisting of the learning of how to recognize objects in the concept, the concept being a subset of objects in universal set of objects or observations. To define inductive concept learning more rigorously the authors employ the concept of a covering of an object, which means essentially that the description of the object satisfies the description of the concept. The object description is thus "covered" by the concept description. Notions of completeness and consistency of hypotheses are then introduced, with completeness being the requirement that the hypothesis cover all positive examples and no negative ones, while consistency meaning that it does not cover any of the negative examples. These ideas are then generalized to the case where background knowledge is present. Inductive logic programming systems are then defined as those that induce hypotheses in the form of logic programs. These systems are partitioned into those that learn predicates from scratch, called empirical ILP systems, and those that learn multiple predicates, called interactive ILP systems. The authors then discuss briefly the systems that were available at the time of writing. Only empirical ILP systems are considered by the authors in the book, with emphasis on the systems LINUS and FOIL, which were the dominant ones at the time of writing.

Because of its popularity and effectiveness in logic programming, the authors employ Prolog to introduce the basic theory of logic programming. Other languages have been developed since then with ILP applications in mind, one of these being Progol. Symbolic programming languages, such as Mathematica and Maple, can also be used, and very effectively. The essentials of logic programming discussed in the book have no doubt been seen by the reader, and some familiar concepts such as Horn clauses and resolution are discussed by the authors. The goal of empirical ILP then is to find a complete and consistent definition for an unknown predicate given a set of examples and background knowledge. Concept learning is viewed as a search problem, with states in the search space being concept descriptions. The goal is to find states that satisfy a quality criterion, and a learning algorithm is characterized in terms of the structure of its search space, its search strategy, and the search heuristics. The structure of the search space is characterized by a "theta-subsumption lattice", which gives the structure of the search space of program clauses, and which can be searched blindly or heuristically. Theta-subsumption provides the basis for a "bottom-up" ILP technique, namely that of the building of least general generalizations from training examples relative to background knowledge, and a "top-down" technique of the searching of refinement graphs. These techniques and the technique of inverse resolution are discussed in detail by the authors. The idea of inverse resolution will seem natural to the reader familiar with the related (but inverted) procedure in deductive (propositional) logic. Inverse resolution inverts the SLD-resolution proof procedure for definite programs.

Most of the book is devoted to an overview of the FOIL system and how it can be implemented to do practical inductive logic programming. The search routines used by FOIL are hill-climbing strategies, and the authors discuss ways that have been used to improve on these. Since this book was written, an ILP system called SFOIL has appeared that takes advantage of the view of induction of hypotheses as an optimization problem. Interestingly, SFOIL uses a generalization of simulated annealing to do this, based on Markovian neural networks. The authors also review the GOLEM ILP programming language, which is based on the notion of relative least general generalization, again a bottom-up search of the theta-subsumption lattice. Other ILP languages, such as MOBAL and MPL are also reviewed. In addition, the LINUS IPL system is reviewed, which exploits background knowledge in learning both propositional and relational descriptions. Deduction plays a major role in the LINUS system, as well as the transformation of relational descriptions to a propositional learning task. Both the FOIL and the LINUS systems are characterized with respect to refinement operators and refinement graphs, which allows a comparison of the expressiveness of their hypothesis languages and the search costs associated with these systems.

The authors also discuss how to handle imperfect data in ILP, and show the role of heuristics in doing this. Random errors in training examples and background knowledge, sparse training examples, inexact description of target concepts, and missing values in training examples all need to be dealt with when using ILP, and various techniques are oultined by the authors to do this. Several interesting applications of ILP are given in the book, including medical diagnostics, finite element methods, qualitative modeling of dynamical systems, and predicting protein secondary structure. The role of ILP in bioinformatics has taken on more importance in recent years, and this trend will no doubt continue.

Review:

Out of date, but still can be useful

Interest in inductive logic programming has waxed and waned over the last decade, but never fallen to zero. This book is a summary of what was known in the field in 1994, and much has changed since then. It can however still serve as an introduction to the field of inductive logic programming, in spite of its publication date. Most of the current research and applications of inductive logic programming has concentrated on introducing stochasticity into logic programming and on how to incorporate reasoning with numerics into the framework.

The authors emphasize the empirical aspects of inductive logic programming and its applications, but do spend the first few chapters detailing the theoretical foundations of the subject. The characterize machine learning paradigms as inductive, deductive, learning with genetic algorithms, and learning with neural nets. They rule out neural net learning as being a true learning system since it does not pass the "Michie strong criterion", i.e. learning must acquire new knowledge which must be understandable by humans. They do not elaborate on why neural nets fail to meet this criterion. Inductive learning of course is what they consider exclusively in the book, with inductive concept learning essentially consisting of the learning of how to recognize objects in the concept, the concept being a subset of objects in universal set of objects or observations. To define inductive concept learning more rigorously the authors employ the concept of a covering of an object, which means essentially that the description of the object satisfies the description of the concept. The object description is thus "covered" by the concept description. Notions of completeness and consistency of hypotheses are then introduced, with completeness being the requirement that the hypothesis cover all positive examples and no negative ones, while consistency meaning that it does not cover any of the negative examples. These ideas are then generalized to the case where background knowledge is present. Inductive logic programming systems are then defined as those that induce hypotheses in the form of logic programs. These systems are partitioned into those that learn predicates from scratch, called empirical ILP systems, and those that learn multiple predicates, called interactive ILP systems. The authors then discuss briefly the systems that were available at the time of writing. Only empirical ILP systems are considered by the authors in the book, with emphasis on the systems LINUS and FOIL, which were the dominant ones at the time of writing.

Because of its popularity and effectiveness in logic programming, the authors employ Prolog to introduce the basic theory of logic programming. Other languages have been developed since then with ILP applications in mind, one of these being Progol. Symbolic programming languages, such as Mathematica and Maple, can also be used, and very effectively. The essentials of logic programming discussed in the book have no doubt been seen by the reader, and some familiar concepts such as Horn clauses and resolution are discussed by the authors. The goal of empirical ILP then is to find a complete and consistent definition for an unknown predicate given a set of examples and background knowledge. Concept learning is viewed as a search problem, with states in the search space being concept descriptions. The goal is to find states that satisfy a quality criterion, and a learning algorithm is characterized in terms of the structure of its search space, its search strategy, and the search heuristics. The structure of the search space is characterized by a "theta-subsumption lattice", which gives the structure of the search space of program clauses, and which can be searched blindly or heuristically. Theta-subsumption provides the basis for a "bottom-up" ILP technique, namely that of the building of least general generalizations from training examples relative to background knowledge, and a "top-down" technique of the searching of refinement graphs. These techniques and the technique of inverse resolution are discussed in detail by the authors. The idea of inverse resolution will seem natural to the reader familiar with the related (but inverted) procedure in deductive (propositional) logic. Inverse resolution inverts the SLD-resolution proof procedure for definite programs.

Most of the book is devoted to an overview of the FOIL system and how it can be implemented to do practical inductive logic programming. The search routines used by FOIL are hill-climbing strategies, and the authors discuss ways that have been used to improve on these. Since this book was written, an ILP system called SFOIL has appeared that takes advantage of the view of induction of hypotheses as an optimization problem. Interestingly, SFOIL uses a generalization of simulated annealing to do this, based on Markovian neural networks. The authors also review the GOLEM ILP programming language, which is based on the notion of relative least general generalization, again a bottom-up search of the theta-subsumption lattice. Other ILP languages, such as MOBAL and MPL are also reviewed. In addition, the LINUS IPL system is reviewed, which exploits background knowledge in learning both propositional and relational descriptions. Deduction plays a major role in the LINUS system, as well as the transformation of relational descriptions to a propositional learning task. Both the FOIL and the LINUS systems are characterized with respect to refinement operators and refinement graphs, which allows a comparison of the expressiveness of their hypothesis languages and the search costs associated with these systems.

The authors also discuss how to handle imperfect data in ILP, and show the role of heuristics in doing this. Random errors in training examples and background knowledge, sparse training examples, inexact description of target concepts, and missing values in training examples all need to be dealt with when using ILP, and various techniques are oultined by the authors to do this. Several interesting applications of ILP are given in the book, including medical diagnostics, finite element methods, qualitative modeling of dynamical systems, and predicting protein secondary structure. The role of ILP in bioinformatics has taken on more importance in recent years, and this trend will no doubt continue.

http://mihd.net/r87gka

http://rapidshare.com/files/56058637/ILPbook_pdf.zip.html