Bayesian Networks

Historical Background

Bayesian networks originated in the mid-1980s from the quest for mathematically sound and computationally tractable methods for reasoning with uncertainty in artificial intelligence. In the preceding decade, the first applications of computer-assisted decision making had found their way to the medical field, mostly focusing on the diagnostic process. This had required the development of methods for reasoning with uncertain and incomplete diagnostic information.

One popular method was the naive Bayesian approach that required specification of positive and negative predictive values for each of a set of predefined diagnostic tests and a prior (i.e., marginal) probability distribution over possible diagnostic hypotheses. The approach assumed that all test results were mutually independent markers of disease and used Bayes's theorem to compute posterior (i.e., conditional) probabilities on the hypotheses of interest. The approach is simple and fast and requires a relatively small number of marginal and conditional probabilities to be specified. However, the assumption of independence is mostly wrong and leads to overly extreme posterior probabilities.

Another approach arose in the field of expert systems, where algorithms had been devised to reason with so-called certainty factors, parameters expressing the strength of association in if-then rules. The underlying reasoning principles were mostly ad hoc and not rooted in probability theory, but large sets of if-then rules allowed for a domain representation that was structurally richer and more complex than naive Bayesian models. Bayesian networks bring together the best of both approaches by combining representational expressiveness with mathematical rigor.

Model Structure

Bayesian networks belong to the family of probabilistic graphical models (PGMs), graphs in which nodes represent random variables, and the (lack of) arcs represent conditional independence assumptions. Let G = (V(G), A(G)) be a directed acyclic graph, where the nodes V(G) = {V1, …, Vn} represent discrete random variables with a finite value domain. For each node Vi ∊ V(G), let πi denote the set of parent nodes of Vi in graph G. A Bayesian network now is a pair B = (G, Θ), where Θ={θi|Vi ∊ V(G)} is a set of parametrization functions. The function θi describes a local model for node Vi ∊ V(G) by specifying a conditional probability θi(v|s) for each possible value v of variable Vi and all possible value assignments s to its parents πi. The Bayesian network B defines a unique multivariate probability distribution Pr on V1, …, Vn using the

...