Conditional Random Fields: Probabilistic Models
for Segmenting and Labeling Sequential Data
John Lafferty
Andrew McCallum
Fernando Pereira
Papers & Tutorials
Conditional Random Fields: Probabilistic Models for
Segmenting and Labeling Sequential Data
-
Lafferty
-
McCallum
-
Pereira
http://www.aladdin.cs.cmu.edu/papers/pdfs/y2001/crf.pdf
Efficient Training of Conditional Random Fields
-
Hanna Wallach
http://www. cogsci.ed.ac.uk/~osborn...wallach.ps.gz
Directed Graphical Models
HMMs
Cannot represent multiple interacting features or long
range dependencies between observed elements.
MEMMs
Label-bias problem: the probability transitions leaving any
given state must sum to one.
Conditional Random Fields: CRF
Conditional probabilistic sequential models
Undirected graphical models
A single log-linear distribution over the joint
probability of an entire label sequence given
a particular observation sequence.
Weights of different features at different
states can be traded off against each other.
Label Bias Problem
States with low entropy next state
distributions will take little notice of
observations.
Two solutions:
Change the state-transition structure
Start with fully-connected model and let the
training procedure figure out a good structure.