5. Active Learning

(placeholder)

5.1. Main Active Learning Routine

(placeholder)

5.2. Heuristics

(placeholder)

5.2.1. Random Benchmark

(placeholder)

5.2.2. Uncertainty Sampling

(placeholder)

5.2.3. Query by Bagging

The Kullback-Leibler divergence of $Q$ from $P$ is defined as

$$D_{\mathrm{KL}}(P\|Q) = \sum_i P(i) \, \ln\frac{P(i)}{Q(i)}.$$

This KL divergence measures the amount of information lost when $Q$ is used to approximate $P$. In the active learning context, $Q$ is the average prediction probability of the committee, while $P$ is the prediction of a particular committee member.