relative.entropy.md

relative.entropy	R Documentation

Relative Entropy

Description

A generic S3 function to compute the relative entropy score for a classification model. This function dispatches to S3 methods inrelative.entropy() and performs no input validation. If you supply NA values or vectors of unequal length (e.g. length(x) != length(y)), the underlying C++ code may trigger undefined behavior and crash your R session.

Defensive measures

Because relative.entropy() operates on raw pointers, pointer-level faults (e.g. from NA or mismatched length) occur before any R-level error handling. Wrapping calls in try() or tryCatch() will not prevent R-session crashes.

To guard against this, wrap relative.entropy() in a "safe" validator that checks for NA values and matching length, for example:

safe_relative.entropy <- function(x, y, ...) {
  stopifnot(
    !anyNA(x), !anyNA(y),
    length(x) == length(y)
  )
  relative.entropy(x, y, ...)
}

Apply the same pattern to any custom metric functions to ensure input sanity before calling the underlying C++ code.

Usage

## Generic S3 method
## for Relative Entropy
relative.entropy(...)

Arguments

...

Arguments passed on to relative.entropy.matrix pk,qk A pair of <double> matrices of length n of emprical probabilities p and estimated probabilities q. dim An <integer> value of length 1 (Default: 0). Defines the dimension along which to calculate the entropy (0: total, 1: row-wise, 2: column-wise). normalize A <logical>-value (default: TRUE). If TRUE, the mean cross-entropy across all observations is returned; otherwise, the sum of cross-entropies is returned.

Value

A <double> value or vector:

• A single <double> value (length 1) if dim == 0.

• A <double> vector with length equal to the length of columns ifdim == 1.

• A <double> vector with length equal to the length of rows ifdim == 2.

References

MacKay, David JC. Information theory, inference and learning algorithms. Cambridge university press, 2003.

Kramer, Oliver, and Oliver Kramer. "Scikit-learn." Machine learning for evolution strategies (2016): 45-53.

Virtanen, Pauli, et al. "SciPy 1.0: f'undamental algorithms for scientific computing in Python." Nature methods 17.3 (2020): 261-272.

Examples

## generate valid probability
## distributions
rand.sum <- function(n) {
   x <- sort(runif( n-1 ))
   c(x,1) - c(0, x)
}

## empirical and
## predicted probabilites
set.seed(1903)
pk <- t(replicate(200,rand.sum(5)))
qk <- t(replicate(200,rand.sum(5)))

## entropy
relative.entropy(
 pk = pk,
 qk = qk
)

</div>