Confusion matrices compare two classifications (usually one done automatically using a machine learning algorithm versus the true classification done by a specialist... but one can also compare two automatic or two manual classifications against each other).
confusion(x, ...)
# Default S3 method
confusion(
x,
y = NULL,
vars = c("Actual", "Predicted"),
labels = vars,
merge.by = "Id",
useNA = "ifany",
prior,
...
)
# S3 method for class 'mlearning'
confusion(
x,
y = response(x),
labels = c("Actual", "Predicted"),
useNA = "ifany",
prior,
...
)
# S3 method for class 'confusion'
print(x, sums = TRUE, error.col = sums, digits = 0, sort = "ward.D2", ...)
# S3 method for class 'confusion'
summary(object, type = "all", sort.by = "Fscore", decreasing = TRUE, ...)
# S3 method for class 'summary.confusion'
print(x, ...)Arguments
- x
an object with a
confusion()method implemented.- ...
further arguments passed to the method.
- y
another object, from which to extract the second classification, or
NULLif not used.- vars
the variables of interest in the first and second classification in the case the objects are lists or data frames. Otherwise, this argument is ignored and
xandymust be factors with same length and same levels.- labels
labels to use for the two classifications. By default, they are the same as
vars, or the one in the confusion matrix.- merge.by
a character string with the name of variables to use to merge the two data frames, or
NULL.- useNA
do we keep
NAs as a separate category? The default"ifany"creates this category only if there are missing values. Other possibilities are"no", or"always".- prior
class frequencies to use for first classifier that is tabulated in the rows of the confusion matrix. For its value, see here under, the
value=argument.- sums
is the confusion matrix printed with rows and columns sums?
- error.col
is a column with class error for first classifier added (equivalent to false negative rate of FNR)?
- digits
the number of digits after the decimal point to print in the confusion matrix. The default or zero leads to most compact presentation and is suitable for frequencies, but not for relative frequencies.
- sort
are rows and columns of the confusion matrix sorted so that classes with larger confusion are closer together? Sorting is done using a hierarchical clustering with
hclust(). The clustering method is"ward.D2"by default, but see thehclust()help for other options). IfFALSEorNULL, no sorting is done.- object
a confusion object
- type
either
"all"(by default), or consideringTPis the true positives,FPis the false positives,TNis the true negatives andFNis the false negatives, one can also specify:"Fscore"(F-score = F-measure = F1 score = harmonic mean of Precision and recall),"Recall"(TP / (TP + FN) = 1 - FNR),"Precision"(TP / (TP + FP) = 1 - FDR),"Specificity"(TN / (TN + FP) = 1 - FPR),"NPV"(Negative predicted value = TN / (TN + FN) = 1 - FOR),"FPR"(False positive rate = 1 - Specificity = FP / (FP + TN)),"FNR"(False negative rate = 1 - Recall = FN / (TP + FN)),"FDR"(False Discovery Rate = 1 - Precision = FP / (TP + FP)),"FOR"(False omission rate = 1 - NPV = FN / (FN + TN)),"LRPT"(Likelihood Ratio for Positive Tests = Recall / FPR = Recall / (1 - Specificity)),"LRNT"Likelihood Ratio for Negative Tests = FNR / Specificity = (1 - Recall) / Specificity,"LRPS"(Likelihood Ratio for Positive Subjects = Precision / FOR = Precision / (1 - NPV)),"LRNS"(Likelihood Ratio Negative Subjects = FDR / NPV = (1 - Precision) / (1 - FOR)),"BalAcc"(Balanced accuracy = (Sensitivity + Specificity) / 2),"MCC"(Matthews correlation coefficient),"Chisq"(Chisq metric), or"Bray"(Bray-Curtis metric)- sort.by
the statistics to use to sort the table (by default, Fmeasure, the F1 score for each class = 2 * recall * precision / (recall + precision)).
- decreasing
do we sort in increasing or decreasing order?
Value
A confusion matrix in a confusion object.
See also
Examples
data("Glass", package = "mlbench")
# Use a little bit more informative labels for Type
Glass$Type <- as.factor(paste("Glass", Glass$Type))
# Use learning vector quantization to classify the glass types
# (using default parameters)
summary(glass_lvq <- ml_lvq(Type ~ ., data = Glass))
#> Codebook:
#> Class RI Na Mg Al Si K
#> 67 Glass 1 1.522534 13.44932 3.85934116 0.6953009 72.09108 0.10287143
#> 26 Glass 1 1.517912 13.21384 2.96679081 1.0638540 73.09418 0.62446576
#> 32 Glass 1 1.517500 12.79125 3.47000000 1.1625000 73.24375 0.57125000
#> 40 Glass 1 1.519683 13.85617 3.98298320 0.8829080 72.11581 0.14612747
#> 38 Glass 1 1.517241 12.86400 3.56730134 1.3253184 73.03930 0.59471469
#> 17 Glass 1 1.517953 12.75584 3.53146094 1.2048663 73.12350 0.61831472
#> 33 Glass 1 1.516813 13.18249 3.58824194 1.0941608 73.28127 0.54218127
#> 16 Glass 1 1.517770 12.73773 3.56135984 1.2938022 73.27586 0.57345520
#> 106 Glass 2 1.529182 10.95909 0.00000000 2.0300000 70.56727 0.65318182
#> 81 Glass 2 1.517303 12.61100 3.61600000 2.2130000 72.61500 0.36900000
#> 80 Glass 2 1.516683 12.95661 3.59952510 1.5609646 73.12454 0.49140132
#> 93 Glass 2 1.515893 13.06412 3.50044677 1.5688908 73.22450 0.09132098
#> 144 Glass 2 1.518029 12.81731 4.15906227 1.6511718 73.19652 -0.33283271
#> 112 Glass 2 1.527990 12.10090 0.00000000 0.9076119 72.09761 0.06940299
#> 88 Glass 2 1.517411 13.54454 3.91205511 1.3385817 72.50985 0.42145765
#> 101 Glass 2 1.516747 12.58148 3.37065852 1.4143369 73.20617 0.22938759
#> 162 Glass 3 1.518527 13.63163 3.59082381 0.5562626 72.94373 0.07840846
#> 147 Glass 3 1.512463 13.84826 3.91879200 1.0309080 73.18266 -0.24886900
#> 169 Glass 5 1.520131 12.17186 0.42508693 2.1032884 72.81830 0.56922866
#> 179 Glass 6 1.516730 14.30190 2.60431263 1.3851871 73.37313 -0.26252576
#> 203 Glass 7 1.515683 14.68643 -0.02875463 2.4980121 73.35808 -0.08289203
#> 207 Glass 7 1.518196 14.75666 0.19283012 1.8902363 72.92972 -0.09195050
#> 199 Glass 7 1.516146 14.60523 0.00000000 2.6535385 73.52369 -1.36907692
#> Ca Ba Fe
#> 67 9.689596 0.000000000 0.091117034
#> 26 9.242871 -0.145801911 -0.062689222
#> 32 8.583750 0.000000000 0.000000000
#> 40 8.850008 0.007656739 -0.059842505
#> 38 8.487545 -0.004122066 0.052989584
#> 17 8.722567 0.000000000 0.038085891
#> 33 8.184611 0.011694440 0.023366137
#> 16 8.460135 0.000000000 0.037584925
#> 106 13.280909 2.147727273 0.299090909
#> 81 8.732000 -0.357000000 0.000000000
#> 80 8.101031 -0.054141050 -0.023075504
#> 93 8.378534 0.000000000 0.166839013
#> 144 8.334913 0.000000000 0.027098964
#> 112 14.659552 0.000000000 0.054626866
#> 88 8.211648 -0.234078018 0.112684840
#> 101 8.806406 0.047887045 0.085070197
#> 162 8.934824 0.111940999 0.179105599
#> 147 8.256887 0.000000000 0.000000000
#> 169 11.751128 0.010675921 0.005961232
#> 179 8.815785 -0.225619224 -0.063273952
#> 203 8.850959 0.648165274 0.013643891
#> 207 8.476473 1.782818874 0.010110012
#> 199 9.606154 0.942769231 0.000000000
# Calculate cross-validated confusion matrix
(glass_conf <- confusion(cvpredict(glass_lvq), Glass$Type))
#> 214 items classified with 136 true positives (error rate = 36.4%)
#> Predicted
#> Actual 01 02 03 04 05 06 (sum) (FNR%)
#> 01 Glass 3 0 11 6 0 0 0 17 100
#> 02 Glass 1 3 55 12 0 0 0 70 21
#> 03 Glass 2 0 22 51 1 2 0 76 33
#> 04 Glass 5 0 1 6 5 0 1 13 62
#> 05 Glass 6 0 1 2 2 2 2 9 78
#> 06 Glass 7 0 0 4 1 1 23 29 21
#> (sum) 3 90 81 9 5 26 214 36
# Raw confusion matrix: no sort and no margins
print(glass_conf, sums = FALSE, sort = FALSE)
#> 214 items classified with 136 true positives (error rate = 36.4%)
#> Predicted
#> Actual 01 02 03 04 05 06
#> 01 Glass 1 55 12 3 0 0 0
#> 02 Glass 2 22 51 0 1 2 0
#> 03 Glass 3 11 6 0 0 0 0
#> 04 Glass 5 1 6 0 5 0 1
#> 05 Glass 6 1 2 0 2 2 2
#> 06 Glass 7 0 4 0 1 1 23
summary(glass_conf)
#> 214 items classified with 136 true positives (error = 36.4%)
#>
#> Global statistics on reweighted data:
#> Error rate: 36.4%, F(micro-average): 0.494, F(macro-average): 0.486
#>
#> Fscore Recall Precision Specificity NPV FPR
#> Glass 7 0.8363636 0.7931034 0.8846154 0.9837838 0.9680851 0.01621622
#> Glass 1 0.6875000 0.7857143 0.6111111 0.7569444 0.8790323 0.24305556
#> Glass 2 0.6496815 0.6710526 0.6296296 0.7826087 0.8120301 0.21739130
#> Glass 5 0.4545455 0.3846154 0.5555556 0.9800995 0.9609756 0.01990050
#> Glass 6 0.2857143 0.2222222 0.4000000 0.9853659 0.9665072 0.01463415
#> Glass 3 0.0000000 0.0000000 0.0000000 0.9847716 0.9194313 0.01522843
#> FNR FDR FOR LRPT LRNT LRPS LRNS
#> Glass 7 0.2068966 0.1153846 0.03191489 48.908046 0.2103069 27.717949 0.1191885
#> Glass 1 0.2142857 0.3888889 0.12096774 3.232653 0.2830931 5.051852 0.4424057
#> Glass 2 0.3289474 0.3703704 0.18796992 3.086842 0.4203216 3.349630 0.4561043
#> Glass 5 0.6153846 0.4444444 0.03902439 19.326923 0.6278797 14.236111 0.4624929
#> Glass 6 0.7777778 0.6000000 0.03349282 15.185185 0.7893289 11.942857 0.6207921
#> Glass 3 1.0000000 1.0000000 0.08056872 0.000000 1.0154639 0.000000 1.0876289
#> BalAcc MCC Chisq Bray Auto Manu A_M TP FP FN
#> Glass 7 0.8884436 0.81391162 141.7647546 0.007009346 26 29 -3 23 3 6
#> Glass 1 0.7713294 0.51573305 56.9198438 0.046728972 90 70 20 55 35 15
#> Glass 2 0.7268307 0.44762029 42.8778806 0.011682243 81 76 5 51 30 25
#> Glass 5 0.6823574 0.43403526 40.3147336 0.009345794 9 13 -4 5 4 8
#> Glass 6 0.6037940 0.27583060 16.2816592 0.009345794 5 9 -4 2 3 7
#> Glass 3 0.4923858 -0.03502763 0.2625641 0.032710280 3 17 -14 0 3 17
#> TN
#> Glass 7 182
#> Glass 1 109
#> Glass 2 108
#> Glass 5 197
#> Glass 6 202
#> Glass 3 194
summary(glass_conf, type = "Fscore")
#> Glass 7 Glass 1 Glass 2 Glass 5 Glass 6 Glass 3
#> 0.8363636 0.6875000 0.6496815 0.4545455 0.2857143 0.0000000
#> attr(,"stat.type")
#> [1] "Fscore"