Lets imagine that we have some experiments. Each experiment may result in one of the outcomes: A, B, C. So we have probability distributions for each experiment $P_A, P_B, P_C$ which is context-dependent, e.g.:
- $Context_1 \Rightarrow \{P_A^1, P_B^1, P_C^1\},$ experimental outcome is A
- $Context_2 \Rightarrow \{P_A^2, P_B^2, P_C^2\},$ experimental outcome is B
- $Context_3 \Rightarrow \{P_A^3, P_B^3, P_C^3\},$ experimental outcome is A
- $Context_4 \Rightarrow \{P_A^4, P_B^4, P_C^4\},$ experimental outcome is C
These probabilities are calculated by some function $F:Context\rightarrow \{P_A, P_B, P_C\}$
I want to estimate an absolute trust rate of this function. In other words, I want to be able to say "we can trust this function on 86%" like we do when we deal with Pearson's chi-square test.
Any suggestions?