[FRIAM] "epistemic status"

[uǝlƃ ☤>$]

Interesting. Part of the "rationality community" that I find troubling is their tendency to *game* everything, including things like charity (Effective Altruism). But the good news is that your attempt to game this test is what the test *wants* you to do, i.e. play around with your tacit bindings between belief and knowledge. Given that I doubt everything, every single test I've ever taken has been filled with "trick questions" precisely because they don't have various spectra associated.  Each T/F question should look more like those Strongly Disagree ... Strongly Agree questions. Every question needs a "confidence selector". Etc. Such is the curse of the agnostic.

Anyway, it wouldn't be very difficult to replicate this test's measures, but on a large, pseudo-randomized database of questions. I haven't done the search. But my guess is it already exists somewhere. 

On 9/29/21 2:39 PM, Steve Smith wrote:
> 
> I retook the test back-to-back and was a little surprised by the results: 
> 
>  1. I did increase my high confidence correct answers marginally (unsurprising)
>      1. This means I still got a few dead wrong.
>  2. I did lower my overall confidence.
>      1. no-brainer after seeing how overconfident I was first time around
>  3. I lowered my overall correct answers (this is the surprise).
>      1. not sure what this is about, trying too hard to "flip" my guesses and getting them wrong?
>      2. both times, I got 100% of my 50% confidence answers correct (3 or 4 of them?)
> 
> I would have probably been more better at improving my results if I'd paid more attention the first time to how I answered the low-confidence questions...  even though I was "guessing" I quoted a higher-than-50% confidence.. Sounds dumb huh?
> 
> 
>>> Do you know what you know?
>>> A Confidence Calibration Exercise
>>> http://confidence.success-equation.com/
>>
>> I share Glen's interest in retaking such a test under different personal contexts.   I found some of the questions seemingly a little disingenous and was surprised by the modest number that were easy to answer with high confidence.  A randomly selected set from a larger group might give me a slightly different mix of these.
>>
>> Unsurprisingly (to me if not everyone), my Percent Correct was lower than Glen's while my Confidence was higher.
>>
>> The only thing I feel a little proud of was that most of my high confidence answers were in fact correct.
>>
>> I think I might have gotten better scores if I'd followed an intuition that the questions were worded to yield an equal distribution of true/false questions... I definitely allowed my own optimistic nature to bias toward answering "yes" rather than "no" when I had low confidence.   A second pass through the questions with that in mind would probably have had me flipping some of my low-confidence "true"s to low confidence "false"s.  Maybe this is an incorrect assumption about the design of the test.
>>
>> I may take it again to see if that improves my hit rate...   I think my performance *would* be skewed by having seen the evaluation...  knowing that a few of my high confidence answers were *wrong* will surely yield a few more "hedged bets" there...   if I study the results with an eye to improving my scores, I can probably recognize a few other systematic areas for improvement.
>>
>>> "After answering each of the true/false questions below, indicate how confident you are in your answer using the corresponding slider. A value of 50% means you have no idea what the right answer is (the same probability as a random guess between the two choices); a value of 100% means you are completely confident in your answer."
>>>
>>> It seems to present the same questions each time, which is a shame. I'd love to try it fully alert. But my attempt at 4am, with an irritating headache, turned out this way:
>>>
>>> Mean confidence: 61.60%
>>> Actual percent correct: 78.00%
>>> You want your mean confidence and actual score to be as close as possible.
>>> Mean confidence on correct answers: 63.59%
>>> Mean confidence on incorrect answers: 54.55%
>>> You want your mean confidence to be low for incorrect answers and high for correct answers.
>>>
>>> Quiz score
>>> 39 correct out of 50 questions answered (78.00%)
>>> 27 correct out of 38 questions answered with low (50 or 60%) confidence (71.05%)
>>> 5 correct out of 5 questions answered with medium (70% or 80%) confidence (100.00%)
>>> 7 correct out of 7 questions answered with high (90 or 100%) confidence (100.00%)


-- 
"Better to be slapped with the truth than kissed with a lie."
☤>$ uǝlƃ