[FRIAM] Simulation and policy-making

[mgd at santafe.edu]

Quoting Robert Holmes <robert at holmesacosta.com>:

> You want a simulation that shows it's a good idea to
> invade Iraq? No problem, I'll interview a bunch of experts, code up
> realistic micro-rules and give you a simulation that shows yes, that's a
> sensible policy. You want a simulation that shows it's not a good idea to
> invade Iraq? No problem, I'll just interview a different set of experts, get
> some different micro-rules in there and voila, I've shown invasion is a Bad
> Thing.

Two ways to deal with that problem.

1) Data

Experts can suggest micro-rules, but then provide examples when those micro-
rules were used and other cases where different micro-rules were used, and 
historical accounts of what happened in each case.  In other words do 
retrodiction on a past similar system that had a known result.  If there are 
no such systems, and no way to change plans or modify a plan (based on early 
data), then indeed the forecasts are suspect and vulnerable to manipulation.

2) Peer review

Not all experts will agree on micro-rules.  Get a range of micro-rules from 
different experts and compute the range of possibility for all of them.

Marcus