[FRIAM] Chaos Scientist Finds Hidden Financial Risks That Regulators Miss Oxford Professor Doyne Farmer is working with central banks to improve stress testing.

[Marcus Daniels]

Stress testing is the activity of subjecting a potential investment strategy to plausible volatility from the market.    There is a record of what the market has done in the past, so one way to do stress testing is to do retrodiction on many different past periods to see how well the strategy performs.  One doesn't cheat in that testing (peek into the known future to train the model), beyond the recognition that the strategy must integrate knowledge of how markets work.   That kind of circularity exists in science.  Where does a hypothesis come from if not from past experience?

A limitation of this kind of protocol is that every bit of data one uses for training must be kept separate from the data one uses for testing.  While cross validation is possible -- using different training and testing subsets -- at the end of the day history only tells you what has happened not what could happen.    

Generative machine learning techniques can learn many-body joint probabilities of market events like price moves of different instruments.   Using these techniques one can generate not just a tape recording of what happened, but plausible looking trading patterns that share similar patterns -- like might occur in the future.   For ABM to be better, it seems to me, there has to be the possibility that a modeler can imagine subtleties of individual behaviors that, when coupled together, create phenomena that has never been seen on markets where equities can trade dozens of times a second.   Such a simulation could involve hundreds, or even thousands of types of traders each with their own evolving motivations.   Never mind the difficulty of implementing such a model, how could those rules be learned if not inferred from data?   How does one presume to know the motivations and resources of traders that often take active measures to hide themselves, e.g. disguise their price impact?

Perhaps this is more of an explanatory (academic) effort, positing that stress testing is possible using relatively simple models.   That's a nice hypothesis, maybe it even could explain a lot of variation, and the result of that would be satisfying, and a contribution to theory.   I expect practitioners have a different, higher, bar.

Marcus

On 11/15/19, 10:30 AM, "Friam on behalf of uǝlƃ ☣" <friam-bounces at redfish.com on behalf of gepropella at gmail.com> wrote:

    Since Marcus hasn't answered, I think it's important that he pre-pended ML with "generative". There's plenty to argue about with respect to that word (e.g. my complaint that EricC _thinned_ relativism). But in my own work, I've made the somewhat hand-waving argument that mathematical models (e.g. systems of ODEs) are thin and component-based models are thick. But, I'll take the opportunity, here, to argue against myself (and against you, Steve 8^) that not only can mathematical models be "a little bit thick", but that models induced into combined-but-separable probability distributions are also "a little bit thick".
    
    For math models, each term of some equation kinda-sorta represents a component of the model and then various operators are used to integrate those components (+ and - are the most boring). At the next layer, different constraints, mechanisms, components might be modeled by entirely different equations that have to be solved as a system. So, even in this brief conception, it seems clear that these models *can be* mechanistic ... i.e. explanatory, at least to some extent.
    
    The same might be said of an induction method that produces, say, bifurcated components. If you find 2 modes in some output, then it seems reasonable that there might be 2 mechanisms at work.
    
    So, in direct response to your response. Had you said the point of agent models is to yield *more* explanatory results than a generative ML classifier, I don't think there's much room to argue. We could turn the tables and argue that agent models might be more explanatory, but they'll be less predictive. So, maybe the total power is similar and we should all use *both*. (That's what I argue to my clients ... but it's rarely done because the skill sets are a bit different and it's more expensive. [sigh])
    
    
    On 11/14/19 5:56 PM, Steven A Smith wrote:
    > 
    > On 11/14/19 6:10 PM, Marcus Daniels wrote:
    >>
    >> Generative machine learning seems a heck of a lot easier than ABMs for stress testing. 
    >>
    >     Agent-based models, used in fields from biology to sociology, are bottom-up, simulating the messy interactions of hundreds and even millions of agents—human cells or attitudes or financial firms—to explain the behavior of a complex system.
    > 
    > I think the point of Agent Models is to yield *explanatory* not just *predictive results?
    
    -- 
    ☣ uǝlƃ
    
    ============================================================
    FRIAM Applied Complexity Group listserv
    Meets Fridays 9a-11:30 at cafe at St. John's College
    to unsubscribe http://redfish.com/mailman/listinfo/friam_redfish.com
    archives back to 2003: http://friam.471366.n2.nabble.com/
    FRIAM-COMIC http://friam-comic.blogspot.com/ by Dr. Strangelove