[FRIAM] Fwd: [NICAR-L] Machine learning in reporting example

[Marcus Daniels]

Tom,


The random forest method is kind of unsatisfying to me.  It says that one can train many simple experts, trained on subsets of a dataset, to vote and thereby predict as well or better as one big integrated expert.  One might hope this could be a mechanism of democracy... A property of recursive partitioning, that underlies random forests -- and which can work remarkably well by itself -- is that it commits each expert to a main effect, secondary effects, tertiary effects and so on.  Some kinds of decision making don't have this structure, e.g. they could have non-linear composition of factors.   But, if the random subsets they learn on happen to be representative of various sub-populations, that do each have simple hierarchical rules, then one could see how different rules per expert apply to, say, different voting communities.   It would be like if one went to expert pollsters in every voting district who also happened to live there, and ask them each for a prediction


Marcus

________________________________
From: Friam <friam-bounces at redfish.com> on behalf of Tom Johnson <tom at jtjohnson.com>
Sent: Monday, August 7, 2017 6:56:54 PM
To: Friam at redfish. com
Subject: [FRIAM] Fwd: [NICAR-L] Machine learning in reporting example

All:

Perhaps some of you will be interested in these links describing how journalists -- well, at least ONE journalist -- used AI, and specifically the "Random Forest" algorithm, to uncover government agency surveillance activities at home and abroad.  See especially the first and the last link.

Any thoughts on other applications of this methodology?

Tom

============================================
Tom Johnson
Institute for Analytic Journalism   --     Santa Fe, NM USA
505.577.6482(c)                                    505.473.9646(h)
Society of Professional Journalists<http://www.spj.org>
Check out It's The People's Data<https://www.facebook.com/pages/Its-The-Peoples-Data/1599854626919671>
http://www.jtjohnson.com<http://www.jtjohnson.com/>                   tom at jtjohnson.com<mailto:tom at jtjohnson.com>
============================================


On Mon, Aug 7, 2017 at 12:53 PM, Peter Aldhous <peter at peteraldhous.com<mailto:peter at peteraldhous.com>> wrote:
Hi all,

Excuse the shameless self-promotion, but I thought some folks on the list might be interested in this: using the random forest algorithm on flight/aircraft data to identify potential spy planes.

1) https://www.buzzfeed.com/peteraldhous/hidden-spy-planes

Here are the other two recent stories that it spawned:

2) https://www.buzzfeed.com/peteraldhous/us-marshals-spy-plane-over-mexico

3) https://www.buzzfeed.com/christianstork/spy-planes-over-american-cities

And here are the methods/data/code:

4) https://buzzfeednews.github.io/2017-08-spy-plane-finder/

Thanks also to colleagues Christian Stork and Karla Zabludovsky for their excellent reporting on these stories.

Cheers,

Peter

--
Peter Aldhous, PhD
Science journalist
cell: 415 503 7323<tel:(415)%20503-7323>
peter at peteraldhous.com<mailto:peter at peteraldhous.com>
@paldhous<https://twitter.com/paldhous>
www.peteraldhous.com<http://www.peteraldhous.com/>
==================================================================== To unsubscribe from NICAR-L, please send "unsubscribe NICAR-L" in the body of an e-mail message to "listserv at lists.missouri.edu". Please e-mail listmaster at ire.org<mailto:listmaster at ire.org> if you need help or have questions. ====================================================================



--
Jacqueline Kazil | @jackiekazil


==================================================================== To unsubscribe from NICAR-L, please send "unsubscribe NICAR-L" in the body of an e-mail message to "listserv at lists.missouri.edu". Please e-mail listmaster at ire.org<mailto:listmaster at ire.org> if you need help or have questions. ====================================================================

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20170808/dda8f91e/attachment.html>