[FRIAM] Layers

[Stephen Guerin]

Looks cool, Frank.

Surfing around:

a Deep Equilibrium Model paper by Shaojie Bai:
https://proceedings.neurips.cc/paper/2019/file/01386bd6d8e091c2ab4c7c7de644d37b-Paper.pdf

And one of his 12-minute talks:
https://crossminds.ai/video/oral-at-neurips-2020-multiscale-deep-equilibrium-models-5fb82261890833803bc7e7f4/
_______________________________________________________________________
Stephen.Guerin at Simtable.com <stephen.guerin at simtable.com>
CEO, Simtable  http://www.simtable.com
1600 Lena St #D1, Santa Fe, NM 87505
office: (505)995-0206 mobile: (505)577-5828
twitter: @simtable
z <http://zoom.com/j/5055775828>oom.simtable.com


On Wed, Mar 30, 2022 at 2:31 PM Frank Wimberly <wimberly3 at gmail.com> wrote:

> A current CMU dissertation defense:
>
> PhD Candidate: Shaojie Bai
>
> *Title: **Equilibrium Approaches to Modern Deep Learning*
>
> Abstract:
> Deep learning (DL) has become one of the most successful and
> widely-adopted methods in modern artificial intelligence. Accompanying
> these successes are also increasingly complex and costly architectural
> designs, at the foundation of which has been a core concept: *layers*.
> This thesis challenges this fundamental role of layers, and provides an
> in-depth introduction to a new, layer-*less* paradigm of deep learning
> that computes the output as the fixed point of a dynamical system: deep
> equilibrium (DEQ) models.
> First, we introduce the general formulation of deep equilibrium models. We
> discuss how these models express “infinite-level” neural networks, decouple
> forward and backward passes, yet with the cost and design complexity of one
> traditional layer— even in some of the most competitive settings (e.g.,
> language mode
>
>
> ---
> Frank C. Wimberly
> 140 Calle Ojo Feliz,
> Santa Fe, NM 87505
>
> 505 670-9918
> Santa Fe, NM
>
> .-- .- -. - / .- -.-. - .. --- -. ..--.. / -.-. --- -. .--- ..- --. .- - .
> FRIAM Applied Complexity Group listserv
> Zoom Fridays 9:30a-12p Mtn UTC-6  bit.ly/virtualfriam
> un/subscribe http://redfish.com/mailman/listinfo/friam_redfish.com
> FRIAM-COMIC http://friam-comic.blogspot.com/
> archives:
>  5/2017 thru present https://redfish.com/pipermail/friam_redfish.com/
>  1/2003 thru 6/2021  http://friam.383.s1.nabble.com/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20220330/1334c864/attachment-0001.html>