[FRIAM] Layers

[Frank Wimberly]

A current CMU dissertation defense:

PhD Candidate: Shaojie Bai

*Title: **Equilibrium Approaches to Modern Deep Learning*

Abstract:
Deep learning (DL) has become one of the most successful and widely-adopted
methods in modern artificial intelligence. Accompanying these successes are
also increasingly complex and costly architectural designs, at the
foundation of which has been a core concept: *layers*. This thesis
challenges this fundamental role of layers, and provides an in-depth
introduction to a new, layer-*less* paradigm of deep learning that computes
the output as the fixed point of a dynamical system: deep equilibrium (DEQ)
models.
First, we introduce the general formulation of deep equilibrium models. We
discuss how these models express “infinite-level” neural networks, decouple
forward and backward passes, yet with the cost and design complexity of one
traditional layer— even in some of the most competitive settings (e.g.,
language mode


---
Frank C. Wimberly
140 Calle Ojo Feliz,
Santa Fe, NM 87505

505 670-9918
Santa Fe, NM
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20220330/68ea74a7/attachment.html>