[FRIAM] Fwd: Impromptu talk by Vahab Mirrokni (Google) of possible interest

[Frank Wimberly]

---
Frank C. Wimberly
140 Calle Ojo Feliz,
Santa Fe, NM 87505

505 670-9918
Santa Fe, NM

---------- Forwarded message ---------
From: Andrej Risteski <aristesk at andrew.cmu.edu>
Date: Thu, May 22, 2025, 3:32 PM
Subject: Impromptu talk by Vahab Mirrokni (Google) of possible interest
To: <ml-seminar at cs.cmu.edu>


Hi all,

of possible interest: Vahab Mirrokni (Google) will speak tomorrow (Friday
May 23) in GHC 6115 from 1:30-2pm. (This got arranged at the very last
minute, so the last minute advertisement) Talk info below.

*Title: **ML Efficiency for Large Models: From Data Efficiency to Faster
Transformers*

Abstract: Scaling large models efficiently for faster training and
inference is a fundamental challenge. In this talk, we present a number of
algorithmic challenges and potential solutions from theory to practice.
First, we discuss data efficiency and model efficiency problems that can be
formalized as a subset selection problem. For model efficiency, we present
sequential attention for feature selection and sparsification[ICLR'23,
arxiv]. For data efficiency, we present a sensitivity sampling technique
for improved quality and efficiency of the models[ICML'24]. Furthermore, we
discuss the intrinsic quadratic complexity of attention models as well as
token generation. We first discuss HyperAttention; a technique to develop
linear-time attention algorithms under mild assumptions[ICLR'24]. We then
present PolySketchFormer, a technique to bypass the hardness results of
achieving sub-quadratic attention by applying sketching of polynomial
functions[ICML'24]. We also show how to address the complexity of token
generation via clustering techniques[arxiv]. Finally, I will discuss
Titans, which is a family of architectures based on a new neural long-term
memory module that learns to memorize a historical context and helps an
attention attend to the current context while utilizing long past
information.

*Bio*: Vahab Mirrokni is a Google Fellow and VP at Google Research, and now
Gemini data area lead. He also leads the algorithm and optimization
research groups at Google. These research teams include:  market
algorithms,  large-scale graph mining, and large-scale optimization.
Previously he was a distinguished scientist and senior research director at
Google. He received his PhD from MIT in 2005 and his B.Sc. from Sharif
University of Technology in 2001. He joined Google Research in 2008, after
research positions at Microsoft Research, MIT and Amazon. He is the
co-winner of best paper awards at KDD, ACM EC, and SODA. His research areas
include algorithms, distributed and stochastic optimization, and
computational economics. Recently he has been working on various
algorithmic problems in machine learning, online optimization and mechanism
design, and large-scale graph-based learning.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://redfish.com/pipermail/friam_redfish.com/attachments/20250522/57b6eebd/attachment.html>