Invited Vision Talk · Efficiency
Rahim Entezari
Starting something new ... (ex-StabilityAI, ex-Wayve), UK / Austria
AXIOM 2026
Foundations of Efficient Deep Learning
Towards predictive principles for efficient AI connecting deep learning theory, scaling laws, and efficiency
December 13, 2026
Room: Salle 252 · Location: Palais des Congrès de Paris
About the Workshop
Recent advances in deep learning theory suggest that machine learning is gradually evolving from an empirical discipline into a predictive science. Scaling laws, optimization theory, learning dynamics, and representation learning increasingly explain why modern deep learning systems work. Yet these advances have had limited impact on one of today's most pressing challenges: building AI systems that are efficient under realistic constraints of compute, memory, energy, communication, and data. Current efficiency techniques, such as pruning, quantization, sparse computation, adaptive inference, and efficient architecture, remain largely driven by empirical investigation. Conversely, many theoretical advances explain observations only after the fact, rather than predicting which algorithms or architectures will be most effective before expensive experimentation.
The AXIOM Workshop brings together researchers from deep learning theory, machine learning systems, optimization, and efficient AI to explore a central question:
Can we develop predictive principles that guide the design of efficient AI systems?
Our goal is to bridge theory and practice by identifying the mathematical principles underlying efficient learning and by defining the next generation of research challenges for efficient deep learning. The workshop features invited vision talks, contributed papers, posters, panel discussions, and a community-driven Grand Challenges initiative that will collectively shape a research agenda for the foundations of efficient AI.
Invited Speakers
Invited Vision Talk · Efficiency
Katharina Eggensperger
Lamarr Institute and TU Dortmund, Germany
Invited Vision Talk · Efficiency
Michael Kamp
Lamarr Institute and TU Dortmund, Germany
Invited Vision Talk · Scaling Laws
Florian E. Dorner
MPI-IS / ETH Zürich, Germany / Switzerland
Invited Vision Talk · Theory
Bruno Loureiro
CNRS & École Normale Supérieure, France
Invited Vision Talk · Theory
Alexander van Meegen
RWTH Aachen, Germany
Invited Vision Talk · Theory
Hannah Pinson
Eindhoven University of Technology (TU/e), Netherlands
Call For Papers
Topics of Interest
We welcome submissions addressing theoretical, algorithmic, and systems aspects of efficient deep learning, including but not limited to:
- Predictive Theory for Efficient Learning: Scaling laws for efficient models, compute-optimal training and inference, predicting capability under resource constraints, optimization and learning dynamics, generalization under limited compute or data, phase transitions in efficient learning;
- Sparsity, Compression, and Model Structure: Foundations of pruning and quantization, sparse and modular neural networks, lottery tickets and subnetworks, adaptive computation, neural architecture design, representation learning for efficiency;
- Efficient Foundation Models: Efficient LLMs and multimodal models, efficient reasoning and adaptive inference, mixture-of-experts and modular architectures, test-time adaptation, memory-efficient training and inference, distillation and compression;
- Foundations and Future Directions: Theoretical limits of efficient AI, new efficiency metrics and benchmarks, predictive models of training dynamics, interpretability of efficient models, mathematical foundations of efficient deep learning, emerging theoretical paradigms for efficient AI.
Interdisciplinary work connecting theory, algorithms, systems, and hardware is particularly encouraged.
Paper Submission
We invite submissions presenting original research, preliminary results, novel ideas, and emerging research directions that advance the theoretical and practical foundations of efficient deep learning. We particularly encourage work that connects deep learning theory with efficient machine learning, including studies that improve our understanding of resource-efficient learning, reveal new theoretical principles, or bridge the gap between mathematical foundations and practical AI systems.
We invite short paper submissions (4 pages, excluding references) which follow the NeurIPS workshop formatting guidelines. This workshop is non-archival. Outstanding submissions will be invited for a 15-minute oral presentation. All accepted papers will be presented during the poster session and published on the workshop website. A Best Paper Award will be presented during the workshop.
Grand Challenges Track
Beyond research papers, AXIOM introduces a Grand Challenges track designed to identify the most important open questions for the future of efficient AI. Instead of reporting completed research, Grand Challenges submissions should articulate important unanswered questions, theoretical gaps, surprising empirical observations, or future research opportunities. We particularly encourage contributions on:
- Open theoretical questions motivated by efficient AI;
- Efficiency methods lacking theoretical explanations;
- Contradictions between theory and practice;
- Missing benchmarks, evaluation methodologies, or efficiency metrics.
Accepted submissions will be published on the workshop website and synthesized into a community report that will serve as the starting point for an interactive discussion session during the workshop. The insights from this discussion will contribute to a community position paper outlining a research agenda for predictive and efficient AI.
Grand Challenges submissions should consist of a one-page abstract plus references, with no appendix. They will undergo a light review and accepted submissions will be published on the workshop webpage.
Important Dates
- Paper submission
- August 29, 2026 (11:59 PM UTC-0)
- Author notification
- September 29, 2026 (AoE)
- Camera-ready deadline
- November 15, 2026 (11:59 PM UTC-0)
Submission Details
Submission site: OpenReview.
Submission format instructions:
- Papers: 4 pages excluding references and appendix, double-blind;
- Grand Challenges: 1 page excluding references, no appendix, double-blind.
Please use the NeurIPS 2026 LaTeX template. All papers will undergo peer review by the Program Committee.
Following the Guidance for NeurIPS Workshop Proposals 2026, workshop submissions must not duplicate work previously published at machine learning or related conferences. Work presented at the main NeurIPS conference must not also appear in the workshop, including as part of an invited talk. All authors, reviewers, presenters, and participants are expected to follow the NeurIPS Code of Conduct.
Camera-Ready Instructions
The camera-ready deadline is November 15, 2026 (11:59 PM UTC-0). The page limits remain the same as for the original submission:
- Short papers: Maximum 4 pages, excluding references and appendix. An appendix of unlimited length is permitted.
- Grand Challenges: Maximum 1 page, excluding references. No appendix is permitted.
Please de-anonymize your camera-ready submission by restoring author names and affiliations and removing anonymization intended for double-blind review. The author list must remain identical to the author list on OpenReview.
For the camera-ready version, add or uncomment the following two lines near the beginning of your neurips_2026.tex file:
\usepackage[dblblindworkshop,final]{neurips_2026}
\workshoptitle{AXIOM Foundations of Efficient Deep Learning}The dblblindworkshop option identifies the paper as a workshop submission, while final enables final formatting, reveals author names, and activates the \begin{ack} acknowledgment environment.
Poster Presentation Guidelines
All posters must be displayed on the provided poster boards; hanging posters on walls is strictly forbidden. Poster clips will be provided. Posters should be in A1 portrait format, or 90 cm (H) × 61 cm (W) / 35.43 inches (H) × 24.02 inches (W), and can be printed via the NeurIPS printing service. Posters must be removed at the end of each poster session (morning and afternoon). Any posters left behind will be discarded.
Workshop Participation Policy
The workshop is intended to be an in-person interactive event. Accepted papers are therefore expected to be presented in person by at least one author or designated presenter. Remote oral presentations may be permitted only in exceptional circumstances (e.g., participation from another official NeurIPS location, visa or medical issues), subject to the workshop's technical constraints. If no presenter can attend in person, the paper will remain accepted but will be identified as "not presented" on the workshop website.
Program
Times shown in CET
Time
Session
Speaker(s)
Opening Remarks
Organizers
Invited Vision Talks · Efficiency
Rahim Entezari · Katharina Eggensperger · Michael Kamp
Contributed Oral Presentations I (3 talks · 10 min each + Q&A)
Speakers TBA
Poster Session 1
—
Invited Vision Talks · Scaling Laws & Theory
Florian E. Dorner · Bruno Loureiro · Alexander van Meegen · Hannah Pinson
Lunch
—
Contributed Oral Presentations II (3 talks · 10 min each + Q&A)
Speakers TBA
Grand Challenges & Research Directions · Panel Discussion
Organizers + Panelists
Closing Remarks & Best Paper Announcement
Organizers
Poster Session 2
—
Accepted Papers
Accepted Grand Challenges (7)
- Grand Challenge: Gaming the Hardware Lottery
- Grand Challenge: Measuring Severity
- Grand Challenge: On the Loss-Landscape Connectivity in Federated Learning
- Grand Challenge: Predict Task-Relative Resource Frontiers Before Deployment
- Grand Challenge: Predict Where the Next Bit Should Go, Before You Quantize
- Grand Challenge: Towards Deep-Learning Architectures That Respect Causal Principles
- Grand Challenge: Towards Financial World Modeling: An Open Efficiency Question
Accepted Oral Presentations (6)
- Estimating Optimal Hyperparameter Scaling Laws Efficiently through Power-Law Entropy Search
- Hot training errors predict cold test errors in the interpolation regime
- Learning Dynamics of Chain-of-Thought State Tracking in a Solvable Transformer Model
- MegaTabICL: Efficient Scaling of Tabular Foundation Model Pretraining
- Towards Efficient Diffusion Language Modeling via Parallel Reservoir Computing
- Understanding and Exploiting Weight Update Sparsity for Communication-Efficient Distributed RL
Accepted Poster Presentations (84)
- A Boundary-Layer Mechanism for One-Third Scaling in Online Softmax Classification
- A Closed Form Depth-Width Law for Neural Collapse in Deep Linear Networks
- A First-Order Theory of Decision Flips Under Post-Training Quantization: What It Predicts, and Where It Breaks
- ACE: An Analytical Rate Proxy for Compression-aware Optimization
- Adaptive-Depth Is a Finishing Stage
- AdaqL: Adaptive Mixed-Precision Layer-wise Quantization for Large Language Models
- Adversarial Training Cannot be Reduced to Regularization in Non-Linear Models
- Amortizing Scaling Law Construction Costs
- Back to Basics: Testing Hessian-Based Sensitivity Scores for PoT Weight-Quantization Damage
- Balancing Performance and Forgetting via Intermediate Principal Components
- Beyond Gaussian Initialization: Activation-Induced Gibbs States for Critical and Width-Efficient Networks
- BiRRM: Bidirectional Token Mixing for Recurrent Reasoning Models
- C3: Speech-Guided KV Retrieval for Efficient Long-Context SpeechLLM Inference
- CAGE-NAS: Certified Functional Descent for Efficient Model Growth
- Circular Models Trained with Equilibrium Propagation Unlock Test-Time Computations
- COALA 2: Adaptive Rank Selection for Context-Aware Low-Rank Approximation
- Component-Weighted Centroid Search for Exact Incremental BPE
- Compute-Optimal Scaling Laws for Spiking Language Models
- Contrastive Activation-Guided Zeroth-Order Preference Optimisation for Memory-Efficient LLM Alignment
- Curriculum Learning with Compositional Data
- DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation
- Delta-SSM: Exact parallelization of spiking State Space Models for Efficient Sequence Modeling
- Efficient and Fast Diffusion Language Models with Structured State Space Models
- ESPL: Efficient Block Sparse Plus Low-Rank Attention
- Exact Solutions and Saddle-to-Saddle Dynamics in Nonlinear Matrix Factorization
- Fewer Expert Transfers Without Changing Routing: Expert Eviction for MoE Offloading
- Functional Degeneracy in Neural Networks: Measurement and Pruning
- GASP: Group-regularized Adaptive Structured Pruning
- Gating Enables Long-Range Recall in Recurrent Logic Gate Networks
- HSWA: Hierarchical Sliding Window Attention
- ICICLE: In-Context Inference for Curve-Learning Extrapolation
- IrekoGPT: Turning Structured Pruning into Post-Hoc Slimmable LLMs
- Joules-to-GDP: How Much Economic Value Do AI Systems Produce per Unit of Energy?
- LeanTTA: A Backpropagation-Free, Stateless Approach to On-Device Test-Time Adaptation
- Learning Structured Sparsity Beyond SALAAD: Bridging Model Compressibility and Efficient Inference
- Linearized 2-Simplicial Attention
- LRCC: Generalizing Low-rank Compression with Conditional Computation
- Manifold-Constrained Hyper-Connections for Parameter-Efficient Finetuning
- MARCH: Content-Routed State Anchors for Efficient Recurrent Memory
- Match the Distribution, Not the Compute: Post-Training Multi-Token Prediction Heads
- Memory-Efficient Speculative Decoding with Quantized Draft KV Cache
- Mixtures of Embedding Dimension Experts
- Model Capacity Dominates Language Resourcing in Multilingual ASR Quantization
- On the Approximation of Chebyshev Polynomials with Looped ReLU Networks
- Output-Aware Block Influence: Jacobian-Lens Weighting for Depth Pruning
- Per-Symbol Exposure, Not Vocabulary Size, Governs Symbol Learnability in Transformers
- Port-Hamiltonian Neural Networks for Systems with Multiple Asymptotically Stable Equilibria
- Prism Transformer: Progressive Head Schedules for Hierarchical Attention Processing
- Probing SAM Perturbations Through Statistical Replay
- Reconstruction-Optimized Expert Pruning for Mixture-of-Experts Language Models
- Resource-Aware Joint Kernel Fusion for Ahead-of-Time Inference
- Rethinking Edge-of-Chaos Initialization for Sparsifying Activations
- RiPPL: Rigging Post-hoc Pruning Layerwise
- Router Sensitivity Under Lightweight Fine-Tuning Identifies Prunable Experts in Mixture-of-Experts Models
- S-DPQ: Block-Level Angular and Norm Matching for Data-Free Ternary Quantization
- SACQ: Structure-Aware Codebook Quantization for Sub-2-Bit Vision-Language-Action Models
- Sample-Efficiency of Kolmogorov–Arnold Networks
- Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models
- Sink vs. Diagonal Attention: Sharpened Cost Bounds and Comparison Regimes
- Sketching Attention Sinks in LLMs
- SSD: Spatially Speculative Decoding Accelerates Autoregressive Image Generation
- Stuck on "A": Diagnosing and Repairing Interface Injury in Attention-to-KDA Linearization of a 0.6B Language Model
- Surfing the Activation Wave: An Information-Theoretic and Geometric Analysis of LLM Hidden States Applied to Speculative Decoding
- SVD-Surgeon: Optimal Singular-Value Surgery for Large Language Model Compression
- Task Alignment as a Data-Level Predictor of Inter-Task Dynamics in Linear MTL
- The Price of Precision: A Calibrated Cost Model for Mixed-Precision LLM Serving
- The Spectral Anatomy of Plasticity Loss
- The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globally
- Threshold Renormalization Explains Weak-to-Strong Generalization in Diagonal Linear Networks
- ThriftAttention: Selective Mixed Precision for Long-Context FP4 Attention
- TopK-Guided: Adaptive, Budget-Aware Activation Sparsity for Efficient LLM Inference
- Towards an Expressivity-Normalized Energy-Demand Comparison of ANNs and SNNs
- Transmuting prompts into weights
- TreeProp: Training $N$-Layer Deep Networks with $\mathcal{O}(\log N)$ Parallel Time Complexity
- Trust the Mass: Forced Weights in KV-Cache Eviction
- Unsupervised Recursive Reasoners for Combinatorial Problems
- Varying-Width Vision-Transformers for Efficient Vision–Language Models
- Vision-TabPFN: An Efficient Tabular Prior for Few-Shot Vision
- Voronoi-Calibrated Order Selection for Bayesian Softmax-Gated Mixtures of Experts
- Weight Pair Encoding: Inducing A Smaller Grammar in Neural Network Weights
- What is the role of Learning Rate in a Mixture-of-Experts ResNet
- What Makes Recurrence Generalize Across Lengths?
- Why Classifier-Free Guidance Is Fragile Under Quantization: A Mechanistic Study
- xRRMs: Explorative Recurrent Reasoning Models
Organizing Team
The workshop originates from the ELLIS Mathematics and Efficiency of Deep Learning Reading Group.
Olga Saukh
Graz University of Technology, Austria
Linara Adilova
TU Dortmund and RC Trust, Germany
Bernhard Geiger
Graz University of Technology, Austria
Yedi Zhang
University College London, UK
Flavio Martinelli
EPFL Lausanne, Switzerland
Contact
For questions or further information regarding the workshop, please contact the organizing committee: axiom.neurips2026@gmail.com