Google Research 推出了“嵌套学习”（Nested Learning），这是一种新颖的机器学习范式，旨在解决持续学习中的灾难性遗忘问题。该方法将模型视为相互关联的优化问题，使它们能够在不丧失先前任务熟练度的情况下获取新知识。一个名为“Hope”的概念验证架构已通过该范式在语言建模和长上下文记忆管理方面展示了卓越的性能。OpenAI 还发布了关于元学习算法的研究，包括 Reptile，该算法专注于学习如何高效地为新任务学习，以及一种分层强化学习算法，通过将复杂问题分解为高级动作来加快任务完成速度。 AI
排序理由 该集群包含来自主要 AI 实验室的关于新机器学习范式和算法的多个研究论文和博客文章。
AI 生成摘要 · Google Gemini · 来自 13 个来源。 我们如何撰写摘要 →
Algorithms & Theory
Lilian Weng works on Applied AI Research at OpenAI.
We’ve developed a simple meta-learning algorithm called Reptile which works by repeatedly sampling a task, performing stochastic gradient descent on it, and updating the initial parameters towards the final parameters learned on that task. Reptile is the application of the Shorte…
We’ve developed a hierarchical reinforcement learning algorithm that learns high-level actions useful for solving a range of tasks, allowing fast solving of tasks requiring thousands of timesteps. Our algorithm, when applied to a set of navigation problems, discovers a set of hig…
<p>Here comes the Part 3 on learning with not enough data (Previous: <a href="https://lilianweng.github.io/posts/2021-12-05-semi-supervised/">Part 1</a> and <a href="https://lilianweng.github.io/posts/2022-02-20-active-learning/">Part 2</a>). Let’s consider two approaches for gen…
<!-- The performance of supervised learning tasks improves with more high-quality labels available. However, it is expensive to collect a large number of labeled samples. Active learning is one paradigm to deal with not enough labeled data, when there are resources for labeling m…
<!-- The performance of supervised learning tasks improves with more high-quality labels available. However, it is expensive to collect a large number of labeled samples. There are several paradigms in machine learning to deal with the scenario when the labels are scarce. Semi-su…
<!-- Meta-learning, also known as "learning to learn", intends to design models that can learn new skills or adapt to new environments rapidly with a few training examples. There are three common approaches: 1) learn an efficient distance metric (metric-based); 2) use (recurrent)…
arXiv:2602.12469v2 Announce Type: replace Abstract: Deep ensemble methods often improve predictive performance, yet they suffer from three practical limitations: redundancy among base models that inflates computational cost and degrades conditioning, unstable weighting under mult…
Examining the broad strokes of NLP progress and comparing between models