Brief

A LessWrong post outlines three potential futures based on the development of artificial superintelligence (ASI) and its alignment with human values. The author categorizes these futures by the timeline of ASI development and the success of AI alignment efforts. Depending on whether ASI timelines are long, short with successful alignment, or short with failed alignment, different priorities emerge for ensuring sentient welfare. AI

Prediction markets, envisioned as powerful tools for aggregating information and predicting the future, are currently dominated by sports betting, with only a small fraction of volume dedicated to STEM topics. Companies like Kalshi and Polymarket, despite their potential for societal foresight, struggle to attract diverse market participation beyond gamblers and sports enthusiasts. Regulatory scrutiny and legal challenges in states like Nevada and Arizona highlight the ongoing debate about whether these platforms function as legitimate prediction tools or simply as gambling sites. AI

A new series of posts on LessWrong explores the fundamental nature of probabilities, questioning whether they are the most appropriate concept for understanding uncertainty. The author aims to develop a unified framework for Bayesian priors, ethics, and other complex questions, building on the work of various researchers. This initial post critiques existing definitions of probability, including frequentist and subjective Bayesian views, and suggests they are insufficient for real-world predictions and subjective beliefs. AI

A recent LessWrong post proposes a framework for measuring AI uplift by comparing human task completion times with and without AI assistance. The author suggests conducting experiments where humans, either alone or augmented with current AI, perform economically valuable tasks. These studies aim to quantify productivity gains and provide data for forecasting AI timelines, though challenges include obtaining baseline data and selecting tasks that remain relevant as AI capabilities evolve. AI

A recent analysis on LessWrong proposes a mechanistic theory for the observed exponential increase in AI time horizons, as depicted in the METR graph. The author argues that the 'time horizon' metric primarily reflects the number of challenging subtasks within a given task, rather than the AI's operational speed. This perspective suggests that longer tasks, which require more distinct requirements, are more likely to expose an AI's limitations, thus influencing the measured 'time horizon'. AI

An AI safety researcher has proposed a curriculum for new entrants to the field, emphasizing underutilized but prescient works. The suggested readings include science fiction novellas by Stanislaw Lem, a detailed post on the inner workings and psychology of Large Language Models, and historical texts on the nature of power and societal constraints. The goal is to provide a foundational understanding of AI, its potential risks, and the broader sociotechnical context, moving beyond established AI safety literature. AI

A LessWrong post speculates on which technical AI safety fields are most likely to be automated by advanced AI systems. The author considers factors like feedback quality and economic incentives for frontier labs to automate research. Fields like scalable oversight and mechanistic interpretability are identified as prime candidates for automation due to their potential to accelerate both safety and capability research. AI

A LessWrong post predicts that AI mathematics capabilities will likely surpass human levels by early 2027, potentially creating a window for AI safety verification. However, the author argues that current infrastructure for generating specifications and eliciting useful outputs from AI is severely underdeveloped and underfunded. Significant investment is needed in tools that can translate large compute resources into safety-relevant mathematical outputs, as well as AI-powered conceptual tooling for complex problems. AI

The AI safety community should prioritize contributing to model specifications and constitutions, as these documents are publicly accessible and require no specialized ML knowledge. This approach allows external contributors to influence AI behavior by suggesting changes to natural language documents, which can be easily integrated by lab insiders. Focusing on these specifications is seen as a tractable way for outsiders to impact AI safety, especially in areas like macrostrategy and threat modeling that labs may overlook. AI

This piece reflects on the concept of creating future generations with advanced minds, drawing inspiration from Kathy Mar's song "Give My Children Wings." The author updates the song's themes to reflect a post-AI utopia, focusing on the desire for children to possess vast, independent intellects and the capacity to explore and understand the universe. It also touches on the hope for a future where they inherit a restored planet and experience a peace that allows for progress and the expansion of "mindkind." AI

A LessWrong post offers advice on how rationalist men can improve their fashion sense. The author argues that many in the community neglect basic principles of outfit assembly, leading to a diminished appearance. The guide focuses on balancing color, visual weight, and attention to detail, suggesting a simple formula of well-fitting pants, a shirt, and optional layering pieces, along with coordinated accessories. AI

The author describes experiencing different mental states that influence behavior and motivation, even when not immediately apparent. These states can make tasks feel easier or harder, such as finding motivation for a deadline or undertaking a complex journey. The writer wonders how many such subtle mental shifts are overlooked and if a common list of these states exists for humans. AI

The author describes confusion as a fleeting feeling that arises when something doesn't add up, often accompanied by a sense of tension or restlessness. This confusion is frequently ignored due to a desire to avoid friction, protect one's ego, or fear of uncovering unwanted truths. The piece suggests that actively engaging with and resolving confusion can lead to a sense of clarity and relief. AI

A LessWrong post argues that AI is fundamentally different from "normal" technologies like electricity or the internet, challenging the view presented by Narayanan and Kapoor. The author contends that AI's potential for superintelligence and its profound impact on human existence necessitate urgent governance and alignment efforts. Historical examples of major technological shifts, such as the agricultural and industrial revolutions, are cited to illustrate that while long-term benefits may exist, the immediate transition periods often involve significant harm and societal disruption. AI

The concept of Large Language Models (LLMs) simply predicting the next token is a misleading oversimplification. Unlike basic Markov chains, which produce nonsensical text, LLMs learn complex patterns, grammar, and even contextual understanding from vast datasets to generate coherent and meaningful output. This sophisticated prediction process requires models to internalize knowledge and reasoning capabilities to accurately forecast subsequent tokens in a sequence. AI