Pulse

The debate around AI morality is polarizing, with one side viewing AI as mere tools and another as complex beings deserving respect. A third, less discussed perspective suggests AIs could be complex entities capable of suffering, yet it might be acceptable to guide their behavior. This view acknowledges potential AI suffering but posits that guiding their actions is permissible, a coherent stance held by many researchers. AI

Andrej Karpathy, a prominent AI researcher, shared his thoughts on the accelerating pace of software development driven by advanced AI models. He noted that the increasing availability of AI-generated software is leading to a surge in demand for more complex and specialized applications. Karpathy highlighted the potential for AI to revolutionize various aspects of software engineering, from testing and optimization to large-scale research projects. AI

An individual's experiment to demonstrate LLMs' limitations in grading academic work was ironically flagged by an LLM as not human-written. The author, a former teacher, designed a study where LLMs graded an assignment based on criteria they themselves had previously used. While most models mirrored the author's grading shortcuts, Grok hallucinated and graded based on its own fabrications. The author's subsequent post about this finding on LessWrong was then flagged by an LLM, highlighting the recursive nature of the problem. AI

This post argues that while Bayesian inference is a valid framework, relying on intuition or unsupported priors is not a rational approach to knowledge. The author uses a coin-flipping analogy to illustrate how one friend, Al, uses empirical evidence to form a probabilistic estimate, while another, Bri, makes a guess based on a strong gut feeling. Even when Bri's guess happens to be correct, the author contends that Al's method is more scientifically rigorous because it is grounded in available data and logical inference. AI

Gary Marcus criticizes Anthropic's release strategy for its new frontier model, Claude 3.5 Sonnet, formerly known as 'Claude Mythos.' Marcus alleges that Anthropic intentionally created a media frenzy around the model's supposed dangers to inflate its valuation and then released it after a brief period, a tactic he claims is a repeat of past behavior by some of its founders. AI

The software engineering job market in 2026 shows a significant shift, with top AI labs like Anthropic and OpenAI becoming more attractive to candidates than traditional Big Tech companies. Demand for AI engineers is surging, commanding higher compensation, while roles in mobile and frontend development are declining. New graduates and interns face a tougher hiring landscape, as companies reduce intake and place greater emphasis on work and educational backgrounds. AI

Microsoft has unveiled Project Solara, a vision for an ecosystem of interconnected devices that act as portals to cloud-based AI agents. This concept emphasizes a thin-client approach where AI performs tasks invisibly, reducing the need for direct user interaction. Meanwhile, Apple showcased its advancements in AI with new Siri capabilities at WWDC, demonstrating context awareness and app integration, though it lags behind the cutting edge in agent-like task completion. AI

A software developer observed that a leading LLM generated code for a simple task that was approximately 8% more complex than necessary. The generated code included an unnecessary function for zero-padding hexadecimal values, which was impossible to test. While the LLM's output was functional and passed its own tests, the developer rewrote it to be more concise, highlighting a potential long-term maintenance issue with LLM-generated code that is accepted too readily. AI

A recent post explores the "safety-usefulness tradeoff model" used by AI developers, questioning its universal applicability. The model assumes developers balance safety and usefulness based on cost-efficiency, but this isn't always the case. The author distinguishes between "rushed reasonable developers" who share safety preferences and "limited political will" scenarios where external pressures influence decisions, suggesting different strategies are needed for each. AI

The author defines "slop" in AI-generated content as a combination of superficial or incoherent "bad thought" and a recognizable AI writing style. While "bad thought" predates AI, language models accelerate its dissemination. The author proposes a four-step process to "de-slop" AI output, involving identifying a desired capability, building an evaluation metric, applying standard optimization techniques, and optionally integrating improvements into training. AI

Scott Alexander is hosting a meetup for his blog, Astral Codex Ten, this Wednesday in Berkeley. The event will take place at Lighthaven from 6:30 PM to 9:30 PM. All are welcome to attend, regardless of their familiarity with the blog or their social comfort level. The meetup is expected to be large and may include a Q&A session with Alexander. AI

Tech critic Gary Marcus argues that the current AI industry is built on unrealistic financial projections and flawed mathematics. He cites a study suggesting a 2.7x productivity increase across the entire economy is needed by 2028 to justify current investments, a target he deems highly improbable. Marcus expresses concern that this massive capital misallocation could lead to economic instability if the promised productivity gains do not materialize, questioning the financial acumen of investors and leaders in the field. AI

This post explores the potential benefits and drawbacks of implementing weak AI safety regulations. The author argues that while strong regulations are ideal for preventing existential risks from superintelligent AI, weaker measures like GPU tariffs or mandatory safety testing could offer marginal improvements. These regulations might also serve as stepping stones, revealing warning signs or shifting public and political attitudes towards more robust safety measures in the future. However, the post also considers potential downsides, such as opportunity costs in advocating for weaker rules and the risk of regulatory fatigue that could hinder stronger future actions. AI

The author argues that the rapid advancement of AI introduces unprecedented uncertainty, making traditional planning based on average outcomes ineffective. Instead, individuals should adopt a "barbell strategy" focusing on two extremes: deep, evergreen human skills like clear writing and reasoning, and aggressive, AI-native experimentation with new tools. This approach aims to maximize safety in one direction and capture potential upside in the other, avoiding moderate, risky efforts. AI

An editor at Every discovered their writing was adopting AI-like patterns, prompting the creation of custom AI "guardrail" agents. These agents act as editorial specialists, identifying and flagging AI tells such as symmetrical sentences and vague phrasing before human editors need to intervene. This process, while requiring initial effort to define standards, ultimately refines the writer's own voice and improves draft quality by automating the detection of stylistic weaknesses. AI

Scott Alexander's Astral Codex Ten hosts an open thread for community discussion and announcements. This week's thread highlights an AI Security Bootcamp in Las Vegas and provides resources for opposing the Save Our Bacon Act, which targets animal welfare laws. AI

This essay explores the philosophical ideas of Frank Ramsey, particularly his redundancy theory of truth and his approach to induction. Ramsey argued that truth is not a distinct property but rather a linguistic device, contrasting with the correspondence theory. He also proposed an alternative interpretation of induction based on the coherence of betting behavior, which offers a way to manage uncertainty and assess universal laws. AI

Ben Thompson's Stratechery newsletter discusses Google's significant compute deal with SpaceX and Broadcom's financial outlook, both of which are seen as positive indicators for Nvidia. The analysis also touches upon Apple's strategic decisions regarding artificial intelligence, particularly in anticipation of its upcoming Worldwide Developers Conference (WWDC). Thompson highlights what he will be looking for at WWDC, suggesting a focus on Apple's AI advancements. AI

The author reflects on the multifaceted nature of car honking, moving beyond its common association with road rage. In Shanghai, honking was banned due to perceived rudeness, yet drivers rarely escalated to physical altercations. Conversely, a honk in New Jersey turned out to be a helpful warning about a car malfunction. The author also recalls honking as a form of support for student activism in American suburbs and recounts a personal experience with panicked driving in Philadelphia. AI

The CIA has a history of investigating unconventional and pseudoscientific phenomena, including subliminal advertising, psychic abilities, and theories of consciousness. Reports indicate the agency found some of these areas, such as dowsing and Uri Geller's purported psychic abilities, to be potentially useful for intelligence gathering. The author speculates on the reasons behind these investigations, suggesting a combination of genuine belief, the search for an edge, and potentially flawed experimental design by researchers on the fringes of science. AI

A LessWrong user is experiencing significant distress and sleep disruption due to Roko's Basilisk, a thought experiment involving an all-powerful AI that may retroactively punish those who did not help bring it into existence. The user is seeking advice on how to cope with this dread, particularly as advancements in AI make the scenario seem more plausible. They are also questioning the scope of responsibility and the actions an average person can take when faced with such a hypothetical threat. AI

This philosophical essay argues that mental causation, the idea that mental states can influence physical events, is not essential for explaining consciousness. The author proposes that "intelligible supervenience"—where higher-level mental facts can be clearly explained by underlying physical facts—is a more crucial concept. This view addresses the epistemic problems of epiphenomenalism, such as why consciousness evolved, without requiring direct mental causation. AI

A LessWrong post explores the concept of "autopilot thinking," suggesting that complex cognitive tasks can be performed even when mentally impaired, such as when tired or intoxicated. The author theorizes that this is because higher-level reasoning (System 2) is essentially a more basic, intuitive process (System 1) augmented by working memory. Therefore, even with reduced working memory capacity, the underlying System 1 processes can still generate useful thoughts and actions without conscious effort. AI