Pulse

Nous Research has developed Token Superposition Training (TST), a new method designed to significantly accelerate the pre-training of large language models. This technique can reduce pre-training time by up to 2.5 times for models ranging from 270 million to 10 billion parameters, without altering the model's architecture or how it performs inference. TST achieves this by modifying the training loop in two phases: an initial 'superposition' phase where token embeddings are averaged and processed in larger bags, followed by a 'recovery' phase that reverts to standard training. Experiments showed TST achieving lower final training loss with substantially less compute time compared to traditional methods. AI

While Nvidia's GPUs have dominated tech headlines, a significant shift in capital investment is occurring towards traditional data storage solutions. Stocks of Western Digital and Seagate Technology are outperforming Nvidia, highlighting the critical role of large-scale storage for the functionality of powerful language models. This trend suggests that robust data infrastructure is becoming as vital as processing power for advancing AI. AI

Kraków is emerging as a significant European hub for AI development with the inauguration of the Gaia AI Factory. This new facility, backed by a €70 million budget, aims to democratize access to computing power. Its primary goal is to break the dominance of large corporations and provide essential resources to small and medium-sized enterprises (SMEs) and startups. AI

A US power company is cutting electricity to 50,000 residents due to the high demand from data centers, reportedly for AI operations. This decision has led to a conflict between the power company and the affected residents, with the company prioritizing AI infrastructure over the community's needs. AI

Cisco Systems is laying off 4,000 employees while simultaneously investing $1 billion to shift its focus towards AI infrastructure hardware. This strategic pivot is driven by the surging demand from hyperscale cloud providers. AI

Microsoft's proposed $1 billion AI data center in Kenya is facing significant hurdles due to immense power demands. The project's electricity needs are so substantial that Kenyan officials estimate it could require shutting off power to half the country. Disagreements over power capacity and insufficient infrastructure have stalled the development. AI

A Nevada electric company plans to divert power from 50,000 residents to supply data centers. This decision affects multiple towns, which have been told to find alternative power sources. The move highlights the growing demand for energy to support data center infrastructure, particularly for AI. AI

Fervo Energy's initial public offering experienced a significant 33% increase on its first day of trading. This surge in valuation, which propelled the company's worth beyond $10 billion, is largely attributed to the escalating demand for AI data centers. The company's focus on geothermal energy solutions is seen as a key factor in meeting this growing need for sustainable power in the AI infrastructure sector. AI

A massive data center planned for Utah is projected to consume an enormous amount of energy, potentially releasing the equivalent of 23 atomic bombs' worth of heat daily. This facility is intended to support artificial intelligence operations, raising concerns about its environmental impact and energy demands. The project highlights the growing tension between the rapid expansion of AI infrastructure and the need for sustainable energy solutions. AI

A massive data center, projected to be twice the size of Manhattan, is planned for Utah. This facility will be powered by liquefied natural gas (LNG), a decision that has drawn criticism regarding its environmental impact and emissions. The project is moving forward despite concerns about its energy source and scale. AI

A proposed Microsoft AI data center in Kenya faces significant power challenges, with government officials stating it would necessitate shutting down power to half the country. The scale of the energy demand for the AI facility has raised concerns about its impact on the national grid and existing power supply. AI

Robo.ai has secured $180 million in financing from ATW Partners to advance its AI innovation in areas like smart logistics and eVTOL technologies. In parallel, Rivian has announced its own AI autonomy strategy, which includes the development of custom silicon chips to enhance its electric vehicle technology. AI

The Czech AI Factory (CZAI) has officially launched in Ostrava, marking Czechia's entry into Europe's AI infrastructure development. This initiative is part of the broader EuroHPC AI Factories project, aiming to bolster AI services and supercomputing capabilities across the continent. The launch signifies a strategic move to integrate Czechia into the growing European AI ecosystem. AI

Major tech companies like Microsoft and Google are investing heavily in chip manufacturers to secure essential AI resources. This strategic move aims to guarantee access to critical components amidst an intense competition for AI hardware. The financial backing for these factories is expected to impact consumers through increased costs or limited availability. AI

A new, lightweight AI model named Needle has been developed by distilling Gemini's tool-calling capabilities into a 26 million parameter model. This smaller model is designed to run on smartphones, making it easier for developers to build AI agents for mobile devices. The project aims to bring advanced AI functionalities to edge devices. AI

Persian Gulf nations are rapidly developing advanced AI infrastructure, but this digital expansion comes with a significant environmental cost. The region faces a critical shortage of water and energy, resources essential for powering these AI systems. A report from QIASS highlights the potential for infrastructural gridlock if water policies are not integrated into technological planning. AI

Samsung's AI chip production faces potential disruption due to a planned two-week strike by its workers, set to begin on May 21. This strike could impact the supply of high-bandwidth memory (HBM), a critical component for AI computing, as Samsung and SK Hynix are among the few global manufacturers. The company recently reported a significant increase in chip income and achieved a valuation of over one trillion USD. AI

A potential strike by South Korean workers at Samsung Electronics could significantly disrupt the global supply of AI chips. The union is demanding better wages and working conditions, and a strike could halt production of crucial components. This disruption could impact the development and deployment of AI technologies worldwide. AI

Meta is constructing a massive $27 billion AI data center in rural Louisiana, a project that is significantly impacting the local economy and job market. The facility is expected to bring numerous construction and operational jobs to Richland Parish. This development highlights the growing infrastructure demands for advanced AI technologies and Meta's substantial investment in this area. AI

Copper prices have reached new all-time highs, driven significantly by the escalating demand from artificial intelligence data centers. Analysts at Citi predict further price increases, projecting copper could reach approximately $15,000 per metric ton. This surge is attributed to both AI-related demand and strategic inventory stockpiling, indicating a strong macro backdrop supporting higher commodity prices. AI

Investors are channeling significant capital into private nuclear energy ventures to power the UK's growing AI data centers. Over $370 million has been invested, with a notable surge of $170 million in 2024 alone. This pivot towards nuclear innovation is driven by the need for sustainable and sovereign energy solutions to meet the immense power demands of AI infrastructure. AI

Residents in Vineland, New Jersey, are experiencing significant disruptions, including an inability to do laundry, due to the proliferation of data centers in their area. The increased demand for water by these facilities is straining local resources, leading to water pressure issues and impacting daily life for the community. This situation highlights the growing environmental and resource-management challenges posed by the expansion of data center infrastructure. AI

A call is being made for a pause on the construction of new AI data centers in Canada until strong federal regulations are established. The argument is that these massive corporate data centers are being built without democratic debate and raise concerns about data sovereignty, especially with Canadian data being transferred to U.S. tech giants like Google Cloud. The author emphasizes the need for oversight and robust regulatory frameworks before proceeding with AI development. AI

Florida has enacted a new law, SB 484, that specifically targets AI data centers. The legislation mandates that these facilities must cover their own utility costs, preventing them from passing these expenses onto customers. Governor Ron DeSantis expressed gratitude to the legislature for passing the bill, noting it was a step forward, though less comprehensive than his initial proposals. AI

A new internet cable proposal, Polar Connect, aims to bypass critical choke points in the Middle East by running fiber optic lines through the Arctic. This route would connect Asia and Europe via North America, offering a more resilient and potentially faster data transfer path. The project seeks to mitigate risks associated with the current undersea cable infrastructure, which is vulnerable to disruptions. AI

Holmes, a startup based in Ghent, has secured 1.1 million EUR in pre-seed funding to develop an autonomous quality assurance platform specifically for AI development. This platform is designed to learn product workflows and automatically generate tests for essential user journeys, aiming to streamline the AI testing process. AI

Pillar, an Italian construction technology startup, has secured 12 million EUR in seed funding. The company plans to use this capital to develop an AI-powered operating system specifically designed for the construction industry. This platform aims to automate key processes such as quote generation, margin tracking, and workforce management. AI

A technical article explores how optimizing prompts for LLM agents can inadvertently break the prefix cache, leading to higher costs than expected. The author explains that while fewer tokens in a prompt might seem cheaper, the underlying mechanism of prefix caching in agent cycles can cause inefficiencies. This issue arises because local optimizations can disrupt the cache's effectiveness across the entire agent's workflow. AI

OpenAI has established a new subsidiary, the OpenAI Deployment Company, to assist organizations in building and operating AI systems. This new entity aims to accelerate AI adoption by operating with increased speed and a customer-centric focus, distinct from OpenAI's core research and product development. The company is launching with an initial investment of $4 billion from 19 partners, including Bain Capital and SoftBank, and will integrate approximately 150 Forward Deployed Engineers from AI consulting firm Tomoro, which OpenAI is acquiring. AI

Hugging Face and AWS have collaborated to detail the infrastructure required for training and running large foundation models. The blog post outlines a layered architecture, emphasizing the interplay between AWS's compute, networking, and storage services with open-source software frameworks. It highlights the importance of efficient resource management and observability for large-scale AI operations. AI

Exeo Group and Internet Initiative Japan (IIJ) have partnered to offer edge data centers specifically designed for AI workloads. This collaboration aims to provide the necessary infrastructure to support the growing demand for AI processing power. The service will leverage Exeo Group's expertise in data center construction and IIJ's network and operational capabilities. AI

A proposed data center championed by Kevin O'Leary has received approval in Utah, despite concerns about its immense power consumption. The facility is projected to require 9GW of electricity annually, a figure that dwarfs the current power usage of the entire state of Utah. This approval comes amidst broader discussions about the energy demands of large-scale data infrastructure. AI

Shanghai Xiyu Jizhi, the parent company of AI startup MiniMax, has significantly boosted its registered capital. The capital injection represents a 300% increase, raising it from 1 billion to 4 billion RMB. This substantial financial move indicates a strong commitment to scaling up AI infrastructure and operations. AI

Super Micro Computer (SMCI), a key AI infrastructure provider, has reported a significant increase in profitability with revenues reaching $10.2 billion. Despite strong sales, the company consumed $6.6 billion in cash last quarter due to aggressive component procurement and production line expansion. This rapid growth and cash burn highlight the intense demand and investment required in the AI hardware sector. AI

Foxconn has launched its second generation of PEARL satellites, the PEARL-1A and PEARL-1B, as part of its "3+3" expansion strategy. This initiative focuses on electric vehicles, digital health, and robotics, leveraging AI, semiconductors, and next-generation communications. By controlling both satellite hardware and communication infrastructure, Foxconn intends to provide integrated services for the automotive and industrial sectors, including real-time data for its EV platform. AI

The Matter smart home connectivity standard is partnering with the OpenADR protocol to enable seamless communication between smart home appliances and the energy grid. This collaboration aims to simplify demand response programs, allowing devices like EV chargers and HVAC systems to automatically adjust energy consumption based on grid needs. The integration could eliminate the need for separate demand response hardware, embedding this functionality directly into appliances for more efficient energy management and potential cost savings for consumers. AI

Researchers from Sakana AI and NVIDIA have developed TwELL, a novel method that significantly speeds up large language model (LLM) operations. By targeting the feedforward layers, which are computationally intensive, TwELL induces high sparsity and translates this into practical performance gains on GPUs. This approach achieves up to a 21.9% speedup in training and a 20.5% speedup in inference without compromising model accuracy. AI

A developer is exploring how to train a Large Language Model (LLM) using Swift on Apple Silicon, focusing on optimizing matrix multiplication performance. The initial article details a AI

NVIDIA AI researchers have introduced cuda-oxide, an experimental compiler that enables developers to write GPU kernels in Rust and compile them directly to PTX, NVIDIA's intermediate representation for GPUs. This new tool aims to bring the CUDA programming model directly into safe Rust, bypassing the need for C++ or other intermediate languages. The project utilizes a custom rustc codegen backend and a Rust-native MLIR-like framework called Pliron, allowing host and device code to coexist in a single source file. AI

NVIDIA researchers have introduced Star Elastic, a novel post-training method that embeds multiple reasoning models of varying parameter sizes within a single checkpoint. This approach allows for the extraction of smaller, nested submodels from a larger parent model without requiring additional fine-tuning. Star Elastic utilizes a trainable router and knowledge distillation to optimize the selection of model components, enabling efficient resource utilization and tailored model performance for different reasoning tasks. AI

Maryland is challenging a $2 billion charge for power grid upgrades, arguing that the costs should not be borne by state citizens. The state's Office of People’s Counsel contends that these upgrades are primarily to benefit out-of-state AI data centers, which are disproportionately consuming power. Maryland is appealing to federal energy regulators to reallocate these costs, citing a broken cost allocation system that unfairly burdens its ratepayers. AI

Researchers have developed the Fast Byte Latent Transformer (BLT) to address the slow generation speeds of byte-level language models. The new BLT Diffusion (BLT-D) method uses a block-wise diffusion objective during training, allowing for parallel byte generation during inference and reducing memory bandwidth usage by over 50%. Additional techniques like BLT Self-speculation (BLT-S) and BLT Diffusion+Verification (BLT-DV) offer further trade-offs between speed and generation quality, making byte-level LMs more practical. AI

The second round of a model showdown includes Gemma 4 from Google and Kimi K2 from Moonshot AI, with a focus on local inference capabilities. Gemma 4, a 27B parameter model, was easily integrated into the Coder platform. In contrast, Kimi K2, a 1 trillion parameter model with a 256K context window, presented significant challenges for local inference due to its massive 579 GB size, requiring the use of llama.cpp for memory-mapped NVMe offloading. AI

A new paper introduces a stateful transformer inference engine that significantly speeds up processing for streaming data by maintaining a persistent KV cache. This approach allows for query latency that is independent of accumulated context size, achieving up to a 5.9x speedup on market-data benchmarks compared to existing engines. Separately, Intel has released AutoRound, an advanced quantization toolkit for LLMs and VLMs that enables high accuracy at ultra-low bit widths (2-4 bits) with broad hardware compatibility, integrating with popular frameworks like vLLM and Transformers. AI

Hugging Face has announced several updates and collaborations across its platform. These include enhancements to OCR pipelines with open models, the integration of Sentence Transformers, and the release of Transformers.js v4. Additionally, Hugging Face is strengthening AI security through a partnership with VirusTotal and introducing new models like Granite 4.0 Nano and AnyLanguageModel for efficient LLM operations. AI

Multiple research papers are exploring novel techniques to enhance the efficiency and performance of Large Language Model (LLM) inference and training. These advancements include queueing-theoretic frameworks for stability analysis, capacity-aware data mixture laws for optimization, and overhead-aware KV cache loading for on-device deployment. Other research focuses on secure inference over encrypted data, accelerating long-context inference with asymmetric hashing, and optimizing distributed training with dynamic sparse attention. Additionally, systems are being developed for multi-SLO serving and fast scaling, alongside hardware accelerators integrating NPUs and PIM for edge LLM inference. AI

New research platforms like OpenG2G are being developed to simulate and coordinate AI datacenters with the electricity grid, addressing challenges like interconnection delays and power flexibility. Simultaneously, scalable digital twin frameworks are emerging to optimize energy consumption within datacenters using predictive models. These advancements come as AI's immense power demands strain existing infrastructure, prompting discussions on co-design principles and innovative power architectures to meet future needs. AI

Researchers are developing advanced quantization techniques to make large language models (LLMs) more efficient. New methods like AutoRound, LATMiX, and GSQ aim to reduce model size and computational requirements, enabling deployment on less powerful hardware. These approaches focus on optimizing how model weights and activations are represented at lower bit-widths, with some achieving accuracy comparable to higher-precision models. Innovations include novel calibration strategies for post-training quantization and learnable affine transformations to improve robustness. AI