In a groundbreaking partnership poised to redefine the landscape of AI acceleration, Andes Technology and Fractile have announced their collaboration aimed at developing revolutionary AI inference accelerators. Andes Technology, known for its low-power 32/64-bit RISC-V processors, and Fractile, an Oxford-based AI IC startup, plan to leverage in-memory computing to create accelerators that execute large language, vision, and audio models at unprecedented speeds. These accelerators promise to deliver performance enhancement two orders of magnitude faster than existing hardware while simultaneously slashing costs by tenfold.
Fractile’s primary focus lies in model inference—the computationally intensive process of using trained models to make predictions or decisions. This aspect of AI computing has become increasingly significant, with inference costs now surpassing those associated with training. In an ambitious move to enhance efficiency and speed, Fractile plans to integrate Andes’ AX45MPV RISC-V vector processor along with the ACE (Andes Custom Extension) and Andes Domain Library into their first-generation data center AI inference accelerator. The collaboration aims to harness the robust performance of Andes’ high-performance vector processors, aligning it perfectly with Fractile’s novel in-memory computing framework.
The Significance of In-Memory Computing
One of the most compelling aspects of Fractile’s technology is its novel approach to in-memory computing, which allows for 99.99% of model inference operations to be conducted within on-chip memory. This architectural innovation eliminates the need for frequent data transfers between the processor and memory, thereby significantly enhancing energy efficiency and reducing latency. By embedding computational operations directly into the memory, Fractile’s architecture sets a new benchmark for energy efficiency, delivering superior TOPS/W (tera operations per second per watt) performance.
This architectural paradigm shift offers substantial advantages, particularly in reducing latency for large language models (LLMs). Faster token processing is critical for improving the responsiveness and efficiency of AI models, which is increasingly pivotal as the AI industry—exemplified by companies like OpenAI—continues to scale inference capabilities. The ability to make AI model responses nearly instantaneous, an area where current hardware often falls short, marks a significant leap forward for the industry.
Addressing the Challenges of AI Acceleration
The collaborative effort between Fractile and Andes is more than a mere technical partnership; it represents a strategic alignment to overcome the growing challenges of AI acceleration hardware development. One of the major hurdles is the rapidly evolving nature of leading AI models, which often outpaces the development cycle of specialized chips. By incorporating Andes’ software-programmable vector processors, which offer considerable flexibility, Fractile positions itself to adapt swiftly to these changes and stay ahead in the competitive AI landscape.
Furthermore, as highlighted by Fractile’s CEO Walter Godwin, existing GPU-based solutions like those using Nvidia’s CUDA are not inherently optimized for AI tasks. These systems often require redundant computational adjustments that diminish overall efficiency. Fractile’s approach eliminates such inefficiencies, offering a more streamlined and effective software solution. This combination of advanced hardware and efficient software underscores the transformative potential of the Fractile-Andes partnership.
Strategic Funding and Future Prospects
In a groundbreaking collaboration set to transform the field of AI acceleration, Andes Technology and Fractile have joined forces to develop state-of-the-art AI inference accelerators. Andes Technology, recognized for its low-power 32/64-bit RISC-V processors, and Fractile, an innovative AI IC startup from Oxford, plan to harness in-memory computing to create accelerators capable of running extensive language, vision, and audio models at exceptional speeds. These new accelerators promise a performance boost two orders of magnitude faster than current hardware while cutting costs by ten times.
Fractile’s main focus is on model inference, the intensive process of using trained AI models for predictions or decision-making. As inference costs now exceed those of training, this area of AI computing has gained immense importance. In a bold effort to enhance speed and efficiency, Fractile intends to integrate Andes’ AX45MPV RISC-V vector processor, along with the ACE (Andes Custom Extension) and Andes Domain Library, into their first-generation data center AI inference accelerator. This alliance seeks to combine the high performance of Andes’ vector processors with Fractile’s innovative in-memory computing framework, aiming to redefine the future of AI technology.
