A new technique to compress AI models 3x smaller—without losing smarts—and it doesn't need calibration data.

Researchers have introduced a new AI model compression technique called SPECTRA that promises to compress large language models (LLMs) by 3x to a size under 50 MB, while maintaining near-lossless performance. It achieves this by using a 'calibration-free topological saliency mapping' method. This means it figures out which parts of the model's internal weights are most important for its reasoning abilities by analyzing the network's own structure, instead of requiring a separate calibration step with example data.

The breakthrough could drastically reduce the memory footprint required to run powerful AI models, making them viable on standard consumer hardware like laptops and smartphones without sacrificing their problem-solving capabilities. The paper claims the compressed model maintains over 99.6% similarity to the original and performs almost identically on complex reasoning benchmarks. This represents a significant step towards efficient, accessible, and portable AI.