The options for playing audio on the Nintendo 64 were limited. The PlayStation could stream uncompressed CD audio, but the N64 relied on cartridges, where space was at a premium and the 93 MHz CPU had limited time to spare. Nintendo's Libultra library used the RSP vector co-processor to decompress ADPCM audio, which cut sample sizes by a factor of 4. Even so, a five-minute stereo track would take about 12MB, far too much for a cartridge where Mario 64 fit into just 8MB. As a result, N64 games generally produced music in a tracker-like style using short, scheduled samples.
Later, some developers were allowed direct RSP access. Conker's Bad Fur Day used it to decode MP3 speech samples, but the decoding was too costly for continuous background music or gameplay sound effects. Open source Libdragon offers a real-time ADPCM decoder, and the author compressed all music for Xibalba64 with it, using about 31MB of a 32MB ROM. Libdragon also includes an RSP-accelerated Opus decoder, which is impressive but still too slow during gameplay.
An MP3 decoder would offer only marginal gains over Opus, and QOA is hard to vectorize because each sample depends on the previous one. A simpler transform codec might work better. While researching, the author found a Game Boy Advance demo by Aikku93 that plays recorded music at 70kbps on a 16 MHz ARM CPU with no vector instructions.
Aikku93 wrote a decoder in ARM assembly and created a new codec, the Ultra Low Complexity Codec (ULC), which the author believes could be a strong fit for the N64.
Source: Hacker News · Summarized by HeadlinesBriefing