https://stream.andersonr.net/videos/watch/faf01cfa-a042-4665-9320-9062556a21e5 Fitting a 27-billion-parameter model onto one 24GB gaming card usually means stopping at "it loads." This recipe pushes past that to 417 tokens a second batched, and the wins are unglamorous in the best way. Qwen's untied embeddings ship as two unquantized 2.5GB matrices nobody bothered with, so requantizing both to int8 hands back 2.6GB, and on this architecture spare memory converts straight into batch size. #github #opensource https://github.com/syv-ai/qwen38-27b-rtx3090 Tue, 01 Sep 2026 11:59:14 GMT https://validator.w3.org/feed/docs/rss2.html PeerTube - https://stream.andersonr.net https://stream.andersonr.net/client/assets/images/icons/icon-1500x1500.png https://stream.andersonr.net/videos/watch/faf01cfa-a042-4665-9320-9062556a21e5 All rights reserved, unless otherwise specified in the terms specified at https://stream.andersonr.net/about and potential licenses granted by each content's rightholder.