Alibaba's XuanTie C950 chip hits 30 tokens/second, with a 1.9-second first-token latency, on Qwen-3.8 27B - no emulation layer required.
  • Eager Eagle@lemmy.worldEnglish
    4·
    1 hour ago

    Is it just me or it’s really underwhelming that they seem to remove a lot of layers and still only get 30 tokens per second? I hope they’re at least more energy efficient.

    • ☆ Yσɠƚԋσʂ ☆@lemmy.mlOP
      3·
      54 minutes ago

      I think it’ll depend on how much these chips cost. If it’s gonna be something like a Raspberry Pi that works as well as a 500 dollar GPU, that’s gonna be pretty amazing.

      • Asafum@lemmy.world
        1·
        2 minutes ago

        as well as a 500 dollar GPU

        (Cries in $1,300 Intel arc fighting tooth and nail to break 25t/s…)