Breakthrough of Local AI - KIMI K3

Comments

• concept Member

  July 30

  Theo’s video on Kimi K3 is pretty good.

  

  Thanked by 2lala_th BBTN
• 1q1 Member

  July 30

  @praburam said: kimi k3 to 1bit

  how much VRAM needed for that?
• Neoon Community Contributor, Veteran

  July 30

  @1q1 said:

  @praburam said: kimi k3 to 1bit

  how much VRAM needed for that?

  Someone on Reddit got it running with 768GB REM or 768GB VREM.

  Apparently https://github.com/JustVugg/colibri has already support for Kimi K3, you could run the 1bit quant, but then you could run a 4bit GLM 5.2 quant, they are the same size, so choose wisely.
• dbadude Member

  July 30

  I would recommend bonsai llm too, very responsive on cpu based systems. Start short selling Nvidia the cpu/mobile based llms are coming.
• yokowasis Member

  July 30

  @Neoon said:

  @1q1 said:

  @praburam said: kimi k3 to 1bit

  how much VRAM needed for that?

  Someone on Reddit got it running with 768GB REM or 768GB VREM.

  Apparently https://github.com/JustVugg/colibri has already support for Kimi K3, you could run the 1bit quant, but then you could run a 4bit GLM 5.2 quant, they are the same size, so choose wisely.

  What is REM ?
• oloke Member, Host Rep

  July 30

  @yokowasis said:

  @Neoon said:

  @1q1 said:

  @praburam said: kimi k3 to 1bit

  how much VRAM needed for that?

  Someone on Reddit got it running with 768GB REM or 768GB VREM.

  Apparently https://github.com/JustVugg/colibri has already support for Kimi K3, you could run the 1bit quant, but then you could run a 4bit GLM 5.2 quant, they are the same size, so choose wisely.

  What is REM ?

  @buggedout

  https://lowendtalk.com/discussion/comment/4781365/#Comment_4781365

  it seems to be how @Neoon calls RAM

  Thanked by 1buggedout
• Neoon Community Contributor, Veteran

  July 30

  @oloke said:

  @yokowasis said:

  @Neoon said:

  @1q1 said:

  @praburam said: kimi k3 to 1bit

  how much VRAM needed for that?

  Someone on Reddit got it running with 768GB REM or 768GB VREM.

  Apparently https://github.com/JustVugg/colibri has already support for Kimi K3, you could run the 1bit quant, but then you could run a 4bit GLM 5.2 quant, they are the same size, so choose wisely.

  What is REM ?

  @buggedout

  https://lowendtalk.com/discussion/comment/4781365/#Comment_4781365

  it seems to be how @Neoon calls RAM

  Kinda.

  

  Thanked by 6oloke forest tux buggedout BasToTheMax ebietsy
• rpqu Member

  July 30

  @Neoon said:

  @oloke said:

  @yokowasis said:

  @Neoon said:

  @1q1 said:

  @praburam said: kimi k3 to 1bit

  how much VRAM needed for that?

  Someone on Reddit got it running with 768GB REM or 768GB VREM.

  Apparently https://github.com/JustVugg/colibri has already support for Kimi K3, you could run the 1bit quant, but then you could run a 4bit GLM 5.2 quant, they are the same size, so choose wisely.

  What is REM ?

  @buggedout

  https://lowendtalk.com/discussion/comment/4781365/#Comment_4781365

  it seems to be how @Neoon calls RAM

  Kinda.

  

  

  Thanked by 2forest buggedout
• buggedout Member

  July 31

  @oloke said:

  @yokowasis said:

  @Neoon said:

  @1q1 said:

  @praburam said: kimi k3 to 1bit

  how much VRAM needed for that?

  Someone on Reddit got it running with 768GB REM or 768GB VREM.

  Apparently https://github.com/JustVugg/colibri has already support for Kimi K3, you could run the 1bit quant, but then you could run a 4bit GLM 5.2 quant, they are the same size, so choose wisely.

  What is REM ?

  @buggedout

  https://lowendtalk.com/discussion/comment/4781365/#Comment_4781365

  it seems to be how @Neoon calls RAM

  You still remember that? Thanks for reminding me, even I had forgotten !!
  

  Thanked by 3rpqu oloke ebietsy
• farsighter Veteran

  July 31

  @yokowasis said:

  @Neoon said:

  @1q1 said:

  @praburam said: kimi k3 to 1bit

  how much VRAM needed for that?

  Someone on Reddit got it running with 768GB REM or 768GB VREM.

  Apparently https://github.com/JustVugg/colibri has already support for Kimi K3, you could run the 1bit quant, but then you could run a 4bit GLM 5.2 quant, they are the same size, so choose wisely.

  What is REM ?

  REM (rapid eye movement sleep) is a unique sleep stage defined by fast eye movements, vivid dreaming, and temporary muscle paralysis.
• Levi Veteran

  July 31

  REM is manerism used by femboys to name RAM modules.
• elliotc Member

  July 31 edited July 31

  I can’t even afford a 16 GB REM these days, let alone a 768—that’s something I wouldn’t even dare to dream of. I don’t understand how Kimi or GLM could possibly put an end to OpenAI and Claude, because it seems to me they’re actually matching those companies’ prices—in terms of being expensive. The real hope lies with Deepseek and gpt luna, now that its prices have come down.
  If you disregard benchmark, the Opus for planning and the Luna Max for coding offer the best balance of value for money.
• rpqu Member

  July 31

  @buggedout said:

  @oloke said:

  @yokowasis said:

  @Neoon said:

  @1q1 said:

  @praburam said: kimi k3 to 1bit

  how much VRAM needed for that?

  Someone on Reddit got it running with 768GB REM or 768GB VREM.

  Apparently https://github.com/JustVugg/colibri has already support for Kimi K3, you could run the 1bit quant, but then you could run a 4bit GLM 5.2 quant, they are the same size, so choose wisely.

  What is REM ?

  @buggedout

  https://lowendtalk.com/discussion/comment/4781365/#Comment_4781365

  it seems to be how @Neoon calls RAM

  You still remember that? Thanks for reminding me, even I had forgotten !!
  

  Is forgetting something that had happened 3 months ago a sign of early alzeheimer's diseases?

  Thanked by 1buggedout
• Neoon Community Contributor, Veteran

  July 31 edited July 31

  You know the worst part, since we got 2.8T now.
  They started dropping 300B as "small".

  https://huggingface.co/thinkingmachines/Inkling-Small

  fk
• buggedout Member

  July 31

  @rpqu said:

  @buggedout said:

  @oloke said:

  @yokowasis said:

  @Neoon said:

  @1q1 said:

  @praburam said: kimi k3 to 1bit

  how much VRAM needed for that?

  Someone on Reddit got it running with 768GB REM or 768GB VREM.

  Apparently https://github.com/JustVugg/colibri has already support for Kimi K3, you could run the 1bit quant, but then you could run a 4bit GLM 5.2 quant, they are the same size, so choose wisely.

  What is REM ?

  @buggedout

  https://lowendtalk.com/discussion/comment/4781365/#Comment_4781365

  it seems to be how @Neoon calls RAM

  You still remember that? Thanks for reminding me, even I had forgotten !!
  

  Is forgetting something that had happened 3 months ago a sign of early alzeheimer's diseases?

  Canbe but here it was just a retrieval failure my brain archived it too efficiently.... for AD we also look loss of learned motor actions(I forgot the term for it), aphasia and few other things too !!

  Even if I get AD when REM finally gets her consciousness back in re:zero would cure me out of big shock 😁😂 !!

  Thanked by 1rpqu
• hyperblast Member

  July 31

  is rem chap?