New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
And SOTA is how many T’s?
only led to more providers at same or higher prices though
Kimi K3 right now, but I doubt you can run that.
One guy had 80x 5090 and got 20t/s.
https://x.com/totheagi/status/2081855316443205717
Otherwise https://github.com/JustVugg/colibri
GLM 5.2 can run on a 32gig potato
80x?? At >500 W each, that's over 40 kilowatts just to run the damn thing for one person!
Probably you can run a few sessions until the t/s goes below 15 or lower.
But we shall wait for a destilled Kimi K3 8B gguf.
https://huggingface.co/unsloth/Kimi-K3-GGUF
Not there yet, might be a bit chonky to quantize that chonkyness.
So, persons will be limited not only by hardware but also by power price.
Not really, he's saying that he's expecting major optimization improvements and then he'll launch it on https://engy.ai as pay per token
They currently run glm-5.2 on 32x RTX 5090 (32GB)
No idea why they're not on OpenRouter because they appear to be cheaper than the competition
Given that one of these fancy NVIDIA racks is like 6 mil, 80x 5090 sound cheap.
Rich-bitch.
In comparison.
Neither is a single 5090 cheap.
I think the Kimi K3 weights are publicly available now:
http://huggingface.co/moonshotai/Kimi-K3
We know, because it was said so in this thread yesterday.
But can I run it on a PIC18?
Yes you can. The other question will it run?
1bit is available for Kimi K3
https://huggingface.co/unsloth/Kimi-K3-GGUF
Just 649 GB
Its so jover.
I'd prefer an 8051, among other reasons massive RAM g
*(also an assembler I like)
That was the question. You likely meant "work".
Y> @Neoon said:
Broken image?
Everything can run as long you have fat enough storage.
You can stream models from local disks.
Which if the engine isn't optimized for this, is a terrible idea.
idk, reddit doing imgurl things.