New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Comments
Just setup Chicken Nugget for Frankfurt, its gonna take a few days prob a week for restock though.
Anyone took a gamble with a KS-4 in Warsaw? 2x 4TB instead of 2x 2TB?
BHS is better for me, but I am in to gambling 🎰
16TB KS-2s are like 50% in LIM and 10% in BHS
Is anyone handing in their KS-Mystery in Europe?
Hmmm
Any with 10g symmetrical with 4tb so far?
Yesterday I got my server
Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz (2.10 GHz)
32 GB RAM
4x4 TB
10 GBiT down and 500 MBiT up.
Location: Limburg
Haven't been that lucky
Welcome to the club.
16TB rulez.
go for Frankfurt, its like 100ms away, well connected. good enough for linux iso's.
Yes 16 TB of Linux ISO's
Raid 1 is gonna be like 7TB ish usable.
Plus you got plenty of bandwidth and a CPU to transcode the linux iso's.
@Neoon howz llms treating the le‐b?
Whats the word on the street?
it gets molested by a 122B model.
O spicy
Which 122b model ?
New Qwen 3.5, with vision.
Pictures work fucking great.
Have you tried the 9B version? Meant to be quite impressive.
It is, actually. Does 3 t/s on an E5-1620v2.
Why try 9B if you can have 122B.
I paid for 64gig, I am gonna use it.
Yea, LLM's on E5 isn't great, neither EPYC.
Xeon G as mystery was actually the best you could get.
But for 10$ running a 122B model, is fine for me.
E3-1270v6 is not very fast running LLM models.
I mostly run optimized quants, can't complain for 10€/m.
Its never gonna be fast compared to a GPU.
out of curiosity, what kinds of optimisations have you done to run these better on pure cpu? software and prompt related
for reference, i noticed a "big" speedup switching to ik_llama.cpp, but im unsure if its purely for the newer qwen models or related to the cpu model im using. i feel like i still need to do something to the prompt prosessing/context, as it starts feeling quite slow couple messages in as it seems to be quite verbose
Freshly baked baguette on this monday morning?
The main issue, is the memory bandwidth, at least we got DDR4.
DDR5 would be nicer, but cost more way more with a recent CPU.
@crunchbits I think they sold a VM with a dedicated 3090 for 150$/m if its in stock.
Hetzner sold a 1080 Dedi for 90€/m.
We are sitting on 64gig DDR4 dedis for 12$/m.
That's the best we can do.
I did compile it with different parameters, changed the batch sizes and so on.
And, always use quants, especially on CPU, never go full model on these.
GLM-4.7 is currently the best self-hosted in the coding area I'd say (yes, I'm not at 5 atrm).
Newer CPUs gives much better performance when running LLM models than old Kaby Lake server CPU from 2017.
Sure, power is expensive, its cheaper for me to rent a baguette with subsidized nuclear power than hosting it yourself, at least here.
Maybe we get some cheap Ryzens soonTM.
KS-MYSTERY 2026?
Will there be any surprises this April Fools' Day?