New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
[2024 EXTENDED] Black Friday / Cyber Monday: FLASH SALE & MEGATHREAD
This discussion has been closed.
Comments
Crazy happy. Glad for him. I was gonna offer mine for transfer to him but guess that's not needed.
If only @kuroit could move my Singapore VM to a Japan node...
yeah this is the first link google spit out. checking that.
https://github.com/ggerganov/llama.cpp
just in case you lost the link
@kuroit
Order Number: 2785987705
Of course. Double bookmarked....
Congrats 👏 you've coped enough
Wait. Lost it again. Can you please share it?
Very good coper
Why are people posting order ids
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
https://github.com/ggerganov/llama.cpp
bookmark them all please
idk but it boosts the thread stats
Got it now. Finally. I hope not to loose it. I knew i can always count on you.
of course
Congrats!! 🎉🍾🎊
👍🏼👍🏼
I just tested the new gemini
@Saragoldfarb what do you think
I am trying to get approval for ChatGPT Pro subscription at work. $200/month seems to be pretty high.
source: openai, lol
o1 pro very worth
Isn't there a standard 20/ seat option?
My use case
https://community.openai.com/t/any-other-pro-users-using-o1-for-math/1055094
why make the llm do math?
make llm do python make python do math
Good sexting doesn't come cheap.
Is there still stock available? I missed it, and the promo code shows as expired.
Do you even know where you are!?
@plumberg said:
You need quite a lot of
For a self hosted LLM you need a lot of VRAM especially for coding. For home PC probably the best model you can run is currently Qwen2.5-32B-Coder . For coding, unlike chat you need at least half(8 bits) precision, so that means you will need around 40GB Vram, e.g. Dual rtx 3090, which is a setup I am using. Bigger models, like Qwen2.5-72B or LLama-Nemotron-70B are better, but you won't run it at home at that precision unless you build 4xGPU rig. For the largest open weight models like Mistral Large, you need dual H100 to run it, so you definitely need to rent GPU and won't be cheap but you get around a performance of GPT-4o in coding there, probably slightly less since you will still run it in half precision only.
Tl;dr: Don't
First off, thanks for the detailed post.
So here is the deal. I have 0 GPU. But have a pair of dual E5-2699v4 pair with decent RAM (384 gb ddr4 2400 speeed or something).
I am not really interested in getting fastest responses. As long as it spits out decent I am game.
Or am I dreaming of hosting a llm ? What are your thoughts?
Place 10 for comments now