New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
Python is pretty easy for someone who already knows programming
LAST DAY! SAD BUT HAPPY AT THE SAME TIME
Yah, AI is the reason why I wanna get into Python. Else, I'd have stuck with PHP forever.
Let's hope so. I really need to get into Python and understand the basics of AI and LLMs soon.
Wanna do AI projects?
Btw, when RN BW limits reset? It's been decades since i've consumed the same thing (month after month not using the VPS)
I like nodejs
Not anything mainstream. It intrigues me. I recently started using Groq and OpenAI's chat assistant. What it's capable of, does amuse me and yet, scares me a bit. But, won't wanna be too late to join the AI bandwagon.
Did y'all tested Deepseek btw? It's a copy of OpenAI lol. I've used it to troll asking for what politician has winnie the pooh as nickname
I should stop selfhosting ai models.
and it didn't answered me, of course.
Yah, just tested the local version of the latest R1 (7b) on my PC. It's alright, but, not the best. Did give me some wrong information. Maybe the 671b variant has more accurate information, but, that's 404GB!
Moreover, here's something interesting that I saw today - https://www.wiz.io/blog/wiz-research-uncovers-exposed-deepseek-database-leak Haven't had the time to read it yet.
BTW, what did it reply to the Winnie question?
They are fine for testing on your local environment, but, if you do go above the 3b models, you will need some serious RAM and CPU specs, whether on a local PC or on a VPS.
their answer: "Sorry, that's beyond my current scope. Let’s talk about something else."
lolll
As expected. Chinese AIs have the habit of being extremely biased to toe the CCP's line. You ask them about any territorial disputes and they will take the CCP's stance. AI and LLMs are supposed to be unbiased.
The truth is, every AI and LLM out there presently, if I am not wrong, is biased towards a side or the other. This was pretty evident during the recent American elections.
True. Sadly all AIs are biased. That's 100% true
I host 8-9B model on my vps. It generates like 1 word per second.
My vps has 16 GB ram
Yah, it is totally understandable since LLMs depend on pre-existing data for training and the existing data is biased.
I have pretty good speeds like 3-5 wps with deepseek r1 7b on my Ryzen 5700G with 32GB DDR4 3200MHz RAM
You can probably run 16-30B models too might bee slow but usable still
try ollama with webui or lmstudio
In order to run any model you have to like convert the model storage*1.2/1.5 for better running in VRAM that is GPUs BTW one can do as you are doing it with ram but yaa the performance would be slower
Yah tried pulling Llama 3.3 10+b yesterday. Woke upto some weird error I don't remember. Now, I have to redownload 40+GB again to test a 7b+ model
Yup. I was really hoping to save up money to get trh 5070 when it launches in India. But, looking at the current scalping scene, I don't think that's gonna be possible anytime soon.
Anyways can't afford Nvidia's more industrial GPUs that are recommended for AI.
I have also been considering a Radeon if nothing else works since Ollama supports AMDGPU
I dont know about ollama supporting AMD but generally all the llms are based on nvidias cuda technology only r1 is different making it easier to run on amd gpus too
I would like to double the bandwidth.
Invoice #12397740
thanks.
Yah, it is true. Nvidia was farsighted with its CUDA tech that helps AI a lot. AMD does lack something as authentic as CUDA.
But Ollama with the amd rocm package should do some good, though nothing as good as NVIDIA's CUDA. https://github.com/ollama/ollama/blob/main/docs/linux.md (Check under AMD GPU install)
Hi everyone
hello mate