Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

RackNerd’s BLACK FRIDAY TAKEOVER! NEW DEALS, NEW TECH, AdminBolt HOSTING + HUNDREDS OF GIVEAWAYS!

1…2627293132…104

Comments

  • op23op23 Member

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

  • @Jamie96 said:

    @noob404 said:

    @Jamie96 said:

    @noob404 said:

    @Jamie96 said:
    So, I understand how a provider like this manages to get these prices. That's not a problem. But anyone that has one, maybe even cut to the chase and anyone that has one in Ashburn, what CPU are they using to fuel this offer, and otherwise how's your performance so far? I thought I'd ask here for the real user reviews and because RackNerd's sales chat is understandable overwhelmed right now so I can't ask them anything. I am getting one from this offer but I'm trying to temper my expectations and make sure I get the right one since there's no refund policy.

    My goal is docker apps, a variety of them as I find myself in a bleak economy not really looking forward to paying repair costs on a homelab. So docker, all sorts, a Postgres or two as they require. Thinking of going all the way for the memory but I'll be sad to find the CPU is lackluster and I should have gotten multiple smaller ones to spread the nodes.

    TL;DR, users what CPU you got, how's it going, would you think I survive on one monolith VPS or prefer split load?

    For your purposes, I'd recommend considering RackNerd's Ryzen VPSes. https://www.racknerd.com/ryzen-vps

    That bad, huh? I'll assume decade old Xeons then, the original budget driver.

    Not my point. I have used both the Xeons and Ryzens from RackNerd. Some of my projects were really happy on the Xeons, but I can't remember which versions there, only remember I never faced any issues with them.

    I moved to their Ryzen cause I got a good deal a few years ago. I just meant if you can get the Ryzen go for it. Else, the Intel-based VPSes from RackNerd will also serve your purpose well.

    I mean I don't disagree with you, and if they put the Ryzen's on sale that would be good, but as of today it's a 400% increase along the same RAM line with less cores, storage, etc....of course those less quantity cores are probably orders of magnitude better at the job..and it's not a critical or profitable workload so I'm hard pressed to justify all costs, hence the deal hunting and interest in what the sacrifices are to get to this cost.

    Alright, for anyone following along I did get a quick bite from sales, they've got a mix of Xeon 2680 V2s and Gold 6252. So some old and some newer, luck of the draw.

    Yes, I totally understand. I would urge you to sticka round for the flash sales (expected on Black Friday) on this thread.

  • LET’S GO! RACKNERD BLACK FRIDAY 2026 IS LIVE: https://www.racknerd.com/BlackFriday/ ⚡

  • 我们准备好迎接更多赠品啦!RACKNERD 最棒了!

  • @op23 said:

    @DaDeveloper said:

    @op23 said:

    @DaDeveloper said:
    As a side note for peeps that are still awake and kicking, Local AI is badass! Had a agent working for damn near 24 hours straightht off of my hardware and Hermes agent, and its used over 498M tokens in input and 4.5M in output and it only cost me electricity and saved me time of doing what its doin. that and I don't have to worry about big bro snooping or a company harvesting my data. Soon I will be moving off of Hermes tho.

    It’s between paying $20 for a vibecoded generic website through Claude and $12000 to do the same thing locally by buying a GPU. Or lots of ram (slower option). Yeah I think I’ll let big brother snoop into the 157373373737th react offering.

    1. Not using it to vibe code anything
    2. Cost does not matter as much for me compared to no caps, convenience, data sovereignty and privacy
    3. It generates things faster than I could have done them on my own
    4. IT allows me to get tasks done when I am working my day job so that I can spend time doing what I love and enjoy rather than these tasks
    5. What does react have to do with it? I wouldn't have used react anyways... it sucks

    Just curious, what else are you doing/generating and what are your machine specs? I am not stranger to AI (use it at my day job and during personal projects), but I am also not rich to buy hardware for anything better than a 9b that fits in locally. And during my playing with a 27b model I was very disappointed.

    Automations both Homelab and personal admin, software factories, personal prose review, tune and deployed on device neural text to speech with response streaming (first audio under a second on apple silicon, personal daily breifing using sources both specified and found by the agent, prototyping applications, hammering out specs, automating research that I can trigger when on the job (via voice command), constant scanning for vehicles that match a very strict set of parameters (looking for a tow vehicle for a camper) is just some off the top of my head

    Thanked by 2noob404 op23
  • @op23 said:

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

    True. I still regret not buying two sticks of 32GB RAM when they would have cost me just $200. Now, I am even scared to look their prices up.

    Woulda helped a lot with CPU inference.

    Thanked by 1op23
  • LET’S GO! RACKNERD BLACK FRIDAY 2026 IS LIVE: https://www.racknerd.com/BlackFriday/ ⚡

  • Order Number: 7701732637 , please double my bandwidth , Thanks!

  • LET’S GO! RACKNERD BLACK FRIDAY 2026 IS LIVE: https://www.racknerd.com/BlackFriday/ ⚡

  • @op23 said:

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

    Yea I panic bought in the 4k range for the sparks now they are more expensive, 7k if you can find them cheap, 9-11k otherwise XD

    Thanked by 1op23
  • @op23 said:

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

    Also op23 which 27b model did you try?

  • Hi @dustinc, quick question — is Los Angeles DC-02 sold out for the Black Friday KVM plans, or will more stock be added during the event? I'm looking at the 4GB plan and DC-02 is my first choice, with San Jose as backup. Thanks!

  • noob404noob404 Member
    edited November 20

    @DaDeveloper said:

    @op23 said:

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

    Yea I panic bought in the 4k range for the sparks now they are more expensive, 7k if you can find them cheap, 9-11k otherwise XD

    Dang! Anyone has any idea if/when AI hardware prices are going back to sane?

  • LET’S GO! RACKNERD BLACK FRIDAY 2026 IS LIVE: https://www.racknerd.com/BlackFriday/ ⚡

  • One of the biggest benefits aside from the privacy/soverignity aspect is the way it can keep trooping along and not worrying about it hitting some arbitrary cap lol one job is now at 26 hours (continuous working)

    Thanked by 1op23
  • @noob404 said:

    @PuDLeZ said:

    @gritty said:
    will RackNerd still be hosting flash sales this bf?

    Chef always bakes some tasty treats on BF!

    Hey dude. You still here?

    yes/no, glancing at the thread every now and then :)

  • @noob404 said:

    @DaDeveloper said:

    @op23 said:

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

    Yea I panic bought in the 4k range for the sparks now they are more expensive, 7k if you can find them cheap, 9-11k otherwise XD

    Dang! Anyone has any idea if/when AI hardware prices are going back to sane?

    prob not for a good few years if ever, companies tend to be greedy like that

    Thanked by 1op23
  • op23op23 Member

    @DaDeveloper said:

    @op23 said:

    @DaDeveloper said:

    @op23 said:

    @DaDeveloper said:
    As a side note for peeps that are still awake and kicking, Local AI is badass! Had a agent working for damn near 24 hours straightht off of my hardware and Hermes agent, and its used over 498M tokens in input and 4.5M in output and it only cost me electricity and saved me time of doing what its doin. that and I don't have to worry about big bro snooping or a company harvesting my data. Soon I will be moving off of Hermes tho.

    It’s between paying $20 for a vibecoded generic website through Claude and $12000 to do the same thing locally by buying a GPU. Or lots of ram (slower option). Yeah I think I’ll let big brother snoop into the 157373373737th react offering.

    1. Not using it to vibe code anything
    2. Cost does not matter as much for me compared to no caps, convenience, data sovereignty and privacy
    3. It generates things faster than I could have done them on my own
    4. IT allows me to get tasks done when I am working my day job so that I can spend time doing what I love and enjoy rather than these tasks
    5. What does react have to do with it? I wouldn't have used react anyways... it sucks

    Just curious, what else are you doing/generating and what are your machine specs? I am not stranger to AI (use it at my day job and during personal projects), but I am also not rich to buy hardware for anything better than a 9b that fits in locally. And during my playing with a 27b model I was very disappointed.

    Automations both Homelab and personal admin, software factories, personal prose review, tune and deployed on device neural text to speech with response streaming (first audio under a second on apple silicon, personal daily breifing using sources both specified and found by the agent, prototyping applications, hammering out specs, automating research that I can trigger when on the job (via voice command), constant scanning for vehicles that match a very strict set of parameters (looking for a tow vehicle for a camper) is just some off the top of my head

    Very cool. This is the stuff that I like to see. Thanks so much for sharing.

    I have also been working on recreating some services of my own instead of commercial (faster than taking years nowadays thanks to AI) and a dashboard like setup for family etc. Sadly got no choice but to use Anthropic offerings though :(

    Now back to wishing for price errors on hardware lol.

  • Hey, @MrEd sorry to bother you. Can you update https://racknerd.projecthive.eu/ ?

  • @DaDeveloper said:
    One of the biggest benefits aside from the privacy/soverignity aspect is the way it can keep trooping along and not worrying about it hitting some arbitrary cap lol one job is now at 26 hours (continuous working)

    Damn. Have you tried running something like Craft Anything with it?

  • @PuDLeZ said:

    @noob404 said:

    @PuDLeZ said:

    @gritty said:
    will RackNerd still be hosting flash sales this bf?

    Chef always bakes some tasty treats on BF!

    Hey dude. You still here?

    yes/no, glancing at the thread every now and then :)

    Did you get the Toronto VPS giveaway? I missed it.

  • Order #9072622518 - please double the bandwidth, thanks!

  • op23op23 Member

    @DaDeveloper said:

    @op23 said:

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

    Yea I panic bought in the 4k range for the sparks now they are more expensive, 7k if you can find them cheap, 9-11k otherwise XD

    You did the right thing. Many of us didn’t.

  • @DaDeveloper said:

    @noob404 said:

    @DaDeveloper said:

    @op23 said:

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

    Yea I panic bought in the 4k range for the sparks now they are more expensive, 7k if you can find them cheap, 9-11k otherwise XD

    Dang! Anyone has any idea if/when AI hardware prices are going back to sane?

    prob not for a good few years if ever, companies tend to be greedy like that

    Yah, I guess, the only way is to become rich enough to afford them

  • @noob404 said:

    @DaDeveloper said:
    One of the biggest benefits aside from the privacy/soverignity aspect is the way it can keep trooping along and not worrying about it hitting some arbitrary cap lol one job is now at 26 hours (continuous working)

    Damn. Have you tried running something like Craft Anything with it?

    Not yet, may look into that and Reverse Engineer Anything (REA) later this week though.

  • @DaDeveloper said:

    @noob404 said:

    @DaDeveloper said:

    @op23 said:

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

    Yea I panic bought in the 4k range for the sparks now they are more expensive, 7k if you can find them cheap, 9-11k otherwise XD

    Dang! Anyone has any idea if/when AI hardware prices are going back to sane?

    prob not for a good few years if ever, companies tend to be greedy like that

    Hate to agree here but this is it. As long as people cave and buy or companies too, they prove this is a viable market with these prices, so they are here to stay. They don't care if we start taking something like auto loans to buy our PCs, as long as someone is buying.

    Thanked by 2op23 noob404
  • @op23 said:

    @DaDeveloper said:

    @op23 said:

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

    Yea I panic bought in the 4k range for the sparks now they are more expensive, 7k if you can find them cheap, 9-11k otherwise XD

    You did the right thing. Many of us didn’t.

    Wanted 4, wish I had the cash at the time XD

    Thanked by 1op23
  • op23op23 Member

    @noob404 said:

    @op23 said:

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

    Also op23 which 27b model did you try?

    Qwen 3.6-27B

  • @op23 said:

    @DaDeveloper said:

    @op23 said:

    @noob404 said:
    At this point, is it better to get gaming GPUs or NPUs like Ryzen AI Max for local LLM inference?

    At this point you need to get lucky.

    Any hardware with capability to run something decent including strix halo is crazy expensive.
    I wish I bought the 128gb one when it was $1800 USD late 2025. Dumb mistake. Now it’s 3-4x.

    Gaming GPUs or Macs are even more expensive for same memory capacity.

    Yea I panic bought in the 4k range for the sparks now they are more expensive, 7k if you can find them cheap, 9-11k otherwise XD

    You did the right thing. Many of us didn’t.

    I still look at prices of 5070ti atleast in hopes that it has gone down or by some miracle someone wants to get rid of theirs at cheap. But, no luck yet.

  • @DaDeveloper said:

    @noob404 said:

    @DaDeveloper said:
    One of the biggest benefits aside from the privacy/soverignity aspect is the way it can keep trooping along and not worrying about it hitting some arbitrary cap lol one job is now at 26 hours (continuous working)

    Damn. Have you tried running something like Craft Anything with it?

    Not yet, may look into that and Reverse Engineer Anything (REA) later this week though.

    Sure. Let us know how it goes if you can.

Sign In or Register to comment.