Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Register

Quick Links


Shells Virtual Desktop
BMail.ag - Secure Email Service
Server.net
CPLicense.net
VPS Server
Buy VPN
Vultr
VMs for AI
HostDare
ReliableSite White-Label Dedicated Hosting for Resellers
InterServer VPS
BMail.ag - Secure Email Service
Best VPN
High-Performance Bare Metal Server Solutions
Karvl.com
Server Mania Cloud Hosting
DataWagon Hosting
AlphaVPS Hosting
Evoxt.com
Clouvider
VPS Hosting with NVMe
Residential IPs in the US & 4G Mobile Proxies in EU & US with Unlimited Bandwidth
ReliableSite White-Label Dedicated Hosting for Resellers
Rabisu - Hosting Solutions
Shells Virtual Desktop

Categories

In this Discussion

Home › Announcements
New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

[2024 EXTENDED] Black Friday / Cyber Monday: FLASH SALE & MEGATHREAD

1…10161017101910211022…1337

Comments

  • dev_vpsdev_vps Member

    @emgh said:

    @dev_vps said:

    @emgh said:

    o1 pro very worth

    My use case
    https://community.openai.com/t/any-other-pro-users-using-o1-for-math/1055094

    why make the llm do math?

    make llm do python make python do math

    Because I teach jee calculus

    Thanked by 2emgh r3k
  • emghemgh Member, Megathread Squad

    @Savvy said:
    Place 10 for comments now :*

    good job

    Thanked by 2Savvy r3k
  • tanseltansel Member

    @donli said:

    @tansel said:

    @kuroit said:

    @raza19 said:
    Dear @kuroit we r sincerely missing ur presence. I realize u r preparing for Christmas/boxing Friday deals but wt a pleasant surprise it wud be if u offered 7 Dollah Singapore deals at the auspasciois occasion of having reached 1000 pages. That's just good business and karma

    You have been heard!

    1 vCore
    1GB DDR4 RAM
    15GB SSD/NVMe Disk
    1TB Bandwidth @ 1/10Gbps Uplink

    Supported Locations:
    West Midlands, UK
    Dallas, USA
    Tampa, USA
    Los Angeles, USA
    Ashburn, USA
    The Netherlands
    Singapore

    Price: 7.77GBP/Year with promocode
    Promocode: YOU-KNOW-WHO!
    Stock: 5

    Order: https://my.kuroit.com/store/sale-offers/bf-worldwide-1vcore-1gb-ram-15gb-disk

    Is there still stock available? I missed it, and the promo code shows as expired.

    Do you even know where you are!?

    Am I causing you any trouble by asking if there’s still stock available?

    Thanked by 1r3k
  • JohnFilch123JohnFilch123 Member

    @tansel said:

    Am I causing you any trouble by asking if there’s still stock available?

    Nope but the answer is kinda obvious.

    Thanked by 2donli r3k
  • SavvySavvy Member

    @tansel said:

    @donli said:

    @tansel said:

    @kuroit said:

    @raza19 said:
    Dear @kuroit we r sincerely missing ur presence. I realize u r preparing for Christmas/boxing Friday deals but wt a pleasant surprise it wud be if u offered 7 Dollah Singapore deals at the auspasciois occasion of having reached 1000 pages. That's just good business and karma

    You have been heard!

    1 vCore
    1GB DDR4 RAM
    15GB SSD/NVMe Disk
    1TB Bandwidth @ 1/10Gbps Uplink

    Supported Locations:
    West Midlands, UK
    Dallas, USA
    Tampa, USA
    Los Angeles, USA
    Ashburn, USA
    The Netherlands
    Singapore

    Price: 7.77GBP/Year with promocode
    Promocode: YOU-KNOW-WHO!
    Stock: 5

    Order: https://my.kuroit.com/store/sale-offers/bf-worldwide-1vcore-1gb-ram-15gb-disk

    Is there still stock available? I missed it, and the promo code shows as expired.

    Do you even know where you are!?

    Am I causing you any trouble by asking if there’s still stock available?

    You forgot to close your comment with "reguards" and that is considered rude here

    Thanked by 3plumberg donli r3k
  • stenysteny Member

    @plumberg said:

    @steny said:
    @plumberg said:

    any recommendations for selfhosted llm which would help with code generation? Claude does amazingly well, but I end up making it work so much that I am rate-limited.

    You need quite a lot of

    @plumberg said:
    any recommendations for selfhosted llm which would help with code generation? Claude does amazingly well, but I end up making it work so much that I am rate-limited.

    For a self hosted LLM you need a lot of VRAM especially for coding. For home PC probably the best model you can run is currently Qwen2.5-32B-Coder . For coding, unlike chat you need at least half(8 bits) precision, so that means you will need around 40GB Vram, e.g. Dual rtx 3090, which is a setup I am using. Bigger models, like Qwen2.5-72B or LLama-Nemotron-70B are better, but you won't run it at home at that precision unless you build 4xGPU rig. For the largest open weight models like Mistral Large, you need dual H100 to run it, so you definitely need to rent GPU and won't be cheap but you get around a performance of GPT-4o in coding there, probably slightly less since you will still run it in half precision only.

    First off, thanks for the detailed post.

    So here is the deal. I have 0 GPU. But have a pair of dual E5-2699v4 pair with decent RAM (384 gb ddr4 2400 speeed or something).

    I am not really interested in getting fastest responses. As long as it spits out decent I am game.

    Or am I dreaming of hosting a llm ? What are your thoughts?

    Running on Ram would be awfully slow, especially those large models you could theoretically run with that amount of ram.

    Thanked by 2plumberg r3k
  • emghemgh Member, Megathread Squad

    regar

    Thanked by 2Savvy r3k
  • emghemgh Member, Megathread Squad

    ds

    Thanked by 2Savvy r3k
  • emghemgh Member, Megathread Squad

    how many minutes of music did you guys listen to 2024

    Thanked by 2_MS_ r3k
  • plumbergplumberg Veteran, Megathread Squad

    @steny said:

    @plumberg said:

    @steny said:
    @plumberg said:

    any recommendations for selfhosted llm which would help with code generation? Claude does amazingly well, but I end up making it work so much that I am rate-limited.

    You need quite a lot of

    @plumberg said:
    any recommendations for selfhosted llm which would help with code generation? Claude does amazingly well, but I end up making it work so much that I am rate-limited.

    For a self hosted LLM you need a lot of VRAM especially for coding. For home PC probably the best model you can run is currently Qwen2.5-32B-Coder . For coding, unlike chat you need at least half(8 bits) precision, so that means you will need around 40GB Vram, e.g. Dual rtx 3090, which is a setup I am using. Bigger models, like Qwen2.5-72B or LLama-Nemotron-70B are better, but you won't run it at home at that precision unless you build 4xGPU rig. For the largest open weight models like Mistral Large, you need dual H100 to run it, so you definitely need to rent GPU and won't be cheap but you get around a performance of GPT-4o in coding there, probably slightly less since you will still run it in half precision only.

    First off, thanks for the detailed post.

    So here is the deal. I have 0 GPU. But have a pair of dual E5-2699v4 pair with decent RAM (384 gb ddr4 2400 speeed or something).

    I am not really interested in getting fastest responses. As long as it spits out decent I am game.

    Or am I dreaming of hosting a llm ? What are your thoughts?

    Running on Ram would be awfully slow, especially those large models you could theoretically run with that amount of ram.

    How slow are we talking about? Any idea?

    And will that change the quality of the output?

    Reguards

    Thanked by 3Savvy emgh r3k
  • _MS__MS_ Member

    This place is dry like the desert, anybody lubin' the deals or what?

    Thanked by 3Savvy emgh r3k
  • emghemgh Member, Megathread Squad

    MS said:
    This place is dry like the desert, anybody lubin' the deals or what?

    East Bound and Down!!!

    Thanked by 2_MS_ r3k
  • _MS__MS_ Member

    @emgh said:
    how many minutes of music did you guys listen to 2024

    Miss the guy. :cry:

    Thanked by 3emgh FAT32 r3k
  • tanseltansel Member

    @tansel said:

    @donli said:

    @tansel said:

    @kuroit said:

    @raza19 said:
    Dear @kuroit we r sincerely missing ur presence. I realize u r preparing for Christmas/boxing Friday deals but wt a pleasant surprise it wud be if u offered 7 Dollah Singapore deals at the auspasciois occasion of having reached 1000 pages. That's just good business and karma

    You have been heard!

    1 vCore
    1GB DDR4 RAM
    15GB SSD/NVMe Disk
    1TB Bandwidth @ 1/10Gbps Uplink

    Supported Locations:
    West Midlands, UK
    Dallas, USA
    Tampa, USA
    Los Angeles, USA
    Ashburn, USA
    The Netherlands
    Singapore

    Price: 7.77GBP/Year with promocode
    Promocode: YOU-KNOW-WHO!
    Stock: 5

    Order: https://my.kuroit.com/store/sale-offers/bf-worldwide-1vcore-1gb-ram-15gb-disk

    Is there still stock available? I missed it, and the promo code shows as expired.

    Do you even know where you are!?

    Am I causing you any trouble by asking if there’s still stock available?

    Am I causing you any trouble by asking if there’s still stock available? Regards.

    Thanked by 3Savvy donli r3k
  • emghemgh Member, Megathread Squad

    @MS said: Miss the guy.

    Yes ;(

    I did about 52k minutes 24! B)

    Thanked by 2_MS_ r3k
  • emghemgh Member, Megathread Squad

    @tansel said:

    @tansel said:

    @donli said:

    @tansel said:

    @kuroit said:

    @raza19 said:
    Dear @kuroit we r sincerely missing ur presence. I realize u r preparing for Christmas/boxing Friday deals but wt a pleasant surprise it wud be if u offered 7 Dollah Singapore deals at the auspasciois occasion of having reached 1000 pages. That's just good business and karma

    You have been heard!

    1 vCore
    1GB DDR4 RAM
    15GB SSD/NVMe Disk
    1TB Bandwidth @ 1/10Gbps Uplink

    Supported Locations:
    West Midlands, UK
    Dallas, USA
    Tampa, USA
    Los Angeles, USA
    Ashburn, USA
    The Netherlands
    Singapore

    Price: 7.77GBP/Year with promocode
    Promocode: YOU-KNOW-WHO!
    Stock: 5

    Order: https://my.kuroit.com/store/sale-offers/bf-worldwide-1vcore-1gb-ram-15gb-disk

    Is there still stock available? I missed it, and the promo code shows as expired.

    Do you even know where you are!?

    Am I causing you any trouble by asking if there’s still stock available?

    Am I causing you any trouble by asking if there’s still stock available? Regards.

    You asking yourself?

    Thanked by 2Savvy r3k
  • tanseltansel Member

    @Savvy said:

    @tansel said:

    @donli said:

    @tansel said:

    @kuroit said:

    @raza19 said:
    Dear @kuroit we r sincerely missing ur presence. I realize u r preparing for Christmas/boxing Friday deals but wt a pleasant surprise it wud be if u offered 7 Dollah Singapore deals at the auspasciois occasion of having reached 1000 pages. That's just good business and karma

    You have been heard!

    1 vCore
    1GB DDR4 RAM
    15GB SSD/NVMe Disk
    1TB Bandwidth @ 1/10Gbps Uplink

    Supported Locations:
    West Midlands, UK
    Dallas, USA
    Tampa, USA
    Los Angeles, USA
    Ashburn, USA
    The Netherlands
    Singapore

    Price: 7.77GBP/Year with promocode
    Promocode: YOU-KNOW-WHO!
    Stock: 5

    Order: https://my.kuroit.com/store/sale-offers/bf-worldwide-1vcore-1gb-ram-15gb-disk

    Is there still stock available? I missed it, and the promo code shows as expired.

    Do you even know where you are!?

    Am I causing you any trouble by asking if there’s still stock available?

    You forgot to close your comment with "reguards" and that is considered rude here

    Thank you for your guidance, I have learned it. Regards.

    Thanked by 3emgh Savvy r3k
  • _MS__MS_ Member
    edited December 2024

    @emgh said: Yes ;(

    His documentary was always painful to watch.
    Avicii: True Stories (2017)

    @emgh said: I did about 52k minutes 24!

    Don't know. Still use offline media like a chad data hoarder. :smiley:

    Thanked by 2emgh r3k
  • tanseltansel Member

    @JohnFilch123 said:

    @tansel said:

    Am I causing you any trouble by asking if there’s still stock available?

    Nope but the answer is kinda obvious.

    I don’t think there’s anything wrong with inquiring about the stock. Did I violate any forum rules? If so, I apologize to you. Regards

    Thanked by 1r3k
  • emghemgh Member, Megathread Squad

    @tansel said:

    @JohnFilch123 said:

    @tansel said:

    Am I causing you any trouble by asking if there’s still stock available?

    Nope but the answer is kinda obvious.

    I don’t think there’s anything wrong with inquiring about the stock. Did I violate any forum rules? If so, I apologize to you. Regards

    He meant because it's such a good price, and only 5 in stock, it'll obviously be sold out within minute(s)

    This discussion isn't worth continuing regards

    Thanked by 5stackr raza19 FAT32 JohnFilch123 r3k
  • stenysteny Member
    edited December 2024

    @plumberg said:

    @steny said:

    @plumberg said:

    @steny said:
    @plumberg said:

    any recommendations for selfhosted llm which would help with code generation? Claude does amazingly well, but I end up making it work so much that I am rate-limited.

    You need quite a lot of

    @plumberg said:
    any recommendations for selfhosted llm which would help with code generation? Claude does amazingly well, but I end up making it work so much that I am rate-limited.

    For a self hosted LLM you need a lot of VRAM especially for coding. For home PC probably the best model you can run is currently Qwen2.5-32B-Coder . For coding, unlike chat you need at least half(8 bits) precision, so that means you will need around 40GB Vram, e.g. Dual rtx 3090, which is a setup I am using. Bigger models, like Qwen2.5-72B or LLama-Nemotron-70B are better, but you won't run it at home at that precision unless you build 4xGPU rig. For the largest open weight models like Mistral Large, you need dual H100 to run it, so you definitely need to rent GPU and won't be cheap but you get around a performance of GPT-4o in coding there, probably slightly less since you will still run it in half precision only.

    First off, thanks for the detailed post.

    So here is the deal. I have 0 GPU. But have a pair of dual E5-2699v4 pair with decent RAM (384 gb ddr4 2400 speeed or something).

    I am not really interested in getting fastest responses. As long as it spits out decent I am game.

    Or am I dreaming of hosting a llm ? What are your thoughts?

    Running on Ram would be awfully slow, especially those large models you could theoretically run with that amount of ram.

    How slow are we talking about? Any idea?

    And will that change the quality of the output?

    Reguards

    The inference speed is mainly dependend on memory bandwidth, Dual rtx 3090 runs 70B model around 15-20 tokens/second. There is some speed loss due to dual setup, yet DDR 2400 bandwidth is about 50xtimes less of 3090, So expect bellow 1 Token per second, where token is like 3-4 characters. And that is just a middle sized models, the large ones would be in fractions of token per second. The quality would be the same though.

    Thanked by 3emgh FAT32 r3k
  • stackrstackr Member

    @emgh said:
    This discussion isn't worth continuing regards

    Can continue just for the regards

    Thanked by 2emgh r3k
  • plumbergplumberg Veteran, Megathread Squad

    @steny said:

    @plumberg said:

    @steny said:

    @plumberg said:

    @steny said:
    @plumberg said:

    any recommendations for selfhosted llm which would help with code generation? Claude does amazingly well, but I end up making it work so much that I am rate-limited.

    You need quite a lot of

    @plumberg said:
    any recommendations for selfhosted llm which would help with code generation? Claude does amazingly well, but I end up making it work so much that I am rate-limited.

    For a self hosted LLM you need a lot of VRAM especially for coding. For home PC probably the best model you can run is currently Qwen2.5-32B-Coder . For coding, unlike chat you need at least half(8 bits) precision, so that means you will need around 40GB Vram, e.g. Dual rtx 3090, which is a setup I am using. Bigger models, like Qwen2.5-72B or LLama-Nemotron-70B are better, but you won't run it at home at that precision unless you build 4xGPU rig. For the largest open weight models like Mistral Large, you need dual H100 to run it, so you definitely need to rent GPU and won't be cheap but you get around a performance of GPT-4o in coding there, probably slightly less since you will still run it in half precision only.

    First off, thanks for the detailed post.

    So here is the deal. I have 0 GPU. But have a pair of dual E5-2699v4 pair with decent RAM (384 gb ddr4 2400 speeed or something).

    I am not really interested in getting fastest responses. As long as it spits out decent I am game.

    Or am I dreaming of hosting a llm ? What are your thoughts?

    Running on Ram would be awfully slow, especially those large models you could theoretically run with that amount of ram.

    How slow are we talking about? Any idea?

    And will that change the quality of the output?

    Reguards

    The inference speed is mainly dependend on memory bandwidth, Dual rtx 3090 runs 70B model around 15-20 tokens/second. There is some speed loss due to dual setup, yet DDR 2400 bandwidth is about 50xtimes less, So expect bellow 1 Token per second, where token is like 3-4 characters. And that is just a middle sized models, the large ones would be in fractions of tokens per second.

    Gotcha. Well I wanna try it out though and see where it takes me. Thanks.

    Thanked by 3emgh steny r3k
  • emghemgh Member, Megathread Squad

    @stackr said:

    @emgh said:
    This discussion isn't worth continuing regards

    Can continue just for the regards

    regards

    Thanked by 1r3k
  • donlidonli Member

    @emgh said:

    @tansel said:

    @JohnFilch123 said:

    @tansel said:

    Am I causing you any trouble by asking if there’s still stock available?

    Nope but the answer is kinda obvious.

    I don’t think there’s anything wrong with inquiring about the stock. Did I violate any forum rules? If so, I apologize to you. Regards

    He meant because it's such a good price, and only 5 in stock, it'll obviously be sold out within minute(s)

    This discussion isn't worth continuing regards

    And by "minutes" he meant "seconds". Regards.

    Thanked by 2emgh r3k
  • plumbergplumberg Veteran, Megathread Squad

    Regards
    Reguards
    Re guards
    Re guar ds

    Wich it s corect?

    Thanked by 5Savvy emgh FAT32 error12358 r3k
  • emghemgh Member, Megathread Squad

    @plumberg said:
    Regards
    Reguards
    Re guards
    Re guar ds

    Wich it s corect?

    reg

    Thanked by 3FAT32 plumberg r3k
  • stackrstackr Member

    @donli said:

    @emgh said:

    @tansel said:

    @JohnFilch123 said:

    @tansel said:

    Am I causing you any trouble by asking if there’s still stock available?

    Nope but the answer is kinda obvious.

    I don’t think there’s anything wrong with inquiring about the stock. Did I violate any forum rules? If so, I apologize to you. Regards

    He meant because it's such a good price, and only 5 in stock, it'll obviously be sold out within minute(s)

    This discussion isn't worth continuing regards

    And by "minutes" he meant "seconds". Regards.

    Every out of stock deal saves at least $7 . Regards

    Thanked by 6emgh FAT32 Saragoldfarb tansel uptown r3k
  • emghemgh Member, Megathread Squad

    @stackr said:

    @donli said:

    @emgh said:

    @tansel said:

    @JohnFilch123 said:

    @tansel said:

    Am I causing you any trouble by asking if there’s still stock available?

    Nope but the answer is kinda obvious.

    I don’t think there’s anything wrong with inquiring about the stock. Did I violate any forum rules? If so, I apologize to you. Regards

    He meant because it's such a good price, and only 5 in stock, it'll obviously be sold out within minute(s)

    This discussion isn't worth continuing regards

    And by "minutes" he meant "seconds". Regards.

    Every out of stock deal saves at least $7 . Regards

    onidel saved you just a little over 3 bux reg

    Thanked by 5FAT32 plumberg Saragoldfarb uptown r3k
  • emghemgh Member, Megathread Squad

    reg my i m hard

    Thanked by 2uptown r3k
This discussion has been closed.