New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
To me intelligence is the capability an agent has to run a mental model of the real world, the better that model is at describing and predicting the events that happen in the real world the more intelligent that agent is.
That'd be an Oracle®
Isn’t AI bottleneck is electricity? Hardware production will ramp up but with electricity it is harder.
Currently, it's memory BW (then size).
No, not an oracle.
In technical fields an oracle provides flawless answers to inquiries in a very specific and limited subject so it is not general and as such it is not a true general intelligence. In non technical fields an oracle offers prophecies, or answers to inquiries generally in a cryptic fashion and it is a fraud, so that does not count either
An intelligence is a mechanism that can model multiple facets of the real world and how it works, it has to be general and it is always fallible.
AI as we have it today has multiple bottlenecks the most serious one being reality shock.
I think the main one would be inter-GPU communication, such as nvlink (although that may count as memory BW, since they're exchanging memory contents to "pool" multiple GPU's memories together).
I suggest to watch the (formerly movie, but now) docu "idiocracy" to get an impression what ai is fed.
Unless the response is "I don't know" which doesn't count as a hallucination
An LLM is an advanced probabilistic prediction engine, it operates by estimating the most likely next token (word, character, or subword) based on the context provided by preceding text. There are cases where all the probabilities are very low and then the LLM "knows" that it does not know the answer but that is not always the case because LLMs of real value work with imperfect datasets, that is incomplete data or data that contains errors, in may cases the probabilities look good but the answer is an hallucination.
An outside agent, one in possession of a complete dataset without errors and also in possession of the imperfect dataset used to train the LLM will be able to formulate a malicious question and lead the LLM to hallucinate.
An LLM that has been trained on a dataset that is both complete and without errors is not expected to hallucinate, you can't trick it to hallucinate, but low uncertainty cases are special and not of great value when compared to the general case where uncertainty is high.
You only find real value in those cases where you are able to make the right call despite the uncertainty, not when there's no uncertainty. Being right when there's no uncertainty is not as good as being right when uncertainty is high.
So, all LLMs trained on imperfect data hallucinate.
EDIT: This is a milder statement but it still covers all situations of real value because real value arises when you deal successfully with uncertainty.
To avoid critical "noise", let's be perfectly correct. llms are not trained; that phase is called "pre-training".
The critical point however is that all pre-training is done on ridiculously huge data collections, ideally on sensibly selected collections, practically though on pretty much everything and anything available, including massive web scraping and its own excrements.
But then, ai is based on a brute force approach rather than on an intelligent approach (pun not unwelcome). And that trend is bound to continue due to reasons I already hinted at and do not wish to elaborate ("thou shall not anger and excite the dumb ones"...).
Which to the best of my knowledge is all ais.
So, the true race is to not get even more flawed. A race that they are very likely to lose. But maybe ever more colourful nonsense images will sooth the ai victims ... uhm, user crowd.
For a modern transformer model? Of course you can. They are not perfect even if their training data is.
When in doubt nationalize
Not anymore, these days everything is being privatized: public property, the commons, even some people's lives are now owned by some corporation.
Nationalize? That was then this this is now.
The world will never be the same! AI is consuming the world.. HDD/RAM/Electricity and now jobs
AI job loss was always a lie. They are just trying to cut jobs and AI is an excuse.
We need to fight back. Is there some script to use on idling machines, by loading multiple AI platforms using lots of nonsensical requests? Such a project deployed on all idling machines would make corporations spend some money. Even though investors will be saved by governments, at least this load might help for this AI bubble to end.
https://iocaine.madhouse-project.org/
I don't know if loading it with garbage is the solution. I am thinking of many people having idlers. Using those idlers to generate AI requests, might make corporations lose more money on AI.
Loading it with garbage burns through their money by wasting training time.
Then a solution could be a combination between the 2. Have some site generate random nonsensical text using AI every hour, afterwards have multiple idlers use AI to interrogate that website with random garbage questions.
https://www.pcgamer.com/hardware/memory/relief-that-the-worst-is-over-or-just-the-calm-before-the-hurricane-continues-some-reports-suggest-that-dram-prices-have-plateaued-or-even-fallen/
LOL, HBM3E platform will soon be obsolete
Yeah, but I was mostly referring to what would need to be done, not what's likely to happen
buy chinese ram they said… great deal they said… until one dimm decides 3am is reboot o’clock 😅
Nand makers need a "No Cash, No Chips" policy to prevent AI CEOs from overbooking supply to stifle competition and by proxy choking the entire consumer PC/electronics industry.
Thankfully HBM is not particularly relevant to the average person due to its high latency, so I don't care if Nvidia buys up all the HBM.
Plot twist: you juice the numbers so much and they get billions in buyouts.
It could still go like that. They won't be shelving all their half-finished chips in a warehouse - they'll still be making the chips but stockpiling them for OpenAI. When OpenAI doesn't pay because they're broke, those chips will go on the general market and prices will return to normal.
We might find that enterprise hardware is cheaper than consumer for a bit. But there's nothing special about enterprise hardware. It's still just hardware, and you can game on a pre-owned rack server if you want to.
Only if you have really good noise cancelling headphones.