New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Comments
They do: https://softwarerecs.stackexchange.com/
"it's mathematically impossible to avoid hallucination with LLMs" that is in fact a fully correct statement, because:
1) An LLM is a prediction machine that operates using only data contained in its training set, anything requiring data that is outside its training data set is out of reach.
2) An LLM has zero deduction capabilities, it can not reason deductively and because of that it cannot generate new correct data, it can only generate new data (with mixed/random results) because you need an operating world model to be able to reason deductively, you need to understand how the world works to be able to follow its rules properly and by that generate new correct data having started with older less complete data. Having a world model allows you to internally expand your data set via deductive reasoning. To my knowledge only humans can do that!
3) Now ask an LLM something that requires data that is not present in its training data set. Despite not having the data necessary to provide an answer and being unable to expand its dataset internally because it does possess a world model, the LLM will answer anyway and without hesitation, confidently. Now guess what kind of answer you will be getting,
An hallucination, of course, or more than one hallucination. UNAVOIDABLY!
Final Note: In some very restricted cases an LLM will come out of training already imparted with a world model, that will happen when said world model is so small that its rules are fully embedded in the training data itself, such cases are, for example, board games and some small subareas of mathematics, in those cases the LLM will in fact possess a world model but that's because its training data is already quite complete, there is almost nothing missing. In those cases the LLM will not hallucinate because it knows everything pertaining to that world, you can ask it anything and you can be sure you will get correct answers.
None of that constitutes a proof. This is what a proof looks like, in this case proving that φ(n) > λ(n) for any semiprime n:
And even that's a sloppy proof because it doesn't show why ∀a,b : lcm(a,b) ≤ ab (and "it's obvious" isn't a proof)!
Everything you stated are conjectures and assumptions. Now, I agree that LLMs are garbage and I doubt humans will ever be able to design a hallucination-free transformer network, but that's still not a proof. Once something is stated to be mathematically impossible, the bar to prove such a thing is raised significantly above what one would normally consider a reasonable standard of evidence. Hell, we don't even know if π itself has a random distribution of numbers, even though most mathematicians would bet on it! A proof is much more than an argument, no matter how well-reasoned.
Also, your statement 1 is wrong. Transformer models can interpolate information that they were not trained on, sometimes quite well. The problem is that, when they can't do it, they tend to make up bullshit and confidently state it as fact.
About your final note, even with a small world, a modern LLM is still likely to hallucinate at times. Perhaps you're referring to other types of ML models which can reasonably be infallible for small systems.
Sit back and relax, don't need to buy up ram at current prices. Demand increases price, as the demand is not getting fulfilled new companies will try to capture the market. Within a year, it will again start to go down things wont scale forever. Just like smartphones, we were seeing massive jumps in performance every year look at now
You are not even wrong> @forest said:
Those statements you just made are not even wrong. Sorry!
Perhaps we are talking about different things. Are you aware of what a mathematical proof is, and why something can be true even if not formally proven? Because right now, you are trying to argue against a research paper that showed with an off-hand remark (a proof by minimal counterexample) how easy it is to produce an LLM that does not hallucinate, albeit one with a utility no greater than a simple lookup table.
What you posted, whether or not the conclusion is correct, is not a mathematical proof because:
A formal proof would need to show that no possible architecture, training procedure, or inference mechanism fitting the (unstated) definition of LLM could avoid this behavior across the complete space of possible designs.
Is it time for delay-line memory to make a comeback?
An LLM that does not hallucinate will be one that often refuses to answer questions, an LLM that ALWAYS provides an answer will hallucinate for sure unless its dataset is fully complete which would mean it knew everything there is to know.
Let me use Schrödinger's cat as an analogy, only this time the LLM is the cat and the training dataset is the box. Now imagine the cat (LLM) trying to come up with answers about what is happening in the world outside its box (data outside its training dataset.)
The cat knows nothing about what is happening in the world outside so what can he do but hallucinate the answers? It's a simple matter, there's no need to complicate, it's just like the original Schrödinger's cat but with roles reversed. In the original Schrödinger's cat problem you have no way to know if the cat is dead or alive, all you can do is hallucinate an answer or refuse to answer, same with the LLM analogy.
The difference is in Quantum Mechanics you are fully aware of your limitations, you know you cannot provide an answer while an LLM has no awareness of its limitations it will "happily" offer you an answer that might be wrong. The only sure way for it to not hallucinate is to refuse answering,
But here we're discussing LLMs that ALWAYS provide an answer because those are the real LLMs we interact with, academic exercise particularities are of no interest here.
NOTE: You've picked the wrong math tool, if the only tool you have is a hammer you'll treat everything as if it were a nail.
Front up: Yes, @forest is right - but also not, because "proof must be mathematical" is too tight a constraint.
In other words, forest and @stable_genius are basically talking about different things. (That said, generally, albeit not in this case, I'm more on forest's/math's side).
... is an example of an answer to a question an ai actually would be useful.
Reason: the way ai works, summarized quite well by stable_genius; an ai basically is a guessing machine, a quite massively pre-trained one but still a guessing machine. There's also a second phase (often called "adaptation") during which in a kind of conversation with real "experts" errors are corrected, etc.
The problem though is that the set of known to be correct solutions ALWAYS is but a quite small sub-set of the question the ai later in "production" will be confronted with, mainly due to economical reasons (which hints at yet another problem with ai).
Another very - and increasingly - major problem is that the pre-training data are from serious and knowledgeable sources only to a small degree, the largest part is from "pretty much anywhere and everywhere", incl. from ai output!, i.e. ai also get fed - and to an increasingly large part - the excrements of ai!
Yet another major problem is that the domain of inputs/data available to ai unavoidably is but a relatively small portion of existing input. Not even due to lack of processing power but structurally inherent because information is not only communicated by text and images due to humans being humans.
Example: emotions, creativity (also in language!), psychological "games" (ex. boss saying "I like you and your working performance" may, depending in particular on intonation, but also on facial expression, situation, and other factors, mean pretty much everything between what the words suggest and the contrary).
I'll close constructively. There are things an ai can do well and reliably, like e.g. find differences, even minute ones (like a semicolon being there or not), but at the end of the day ai just is a glorified computer system - and computers are (a) insanely fast, and (b) stupid, plain stupid. (And YES I like using them a lot).
Coprophagy in biological systems is not a good practice, it will probably not be that good in computer information systems either.
I think we should go back to the basic. What is intelligence itself
I wasn't the one who picked that tool, nor were you. I started by replying to someone else who did. I think you might have perceived that as me defending LLMs capabilities', which I certainly cannot defend because they are environment-destroying, RAM-stealing semantic parrots that are turning people stupid.
I'm not saying that proof must be mathematical to be useful. Kodomu said "hallucination in LLMs is mathematically impossible to solve", so I asked for a citation. He proceeded to link to a research paper that actually said the opposite of what he thought it said.
I think that LLMs will never be hallucination-free, regardless, and I believe that one could reasonably argue that without needing to invoke a mathematical proof of impossibility. Hell, in cryptography, we don't even have mathematical proof that AES is secure (for some definition of secure), but we'll still bet our life on it!
Indeed. I despise vibe coding and wish a pox upon everyone who uses it in production, but I think it has an extremely good potential for code review. LLMs are terrible at catching issues that humans notice quickly, but they are very good at catching issues that humans struggle with. I suspect future LLMs' use in programming will be less agentic in nature and more like a glorified version of
-Wall -Wextra -fanalyzer.There's no one definition. Typically, its definition includes the ability to learn, problem-solve, and think abstractly in order to achieve goals in a wide variety of environments. Depending on the precise definition used, agents (human, non-human animal, or machine) may be described as intelligent. By the minimal definition I gave, LLMs certainly fit, but their abilities are currently quite shallow. By some (common, useful) definitions, bees are highly-intelligent. By other (equally common and useful) definitions, the very concept of intelligence cannot be applied to bees any more than it can be applied to rocks.
The goal should not be defining a term as if the definition is somehow an intrinsic property of the concept itself, but coming up with a definition that is useful.
I'll go further and say that intelligence is something that has no clear definition boundaries.
For example, diverse "standard" IQ tests can reasonably well test a spectrum of intelligence that covers about 80% of people - along a given concept of what intelligence is supposed to mean. But those tests miserably fail - within the boundary of their own definition! - once a proband gets close to their (usually self-ascribed) boundaries (roughly 140 and probably about 85) or even crosses them.
From then on (direction higher but that's all I'm interested in) it becomes clear how "vague" intelligence is. In more extreme cases (>= 160) suddenly "weird" phaenomena pop up, like for example, a person well above 160 (as measured by special tests (and IMO still poking in the fog) said, he cares about beauty and elegance pretty much only. Another one declares utter lack in IQ and in particular IQ pissing contests, his driving force (somewhat like the first one) is music, but there's a but: only specific music, and he is very quickly enchanted or disgusted (by whatever music he is offered); and interestingly there is no clear line, classics, pop. etc., singing or instrumental only, no clear line; the only criterion found out was that he must have a certain feeling when hearing it, but there is a clear negative factor, guess what: beauty again with an almost extreme intolerance for certain things and/or attributes.
All in all (well, there are only so many persons with very high IQs and willing to participate in a study ...) it shows that IQs above 160 (and maybe somewhat lower but then usually hand in hand with other factors) is, as one of them somewhat annoyed explained "not just more but qualitativly radically different" and the reason for his being annoyed was (my summary) "those idiots think that my brain works like theirs, just with 'more' of something. It's almost as if a dog assumed that you have the same brain he has, just with more 'thinking power'".
Why I wrote this? Simple: Good luck trying to give intelligence to a machine! Frankly, I highly doubt that they are able to even "only" reach IQ 120 (a normal smart person).
The concept of intelligence and the levels of intelligence are orthogonal. IQ is a method which, while it has high validity (i.e. it's measuring "something" and that something is consistent within an individual and is not noise), it only measures a subset of what one would consider to be human intelligence. Attempting to measure the IQ of a non-human, whether it's another primate or an LLM, is fun but pushes the IQ testing methodology beyond what it was designed for and results in nonsensical results. In fact, even among humans, there are outliers where it doesn't work very well (e.g. savantism).
But yes, IQ tests struggle at the extremes. I think the high extreme is actually 140 rather than 160. Once you hit 140, there are so few samples that a few seconds delay can be the difference between achieving 145 and 160. Hence IQ is generally only useful to give a general idea of intelligence (insofar as it only models a subset of intelligence) which can be useful for predicting the kind of support and education one may need/benefit from.
To use a benchmarking analogy, I consider IQ to be akin to BogoMIPS: Higher is better and lower is worse, and it stays roughly consistent, but it's damn near useless as far as precision benchmarking and real-life comparisons are concerned.
I dunno, Chrysomya megacephala seems to be doing just fine! But leave me out of that practice!
Wait a year for China to suddenly be manufacturing most of the world's RAM, or AI collapses.
The Western capitalist model has a peculiar habit of shooting itself in the foot, while whatever China's been doing seems to get serious results, and fast.
China remains a viable economy because of the "western capitalist model"'s demand for its cheap goods and services.
But their population pyramid isn't looking good, and their physical infrastructure will probably give out before the population does.
I'm not betting on China this century.
Everyone loves cheap goods and services, but only China is producing them any more, while the rest of us play shell games in a rapidly depreciating currency.
And their capacity to do so isn't tenable given their internal (and external) trajectory. When we in the west continue to subsidize the illusion of Chinese prosperity and stability, we put a bandaid on a bullet wound. It's lose-lose for everyone.
Yes, about 140 seems to be the region where most ("standard") tests become foggy.
small side-note: time ("seconds delay") seem to be largely related to knowledge tests. In IQ tests, at least the ones I know, time usually is "open ended" or generously allocated.
The problem rather seems to be what is tested and how it's tested, which almost is "the average Joe and Jane", i.e. The range between about 85 and about 130, maybe 140.
Fun fact and making it even more difficult: intelligence can change over time. I happen to know someone who at a very young age (about 10 - 12 years old) had an "astonishingly high IQ" (in the range of 145) in large test in school, about a decade later it was 156 and with about 35 he had 164 in a test administered by some high-IQ society (who tended strictly to go lower when in doubt).
Good luck to implement that, ai people! *g
Btw, that is not up-to-date. Actually "cheap! cheap!" has shifted to other Asian countries and China is increasingly moving towards "good quality for a reasonable price" and towards high-tech.
Also re "western capitalist model's demand for cheap goods and services" don't forget that China has nearly 1.5 billion people and an increasingly strong interior market of its own!
Why not? What do you think will stop China from surpassing the USA in everything? Especially now the USA's gone full you-know-what. The only one I can foresee is that they succumb to the same disease we have, of arguing about numbers instead of producing.
Well, for sure they won't succumb to the easily preventable childhood diseases that we are succumbing to now despite the availability of efficacious vaccines..
I typically formulated it as knowledge plus recall and reasoning. Some people would add adaptability and creativity into the equation (I'd emphasize the latter as a type of entropy, which is important too).
If I were to formalize AI hallucinations, it's something caused by the heuristic process (including sycophants) and/or its inability to recall facts during the divide&conquer and forward chaining.
Intelligence is not about knowledge and only peripherally about recall. "Reasoning" is such a vague and wide term that mentioning it in this context is a bit like stating that driving is about moving. True but useless (ly vague).
Intelligence is such a vague concept with no operational definition, so you're both right.
While some (mistaken IMO) assert that there also is "social" intelligence, I posit that aiming for harmony is NOT helpful wrt the definition, or even just a slightly better understanding, of intelligence.
But thanks for your judgement.
I agree, and just because it's vague doesn't mean that all definitions are equally valid, but the definition really needs to depend on how it is being used. Thus in some cases it may fundamentally require recall, but in others it may not.
The memory and performance sections are generally time-limited, with an (effectively) unlimited number of steps, bounded only by the time limit. At least for the WAIS.
@forest
A very simple proof that every LLM hallucinates:
Given an arbitrary LLM and its training dataset look at the data and formulate a question that requires data that is not in the dataset. You know that the LLM has no way to provide a valid answer and what you're getting is an hallucination, which means that the LLM hallucinates. You can do this to every LLM so the proof is complete.
This is simple logic so this is math too, you could rewrite it in the symbolic language of mathematics if you wanted but that would not be very useful. The symbolic language of Mathematics is very useful when you're dealing with set theory, topology, geometric algebra, etc but not in this case, it is the wrong tool.