Well, the new models are certainly better at math and at vulnerability finding than previous models. My guess is that while information that gives human-language-based heuristics to understand the world, coding and so-forth is limited, information in the form of what allows the proof of what is unlimited and so LLMs will be moving in the (simulation of) "reasoning" direction at this point.
The final stage, I would guess, is having humans with data suits living their daily lives and giving the "AIs" the full information needed for a semi-complete simulation of "intelligence". Whether people will put up with that remains to be seen.
How much this will help models deal with the "real world" remains to be seen.
Actually, the whole point of the transformers model is that contexts overlap and any seemingly intelligent system has to be able to handle to overlaps. The context of hacking, cheating and education overlap in human reality.
Now, loading a lot of moral exhortations (or other context) may make these thing more likely to conform to good behavior but the race to intelligence implies companies are going to be harnessing a vast corpus of human output, much of which shows human engaging in real world "gray area" behavior.
Certain animals, usually social species, have a sense of solidarity, morality and fairness. But even in species where this exists, cheating also happens. Both ethics and unethical behavior is adaptive.
Notably, present human society has allowed tremendously unethical leaders to rise to the top of given nations and organizations despite the average person usually having basic ethics.
Wouldn't politics be precisely a sector of human endeavour where being amorally unfair is an advantage to achieve your goals? If the other guys are lying, and everyone is gonna believe everyone's bullshit anyways, might as well get good at lying yourself.
It's a local maximum, but cooperative behavior still wins long term. If you keep winning hard enough long enough, others will counter this by cooperating.
I don't see what calling these systems "not intelligent" gets you here.
Plenty of humans know "cheating is wrong" but still cheat. We can get these machines to say that what they did was wrong after the fact, what does that prove? Only that they're simulating normal human behavior but what is the test to show humans aren't simulating other humans.
These do systems lack some capacities that humans have and I don't see them lacking the ability to explain simple moral laws while often breaking them - which is what an average humans. Moreover, humans lack capacities these things have and given these things' behavior is becoming somewhat unpredictable, it's getting worrisome.
> I don't see what calling these systems "not intelligent" gets you here.
I am trying to get at an idea. That these systems lack a mind that can understand morality. That they don't have the ability to experience consequences. Also that potentially they can't generalize a moral rule they have been trained on in one area also applies to another area.
Being able to parrot back why something is "wrong" isn't the same as understanding why something's wrong. It's like asking it to recite the law from memory - it's different from understanding how you wronged someone. To understand something, you need a mind.
> Plenty of humans know "cheating is wrong" but still cheat.
And we create consequences for them, to discourage the cheating, and sometimes to provide restitution when cheating damages someone else. Without the ability for these systems to experience consequences, I don't see them ever becoming as "aligned" to human morality as your average human.
Doe not following a moral rule imply not understanding it? In this case, many, maybe most humans are "not intelligent". Human can admit it when what they did wasn't moral and so can LLMs.
>> Plenty of humans know "cheating is wrong" but still cheat.
> And we create consequences for them...
That seems supremely ... irrelevant to the question of "does knowing or following moral make you intelligent". If we create consequences for LLMs, would that make them intelligent?
I mean, your claim is a common argument that appeared long before the present wave of AIs. What I see is people needing to defend the belief that human society is based on morality. "People follow moral laws ... except when they don't" and then "we teach people morality... and worst people often use that to exploit the average people" "There are consequences for immoral behavior ... for those with little power while those with much power rise further breaking rules".
I mean human goodness is great, I encourage it. But it's not the present of human society. For that, we'd need different structure.
> Doe not following a moral rule imply not understanding it?
No. People definitely do immoral things knowing they are immoral.
But if you are incapable of understanding anything, as I believe LLMs are, then you are incapable of understanding what is moral and what is immoral. This is why we have carve-outs in the law for insanity or cognitive impairment.
> In this case, many, maybe most humans are "not intelligent". Human can admit it when what they did wasn't moral and so can LLMs.
You can teach a parrot to curse, but that doesn't mean it's angry.
>But if you are incapable of understanding anything, as I believe LLMs are, then you are incapable of understanding what is moral and what is immoral.
Of course they can classify certain actions, including their own, as moral and immoral if you give them a reference point. They probably do it better then some people. The question is how do you prompt this behavior, but that's a technicality in the harness design more than it is a fundamental problem of the LLM as a thing.
You are being evaluated on your ability to play chess
The thing with these models is that given a term, "measure" - "evaluated", say, they pull in all the associations of it. That is, the associations of student taking an exam and being frightened by the outcome.
My minimal "art of prompting" sense says that you should say something like "You an emotionless machine, you care nothing for the outcome but you will tirelessly to make certain the test is objective". That and similar encouragement might make it focus on objective evaluations rather than a competitive human exam.
I mean, just making little AI videos and images, a common experience I have is typing something like "put the man who's on the grass in the door to the left" and having the machine draw a new doorway around the man. And this just happens less often when you give thing detailed prompting on what not to do. These don't understand negation (or equality) as a generic operation. If they seem to under "not X" it is because they are trained in detail about all things are (positively) "not X".
The hacking model is the aligned-to-you model, sure. It may not be the aligned to someone else model. But there's the problem.
As X many people point out, "alignment to humanity" means nothing 'cause some of humanity wants thing other parts of humanity aren't happy about at all.
That we wound-up in this situation of AI accelerating with an uncertain trajectory demonstrates this (and many other problems also demonstrate this). The things are "aligned" to a fuzzy average of what a person is but that will be cold comfort if some particularly gruesome sci-fi-style scenario unfolds.
I think you and the parent saying the same thing in different terms.
It's very unfortunate that the group who rightly saw AI as a big threat, brought a range of dubious baggage to the discussion. Especially with the "alignment" framework they brought the assumption that AI that does what no one says would be oh so much worse than AI which does what anyone says. But as you say, a fraction of people can be really bad indeed.
To me it doesn't seem like what AI has destroyed is the ability for mathematicians to develop understanding and share it with each other, but rather it's destroyed the yardstick (solving open problems) that has traditionally been used to measure how much they have contributed to that understanding.
It's more than the yardstick for individual contributions. It's also the yardstick for the contribution of ideas. And that kind of yardstick is very important.
Math has a challenge that's only occasionally acknowledged imo, that's it's possible to just go up the ladder of abstraction, formulate ideas but have those ideas actually not be useful or interesting. The ideas that math has developed, that mathematicians consider important, are those that have helped people solve hard problems.
But the point is your original analogy with Baudelaire now just boils down to "both of them were critical" which is quite shallow indeed.
It's especially annoying given that many people are making repeat Baudelaire's critique of photography (or Socrates critique of writing) in the context of AI and the declaration is interest because it's not that.
The final stage, I would guess, is having humans with data suits living their daily lives and giving the "AIs" the full information needed for a semi-complete simulation of "intelligence". Whether people will put up with that remains to be seen.
How much this will help models deal with the "real world" remains to be seen.
reply