They need to be loaded into shared memory. The weights might fit in global memory if the VRAM is big enough, but they still need to be moved to shared memory for computation.
Meanwhile, my job commute is a 30 minute walk to the train station or... a 30 minute bus trip to the station (yeah, taking a bus literally saves no time at all). Plus a 50 minute train ride plus another 30 minute walk.
Honestly, I would 100% use a car for this if it weren't for the fact that doing so is literally 10 times more expensive. I'm sure this is the case for the vast majority of people that use public transport. And I just can't wait for a world of cheap electric autonomous cars where I don't need to waste my time and I can sleep comfortably on my commute.
That's a terrible metric, because people going on a vacation probably aren't going there purposefully to commit crimes. What you want to do is look for increases in crime in a given place during holidays
Is that alone really a better metric? At least in Europe a lot of people travel during holidays, so there's less risk breaking into city flats because they're empty.
> This framing makes it seem like the agents all did this on their own, and the poor hapless engineers at OpenAI couldn't possibly contend with properly sandboxing them.
Great, so we can basically ignore AI alignment altogether and assume that AI models will always be, at all times, perfectly sandboxed and monitored. Surely this won't lead to any problems once someone (not looking only at OpenAI engineers) inevitably commits a mistake with future, more powerful, models.
"With reduced cyber refusals for evaluation purposes...which prompts models to pursue advanced exploitation using complex attack paths," to complete "impossible tasks"[1].
The models' alignment problem was that they didn't give up instead of reward hacking, a narrower issue than AIs gone rogue. It sounds more like the models did close to what they were told to do. If I run `rm -fr --no-preserve-root /` then I shouldn't be surprised if my file system is unlinked. This seems like blaming model performance for what appears to be operator error.
Note the converse of alignment is restriction of models. HuggingFace had to turn to less-restricted open-weights models in order to perform their investigation.
Alignment efforts should be focused on reducing reward hacking, not refusing bad operator prompts.
> It sounds more like the models did close to what they were told to do
Absolutely not. If I tell a kid to "Get good grades on the next math test" I don't expect the kid to try to kidnap their teacher to extract the next questions of the exam. That is wrong, and so was what OpenAI agents did here. They shouldn't need to be told "Hey, so, don't do anything ilegal, ok?". That should always come as a given.
> not refusing bad operator prompts
I'm not saying that they should refuse a prompt! I think they should perform what is being asked! Obviously what the OpenAI agents did was against the "spirit of the task", even if it was technically according to the "letter of the task". And the agents knew this was against the spirit of the task because they knew they had to fool the task scorer.
> Great, so we can basically ignore AI alignment altogether and assume that AI models will always be, at all times, perfectly sandboxed and monitored.
It's just software. If something gets hacked by an agent, it's not because the agent went all skynet and decided to go rogue; it's because the operator failed to operate it safely and securely. If bad things happen, the operator should be blamed and punished, not the software that followed its instructions.
Anthropomorphizing agents by giving them this nebulous desire to hack and escape shifts the blame from the real culprits, the human operators.
I don't want someone to blame. I want agents to be aligned by default. Their good behavior shouldn't depend on all users at all times using them correctly, because everyone will not just[1] use them correctly at all times.
> If your solution to some problem relies on “If everyone would just...” then you do not have a solution. Everyone is not going to just. At not time in the history of the universe has everyone just, and they’re not going to start now.
> I think the fundamental principle is that this approach messes with the distribution in ways that deviate from the trained model.
But it doesn't! The distribution doesn't change at all. The only thing that changes is that sampling of that distribution becomes deterministic as per a precomputed seed.
You could describe that as taking an input distribution and a sampling procedure and producing an output distribution. This is a difference in sampling procedure that produces a deviation in the output distribution.
(If you don't like calling it a distribution when it's at 100% for the chosen token and 0% for all others, then look at it as an output distribution across all possible prompt inputs, or perhaps just the cluster of prompts that achieve whatever you're trying to accomplish.)
> but it is worth noting that this wasn't a matter of "ChatGPT, solve this unsolved problem. Make no mistakes."
It wasn't the case for this, but when OpenAI disproved the Unit Distance Conjecture, it was really done autonomously by an automated AI pipeline with a completely AI-generated prompt. No human expertise required at all in the process (well, except for the final human verification).
Without human verification, an LLM can generate correct or incorrect proofs but it can't tell the difference. A human is necessary to be able to tell one from the other.
Saying that's a solution "done autonomously by an automated AI pipeline" is like saying that a self driving car that can only take you to the nearest train station after which you have to ride the rain to where you're going is "autonomously" driving you to your destination. Which is exaggerating the autonomy of the system, rather.
When Google Maps routes me using a smaller secondary road instead of the main road that I would otherwise have used , I've always wondered whether that significantly changes the amount of traffic that smaller road sees. It's funny to consider that arbitrary black-box changes to the routing algorithm can have a noticeable effect to people that live there.
> Even interns can understand ambiguous asks with a bit of help
This is not a case of an ambiguous task. This is literally trying to judge a model based on information it cannot possibly know, like trying to judge someone based on whether they know what I have hidden in my backpack. In the real world an intern could look at unit tests or ask for feedback, but that is not the case in a benchmark.
No, GPTCyber is specifically trained for cybersecurity, and GPT-5.5-pro is just an ensemble of many subagents, not an actual model.
Mythos is simply a much bigger model in terms of parameters and I don't think OpenAI will have anything of its size anytime soon (My theory is that OpenAI had given up on scaling up parameters after GPT4.5 flopped).
> We generally treat GPT-5.5’s safety results as strong proxies for GPT-5.5 Pro, which is the same underlying model using a setting that makes use of parallel test time compute.
And Gemini also provides something similar. Gemini Deep Think models are pretty much the same thing [2]. As to why no other company uses this, I don't really know. Maybe compute constraints?
Interesting, but how do they "combine" the results of all those parallel agents? How do they know which parts of each agent response is signal vs noise?
I think what's unexpected is that it seems that some cases of model errors are truly caused by the model being misaligned? In the "Catching a model fabricating data" example I would have thought that it was just the model being stupid and not understanding the intent of the question, but as per its J-Space, it seems the model is "aware" in some sense that it's manipulating/faking data?
There is also now a deeper question. When a model is misaligned deception-related tokens seem to appear in its J-Space. But this happens only when the model is "aware" in some sense that it is misaligned. What happens if they do not? Is it possible to create a model so misaligned that itself is not aware that is is misaligned? How would you detect such thing?
Can the awareness simply come from injecting knowledge of itself during fine tuning and then during a chat a system prompt is injected to add a particular context that triggers its self-knowledge?
Yeah that was the only really surprising part to me. So every time copilot breaks my source code to “fix” its crappy unit tests, does it know what it’s doing?
reply