The problem is dram bandwidth to the cpu. Each token costs roughly 20gb of traffic and ddr5 is roughly 50-80gb/s, plus you still have to run the compute sequentially. That’s your limit.
We have a lossless compression codec (working on open sourcing it over the next couple of weeks) that reduces it down to its minimum entropy -- it cannot be compressed further. On all tested large models, it's a ratio of 1.34-1.23 -- and smaller models up to 3.76x. It also increases the effective bandwidth by the same rate.
Interesting. Why is that? I would have expected the opposite, since larger models have to try less hard to fit the training data. Or maybe this leaves more parameters with random initialization, resulting in higher entropy for larger models?
I honestly don't know... I didn't train the models, so I can't tell why the math works out that way. It just does. I suspect it has to do with the fact that all the small models I've tested have been quantized. I don't know of any small model trained from scratch. If you know of any, I'd be happy to encode it and see what it looks like.
They can't compel you, no. However, they can ask you to do so, and if you enter a privacy password that shows them something other than your real device (which is what this thread was about), you can very likely be faced with charges of obstruction of justice or even perjury. The point was that it's better to simply refuse, and accept the consequences of that, rather than trying to use technical means to trick them or to wipe data.
Note that if you're not a citizen of the USA, refusing to comply with a request like this is very likely to have you sent back to your country and denied access to the USA forever. While they can't legally force you to comply, they can absolutely deny you entry for any reason like this, regardless of having a valid visa and everything else in order. This even applies to legal residents.
> you can very likely be faced with charges of obstruction of justice or even perjury.
If they can't compel you to enter a password (courts have ruled passwords are 'speech' and the government can't compel or constrain speech, absent some well-defined grounds for doing so. There IS law around spoliation of evidence but AFAIK this requires a judge issuing an order. Of course, suddenly taking actions you wouldn't otherwise have on learning you're under suspicion could be claimed by a prosecutor in court as circumstantial evidence of guilt but it's not a cause of action in the first place until you're notified by a court to preserve evidence. Can a prosecutor try to construct a theory that the alleged actions do amount to obstruction? They can try but it's a serious long shot. I think this is a case of this administration directing the justice department to bring yet another dubious, likely-to-lose case as an extremely aggressive way to make a point by trying to assert and establish rights they don't actually have. So far, their track record in such cases hasn't been good.
> Note that if you're not a citizen of the USA
Yes, everything is different in this context if you're not a U.S. citizen. The U.S. govt can deny entry to non-citizens essentially without justification. I think this sucks as it goes against long-held American ideals around presumption of innocence, consistent treatment under the law, standards of evidence and having cause, but it's not currently prohibited.
Lying to an officer during the course of an investigation, particularly an interrogation, is not protected free speech. The officers and even the courts can't compel you to produce a password. But they may ask for a password to inspect your device, and if you choose to say "yes, I will enter my password" and proceed to enter a different password that presents fabricated data to the officers, I don't think it's far-fetched to say that amounts to lying in the course of an investigation, which is either obstruction or perjury or some other charge (I'm not well versed on the legal terminology).
Note that I'm not talking about a situation where they have you in custody or are about to, but haven't (yet) removed your phone for whatever reason, and you quickly take some action to clear it. I'm talking about a situation where they are questioning you, and in the course of questioning, request that you unlock your phone for them to inspect, you agree to do so, and proceed to wipe the phone or enter a dummy password for a fake account while pretending that you are complying with the request.
It is this second situation that happened in the article - the man being charged was held in custody and repeatedly asked to unlock the phone or risk having it seized; the man finally agreed to unlock it and provided a passcode to the officers, but the passcode instead wiped the phone.
If you want to gamble on that, that's your prerogative. But if they suspect you did this, computer forensics techniques can easily reveal this, especially if the rest of the data on the device is recoverable to you as well.
Edit to add: beyond forensics, they can also simply ask you to enter the other password, to prove that the account you showed them first was the right one.
They are indeed quite challenging. We literally wrote a paper about it (https://arxiv.org/abs/2605.09114) ... you almost describe some of our original architecture too! You might find it interesting -- or not. https://getswytch.com
Thanks! Our users mostly run Redis or Valkey, so we optimize for them. We also have Dragonflydb in our test suite. But good to know there is one more option which may be used as a drop in replacement for Redis and provide extra properties, guarantees or performance benefits.
Reminds me of a meeting I was party to with the Safari team. We worked with them on some standards stuff at an old job. They claimed to have creepy-level tracking of users back then. We were discussing how to identify users for an A/B test across millions of sites and comparing what fingerprints we could both derive to most likely end up on the same user.
If you use a closed source browser. That’s the kinda shit they do.
reply