No AI org of any significant size will ever disclose its full training set, and it’s foolish to expect such a standard to be met. There is just too much liability. No matter how clean your data collection procedure is, there’s no way to guarantee the data set with billions of samples won’t contain at least one thing a lawyer could zero in on and drag you into a lawsuit over.

What Deepseek did, which was full disclosure of methods in a scientific paper, release of weights under MIT license, and release of some auxiliary code, is as much as one can expect.

By Taiwanese law, TSMC isn’t allowed to move cutting edge processes to its US plant. The overseas operations have to be at least one gen behind.

From a strategic point of view, it makes sense for the Taiwan government to do this. They don’t want the US to suck them dry then cut a deal with the mainland.

Also, the release of R1 under the MIT license means that in principle anyone can use R1 to generate synthetic training sets for improving other (non-reasoning) models. This may be a real game changer.

The one fly in the ointment is that Deepseek didn’t deign to share details of their synthetic data generation procedure. But they are already way more transparent than any other non-academic AI lab, so it’s hard to get mad at them over this.

Try it out for yourself: https://chat.deepseek.com/

It can understand LaTeX as well as outputting it. In my limited testing on sample physics problems, it performs pretty well. It also scored 100% on the 2023 A Level maths exam.

It’s MIT licensed, so anyone is free to go about decensoring it. There are already “abliterated” (decensored) variants uploaded to huggingface, at least for the distilled models.

This procedure also decensors stuff that western models routinely censor. So ironically these Chinese open source models are giving us the most free speech friendly LLMs around.

It’s an interesting subject. If not for Beijing’s heavy hand, could Chinese internet companies have flourished much more and become international tech giants? Maybe, but there is one obvious counterpoint: where are the European tech giants? In an open playing field, it looks like American tech giants are pretty good at buying out or simply crushing any nascent competitors. If the Chinese did not have their censorship or great firewall, maybe the situation would have been like Europe, where the government tries to impose some rules, but doesn’t really have much traction, and everyone just ends up using Google, Amazon, Facebook, etc.

More Americans have TikTok accounts than vote. For a shitload of normies who have only the vaguest notion of politics and current affairs, the app they’ve been enjoying gets cut off as the defining event of the waning days of the Biden administration. They are not going to care about how Trump tried to do it first, or it was bipartisan, or whatever. It’s hard not to see how this will cost Dems dearly.

TikTok and its service providers are liable. “No one is enforcing” is meaningless, because they can still be prosecuted retrospectively if the US Government changes its mind.

Google has behind it an incoming US government that puts US economic interests first, and relishes bullying its allies. The EU is weak, divided, and geostrategically boxed in. It will bend the knee.

Lots of commentators seem to be under the impression that the EU is going to stand up against the US and Big Tech. My impression is the exact opposite; they are going to roll over. The EU is pretty good at throwing its weight around in areas where the US is not paying attention and doesn’t feel its interests are at stake. But in areas where the US wants to elbow the EU out of the way, it does so pretty effortlessly, and Brussels just looks embarrassed and tries to forget anything happened.

The Turing Test codified the very real fact that computer AI systems up till a few years ago couldn’t hold a conversation (outside of special conversational tricks like Eliza and Cleverbot). Deep neural networks and the attention mechanism changed the situation; it’s not a completely solved problem, but the improvement is undeniably dramatic. It’s now possible to treat chatbots as a rudimentary research assistant, for example.

It’s just something we have to take in stride, like computers becoming capable of playing Chess or Go. There is no need to get hung up on the word “intelligence”.

Deepseek trained their v3 model for $6M. That’s the AI equivalent of building it in a cave with a pile of scraps. There’s no longer any reasonable way to stop China from developing frontier models.

For the vast majority of military applications, including missiles, you do not want to use bleeding edge chip tech. You use 50nm or higher, anything with smaller feature sizes is not robust enough for a military environment.