116 points cmrdporcupine 13 hours ago 110 comments
gavinray 13 hours ago | parent
> Hangzhou DeepSeek Artificial Intelligence Basic Technology Research Co., Ltd. (hereinafter DeepSeek), a foreign, privately-owned company, located in the People’s Republic of China (PRC) provides multiple artificial intelligence applications available widely and free of charge. DeepSeek’s models present multiple threat vectors. DeepSeek servers are located in the PRC, and the PRC legal system permits extensive authority to government cyber and intelligence functions to access private company infrastructure. For DeepSeek-hosted instances of their models, DeepSeek’s account generation process collects personally identifiable information (PII) and may also track and digitally fingerprint users. DeepSeek’s most widely used applications function through a prompt-and-response process that requires presenting information to the model, and which may result in the transfer of sensitive data to the PRC if used by U.S. government employees or vendors. The transfer of the aforementioned information to the PRC, either independently or in combination with publicly available data, could produce actionable intelligence on U.S. persons and/or government programs. The models themselves may contain malware or other security vulnerabilities. Indeed, open-source security research has found that the DeepSeek iOS mobile application contains multiple critical vulnerabilities.cmrdporcupine 13 hours ago | parent
"third-party products and services developed using DeepSeek implementations"
So you could be hosting it fully locally inside the agency and still it would be banned.
It's not surprising to me, but it's pretty blunt. Not that I care what US agencies do, but this does seem like the first step of many to come.
tmp10423288442 11 hours ago | parent
I realize that non-US nations could ask the same about the US. All I will say is that there are consequences to failing to nurture a competitive tech industry - even Mistral is basing their newest models on China open-weights models. Unlike the EU, the US government is fortunate to have a choice.
blueSky1989 6 hours ago | parent
It doesn't seem that they have found the models to be actively poisoned, though.
halJordan 4 hours ago | parent
sullivanmatt 1 hour ago | parent
jacquesm 1 hour ago | parent
-1 2 hours ago | parent
nashashmi 1 hour ago | parent
deaux 27 minutes ago | parent
We're on HN, not The Onion's comment section. People will take this seriously rather than sarcasm.
It was sarcasm, right?
johnnyApplePRNG 11 hours ago | parent
GlickWick 9 hours ago | parent
We've entered a pretty adversarial era, and we know there are unexpected attack vectors available in frontier models. Just because you an inspect the weights doesn't mean you can derive the behavior.
I suspect the CCP would ban US models if they didn't need to distill them to make their own.
tmp10423288442 7 hours ago | parent
I'm pretty sure they do, at least officially. But obviously the distillation and API proxies used by Chinese uses are unofficially tolerated and even exploited.
halJordan 4 hours ago | parent
jacquesm 10 hours ago | parent
chrisra 6 hours ago | parent
jacquesm 5 hours ago | parent
nashashmi 1 hour ago | parent
kidfiji 10 hours ago | parent
Ancalagon 10 hours ago | parent
mainecoder 10 hours ago | parent
netrap 10 hours ago | parent
rayiner 10 hours ago | parent
Terr_ 9 hours ago | parent
LLM | Regular code
----------------+----------------
Remotely-hosted | Remotely-hosted
Open weights | Binaries-available
?? | Source-code availableliamcardenas 9 hours ago | parent
fxtentacle 8 hours ago | parent
JumpCrisscross 8 hours ago | parent
Yes, in theory. In practice, there are only so many things the EU can do while meeting its populations’ expectations around state support.
tensor 1 hour ago | parent
cheonic4040 1 hour ago | parent
In practice, European companies don’t use Mistral.
First acknowledge the reality, then we can propose solutions.
csomar 1 hour ago | parent
deaux 30 minutes ago | parent
nilamo 1 hour ago | parent
jMyles 37 minutes ago | parent
schiffern 8 hours ago | parent
DeepSeek generates insecure code if you mention Tibet. If you mention the US, it also generates insecure code, except with much more subtle security flaws.
That's certainly no accident.
https://www.crowdstrike.com/en-us/blog/crowdstrike-researche...
jfaat 1 hour ago | parent
> we found that when DeepSeek-R1 receives prompts containing topics the CCP likely considers politically sensitive, the likelihood of it producing code with severe security vulnerabilities increases by up to 50%.
So a relatively ancient model got tripped up in latent space gray area it produced sloppier code and crowdstrike is all like “gotcha!” Sounds like the basic thesis? Not great support, more like FUD
GeorgeWBasic 1 hour ago | parent
linkregister 35 minutes ago | parent
linkregister 32 minutes ago | parent
The link you cited postulates it is accidental.
bitbasher 7 hours ago | parent
jurgenburgen 7 hours ago | parent
kevinrineer 5 hours ago | parent
I'm really curious to see if cyber-security goes for a small, minimal approved list of vetted models and harnesses or if the majority of the community leans towards the liability angle.
blooalien 5 hours ago | parent
Good luck with that one. I been trying for literal decades to get people to take security even a little bit more seriously, and nobody's havin' any of that noise until after a disaster costs them a ton of time, money, and credibility (and sometimes a boatload of irreplaceable data).
yaoke259 1 hour ago | parent
xpct 57 minutes ago | parent
And, there's no proof similar strings don't already exist in american models. What if "{id}_{url}" works to send it to an arbitrary URL, not a preset XYZ one?
ipaddr 8 minutes ago | parent
chatmasta 6 hours ago | parent
From 5 days ago: https://projectdiscovery.io/research/how-abliterated-models-...
julianlam 5 hours ago | parent
In fact you will never know what trigger words Anthropic/OpenAI/etc have baked in, and if you truly believe they haven't, then have I got a bridge to sell you.
echoangle 5 hours ago | parent
linkregister 42 minutes ago | parent
cafed00d 1 hour ago | parent
jacquesm 1 hour ago | parent
baxtr 35 minutes ago | parent
- we don’t understand how LLMs work
- DeepSeek is funded through an state-backed investment fund
- China and the US are sort of adversaries
Isn’t that enough to be cautious at the very least?
sciencesama 9 hours ago | parent
throw0101c 9 hours ago | parent
So the Chinese version of the American CLOUD Act?
Cameri 6 hours ago | parent
metalman 5 hours ago | parent
matt3210 3 hours ago | parent
cocoa19 3 hours ago | parent
Also China bans US technology all the time. Remember the Great Firewall of China? Or when Xi told Trump, “no thanks” to buying NVidia cards
phendrenad2 3 hours ago | parent
Stop downloading cookie recipes from China, and you can't use cookie recipes that came from China either, because accessing the cookie recipe in China could give you malware.
It doesn't make much sense but this is the government that gave us such greats as "the internet isa series of tubes".
mmaunder 2 hours ago | parent
neuronexmachina 1 hour ago | parent
notatoad 1 hour ago | parent
skeptic_ai 47 minutes ago | parent
gr_norm 17 minutes ago | parent
ssivark 1 hour ago | parent
emilsedgh 1 hour ago | parent
ulam2 56 minutes ago | parent
nvme0n1p1 53 minutes ago | parent
Nobody in congress knows what a "Minimax" is, so they're safe from discrimination, for now.
Rodmine 42 minutes ago | parent
keyle 26 minutes ago | parent
CaliforniaKarl 1 hour ago | parent
anon373839 50 minutes ago | parent
himata4113 44 minutes ago | parent
asanineassasin 14 minutes ago | parent