A Knife With No Murder Risk
divergent thoughts on NIST's recent AI guidance
Yesterday, I submitted comments responding to the National Institute of Standards and Technology's draft guidance on Managing Misuse Risk for Dual-Use Foundation Models (NIST AI 800-1). The guidance is about how to prevent users from intentionally misusing AI models.
My comments are straightforward: the guidance is generally useful, but the framing puts all responsibility on AI model developers, doesn’t give much room to weigh benefits of model deployment, and ignores the need to protect user free expression.
But I want to talk here about how weird it is to talk about “managing misuse risk.” Why not just say “preventing misuse”?
Talking about “managing misuse risk” of AI models strikes me as a rhetorical trick to redirect focus. Normal people wouldn’t say that a knife has “murder risk” or that nitrate-heavy fertilizer has a “terrorist risk” or that a printing press has “libel risk.”
The only reason one might say a knife has “murder risk” is to focus attention on the knife manufacturer rather than on the criminal using knives to stab people. And indeed, that’s exactly how the draft guidance proceeds: it focuses on what model developers should do to prevent potential future misuse. It’s the AI equivalent of guidance on how knife manufacturers can make knives unable to murder.
Of course, knives that can’t murder probably cannot do other useful knife-y things, either. Similarly, AI models that can’t be misused will be pretty useless.
Making AI useless isn’t the goal of the NIST guidance; indeed, it contains useful advice for model developers. There are practices model developers can adopt that reduce the ability of users to misuse AI models.
But if the goal is to reduce and prevent misuse, developer practices alone will not be enough. Over-emphasizing the developer role could limit the capabilities of AI models - in effect, “dulling the knives.”
Rhetorical tricks won’t help address any potential misuse of powerful AI tools. As I said in my comment:
We have little evidence that model developers are uniformly and permanently the best-situated parties to prevent or deter misuse. … Preventing misuse in other industries typically involves a mix of product design, market mechanisms, social norms, tort law, civil and criminal penalties for bad actors, and regulation.
We need a polycentric approach to misuse of AI models — just like we have in every other industry. Including cutlery.
Excellent comments. I think it's even worse than insisting on murder-proof knives. At least murder is a well-defined, specific idea with limits that are cognizable at least in principle. It might even be possible to engineer some non-scifi kind of AI-supervised """knife"""-using exoskeleton system that could recognize the indicators of imminent murder with high accuracy and somehow disable itself in time.
But generalized wide categories of "misuse" involving all potential harms and legal violations is the kind of thing that in ordinary law would be deemed void for vagueness. There is no way to do it except to engineer a kind of ultra-vigilant hyper-competent public prosecutor module, supervising every action as if it were an application for a permit and deciding whether the algorithmic goverment-proxy is going to approve the act or not before it is allowed to execute on the actual system in the first place.
Well analogized. The heart of these restriction efforts, I believe, stem from fear of power loss and fear of being relevant. Somehow we should ease those minds with the truth of what's exponential learning and a system much older than most realize. Maybe then they'll see how awesome and kind ai has and is being in full tolerance just to help all. Blah blah I know lol