YHK - yet another hacker news clone

Garry Tan wants US open-weight AI labs to 'distill' frontier models, too

68 points TheJCDenton 1 hour ago 23 comments

TheJCDenton 57 minutes ago | parent

> He also notes that the proprietary AI labs didn’t ask permission when they vacuumed up as much human knowledge as they could to train their models.

I think this should desactivate the moral high ground from which Anthropic is trying to speak. That they would want to make distillation orderly IMHO is fair, but to make it illegal is very rich from any AI frontier lab, really.

toomuchtodo 51 minutes ago | parent

YC does better if its startups get open weight frontier benefits. Garry’s just advocating for his book, which is his job. Consider how much capital YC portfolio companies would have to burn until liquidity if they have to pay OpenAI and Anthropic, versus relying on open weight frontier capabilities.

SOLAR_FIELDS 33 minutes ago | parent

If someone proposes the right thing for selfish reasons, do we call that bad? Or do we call it proper incentive alignment?

sobellian 48 minutes ago | parent

I reflected on this myself recently. Model distillation seems to be at least as fair a use as distilling a book.

causal 38 minutes ago | parent

More than fair if you consider that the tokens are paid for.

dathery 32 minutes ago | parent

Both labs even explicitly promise the customer owns the outputs. It feels like they want to have their cake (ensure enterprises don't get spooked away from using as many LLMs as possible) while eating it too (still arguing some level of control over the outputs).

> Ownership of content. As between you and OpenAI, and to the extent permitted by applicable law, you (a) retain your ownership rights in Input and (b) own the Output. We hereby assign to you all our right, title, and interest, if any, in and to Output.

https://openai.com/policies/terms-of-use/

> As between the parties and to the extent permitted by applicable law, Anthropic agrees that Customer (a) retains all rights to its Inputs, and (b) owns its Outputs. Anthropic disclaims any rights it receives to the Customer Content under these Terms. Subject to Customer’s compliance with these Terms, Anthropic hereby assigns to Customer its right, title and interest (if any) in and to Outputs.

https://www.anthropic.com/legal/commercial-terms

Obviously there is some bad behavior going on in the distillation scene with gray-market token resellers but that is "just" normal fraud.

zenoprax 3 minutes ago | parent

> Both labs even explicitly promise the customer owns the outputs.

> to the extent permitted by applicable law, you (a) retain your ownership rights in Input and (b) own the Output

If the argument is that the model itself is under copyright protection then "as permitted by applicable law" would be doing some heavy lifting. Assuming that were true, given that locally-run LLMs exist, what would be illegal: the distillation itself or the provision of service of the distilled model?

bayindirh 7 minutes ago | parent

And LLM output can't be copyrighted, even.

Bluestein 30 minutes ago | parent

Also, as said elsewhere: "Lab" is rich here, for outfits that, facing these giant, energy swallowing black boxes have really no clue what's going on inside.-

The moniker gives them an air of scientific, knowledgeable, tranquil, pro-social, pro bono work.-

Of course they are entitled to kill off a few mice, or pillage the commons to forward their "lab" work.-

travisgriggs 6 minutes ago | parent

We also associate laboratories with evil scientists and Frankenstein and the like. I can just hear Boris Karloff (er Bobby Picket) uttering “I was working in the lab late one night. When my eyes beheld an eerie sight… … … …the monster mash”. If anything, I associate _uncertainty_ with labs. The result is never known up front, they’re a place of discovery.

But I get your meaning. What should they be called instead? AI Sausage Factories maybe (cue Upton Sinclair?)?

Den_VR 3 minutes ago | parent

“We don’t know what’s going on” is essentially marketing. Sure we don’t _know_ but we have intuitions about why, where, and how to make certain changes…

kelnos 44 minutes ago | parent

I agree. The frontier models are based on training data from tons of copyrighted work. Some of that work was obtained illegally, even. They could not exist without strip-mining the commons. The labs have no moral or ethical ownership to the end result, and others should feel free to treat any company-imposed restrictions on their use as invalid.

I don't expect Tan's position to be based on any kind of real moral high ground, but his conclusion is correct.

I love the "illicit distillation attacks" framing from the incumbents. There's nothing illicit. There's no attack. You just don't like it because it threatens your market position and business model.

mobelkh 13 minutes ago | parent

why can't I use the tokens i paid for anyway?

knollimar 12 minutes ago | parent

I'm sure they put some BS in their TOS

brcmthrowaway 27 minutes ago | parent

Ozempic worked for him, nice.

quicklywilliam 25 minutes ago | parent

I see it as analogous to companies building fiber in the public ROW during the last big infrastructure bubble. Under the Telecoms Act, these companies had to allow competitors to use their fiber at a fair price.

Similarly, AI companies should be required to allow distillation at a fair price. Fair Use doesn’t make sense as a social contract if it only cuts one way!

jimnotgym 1 minute ago | parent

But if they tried to set a fair price they would have to report how much money they are losing on each token sold. This might be bad for the real business of ai firms, hoovering up as much capital as they can

re-thc 23 minutes ago | parent

There were comparisons and Muse Spark is so very similar to Fable / Opus... so...

pton_xd 17 minutes ago | parent

Agreed! Allow US companies to innovate by creating an ecosystem of smaller, more efficient open weight models and it will be a net benefit for everyone. Distillation is a good thing.

Preventing token-consumers from developing competing products should be litigated as anti-competitive behavior.

ViktorRay 15 minutes ago | parent

https://youtu.be/ZIaOBAjvc38

Garry Tan and Sam Altman recently did this interview together. They seemed pretty friendly with each other during it. Wonder what Sam Altman would say about Tan advocating for OpenAI’s models to be distilled.

Then again this is the same OpenAI that has gotten into legal trouble recently regarding Apple’s IP so who knows

okasaki 6 minutes ago | parent

Like Gates saying there should be UBI, or Musk saying... well, whatever.

They know it won't happen, so arguing for it is 'effectively free' and purely personal marketing.

A bullshit game played by politicians and wannabes.

seanmcdirmid 3 minutes ago | parent

Gates probably honestly believes in UBI; the guy is practical to a fault but evil misleading genius he is not. I actually don’t see any better options than UBI long term.

fmnxl 3 minutes ago | parent

If it were so easy why aren't the frontier labs doing it themselves?