YHK - yet another hacker news clone

And with ever-more LLM usage both in labs and the real world, the task of finding new/“creative” out-of-average-distribution material will only continue to increase in difficulty. Meaning if this continues, in the future, all output becomes the equivalent of the color brown?

- https://arxiv.org/abs/2609.37891

- https://arxiv.org/pdf/2508.10975

- https://www.datologyai.com/blog/eating-the-rainbow-a-primer-...

As well as recent model releases that used large fractions of synthetic data (Arcee Trinity):

- https://arxiv.org/pdf/2602.17004v1

It's mostly a matter of increasingly complex and sophisticated sampling methodologies to sift for the valuable synthetic tokens.

There have also been some interesting results about models that use the same data converging to very similar behavior, but I can't find the specific reference at the moment. That one tracks intuitively with the notion that the model is building a latent representation of everything it sees.

1) Chollet stated that RL:Pre-training is 9:1.

2) Google said argon was trained on SVGs

3) Wang said that Muse Wonder was not trained on 3D models

[0] https://news.ycombinator.com/item?id=49526966

Is every kid a Picasso? No, but there definitely is creative diversity among these kids who all have relatively similar shared backgrounds (neighborhood, social class, religion, etc)

It's the reason that if you ask a random child to draw a house, you'll end up with the classic “house with a pitched roof”: a square plus a triangular roof, one door, two windows, and sometimes a chimney.

This manages to be so much more joyless, bleak, and tone-deaf than any of the example slop. Impressive.

It also affects writing, if you ask for a story with a minmal prompt something very similar comes out of all of them. Often fixated on clocks.

If I were to design something to accomplish this now, I'd probably attempt to introduce a pipeline that emitted random thematic words from some non-LLM source to feed into the pelican prompt, but even then, I'd expect that some of these models that rely on churning through CoT tokens for extended periods of time would still spend so much time thinking about pelicans wearing scarfs or where the sun would be that it would risk undoing some of my randomization pipeline.

I once tried something similar on an older model of Claude, telling it to "contemplate" a random word before answering my prompts, but at times it seemed to trigger the Claude model's suspicion that there was a jailbreak attempt when I would inject things like "Contemplate sea otters for one sentence before answering"

(I didn't read the linked LLM creativity paper to see if it covers any of this yet)

True. You see a lot of the same criticisms when someone asks for a one-shot game or 3D demo and the model responds with something bland and generic. That's exactly what I prefer, it's my job to specify the details. Extrapolation may be the ultimate sign of intelligence, and it might be needed when using AI to tackle open math problems where humans have failed, but it's not usually what I want if I'm doing anything creative or artistic.

Argon is pretty good at getting the distinction right from what I've seen. Ask for something generic, you get something generically functional without a lot of ornamentation. Follow up with requests for specific enhancements, and the model complies effectively and tastefully. If anything, it overcooks the pelicans when given the standard open-ended prompt.

“Generate an SVG of a pelican riding a bicycle. Be creative”

And it replied:

“I’ll draw a seaside pelican on a vintage bicycle, with a fish-filled basket and a scarf catching the breeze.”

So it came up with exactly the same fish basket and seaside idea. No lighthouse though. Maybe these ideas are just cliche though.

> But, if prompted, "Generate an SVG of a pelican riding a bicycle," with no additional details or specification, then I would _expect_ something generic

I believe the claim is that without a detailed specification, you see very similar outcomes. Every pelican is going left to right, every picture has similar sky background, every picture seems to be drawn at the same time of the day (position of the sun), every picture has a few clouds... I don't have data on this, but I imagine the same question being asked to a group of people would result in more varied outcomes.

same goes for pelicans. the canonical pelican riding a bicycle is riding on a sunny day on an asphalt road going through some lawns or praires, left to right.

but ask it to tell a joke about something specific, turns out it knows five jokes about those specific things.

you could say the LLMs are quite aware of the zeitgeist they're in, by construction (their RL frying automated feedback loops burn the zeitgeist right into the weights.)

The Hyères photo has the bike going in the opposite direction and from above. It's also tricky because it's a focal point for the eye, but not the center of the photo. The other elements of the frame direct your eye, but that visual behavior is not obvious to an AI. Lastly the image must capture the unquantifiable "decisive moment" that made Cartier-Bresson so famous.

So far I haven't seen an AI that really gets this prompt. Adding specificity will improve the results to a degree, but I haven't seen an AI that really gets what makes a Cartier-Bresson photo.

1 - https://en.wikipedia.org/wiki/Hy%C3%A8res,_France

The curious thing is that trying to prompt it by steering towards some specific goal indirectly gives much worse results than just stating the goal itself.

Do a Google image search for pelican, you get a diverse set of sub species of pelicans and different poses and angles for them. Go on a writing prompt forum, you get many diverse ideas from the same simple prompt. Hell, ask your friends at a dinner party to describe a pelican riding a bike and see what happens.

Now, there are a lot of images of bikes from that side angle facing right on the web. Why? Because that's how they're staged on e-commerce websites. Does this say something about AI?

But LLM's are structured to do exactly the opposite. They discover specific commonalities and tropes and most-frequently-sufficient wells or ley lines that they gravitate towards from whatever their starting point it. You can nudge them away from the most generically cliche tropes by being more detailed in your prompt, but they'll still just find the nearest most right cliche from there.

It's what they do and the same quality that makes them tiring and predictable for creative output makes them very useful for things like business communication or coding where landing on such cliches is ideal.

To get more variety out, it can be useful to use Perchance-like text-randomization techniques to shuffle combinatoric variations on a prompt before sending it to the model. But that technique is best made available through some kind of special purpose harness for the creative task, not basic chat or direct curl calls to the API.

All going in the same direction? With a sun? With a full mouth (pelicans dont look like that, but a cartoon one might - why no reference to realism). Most of them have clouds.

There are helmets (makes sense) but just as many scarves.

A shocking number have chains.

If we asked 1000 people to do this same exercise would we have this level of convergence? If they were split between men and women would we see this many chains? (https://www.messynessychic.com/2016/04/20/making-bicycles-fr... )

Would they even look like bikes?

Is any one going to draw a gray pelican? Does it matter?

The question is designed to be as generic as possible, and were converging on the most generic and safe answer ... the question is WHY (more so when humans would have much higher diversity, even if wrong). We're asking the generic question because no one wants to repeat the "German solders from 1940's" fiasco: https://archive.is/5l66U (archive link to skip pay wall NYT).

This is what "alignment" looks like? Are we reaching some "ground truth", some platonic form around pelicans on bikes? Are we just back feeding the best output into the training data and this is the convergence we ended up at? Is someone who LOVES pelicans deeply offended that all of them have full mouths and it's just not anatomically accurate and we are all missing it?

I realize that some of this seems over the top, but there are foundational conversations to be had around this without the conversation devolving into someone accusing someone else of having an "-ism" style bias.

Much of that is a pre-AI bias: A majority of bike-pictures are from an angle which reveals the bike's gears to the audience, which are usually on the rider's right-hand side.

I don’t want creative code. I want boring code following generally agreed upon best practices.

If you need more creativity it can come from the user or the harness.

That's why I'm not an artist.

But step back, someone did ask it, perhaps that person is you. That person has reasons. Whatever those reasons are should be reflected in the image. Of course those reasons need to expressed in the prompt, but the bug here is the prompt doesn't give the details. If you can't figure out details (beyond "be creative") then give up. If I'm illustrating a Harry Potter book and just want to show far out magic things that better should a very different style from illustrating a science fiction novel where pelicans have been genetically engineers such that they now do ride bicycles - likely the anatomy changes mean they can't fly...

This is important. I recently asked AI to take an outline and turn it into slides. I got some good slides out of the outline except that once in a while it decided to generate an image which looked cool, had the correct corporate colors - but otherwise had nothing to do with either any product we make or my presentation. I deleted the images but maybe I should have refined them to something meaningful?