Yet another Hacker News

WorldTUI: I trained a 360M model to pretend to be a terminal

71 points alphax314 4 hours ago 32 comments

obilgic 1 hour ago | parent

There was a whole fake gui OS with similar concept right

prodigycorp 1 hour ago | parent

athropic had an experiment that did exactly this. we need more experimentation in this space.

andai 49 minutes ago | parent

VibeOS - Fully Hallucinated Operating System

https://www.youtube.com/watch?v=z3pV6FHvcgM

htoz1 1 hour ago | parent

Where is the point of this project if i have terminal ?

dotancohen 1 hour ago | parent

Because it was there

mehmetoguzderin 1 hour ago | parent

Terminal world models have many applications, but empirical observation of successful pretraining or fine-tuning can be significant for encapsulating from rather expensive Bash traces, such as ones obtained from systems with scarce accelerators, etc.

YmFzZTY0ZGFkZHk 1 hour ago | parent

Sometimes it's just fun to do something absurd because you can. This is one of the reasons software is exciting.

good-idea 1 hour ago | parent

because curiosity is the most fun of virtues!

aghuang 36 minutes ago | parent

Was thinking the same thing. I just run my usual terminal and get to work.

antonvs 1 minute ago | parent

This will allow you to delete your terminal app and, eventually, all other binaries in your system!

oilkillsbirds 1 hour ago | parent

Very cool. Fun project. I feel like there’s a wide field to be explored once you understand fine tuning good existing models

psandor 1 hour ago | parent

Reminds me of this project from Microsoft. A Quake II “simulation” model where the whole application was just a model.

https://www.microsoft.com/en-us/research/articles/whamm-real...

soulofmischief 58 minutes ago | parent

These are called world models, which is why the project is called WorldTUI. For another example, take a look at Google's recent Genie models. https://deepmind.google/blog/genie-2-a-large-scale-foundatio...

__mp 1 hour ago | parent

It would be interesting to expose this on the internet. I wonder what the scrapers etc… would do with it.

prodigycorp 1 hour ago | parent

Wow! This this reminds me of when we tried doing this with gpt 3 and.. it worked!

The tmux thing had me confused for a moment until i realized it was emulating tmux. Super fun. Is the model window width aware?

serje1480 18 minutes ago | parent

good old days https://news.ycombinator.com/item?id=33847479

yjftsjthsd-h 1 hour ago | parent

Wait, how did it `sleep 3`? I thought LLMs didn't have a concept of time

peder 52 minutes ago | parent

It’s still rendering frames constantly, no? So it just figures the next X frames are all doing nothing?

simedw 18 minutes ago | parent

Normal only keypresses invoke the next ”frame”. There is a sleep token that the runner could use to indicate that time is moving.

sprobertson 52 minutes ago | parent

from the post:

Time is an input too. If nothing is pressed, the model gets a WAIT token with a duration instead of a key, which is how snake keeps moving on its own.

yjftsjthsd-h 45 minutes ago | parent

That seems as likely as anything, but that just moves the problem to counting, doesn't it? The model gets the duration, but I was under the impression LLMs struggled with the arithmetic that would make that useful.

skohan 37 minutes ago | parent

Idk about Qwen 2.5 Coder that this is based on, but modern models don't struggle with arithmetic. I see Qwen 3.8 do long-hand math in thinking traces (ie. it outputs 17+35+12=64 in the thinking trace) so I would imagine this is something they train into the model specifically. Sometimes it creates and runs a simple python script to check its work.

thorum 56 minutes ago | parent

I regretfully suspect this actually is the future of UI. You just need smart local models running fast enough to keep the framerate high, but that’s inevitable in the long term, isn’t it?

gr_norm 43 minutes ago | parent

Why would you do that instead of having the model do the task you're using the UI for? It could generate UI widgets on-the-fly as needed to take richer inputs.

skohan 21 minutes ago | parent

It might be fine for one-off applications, but it's incredibly inefficient for something that has to be done over and over.

Why would you use a GPU and gigabytes of VRAM running a non-deterministic process for something a program can do on the CPU in milliseconds with kilobytes of memory?

antonvs 7 minutes ago | parent

I don't really disagree, but... why do we use Javascript for so many UIs? Badly designed web pages with ads can easily use gigabytes of RAM.

Edit: oh, I saw you wrote a related comment after I posted this. So you already know that direct LLMs for UIs are all but inevitable...

hnlmorg 6 minutes ago | parent

Probably for the same reasons most of our desktop applications have become slow Chromium instances consuming multiple GBs of RAM, each:

It’s cheaper to hire developers, and quicker to ship software. The user experience be damned.

skohan 42 minutes ago | parent

We had decades to work on making software more performant, but instead most users' space-age 2nm chips spend most of their time running Javascript because it was faster to make software that way.

winrid 12 minutes ago | parent

even my ten year old laptop browses the web fine ^_^

If I had to complain about any site/app it'd be datadog... each tab uses like 1gb in firefox.

andai 43 minutes ago | parent

>I used Qwen2.5-Coder-0.5B as the starting point. I needed a small model to get close to realtime performance. Being a coding model, it had also already seen plenty of CLI usage.

Interesting. I wanted to ask, how that model compares with the fine-tuned version, but I just saw the author provided benchmarks. Interesting that fine-tuning improved the memory so significantly, wouldn't have expected that.

Though, on second thought, I had an experience where I was able to, as a human, fine-tune my memory beyond what I previously thought was possible!

Reminds me of one of my favorite jokes.

Excuse me, can you tell me how to get to Carnegie Hall?

—Practice, man... Practice...

simedw 22 minutes ago | parent

I haven’t run extensive tests yet, but larger models also get better at memory.

Lerc 2 minutes ago | parent

I wonder how much a small model could do if it had a tool calling API for a filesystem, an embedded busybox, and a single output token for 'emit what busybox would do here'. Then presumably the model parameters could learn everything that diverges from what busybox can handle.