37 points qtqtqt 1 hour ago 20 comments
rafram 50 minutes ago | parent
Does GIMP have a very restrictive AI policy or something? I get not wanting to ship a gigantic pile of UI tests without thorough review, but basically any model released in the last six months would be able to write UI tests covering every single control in an afternoon.
samuria 41 minutes ago | parent
Waterluvian 38 minutes ago | parent
forgotmypw17 36 minutes ago | parent
Prod your agent first until it can explain back to you what's wrong with the tests.
Then ask your agent to write a better prompt for writing tests (and save it to an .md file)
Then reference that .md file and iterate on the tests again. It may take a couple of tries, but I've had good results with this strategy.
Ended up with a whole prompting framework.
skydhash 4 minutes ago | parent
To be fair, some people like to prod. But, for me, I'd rather take hours reading manuals than do trial and error prodding for 30 minutes (even if I knew it would be that short). Yes it's not optimal in terms of time, but it's better for my soul and well being.
deskglass 40 minutes ago | parent
locusofself 22 minutes ago | parent
timcambrant 9 minutes ago | parent
hulitu 4 minutes ago | parent
Microsoft also seems to use "models" to "test" its programs. Look how shitty they are.
joegibbs 21 minutes ago | parent
Jolter 11 minutes ago | parent
hulitu 7 minutes ago | parent
AI can write tests now ? For its own software ? Can it evaluate the results ? Correctly ? /s
altmanaltman 5 minutes ago | parent
> *GIMP already has unit tests*. A unit test checks one small part of the code, for example how GIMP reads an XCF file. But our unit tests were not opening a real window. So it coult not tell us if a dialog opens, or if a menu item did its job.
> Note that GIMP *already had tests of GUI code*, usually interacting directly with the core code, and sometimes even simulating input devices’ events (pointer movements, clicks, drags…). But nothing can beat interacting with the real interface. Furthermore such legacy tests directly hijacking core code come with their own problems, such as breaking all too often (and not because they detected a bug), all while not being able to detect whole categories of bugs.
> UI tests check the program as a whole, in the same way as you use it.
qtqtqt 5 minutes ago | parent
slopinthebag 30 seconds ago | parent
dllu 11 minutes ago | parent
alexey-salmin 2 minutes ago | parent
Aozora7 2 minutes ago | parent