246 points yigitdemirag 1 hour ago 236 comments
bloky 1 hour ago | parent
BoldColdHold 1 hour ago | parent
WarmWash 1 hour ago | parent
So now we have
(XL): Carbon/Argon Astra Fable
(L): Neon Sol Opus
(M): Hydrogen Terra Sonnet
(S): Helix(rumored) Luna Haiku
5555watch 38 minutes ago | parent
Also, very curious whether they will ever release the rumored Krypton model, or will it be hidden ala Mythos
WarmWash 19 minutes ago | parent
XCSme 1 hour ago | parent
So Carbon is already deprecated, or?
EDIT: They do explain where it sits in the line-up here:
https://blog.google/innovation-and-ai/models-and-research/ge...
jrflo 1 hour ago | parent
nicoburns 1 hour ago | parent
catigula 1 hour ago | parent
XCSme 1 hour ago | parent
jeffbee 1 hour ago | parent
XCSme 55 minutes ago | parent
ehsankia 50 minutes ago | parent
troupo 55 minutes ago | parent
onlyj 53 minutes ago | parent
thevillagechief 24 minutes ago | parent
drum55 1 hour ago | parent
Gemini 4 Neon (NEW!)
Gemini 4 Hydrogen (NEW!)
Gemini 4 Carbon (NEW!)
Gemini 4 Argon (NEW!
Gemini 3.8 Live
Gemini 3.8 Flash (NEW!)
Gemini 3.7 Flash
Gemini 3.6 Flash
Gemini 3.5 Flash
I've no clue whatsoever how to read anything about the names of them and what they might imply you would use them for though. Within these most of them have LOW MEDIUM HIGH XHIGH, with the default being different for some of them, how anybody is possibly meant to understand this whole matrix of ambiguously named products is beyond me.aenis 56 minutes ago | parent
drum55 54 minutes ago | parent
Hydrogen 1
Carbon 6
Neon 10
Argon 18
Maybe this implies that Neon is smaller than Argon in weight size, but it's newer, but it's also only a little bigger than Carbon? Do the Nobel gas models have more safeguards? Is Carbon 6x the size of Hydrogen and a third the size of Argon? When they release Xeon can we make compounds? Feels like some really cheap Da Vinci Code esque thing if we're supposed to get some sort of meaning out of the names.svnt 49 minutes ago | parent
> In our early evaluations, it matched or exceeded Gemini 4 Carbon on overall quality at 3x the speed and 60% of the cost. We’re excited to bring these capabilities to Glean customers tackling multi-step work that demands accuracy, detail, and speed.
So possibly smaller than Carbon? Or some other axis that is making it faster and cheaper? No idea.
aenis 44 minutes ago | parent
iainmerrick 44 minutes ago | parent
jimbokun 35 minutes ago | parent
prymitive 54 minutes ago | parent
- Gemini (New)
- Gemini
- Gemini (original)
abixb 49 minutes ago | parent
accrual 48 minutes ago | parent
murderfs 39 minutes ago | parent
- Gemini (Not ready yet)
- Gemini (Deprecated)
jimbokun 35 minutes ago | parent
kzrdude 41 minutes ago | parent
genxy 1 hour ago | parent
swasheck 51 minutes ago | parent
CamperBob2 35 minutes ago | parent
moonu 1 hour ago | parent
ahofmann 55 minutes ago | parent
onlyrealcuzzo 1 hour ago | parent
I'm not sure it's much better than 4.1 Flash vs 4 Pro -> if 4.1 Flash is better temporarily when Pro is typically significantly better, people are going to be confused.
I'm not really sure how you get around this problem easily.
It's not going to age well when the next Flash-lite model launches, but maybe they will call that `Gemini 4 <Element>-lite` - who knows...
nicoburns 58 minutes ago | parent
I've found that while Opus 5.5 is often pretty nice for practical work, Fable (even 5.0) is still significantly smarter on hard tasks and makes far fewer "dumb mistakes"
dannyw 51 minutes ago | parent
That would make a lot of sense from an inference optimisation perspective. The same prefill and decode machines can hold one set of weights, and serve both Astra and Sol requests.
At (multi-)terabyte-scale model weights, and when you have to offer not just the latest like Fable 5.2 but still serve 5.1, 5.0, etc; heck I think Opus 4.6 is still around; you really care about overall compute utilisation.
Idle GPUs are wasted GPUs; and even over the fastest interlinks you’re not hotswapping multi terabytes of weights.
With MoEs, there’s probably stuff you can play around with varying the number of active experts; or other clever optimisations.
Maybe Sonnet 5.5 could be just Opus 5.5 with a different router and less active params? Who knows; I’m just theorycrafting now :)
nicoburns 48 minutes ago | parent
willy_k 32 minutes ago | parent
onlyrealcuzzo 37 minutes ago | parent
This doesn't really make sense given how the models released.
OpenAI released Astra long before latent-space looping was a thing at the frontier space.
Then they released Sol 6, which was fast, clearly not looped. Then they release Sol 6.1 which was nearly as good as Astra, but way cheaper, and WAY slower -> this would follow it just being a looped version of Sol 6.
It doesn't seem likely that Astra 6.2 is just a looped Astra 6... It's too fast, and they'd go bankrupt if they're charging the same price... It may be a further looped version of Sol than Sol 6.2 - which would explain why it's not as much better as we would've hoped.
It may also explain why their release is not going as well as they hoped. It's hard to predict how your first looped models are going to fare.
If you look at how long it took Google to recover from their failed 3.5 Pro, it would be surprising if GPT recovered from their rumored failed run on Astra in under a month. That would give them reason to release a looped Sol and call it Astra.
iamdelirium 28 minutes ago | parent
onlyrealcuzzo 25 minutes ago | parent
Even in the open weight Chinese models as small as DeepSeek V4.1 Flash - nothing is looped.
refulgentis 19 minutes ago | parent
refulgentis 20 minutes ago | parent
onlyrealcuzzo 44 minutes ago | parent
I think the rumor-mill consensus is that Google's models are significantly smaller while attempting to swing at the same level - but the ratios AFAIK are thought to be around the same - i.e. Neon is somewhere around half as big as Carbon, big enough that a newer checkpoint can make it temporarily a little bit better.
ml-anon 23 minutes ago | parent
andai 39 minutes ago | parent
weitendorf 15 minutes ago | parent
A GB300 rack has 72x288GB=20TB VRAM. Do the math and 10T parameter models are just about the max size you can easily train and serve (at 1M context) without doing a lot of custom hardware development or limiting your scale.
WarmWash 1 hour ago | parent
I think there are a ton of people who automatically default to the largest model with the highest thinking for everything. Which historically made sense but not nearly as much anymore.
At this point a model like hydrogen can cover just about everything an average person is doing. You don't need Carbon deploying an agent swarm pushing terabytes of parameters through HBM memory to help you plan your birthday party.
DonsDiscountGas 59 minutes ago | parent
ehsankia 49 minutes ago | parent
accrual 38 minutes ago | parent
Tadpole9181 26 minutes ago | parent
adithyareddy 58 minutes ago | parent
(M) Gemini 4 Neon - second fastest - $3/$15
(L) Gemini 4 Argon - second slowest - $2/$10 (till Nov 30), $4/$20 (> Nov 30)
(XL) Gemini 4 Carbon - slowest - $2/10 (till Nov 30), $4/$20 (> Nov 30)
WarmWash 26 minutes ago | parent
There is a Helix model too (in the pipeline) which is the actual small size.
whimsicalism 57 minutes ago | parent
ml-anon 57 minutes ago | parent
It doesn’t correlate with intelligence, speed or price furthermore you have no way of telling what the model sizes of “like for like” named models from oai or Anthropic are.
Mogzol 53 minutes ago | parent
When Argon first came out it seemed like they could be going for noble gasses, like Helium (small), Neon (mid), Argon (big).
But then Carbon came out and it seemed like maybe they were going through the alphabet for model generations. Argon, Boron (internal), Carbon.
And now they've come out with Hydrogen and Neon, and there's no logical way to order them. The actual model sizes are Hydrogen (small), Neon (mid), Carbon (big). This doesn't match sorting by atomic number or alphabetically.
baoooooooooooo 51 minutes ago | parent
I also like the personalities implied by these, with Anthropic saying “human literary achievement is neat”, OpenAI saying “space is neat” and Google being like “science is neat”.
Edit: I was coming back to say “I wish we could talk about stuff more substantive than the naming scheme” but (1) my comment may be objectively incorrect, and (2) I have to apologize that I haven’t helped the situation by… commenting on the naming scheme.
WarmWash 48 minutes ago | parent
dudeinhawaii 3 minutes ago | parent
wat10000 42 minutes ago | parent
georgemcbay 31 minutes ago | parent
But the way google ranks the models by capability doesn't follow periodic table order.
Hydrogen - 1
Carbon - 6
Neon - 10
Argon - 18
It could very well be based on their density (g/cm^3 in standard conditions):
Hydrogen - 0.0000899
Neon - 0.000900
Argon - 0.001784
Carbon - 2.267
This would make more sense than assuming it is based on their periodic table order as it more accurately maps to how Google markets their capabilities, but is too esoteric to really be useful for clearing up confusion for normal users, IMO.
kphorn 44 minutes ago | parent
element/model - atomic number - cost H - 1 - $1.50 & 7.50 per mtok C - 6 - $4 & 20 per mtok Ne - 10 - $3 & 15 per mtok Ar - 16 - $2 & 10 per mtok
Claude - length of story but fable messes it up Haiku - 3 lines Sonnet - 14 lines Opus - a novel? Fable - 30 lines?? Mythos 15700 lines?
Luna - 28 days Sol - 365 days Astra - 5-6000 stars
daemonologist 25 minutes ago | parent
(Although why they'd release all ~four is a mystery. I guess they're having a hard time picking the "best" one and so are just putting them all out to let the public decide.)
ariwilson 1 hour ago | parent
and why can't I use any of these with agy and a $20 plan? unlike openai and anthropic
pilooch 1 hour ago | parent
maplet 1 hour ago | parent
On Artificial Analysis Intelligence, it scores comparably to GPT-6.2 Sol but is significantly more expensive.
onlyrealcuzzo 1 hour ago | parent
This is probably the set of benchmarks most people care about: https://www.vals.ai/comparison?m=claude-fable-5-2,cobalt-c-h...
Overall, performs just under Fable (above Astra or Sol 6.2 or Carbon - slightly ahead of Opus 5.5).
Latency is by far the best for the level of quality.
It's about 2x more expensive than Sol 6.2, but significantly better and faster. You get what you pay for.
catigula 1 hour ago | parent
HDThoreaun 56 minutes ago | parent
catigula 54 minutes ago | parent
Edit: I tested the model and it found several mistakes Astra 6.2 made. It’s possible…
SuperV1234 36 minutes ago | parent
If I wanted to realistically try out Neon in some of my workflows without committing to a paid subscription before seeing if it can actually work as well as Opus, do I have any option, or do I just need to take the gamble?
onlyrealcuzzo 30 minutes ago | parent
IIUC, you also don't get to try Opus unless you have a subscription, right?
user43928 16 minutes ago | parent
"Using third-party software, tools, or services to access Antigravity is a violation of our Terms of Service"
and also: https://antigravity.google/docs/faq?can-i-use-antigravity-cl...
I'd like to keep my Gmail account, so Google can keep their model and will not get my business.
InsideOutSanta 1 hour ago | parent
maplet 54 minutes ago | parent
kristjansson 1 hour ago | parent
e: No, Neon is the cheaper, mid-tier between Hydrogen and Argon. So Argon/Neon are aligned along the noble gasses, and Hydrogen kinda makes sense as the weakest ... but they threw Carbon in without any consideration
Melatonic 23 minutes ago | parent
numpad0 1 hour ago | parent
01 02 ... 13 14 15 16 17 18
H He
Li Be ... B C N O F Ne
Na Mg ... Al Si P S Cl Ar
???clutter55561 1 hour ago | parent
xnx 1 hour ago | parent
hn8726 1 hour ago | parent
agrippanux 1 hour ago | parent
moecables 59 minutes ago | parent
bemtevi 58 minutes ago | parent
What I simply don't get is why people want to push the boundaries of efficiency and knowledge. I used to think as a teenager that all knowledge was good but how good is knowledge when we can't acquire wisdom with a comparable speed?
The greatest thing about the early internet and earlier science was that people shared and communicated with each other. Now they just share what they produce wihouth much humanity at all.
It seems to me that the creation of all these models has just become a race where no one really cares about the finish line and no one really understands why we're doing it, except for the short-term trail of dollar bills on the race course.
For me, each new model release is a sad day for humanity.
dprkh 58 minutes ago | parent
Jorge1o1 58 minutes ago | parent
Maybe the atomic number correlates to model pricing? Nope! Neon (10) is marketed as cheaper than Carbon (14).
Maybe the atomic number correlates to overall intelligence? Nope! Neon (10) and Carbon (14) are both marketed as smarter than Argon (18).
The only thing I can think of right now is that it's in alphabetical order by release... but in that case why the periodic table?
dannyw 57 minutes ago | parent
Unfortunately, their recent change to allow me to opt out of AI training (which would also include connector data like Gmail, photos, etc) hasn’t rolled out to me yet[1]; so I won’t be upgrading my subscription; nor using the Gemini app until I can opt out.
[1]: https://arstechnica.com/ai/2026/11/google-will-let-you-opt-o...
testtestmetest 46 minutes ago | parent
/s not trying to be mean, its just weird hill to die on. Every company is stealing all your data in ways you unknowingly consent to at every second of the day.
5555watch 39 minutes ago | parent
abraxas 57 minutes ago | parent
Computer0 30 minutes ago | parent
abraxas 27 minutes ago | parent
xiaoyu2006 57 minutes ago | parent
artdigital 57 minutes ago | parent
So Neon is the new default, and Carbon the slow deep thinking “when you need it”? A bit like how Opus 5.5 was better than Fable 5.1 in most things, but Fable still had the edge over deeper things (then 5.2 came along)
Hydrogen has already became my default for 80% of small tasks. If Neon is as fast but smarter that would honestly be very cool
gradus_ad 52 minutes ago | parent
So I'd take their hierarchy with a grain of Sodium.
Robdel12 57 minutes ago | parent
bisonbear 56 minutes ago | parent
45sdasf45 55 minutes ago | parent
Say what you like about Apples Pro, Max and Pro Max -- at least it gives some indication of size to the average person.
bluegatty 55 minutes ago | parent
They all need to pick a dam naming scheme and stay with it.
gradus_ad 54 minutes ago | parent
Can't wait to buy Puts on ANT...
impulser_ 54 minutes ago | parent
Is it because Anthropic put a ton more effort into the character of the model that it basically roleplays as a human and that why it tends to think better when giving tasks that it might not be trained heavily on.
I have used Carbon more than most model to try and get the same results but it just feels like i'm talking to a smart AI. Which is fine if you are asking questions, researching an idea, but if you actually want to work on something, build something, it just falls apart. Like Opus can run for days on a task and it feels like it understand what it was doing 2 days ago even when it might not be in it context while other models feel like they breakdown and get lost or falter as the task grows.
baq 53 minutes ago | parent
jorl17 46 minutes ago | parent
(After a long conversation with existing messages)
User Prompt: Ok, so SSH to the unraid server and update the plex db
Reply:
I need to think how to answer this.
The user is asking me to SSH into the unraid server. This seems to imply there is a machine to which we can SSH to. I should look for places where hosts may exist.
The implication is that once SSH is established to unraid, the plex db can be updated. It seems plex is a service that the machine has in which we can run updates.
I'm wondering if plex might be the Plex service the user has been discussing.
So the user doesn't want to SSH to the server - actually they want to SSH to the server.
... Formulating a plan ...
[some messages later]
Aha! THIS IS THE SMOKING GUN! `~/.ssh/config` HAS THE REQUESTED HOST!!! IT'S TIME TO CONNECT!!
[some messages later]
Ok, time to act. I'll ssh to the server and follow coherent rules.
----
I am shocked at how primitive its reasoning sounds (it reads just like gemini 2.5-pro's old reasoning). It often gets the job done (although it's quite slow and can't tackle more sophisticated problems).
Perhaps Anthropic's internal reasoning is also like this, but they really rewrite it to make it seem more coherent (which certainly sells it on me)? I almost want to click stop just by reading this nonsense.
aenis 26 minutes ago | parent
(And it did produce a very competent algorithm, though it took a few days and a metric ton of tokens; I just triggered the same task with Neon).
jorl17 17 minutes ago | parent
iamdelirium 23 minutes ago | parent
jorl17 17 minutes ago | parent
I was not really fabricating the example I gave too much. Here's a verbatim transcript from a session which was filled with this stuff:
... thought LET'S READ RecordReader.cs RIGHT NOW!! RecordReader.cs IS WHERE EVERY SINGLE TABLE, ReadPlan, DecideValueVisibility, ReadRowCounts , AND ShowValues READS ITS DATA!! Let's read RecordReader.cs (lines 1-300)!8note 9 minutes ago | parent
ehsankia 46 minutes ago | parent
No wonder they went for the 5.5 naming, instead of the .1 increase. From just the benchmarks, it seems like neon is basically on par with Opus 5.5 which came out 2 months ago.
aenis 31 minutes ago | parent
Have another session on another machine (say, IOS build on a mac, and server build on a linux?). No problem, they will happily coordinate with each other through the /remote-control protocol, and then you feel like you are in a crowd, with those sessions talking to each other and over you.
No such feel with OpenAI nor Google models for me. They are absolutely great for lots of things, but nothing comes close to the feel of Opus 5.5 and Fable.
(I am biased; i mainly use the anthropic models so i got used to them).
jwpapi 22 minutes ago | parent
eterm 5 minutes ago | parent
pjankiewicz 25 minutes ago | parent
jaccola 46 minutes ago | parent
Just today I was filling out a security questionnaire and it said "I won’t write a PDF that claims zero credentials while this token is in the logs." But we were in the process of removing this from the logs so it was perfectly accurate to write and I wish it had deferred to my explicit request...
I think Anthropic overcorrected on the sycophancy; Google models to me seem a nice balance of highlighting issues / risks while still doing what I actually ask.
enraged_camel 41 minutes ago | parent
aenis 29 minutes ago | parent
willis936 6 minutes ago | parent
epolanski 42 minutes ago | parent
Lots of PR it seems?
I have all of Ant/OAI/OpencodeGo subs and test lots of models and I just cannot understand how and why some people find specific models so ahead, when it seems to me like reality is so nuanced and so specific to the particular task and how you approach it.
WarmWash 35 minutes ago | parent
surround 38 minutes ago | parent
Cursor was popular before any of those three, and just look at how much Grok improved after they acquired Cursor's data.
foohis 30 minutes ago | parent
Other companies would use completely raw internet data for pre-training, which is basically trash. Anthropic's secret sauce is that they filtered and edited down their data for only the highest-quality material. You really don't need quadrillions of tokens to pre-train, as that's just random noise. A smaller-but-higher-quality data set is better in every way.
Meanwhile its obvious that companies like Google would use trash data like email and chat logs or Internet forums for pre-training. Just complete trash data.
This is a classic case of "less is more" and "garbage-in-equals-garbage-out"
quixoticelixer- 17 minutes ago | parent
foohis 8 minutes ago | parent
weitendorf 2 minutes ago | parent
If you look back at GPT2 and the early days of AI safety/risk this was actually something the kinds of AI researchers that joined Anthropic didn't want - RL on coding->using that accelerate development and to bootstrap capabilities across domains was always the plan.
I personally find that Opus 5.5 and Fable 5+ are generally fine to chat with but just very wordy and culturally "LessWrong/essay-guy". Which, if a model is going to have a personality, is close to the one that I want (though it does piss me off when it gets preachy about standards Anthropic clearly doesn't hold themselves against)
I also want to immediately recognize AI generated content on the web or from another person, so I'm very glad it has a distinct voice
impulser_ 7 minutes ago | parent
I honestly think it just their post-training is fantastic and I think it has something to do with them being very opinionated on how they want the model to act that it basically the best at roleplaying a human. Humans are very good at completely and figuring out things that there aren't "trained" on if given basic set of intelligence. So when it working on these long tasks that might not be directly in it training data it still reason through it very well.
They got made fun of for this in the pass because they do a lot of research on this including hiring a philosopher.
This is just my best guess, because you don't see any other labs talk about how they made the model act and Anthropic is basically obsessed with it and talks about it every model release.
I think they just have better "taste" and that taste is their secret sauce.
thanhhaimai 17 minutes ago | parent
I'd recommend trying Gemini Neon on Antigravity, with a trimmed down (or new) agent configs.
dennysora-main 53 minutes ago | parent
But I still can't understand the differences between these models.
Recently I watched Veritasium talking about helium, and I was thinking, will the effort setting of these models eventually have something like "Temperature K" to configure?
ncd 50 minutes ago | parent
Good lord, this is an insane flow.
smallerize 15 minutes ago | parent
dekhn 50 minutes ago | parent
nkmnz 49 minutes ago | parent
horseloverthin 47 minutes ago | parent
xd1936 46 minutes ago | parent
Google Antigravity
Entry rate limits to advanced models, including Gemini 4 Argon and Opus 5.5, in Google Antigravity, our agentic development platform
Yet, here I am, running the `agy` cli, and the model picker says "Unavailable: Gemini 4 Carbon/Argon/Hydrogen/Neon is currently available to paid Pro and Ultra subscribers". It has been this way since the release of Carbon. I guess it's "3.8 Flash" or nothing for me.
xnx 46 minutes ago | parent
This is an underrated quality of Gemini models.
seizethecheese 45 minutes ago | parent
Carbon has a huge problem that it's just way too slow, and outputs way too much per task (on real world tasks, it's the worst at this https://micjm.com/blog/carbon-is-that-slow/).
Without the speed issue, Carbon would be a top model already. It's top on our battle system built on opencode, if you exclude early votes: https://agentduel.com/leaderboard?excludeEarlyVotes=true, though to be fair, small sample size so far.
Conclusion is that it's looking good for Neon. Even if it suffers from similar problems as Carbon, if it's even smarter and way faster, it should be a top model, maybe the best.
Edit: I was wrong about Neon being the larger model. I made this assumption because Neon has higher scores and they're both "v4".
WarmWash 39 minutes ago | parent
Carbon is a Fable sized model
Carbon seems to be a version of Argon that is incredibly spendy with thinking tokens.
aenis 37 minutes ago | parent
WarmWash 30 minutes ago | parent
Also neon seems to be stepping on the toes of Carbon a bit, just like Opus 5.5 did to Fable.
seizethecheese 26 minutes ago | parent
WarmWash 21 minutes ago | parent
seizethecheese 36 minutes ago | parent
Weird because Neon is outscoring Carbon in AA index, Terminal bench science, etc and they're both named "4". Should have released this as Neon 4.1.
With that clarification, I'm sure Neon has the same overthinking problem.
WarmWash 29 minutes ago | parent
Google also has a rumored Krypton internal model, which is also Argon/Carbon class. Neon might very well be distilled from Krypton, not Carbon.
seizethecheese 24 minutes ago | parent
> Neon might very well be distilled from Krypton, not Carbon.
That would make a lot of sense, interesting!
WarmWash 22 minutes ago | parent
comboy 44 minutes ago | parent
declanjackson 42 minutes ago | parent
With the frontier getting more jagged, we might see more of this 'less intelligent overall but useful for the frontier tasks'-type models, meaning picking a model will only get more difficult from here.
(FD: I work at AA)
abixb 40 minutes ago | parent
drivebyhooting 36 minutes ago | parent
iamdelirium 32 minutes ago | parent
refulgentis 25 minutes ago | parent
It's really frustrating to describe. Easiest is you can't score on just the prompt, you need the full context. Ex. "Solve Fermat's last theorem" initial prompt then "continue" followup prompt. Which gets you back in the tarpit of "if I have to pay a model to do an inference run on context, then, the context is cached and I should continue using that model."
The vision of another model thats RL maxx'd on knowing how to delegate has a few problems. One, we can observe externally, it died/didn't work even at OpenAI's scale and willingness to make it work, i.e. the...GPT-5.0? IIRC...router that silently went away. The other, if such a model was built, it'd also be good at regulating it's own thinking enough that it'd naturally use less tokens.
In practice, the frontier hackers have settled on a decision model (i.e. Jev) x calibrating effort level or choosing from a set of models x reasoning effort tuples they specify rules for choosing in their Jev prompt. My guess is this stays stable and the only real choice.
baoooooooooooo 7 minutes ago | parent
I’m out of my depth here but it does seem prompts have a “smell” to their complexity that means we can use a classifier to mate them to a model with better success than random noise, but still don’t essentially all of the savings associated with systems like this just come because there is some “complexity arbitrage” where there is enough of a token cost differential between models that you always save money by trying the cheap model first and then trying a hard model if it turns out to be hard?
seizethecheese 34 minutes ago | parent
WarmWash 23 minutes ago | parent
GodelNumbering 12 minutes ago | parent
I do apologize in advance for putting you in spot like this, but there are two things I have been quite curious about.
1. Since 60% of the benchmarks contributing to AA are now private only, how do you guys ensure transparency? More precisely, how can someone verify the transparency in scoring?
2. There are also disclosures about custom benchmarking services offered to AI labs, what does AA do to keep the custom benchmarking services not getting access to the same/similar data that is used in the private benchmarks?
ByteAtATime 41 minutes ago | parent
aizk 39 minutes ago | parent
rednafi 35 minutes ago | parent
Just give them a version number and an identifier, like:
1. V5 smol 2. V5 mid 3. V5 big 4. V5 yuge
WTF are Hydrogen, Oxygen, and Nitrogen? At least OpenAI and Ant have Luna, Terra, Sol, Astra and Haiku, Sonnet, Opus, Fable. They use those consistently.
TacticalCoder 35 minutes ago | parent
That sounds like serious benchmaxxing for Gemini 4 Neon is ahead on 17 out of 19 benchmarks vs Opus 5.5.
Not so bad for a company that, according to HN, "has lost the AI race".
P.S: I like the Google models but a truce before the Anthropic IPO would have been nice. This is just a very big dick move a few hours ahead of Anthropic's IPO. Shall we see Anthropic launch a new model from now 'till the opening bell?
jaykru 34 minutes ago | parent
atomashevic 21 minutes ago | parent
low_tech_punk 31 minutes ago | parent
origini 29 minutes ago | parent
And then you have big names like Hy-dro-gen haija
thisisauserid 29 minutes ago | parent
jonathanstrange 28 minutes ago | parent
Maybe the people at Google don't realize that but some people need constancy. If a model was good enough for some work 6 months ago, it's not suddenly becoming not good enough for the same task.
inshard 28 minutes ago | parent
hiddenvulkcan 25 minutes ago | parent
I do think they're very well positioned, their small models are smart and fast, and integrating well with Google is a big plus for basic use case stuff. I always like these models!
hendry 22 minutes ago | parent
Nevermind the 20GBP pcm I pay for YouTube.
freecodeio 21 minutes ago | parent
lawandjustice 20 minutes ago | parent
2027 january RSI is achieved. 2027 february first big scary incident (100 million people severely affected by infrastucture collapse for several days) 2027 march a fleet of rogue self replicating agents is discovered. 2027 april first severe fatal incident (1000+ people dead, most likely from a missile created using oss models). 2027 june. Deadly virus created by a terrorist organization is released in Ukraine. Remind me!
stillpointlab 14 minutes ago | parent
I think internally, Google, Anthropic and OpenAI are thinking "It's so easy, you just use Neon for X, Carbon for Y" - but most of the world is trying to map Sol, Astra, Neon, Opus, Fable, Mythos, Carbon, Haiku, etc.
I just hope they finally land on a naming scheme they are happy with and just stick with it. But I have my doubts. Perhaps they are all benefiting from this confusion.
catigula 13 minutes ago | parent
carterpaul 11 minutes ago | parent
Would love "small", "medium", "large", "extra-large", etc...
Nesco 7 minutes ago | parent
jabron 8 minutes ago | parent
bogdan 7 minutes ago | parent
stillpointlab 6 minutes ago | parent
So I'm just hoping for consistency over some period of time to allow me to get familiar.
mstibbard 6 minutes ago | parent
pizzathyme 6 minutes ago | parent
Revanche1367 10 minutes ago | parent
woggy 4 minutes ago | parent