Yet another Hacker News

Our software engineering interviews are changing (a little)

73 points maitland 2 hours ago 56 comments

saghm 2 hours ago | parent

This is the third Jane Street-related thing I've seen on here this week after not seeing much about them that I can remember any time recently beforehand I'm aware of their use of OCaml and probably do other things that might be of interest to the HN community, but it seems like a lot in a very short time out of nowhere. Is there some sort of marketing push or something going on?

MeetingsBrowser 1 hour ago | parent

Probably a self fulfilling prophecy.

Jane street news made it to the front page a couple times and got a lot of attention.

Now more people are recognizing the name and upvoting other Jane Street posts.

dominotw 1 hour ago | parent

> We know this sounds vague,

ok thanks for this slop then. Dont see why this stupid article is on frontpage.

They seem to belive in 'organic' marketing like doing a fake podcasts on dwarkesh. So maybe they have some bots upvoting their name.

chamomeal 1 hour ago | parent

They’re pretty techy, idk why they’re showing up recently though.

Also one of my absolute favorite programming talks ever was hosted by Jane street:

https://www.janestreet.com/tech-talks/algorithm-for-precisio...

It’s like half talk, half story. Got me super interested in logic programming, and minikanren in particular.

vrmiguel 1 hour ago | parent

People like to talk about Jane Street because it is this Very High IQ™ company that pays people astronomical salaries and is extremely hard to get into.

They don't have to do marketing, though. It's not like they sell anything to Hacker News readers, they make their money trading finantial assets

saghm 1 hour ago | parent

You don't think that companies who want to hire engineers ever try to post stuff to Hacker News as a way to market themselves as a potential employer? Where did you get this impression of "Very High IQ" and "astronomical salaries"? I'm guessing at least part of it is from reading stuff from them or from talking to other people who read stuff from them.

The fact that you seem to adamantly disagree that there's any plausible angle here doesn't mean it's not marketing, but it might mean that it is marketing and that it's working on you.

vrmiguel 1 hour ago | parent

> you seem to adamantly disagree

Such strong wording, you got that from me saying "They don't have to do marketing"?

> where did you get this impression of [...] "astronomical salaries"?

From the real world fact that the base salary is US$300k/yr, and often said that the total compensation for the first year ends up being US$700k+. Even for Manhattan + SWE jobs, surely that merits "astronomical". Other devs making that much are usually working at Anthropic or OpenAI, aren't they?

> where did you get this impression of "Very High IQ"?

You didn't copy my ™ there, which was meant to be snarky.

In any case, what I was trying to say is that they don't have to do marketing in the usual sense. They don't have to get the Jane Street logo in your subconscious so you buy a Jane St. product when you go to Kroger's, and you can't sign a Jane St. subscription plan.

People in this thread seem to be implying that the company is botting/astroturfing, but for what? To maintain the image that they're a highly sought after employer?

Anything Paul Graham posts gets upvoted here instantly, because it's Paul Graham who posted it. For Jane Street there's the same halo effect, because many believe Jane Street's work is impressive. Is it really easier to believe the idea of botting to get an even longer list of candidates to interview, than it is to believe people just like to hear what this company has to say?

I don't ever see anyone saying Paul Graham is botting posts about his "essays"

saghm 33 minutes ago | parent

> Such strong wording, you got that from me saying "They don't have to do marketing"?

The only alternative interpretation I can think of for what you said is "they waste resources on marketing they don't need to do". I didn't think that was more likely to be your intended meaning, but you're welcome to correct me on it.

> From the real world fact that the base salary is US$300k/yr, and often said that the total compensation for the first year ends up being US$700k+.

Can you clarify where these numbers are coming from? Real world facts tend to have sources that can verify them.

> In any case, what I was trying to say is that they don't have to do marketing in the usual sense. They don't have to get the Jane Street logo in your subconscious so you buy a Jane St. product when you go to Kroger's, and you can't sign a Jane St. subscription plan.

> People in this thread seem to be implying that the company is botting/astroturfing, but for what? To maintain the image that they're a highly sought after employer?

Yes, that's exactly what I'm implying. I honestly would have stated it explicitly in the first place, but I made the incorrect assumption that it was not controversial to claim that this phenomenon exists at least for some companies and that it can be described as a form of marketing.

> Anything Paul Graham posts gets upvoted here instantly, because it's Paul Graham who posted it. For Jane Street there's the same halo effect, because many believe Jane Street's work is impressive. Is it really easier to believe the idea of botting to get an even longer list of candidates to interview, than it is to believe people just like to hear what this company has to say?

> I don't ever see anyone saying Paul Graham is botting posts about his "essays"

If we were discussing this on a website originally created by Jane Street, I would find this a more compelling counterargument. That said, I also don't bother trying to discuss his writing much because I mostly find it insufferable. I find the motives of Jane Street to be more interesting precisely because my opinion on them is more malleable.

vrmiguel 18 minutes ago | parent

> Real world facts tend to have sources that can verify them.

Sure. What I said is a real world fact is the base salary, which can be verified by any of their job postings, e.g. www.janestreet.com/join-jane-street/position/4274288002, where it says "Base salary is $300,000. Base salary is only one part of Jane Street total compensation, which includes an annual discretionary bonus".

The claim that has less direct sources is how much the TC ends up being. The figure I said is the one I've heard multiple times over, sometimes by JS employees themselves, so I assume it to be true

> If we were discussing this on a website originally created by Jane Street, I would find this a more compelling counterargument.

That's fair, but I used Graham just as an example of source that always gets upvotes here. Uncle Bob hasn't posted anything in a while, but if he did, it'd certainly trend here, so that might be a better example

> I find the motives of Jane Street to be more interesting precisely because my opinion on them is more malleable.

Yes, that's fair. I don't think it's impossible that JS would bot/astroturf like you're implying, I just don't think it's the case. By Occam's razor I find it's more likely that people do care for what they have to say, similar to P. Graham, Uncle Bob, or any sort of blogpost from Anthropic/OpenAI, etc. It's quite possible that they got to this stage due to intentional marketing, though, so I'll give you that

mancerayder 1 hour ago | parent

They sell employment to HN, is the parent comment's point.

myrmidon 1 hour ago | parent

I'd lean towards no on that. Might be a bit of a Baader-Meinhof thing. They've been mentioned/discussed on HN pretty regularly for years:

https://hn.algolia.com/?q=jane+street

spongebobism 48 minutes ago | parent

According to algolia, there were 57 posts with jane street somewhere in the title or url in the last year, so a bit more than one per week. For a Poisson distribution with rate = 57/52, P(X >= 4) ~= 2.5%. So there should on average be ~5 weeks with 4 or more jane street related posts at that rate.

Probably just random fluctuation.

saghm 40 minutes ago | parent

I appreciate a data-focused response here! What I'm more curious about is how many of these reached the front page. My impression is that the rate of how often they reach the front page is probably a good amount lower than once a week, which I think makes the question of whether this is a coincidence or not more reasonable.

parthdesai 1 hour ago | parent

Pretty vague blogpost tbh. Apart from the fact that it's changing, it doesn't really describe much

japhyr 1 hour ago | parent

I had lunch with a couple friends this week, and we each work in different tech environments. Most of the conversation was about the workflows we're each using, and seeing our colleagues use.

It's a really interesting time to work in this field. People who are paying attention to workflows are using a range of tools, and constantly refining their own workflow. For example a whole team might use Claude, but one person collects data about their own usage on an ongoing basis and develops more advanced workflows than everyone else on the team. Their focus isn't just about working faster, but about doing better work.

I don't think the post is vague; I think we're at a time where you need flexibility in your interview process to find candidates who are doing the best work these days.

cyh555 1 hour ago | parent

Thanks, that certainly clears up the doubts

bemtevi 1 hour ago | parent

This sort of announcement has made me very happy I left the software world. I still code as part of my job, but it's not my main part and luckily I don't have to use LLMs are part of my job. I still code 100% by hand.

In some ways, although I detest AI, it has clarified the sort of people and organizations I want to associate with. Any latent characteristics that I abhor have been made obvious with the existence of AI and of course, I would want nothing to do with Jane Street.

username_slop12 31 minutes ago | parent

Upvoted this!

thibsrn 1 hour ago | parent

What's up with the obsession with Jane Street today ?

okdood64 1 hour ago | parent

Money.

fernandotakai 1 hour ago | parent

so, i've gone through a bunch of iterations on how to interview in days of LLMs and i came up with a process that is quite good:

first, a coding problem that is solved live (right now, online) -- it's not a hard one and most LLMs can easily one shot it.

BUT: if you are not a senior developer, you need to do it without LLM, and if you are a senior+ and do it via LLM, the bar is super raised and now you need to explain in details everything the LLM wrote as if you wrote it yourself.

the main component here is to know if the candidate understands coding in general, understands problem solving and how to dive into an existing (although quite small) codebase. solving the problem is not really the goal -- in fact, most candidates we hired did not solve it 100%, but were able to understand and explain everything that they put it on code. this usually takes 1h30.

(we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).

the second is an architecture interview, where we build a system together BUT instead of doing the whole spec, miro board, whatever, we do it in more of a free way: we start with a really simple, MVP level, spec and we ask how the candidate would build it. then, we add another feature, and we ask how they would go for it. during this whole thing, i ask random questions like "why would you use x instead of y?" or "why do you prefer this approach?".

one important thing: we allow the candidate to say "i don't know" and we tell the candidate that there's no gotcha questions (which is true, i hate these kind of questions). this lets the candidate be super comfortable (feedback we had from real people).

in general, this flow helped us a ton and i really really like running it :)

(edit: spelling)

xpct 1 hour ago | parent

> we also ask the candidate to open a PR against the repo just to see if they know the basics of git

I've heard this multiple times now and it still boggles my mind that git is something you have to test for.

fernandotakai 1 hour ago | parent

it's incredibly sad but it's real -- so many senior candidates have problems with commit messages, opening PRs.

not even joking: i've seen people with like 10y of experience trying to push to main :(

cbm-vic-20 1 hour ago | parent

At one job I had, we used git extensively, but not GitHub. We had internal systems for review and branch management. I could tell you all the gory details about various workflows and landmines in git. Is the test here really about git, or git+GitHub?

kerenskiy 1 hour ago | parent

PRs are not only not a basic feature of git - it's not a git feature at all. Though button clicking on GitHub is anyway a crazy thing to test

baud147258 59 minutes ago | parent

Well, I've been using git at work for 6 or 7 years, but I can't out of hand open a PR; at work we very rarely use different branches and the last time I touched Github I was a student.

tempfile 46 minutes ago | parent

Do you mean specifically Github, or any code forge at all? When you say "we rarely use different branches", what do you mean - you're all pushing to master, or you have branches for each team or something? I would regard "code forge" and "separate branch per reviewable unit of work" as extremely normal, nearly universal.

fsnovask 58 minutes ago | parent

I rarely use anything beyond checkout, clone, pull, commit, cherry-pick, revert, and push in a job. If I have to, I'm probably asking the AI what to do by giving it my situation. And hopefully the AI can read about whatever git standards the company would like to be enforced so the engineer doesn't have to interrupt someone for that.

This is more about opportunity cost because instead of asking about git, you could ask about other valuable skills in your limited time with the interviewee, but I guess if git is really that important, then it's justified.

jasonlotito 1 hour ago | parent

Something similar for senior devs: code reviews. Reading and understanding code has always been important. I find offering simple code reviews very telling. In a way, your LLM approach matches that. It's a way to do a code review. It's a test of understanding, and not just memorization.

saghm 1 hour ago | parent

I feel like git operations are one of the few things that I truly don't care at all about whether someone uses an LLM for and doesn't tell me. PR descriptions and commit messages should be written by humans for humans, but like, if someone doesn't know how to manually configure their git remotes and what exact command to use to amend/squash their commits into the shape needed and instead has an agent prepare all that and use the GitHub CLI to open the PR, why should I care? Beyond the code itself, I care about the things that will continue existing in the long run fitting my team's conventions (the number of commits, the commit messages, the PR description and comments, and then whether there's a merge commit at the end), but the process used to create those things doesn't matter to me.

astonex 1 hour ago | parent

> but like, if someone doesn't know how to manually configure their git remotes and what exact command to use to amend/squash their commits into the shape needed and instead has an agent prepare all that and use the GitHub CLI to open the PR, why should I care?

Because LLMs aren't a reason to not learn your tools. A determinist CLI command vs using a statistical machine for it. It's just a waste of tokens at the very least.

tempfile 50 minutes ago | parent

I happen to have memorised how to add new remotes a long time ago, but I would 100% not care if someone I worked with googled it every time. That's how it was for me for several years. I don't think that has anything to do with "learning the tools". And getting claude to do it is not that much different from copy-pasting from google.

wtetzner 42 minutes ago | parent

TBH I usually just search my shell history for the last time I added a remote and edit it.

sokoloff 46 minutes ago | parent

> It's just a waste of tokens at the very least.

Some 6-figure per year developer spending a quarter’s worth of tokens to do something they do once a month or so is not something I’ll ever care to optimize away…

saghm 42 minutes ago | parent

No, the reason not to learn your tools is if they're not your tools anymore, and something else is. I don't spend time trying to manually invoke filesystem APIs to create files in my work projects and instead will often just have my LSP do it when I declare a module and the file isn't found. I could memorize the exact sequence of system calls needed, but my employer would prefer I spend time on things that are more worthwhile to them.

I don't disagree that having a deterministic CLI is useful, but I'd argue that's not the question here, because the CLI is what the statistical machine will be using. The question is whether the time spend doing it manually is worthwhile compared to the tokens spend on it. I'd argue that if you instead learn how to pick the right model for the right job so that you don't use something way more expensive than needed for the task at hand, the token overhead will be extremely minimal for stuff like git operations; knowing that you can use something cheaper than like, Astra or Opus if all you want to do is a quick git cleanup and push is going to save your employer a lot more money in the long run than if you insist on doing every messy rebase by hand.

nightdeployboy 17 minutes ago | parent

While using LLMs is a great idea, it has become crucial to put safeguards in place to ensure that their use doesn't deviate from team practices or compromise security.

bawolff 16 minutes ago | parent

In the before time, this is the sort of thing people would just google.

One time setup commands are the sort of thing people do once when setting up the repo, but dont use day to day. its understandable that people dont have that memorized and will reach for a man page, google or an AI.

syshyk 53 minutes ago | parent

Wasn't this a think before_ai? Enough people are using IDEs which handle this for them. I still have to check a few commands now and then.

Though ideally interviewers should appreciate this - unless its a hard requirement you do it by hand day to day - the rest of us can look this up quick enough.

saghm 21 minutes ago | parent

At previous jobs, I was the one coworkers would come to when they had messy git problems and needed help cleaning them up. I've put in the hours in the past to learn how to do all of that, and then later forgotten it because it's just not worth it anymore. I switched to jj a few years ago and had the majority of the annoying rebase issues go away, and whatever's left goes into my LLM to spit out a potential solution that I can then manually tweak if the result isn't perfect.

My OS professor in college once told us about what he considered the difference between "knowledge" and "information", with "knowledge" being the underlying facts about how something works and "information" being the terminology/labeling/bucketing/etc. of how to refer to it. The point he made was that knowledge was more important than information, and that he would try to test us on knowledge rather than information. To me, knowing how the underlying version control model works and what "shapes" you might want to make with it is useful knowledge. The exact set of commands, interactions with an IDE, or specific prompts you can use to ask an LLM to produce them is information. Arguably the entire benefit of LLMs is that you can ask them to do things even if you don't have the exact information memorized; if you can describe it in a rigorous way, and recognize whether what it gives you as an output fits that or not, you don't need to know any of the actual information of how to do it manually.

philpem 11 minutes ago | parent

That's pretty much where I'm at, so long as they're not filling the code with "///CLAUDE ADDITION" type guards (ie noise).

Personally I ask the LLM to summarise what the commit does and explain it to me, make sure I understand it, then review the PR. I'm getting half the speed the High Speed Sloperators are getting but I was the only person submitting PRs the other day when the LLM service was down.

For nasty multi-way git merges and rebases, it's been a game changer. And of course that's reviewable by doing a delta on the final rebase branch vs. the original branch.

With all that said... having to use LLMs at work is making me want to use them less on my own projects.

bluGill 36 minutes ago | parent

Git is a simple tool. I expect any engineer can learn to use it. While git is common, I would not expect an engineer who worked at a company using a different version control system to know how to use git (which is a really awful - I still miss mercurial). I expect you to know the general concept of version control, but the specific on is a tool I expect you can learn quickly.

ItsYan 34 minutes ago | parent

How would you prepare for the coding problem as a senior? What are the assumptions that underpin this?

Insimwytim 33 minutes ago | parent

  we also ask the candidate to open a PR against the repo just to see if they know the basics of git

That's the basics of Github, not git.

A candidate may be able to open a PR and not be able to make a commit without an IDE.

Another candidate might know plenty of git, but haven't worked with Github.

asmnzxklopqw 1 hour ago | parent

I see much buzz lately about Jane Street. Probably they had a good year and now they’re looking for fresh meat, trying to expand even more.

godockly 58 minutes ago | parent

did you get paid for that slop?

floathub 39 minutes ago | parent

This is a completely side comment, but I don't understand why stories about Jane Street keep making the front page day after day:

Today: https://news.ycombinator.com/item?id=50467297

Yesterday: https://news.ycombinator.com/item?id=50455394

Two days Ago: https://news.ycombinator.com/item?id=50438261

It's a bit odd (?!).

Grimburger 36 minutes ago | parent

Noticing a bit too much

What is the deal with it though? Daily front page content in various forms starts to raise eyebrows.

LPisGood 34 minutes ago | parent

They pay a LOT of money, have some high level of “nerd” prestige for solving interesting technical problems, have good outreach with things like problem of the month, and have not been impacted by massive layoffs.

There are pretty few companies out there like this. Being relatively small probably helps; everyone knows someone that worked at Mag7 and maybe has a bad story to share, but this is more rare.

wmchen 20 minutes ago | parent

I am particularly fond of the assessment that Jane Street is a federation of anarcho-capitalist syndicates disguised as a quantitative trading firm.

locknitpicker 18 minutes ago | parent

> They pay a LOT of money, have some high level of “nerd” prestige for solving interesting technical problems, have good outreach with things like problem of the month, and have not been impacted by massive layoffs.

I never heard of them.

If not for HN, I would never know they existed.

I don't think the remark about "nerd prestige" holds, but beyond that your points boil down to marketing.

And how an unknown company changed its hiring practices is hardly front page news.

LPisGood 14 minutes ago | parent

Paying a lot of money and hard technical problems are not exactly marketing.

Like I said, they are relatively small but they are definitely not “unknown”. Out of curiosity, could you name any quantitative trading firm, or trading firms in general? You specifically may happen to just not be paying much attention to this sector.

maitland 33 minutes ago | parent

I posted this. I'm just a regular dev, without any kind of attachment to Jane Street, in fact i'm a little wary of them, and would subscribe to Evan Czaplicki's understanding of the faustian bargain working there would involve. I think they're banned from India, if you want an example of the kind of raiding they are equipped to do.

In general - I am wondering how engineering interviews change in the age of LLMs, Jane Street, tech-wise is a "thought leader", at least having opinions, sharing them is part of how they recruit "the best", i think oxide does this too.

So in some way, in this time of change it makes sense that they are noisy... It also makes sense that the noises they make may filter down...

sigbottle 23 minutes ago | parent

> I'm just a regular dev, without any kind of attachment to Jane Street, in fact i'm a little wary of them, and would subscribe to Evan Czaplicki's understanding of the faustian bargain working there would involve.

Mind going more into this, since you named a specific person? Did he make some prominent statement?

floathub 7 minutes ago | parent

Yeah, sorry, I didn't mean to imply anything nefarious on your part (or anyone else's part). It's just odd that a single financial firm keeps coming up.

username_slop12 33 minutes ago | parent

Back then, people who were bad at interviews were often undervalued. I think interview culture itself will soon be devalued and eventually become obsolete. People who build real technology and solve real problems will be unaffected, while the glorified Valley slop built around interview performance will become obsolete.

My optimism comes from AI's ability to solve genuinely hard mathematical problems this year. September and October could go down as two of the most remarkable months in the history of AI If, and only if, those proofs are correct and independently verified by real experts. Even if the proofs have been formally verified in Lean, expert scrutiny still matters.

bluGill 25 minutes ago | parent

Does it work? How do they know it works?

There are a lot of people telling you how they interview and why. Almost none of them have shown even the slightly indication that they are aware there there is scientific research on what works for an interview, much less that they apply it. They also are not showing any indication that they have done any testing - to see if their questions help hire better people.

If I said they could get equal results by picking resumes at random and hiring them without any other screening could they refute that? If so have they justified every part of their process or just one step that is useful and the rest they don't know?