I’d even argue that most things operated by tech doesn’t need 24x7x365 availability. If it’s about life-and-death, then yes make it super reliable and available. Otherwise, bring back scheduled downtime please.

I’ve always thought about this as more of a meme than a serious point. Trivially, aren’t most of the network protocol RFCs “sufficiently comprehensive spec that isn’t code”?

This is probably the least important bad thing you could say about Altman.

Also why is a low effort commentary piece of the NYT article on the HN front page?

It’s what the cool kids are doing these days. It’ll pass. Also it’s quite fun to build things around agents and watch them go off to work.

I think there's also something about being able to make it exactly how you want. For example I didn't like openclaw, I felt it was extremely over-engineered and I honestly couldn't even get it running on my first attempt. So I made botctl to be a generalized version where it doesn't rely on complex setup and have every bell and whistle, you just install it, create a directory with a BOT.md file and off it goes.

ARK was all the rage around early pandemic time when wallstreetbets was in the news a lot. Most people probably know it from then.

Weird that this needs to be said. I’m not familiar with the Go ecosystem, but there is usually a natural incentive for library developers to reach more people, which means you’d want to support the oldest feasible version. If you don’t do that then someone will develop a better library which does support an older version. Is that not happening here?

What the article does not say is that if you don't have a recent enough version, by default, Go automatically downloads a more recent toolchain. So, for most users, this is transparent.

However, this behavior can be disabled (for example, when building for a Linux distribution).

The problem is scale. Beyond a certain scale it’s all a net negative: social networks, bitcoin, ads, machine learning, automated trading, big this big that, etc.

Unfortunately for fellow developers, software enables massive scale.

A lot of process and management is about dealing with low performers - by which I don’t mean incompetent people but people lacking motivation, or with the wrong intuitions, etc. Our hiring process can’t reliably filter out low performers, and when they get in it’s difficult to fire them, so we invent ways to raise the bottom line through processes.

And FWIW I don’t think you can solve this by always hiring the “best” either, at least not beyond a certain team size.

I agree in principle. However JSON tooling has also got so good that other formats, when not optimized and held correctly, can be worse than JSON. For example IME stock protocol buffers can be worse than a well optimized JSON library (as much as it pains me to say this).

Yeah the raw parse speed comparison is almost a red herring at this point. The real cost with JSON is when you have a 200MB manifest or build artifact and you need exactly two fields out of it. You're still loading the whole thing into memory, building the full object graph, and GC gets to clean all of it up after. That's the part where something like RX with selective access actually matters. Parse speed benchmarks don't capture that at all.

> The real cost with JSON is when you have a 200MB manifest or build artifact and you need exactly two fields out of it.

There are SAX-like JSON libraries out there, and several of them work with a preallocated buffer or similar streaming interface, so you could stream the file and pick out the two fields as they come along.

You still have to parse half the entire file on average. Much slower than formats that support skipping to the relevant information directly.

yep, this is exactly the kind of use case that caused me to design this format.

as parser: keep only indexes to the original file (input), dont copy strings or parse numbers at all (unless the strings fit in the index width, e.g. 32bit)

That would make parsing faster and there will be very little in terms on tree (json can't really contain full blow graphs) but it's rather complicated, and it will require hashing to allow navigation, though.

yep. I built custom JSON parsers as a first solution. The problem is you can't get away from scanning at least half the document bytes on average.

With RX and other truly random-access formats you could even optimize to the point of not even fetching the whole document. You could grab chunks from a remote server using HTTP range requests and cache locally in fixed-width blocks.

With JSON you must start at the front and read byte-by-byte till you find all the data you're looking for. Smart parsers can help a lot to reduce heap allocations, but you can't skip the state machine scan.

Great post!

The minimized repro seems like something many other eBPF programs will do. This makes me wonder why such kernel issues weren’t found earlier. Is this code utilizing some new eBPF capabilities in recent kernels?

Thanks!

The new spinlock that the problem is in was introduced in kernel 5.15, which is relatively new, you need to be hooking context switches, and you need to be sampling at a high enough frequency that you hit the problem, and you need to be using the ring buffer to emit those events. Outside of CPU profilers like us, I don't think there are many other eBPF applications with this type of setup.