Apple is also a global company and has offices and research labs world wide. At least a couple of the authors seem to work for Apple but at their German lab.

> their main trick for model improvement is distilling the SOTA models

Could you elaborate? How is this done and what does this mean?

I am by no means an expert, but I think it is a process that allows training LLMs from other LLMs without needing as much compute or nearly as much data as training from scratch. I think this was the thing deepseek pioneered. Don’t quote me on any of that though.

No, distillation is far older than deepseek. Deepseek was impressive because of algorithmic improvements that allowed them to train a model of that size with vastly less compute than anyone expected, even using distillation.

I also haven’t seen any hard data on how much they do use distillation like techniques. They for sure used a bunch of synthetic generated data to get better at reasoning, something that is now commonplace.

Thanks it seems I conflated.

Yes. They bounced millions of queries off of ChatGPT to teach/form/train their DeepSeek model. This bot-like querying was the "distillation."

They definitely didn't. They demonstrated their stuff long before OAI and the models were nothing like each other.

Why would OpenAI allow someone to do that?

They don't anymore. They introduced ID verification shortly after, but it's hard to stop completely while also scaling fast.

They didn't, but how do you stop it? Presuming the scale that OpenAI is running at?

I feel very different about AI. I have still no clear idea what crypto is good for except money laundering. AI feels very different. It might not live up to all it’s promises. But it is clearly very capable.

You could increase the battery pack voltage and/or charge several packs in parallel. Why not have 20 charging cables. Shouldn't be a problem for a ship. Would of course be fairly inconvient for a car.

Just because someone is on an H1B visa doesn't mean they know less. It's a bit rich to blame this on foreign workers even though nothing is known about who or what caused this outage.

The problem with H1B is that these people are effectively prisoners. The market is not so hot right now even for those who have leverage, but combine it with the visa system and you get this "gotta do the needful" attitude to please the bosses, rushing broken fixes to production.

I see this directly on my team. The h1bs get bullied by their boss (it's a split team, I work with him but don't report to him) and they don't say anything because he could effectively have them deported. At least 2 of them have kids here and perhaps the others do. So not only does it incentivize the bully to do it, but it traps them to just take it for their family. I openly talk shit back to him because he can't deport me.

Knowledge + tech skills are not the only factor that lead to subpar outcomes with these scenarios. In my experience the thing that causes the most problems with H1Bs is the weak English and related communication issues.

In my experience, the communication problems stem from the Americans who expect perfect English from all others. English is spoken across the entire business world between people for whom it is not their first language. The accents and broken English is epic in many organizations. Yet they work through it and get things done together.

If you work harder at taking the burden upon yourself to understand others, you might be surprised how well people can learn to communicate despite differing backgrounds.

I have the same experience as you. I have been working with many non-native English speakers from different countries, and Americans (and to some degree Brits) are usually the ones who can't follow what is said. This improves over time as they get used to different accents, but it seems it is easier for non-native speakers to understand foreign accents than native speakers in general.

I'm not saying I always understand 100% of what is said. When someone with an accent from a specific part of a country speaks super fast and is on a poor line with lots of street traffic in the background, it can be hard to follow. But usually I catch enough of it to be able to communicate.

Normally I have had very good experiences as well. My colleagues almost always speak very good English and even those who don't are understandable. Everyone is happy to conduct meetings (with many nationalities, as I work in a scientific field) in English.

Only once have I encountered a problem. A colleague berated me in front of others for speaking "difficult English" and accused me of doing this on purpose to cause trouble for them, instead of speaking proper international English like everyone else did. But, I am a native English speaker with an RP accent and we were all in England at the time, working for a British organisation. I was simply speaking normally and otherwise had no issue with this colleague, whose English was very good. I don't recall their having been any misunderstandings between us before.

That’s just saying the same thing. American companies have engineering quality loss when they try to collaborate with people they can’t communicate well with. Whether it’s dumb Americans or poor ESL, it’s not really relevant to the outcome because it’s the same.

I’m blaming it on paying workers less. H1B, location based pay, offshoring, etc. are all ways to pay workers less.

I think the implication is that people with connections who have no business touching production code are getting those jobs through fraud.

We don’t have to completely wipe ourselves out to regress or stagnate. There have been many civilizations that have regressed.

Until it gets widely adopted and people want to have certain feature, etc.

Golang I think (mostly) successfully resisted this temptation

Generics though

Considering language syntax proposals aren’t accepted anymore and they have a culture of being minimal I believe it will remain so.

There's a reason it's in 0.x and will not reach 1.0 for a while.

Solid state batteries and fusion might in the end suffer from a similar economical problem. That they turn out to simply be too expensive.

This seems completely unrealistic nowadays, unless you are the size of China or the United States. The EU could also do this, but there seems to be currently only limited appetite for a more integrated EU.

Without Manchin the Inflation Reduction Act would not have passed. Arguably Biden‘s biggest accomplishment.