The "tech bros" were right: The board absolutely were conspirators wrecking the company based on dogmatic ideology. "Tech bros" had clear evidence of Sutskever, Toner and McCauley's deep and well-known ties to the Effective Altruist (EA) movement and doomerist views. Nobody can doubt Sutskever's technical credentials, whatever his strange beliefs, but Toner and McCauley had no such technical background or even business achievements. These members instead were organizers in EA and researchers in the "field" of AI governance and safety. This is no expertise at all. These "disciplines" are built more on Yudkowsky's rants and hypotheticals and the precautionary principle gone mad than any empirical research. These ideas also come from a cult-like social movement (EA) accumulating power across tech and government with few scruples, as shown by SBF's implosion last year and many smaller incidents. How could Toner and McCauley assess briefings if they couldn't assess the technical fundamentals? How could they foresee the consequences of sacking the CEO if they didn't understand business? If they already believed AGI would kill all humans, how could they judge new advances on the merits without jumping to wild conclusions? Instead, us "tech bros" felt people with this background would fall back on uninformed, reactionary, and opaque decision making, leading to wrecking an $80 billion business with no decent explanation.
This now seems to be exactly what happened. The board saw Q* and decided to coup the company, to put all power in their hands and stop development. This by itself is bad enough if you care about open science or progress, but it gets worse. They didn't even want to hint at capabilities increases to avoid "advancing timelines" i.e. Open-ing knowledge about AI, so they made up some canard about Altman's "lying to the board" to hide their real reasons. This is vile libel for unscrupulous ends. When they realized this excuse wouldn't fly, they obfuscated and refused to explain their true concerns, even to their own handpicked CEO Emmett Shear. However, it turns out that destroying $80 billion and lying about why won't fly in the real world. The board had no second-order or even first-order thinking about the consequences of their actions, and were rolled up by bigger actors. These people were unprepared and unable to follow up their coup.
This failure is exactly what you'd expect from a brilliant scientist with little organizational experience (Ilya) and social-science academics and NGO organizers (Toner and McCauley). I don't care what gender they are, these people neither deserved their authority nor could use it effectively. Dismissing valid criticism of the board as "cyber bullying" or "tech bro sexism" merely underscores why most engineers hate DEI rhetoric in the first place.
“Open”AI already was an unaccountable big corp. They’ve already refused not only to open their weights but to publish most of their research, to create an insurmountable gap with the rest of the world, and to legislate it in with lobbying. “Openness” merely meant “the API is open to your money”, with opaque “content policies” we had no say in. They had the same unaccountable and opaque power and the same commercial drive, but “for our own good” as they define it unilaterally.
Moving to MSFT merely means they’ll do the same thing, but with a bit more reliability, a bit more fear of liability, and a bit less of the doomerist sanctimony of the original leadership. Better to at least have a bigco with coherent and stable values like money. The bigco “nonprofit” led by the current board has made erratic decisions, has insane longtermist and EA values, and refused to give any meaningful statement about why they did what they did. Inasmuch as they resist commercialization, it’s to have more opacity and more control not less. How can we trust these people with control over AGI? Better to junk the board and deal with straightforward greed rather than hubris.
I don't understand why you putting the blame on the board, instead of the CEO, who:
1. is way more responsible for the direction the company deviated to
2. was in fight against the board, who did not like his direction
3. will be in the new company leading everything.
So all the bad that you criticize OpenAI for would leave to MS, and yet people are still cheering for it.
Yes, the EAs and longtermists on the board probably disliked the commercial focus of “Open”AI or its rapid scaling; I’m not blaming them for Laundry Buddy. But make no mistake: the nonprofit board had even less interest in opening up their research or sharing their code. They and the “superalignment” team believe AGI can end the world and needs to be in safe hands (i.e. their own hands). The EA movement which they are embedded in is one of the leading forces advocating shutting down open AI development through regulation.
The board has strong EA and doomerist ties. Within OpenAI, Ilya Sutskever is on record as saying that the world will realize open-sourcing weights is foolish by 2025, and the Atlantic reported he literally burned an effigy of “unaligned” AI at the company retreat. Helen Toner is similarly involved in “AI governance” and not coincidentally took funding early in her career from OpenPhil, one of the key EA slush funds. The new CEO, their appointee, is quite literally a character written into Eliezer Yudkowsky’s rationalist fanfic in a cameo, and believes the probability of AGI killing humanity is 50%.
These people have absolutely no interest in decentralization and accountability, and were more than happy to let OpenAI accumulate power to “protect humanity”—until they pulled the plug for reasons they still refuse to disclose. Let me be clear: this is unacceptable. Not coincidentally, their so-called utilitarianism and altruism merely justifies accumulating all power in their hands, and taking any action (like the backstabbing we saw last week) to make it happen. For all MSFT’s faults, they play by traditional and predictable corporate rules of greed and can be reasoned with. The safety faction are true believers and implacably opposed to openness anywhere, and moreover happily gave the veneer of altruism to the regulatory capture of the commercial faction anyway before they realized they couldn’t control it. I know which one I’d pick.
Is Laundry Buddy actually a controversy? If the mission is to aid humanity, I can think of nothing more helpful. Nobody know what any of that shit on the tags means.
Is this a sheepskin comment or a genuinely naïve one? I can't tell.
Moving to MSFT means they cannot do anything that goes against MSFT's interests.
Everything they ever did, they ever will do belongs to MSFT.
MSFT brings with it all the bloat and risk aversion it needs as a big org, killing the "cutting egde" move-fast, make-it-work nature of OAI that got it to this point in the first place.
Only thing you can be sure of is this thing will now be "closed" forever, with no hope of others benefiting off the hard work of the real people who make it happen.
Phishing emails don’t exactly take AGI.
GPT-NeoX has been out for years, Llama has been out since April, and you can set up an operation on a gaming desktop in a weekend. So if personalized phishing via LLMs were such a big problem, wouldn’t we have already seen it by now?
Just a brief reminder of the last hundred years of copyright: large lobbies like UMG, Disney, or Elsevier have hugely circumscribed the public domain for all but historical purposes (copyright lengths increased from 28 years to *128*). Open access in academic publishing (for public funded research, even!) has been a decades-long fight that is not fully won. The DMCA has created myriad abuses and denied the freedom to control your own hardware, like allowing companies like John Deere to prevent right to repair. Six studios own most movies, a few labels control most music. The cultural heritage of the last hundred years, which should be the collective property of humanity, is locked up in the coffers of a few megacorps: who, by the way, don't much care how well their artists are eating. Yes, there are edge cases, but spare me the sympathy for the poor artist on Fiverr with five images in LAION: this is a fight between tech titans and copyright monopolists, where the little people are always going to be an afterthought. Come out and say the original copyright term of 28 years (well long enough to eat!) should be brought back, condemn the DMCA and the naked abuse of democracy in the Sonny Bono act, and THEN maybe we can get to chiding the lapses of a few researchers just wanting to understand the world better.
I don't support tech titans or copyright monopolists and mostly just think stuff should enter the public domain earlier as that is what both groups don't want to have happen. And in the case of software I think it ought to be illegal to not release source code into the public domain after a reasonable amount of time, say 25 years. In fact I think it's absolutely crazy we allow large companies to sell infrastructure with software in it and then basically light the source code on fire so that people are forced to buy new versions of their product rather than maintain the source for the old version themselves.
I’m not sure who the “we” is in your statement, “we do know with certainty that IVF is immoral”. As evidenced by the entire rest of the thread, this opinion is hotly contested. If you believe IVF is morally wrong, please at least explain why. As it stands, you’re simply condemning families and researchers as evil and holding future research to an arbitrarily high, nebulous ethical bar with no explanation.
"We know with certainty that IVF is gravely immoral." is essentially "I know I'm 100% right and everyone who disagrees is wrong."
Seriously, why even bother engaging with people like that who have made it exceedingly clear they are uninterested in listening. In the words of the always incisive Barney Frank, "I'd rather have a debate with a dining room table."
How does open source compete with the Claude API? Easy: actually let you use the model. From the signup page:
> Anthropic is rolling out Claude slowly and incrementally, as we work to ensure the safety and scalability of it, in alignment with our company values.
> We're working with select partners to roll out Claude in their products. If you're interested in becoming one of those partners, we are accepting applications. Keep in mind that, due to the overwhelming interest we've received so far, we may take a while to reply.
No thanks, I'd much rather not wait months to see if my app deserves their oh-so-limited attention, or "aligns with the values" of a company taking $400m from Sam Bankman-Fried.
To be more charitable to your underlying point, Claude 2 is free to chat with via Anthropic's website, Poe, or Slack, and the GPT-4 API is open to use. If you're building a prototype or just need a chatbot, these do have better results and dev experience, at least for now. But I don't think picking on your Claude API example is unfair. These companies could randomly refuse your prompts via some opaque "moderation API" (that all GPT fine-tuning data goes through!), train on your company's proprietary data, spy on your most intimate questions, or just not find you worth the trouble and cut you off, at any time. THAT is why open source beats proprietary hands down: My device, my data, my weights, my own business.
Awkward tie-ins between SBF and value systems (?) have no effect on practical usage.
A theoretical concern they might train on my API data after saying they won't doesn't either. Amazon might be training on everything not bolted down in S3, not worth wasting brain power on that.
The moderation API isn't some magic gotcha, it's documented. They don't want to deal with people fine tuning for porn. Maybe you have some ideological disagreement on that but it's not of practical relevance when trying to write code.
At the end of the day you're not alone in these opinions. But some of us prefer pragmatism over hype. Until someone catches OpenAI or Anthropic trying to kill their golden goose by breaking their GDPR, HIPPA, and SOC2 certifications, I'm going to take delivered value over theoretical harm.
Governments are far harder to remove than tech firms. You may be able to ditch Google, but there's only one government in your country. And "democracy" doesn't prevent state surveillance.
Most politicians back it, so voting differently makes little difference. Labour and the Conservatives support the Online Safety Bill, the Patriot Act was bipartisan, and voters have very little control over the EU and can’t stop Chat Control. And most of “government” isn’t directly elected: you can’t vote out the NSA, and Congress has little power over them either. The government blunts corporate abuses but doesn’t stop them: revolving doors ensure authorities target small fry while big companies like Visa keep going unimpeded. And finally, most voters don’t mind surveillance that much, since government and media manufacture consent for it. Don’t count on ordinary people to “vote it out” until it’s too late.
Lobbying against government surveillance helps marginally, but it's an eternal struggle. Governments take as much power as they can get, while abuses are exponentially harder to detect and stop than refusing to grant that power in the first place. The “slippery slope” isn’t a fallacy, it’s the record of the last twenty years. Don’t let them track speech and money with a central ID and digital currency, just because you don’t like a few tech bros or online trolls.
> Governments are far harder to remove than tech firms. You may be able to ditch Google
Ditching Google is not the same thing as removing Google from governance of your life, though. I don't use Google search, but I am sure they know who I am and sell that data to anyone who wants it, including government agencies which can't legally obtain that data on their own due (ostensibly) to citizen oversight.
Realistically, the average person has exactly 0 chance of "removing" either a government or a big multinational company.
However, the average person at least has some teeny tiny say in government via democratic processes and oversights. They have zero power against a big company unless they are a major shareholder.
The fundamental difference of "one person, one vote" and "one dollar, one vote" should not be lost in this discussion.
Big bureaucracies are terribly disempowering no matter who runs them, but in government at least you have some tiny amount of representation vs zero in the private sector.
Basic economics: The optimal number of people who die tragically in a burning building is not zero. Every person trades off some amount of safety for cost, that’s just part of living. It’s possible that this particular tradeoff isn’t worth it, but human life isn’t infinitely sacred. I assume if it were in your power, you also would rather let some people burn to death than ban wooden houses, for example. This, well, inflammatory emotive rhetoric really doesn’t lead to good discussion, or good policy for that matter.
The article makes a good point: we should prevent “open-washing” and draw a distinction between well-intentioned restrictive licenses like “Open”RAIL and true open source. However, I worry the name “ethical source” is itself a bit question-begging. While outfits like Bloom may believe in good-faith ethical principles, their definition of ethics isn’t necessarily everyone’s. If restricted models are “ethical”, is releasing open weights “unethical”? Conversely, is releasing a model with PII or artist styles in it “ethical” if a few known use cases are forbidden? There’s no one right answer. Labeling any one set of restrictions as “ethical” off the bat makes discussion harder and puts open source on the back foot to justify “not being ethical”. Better to just call them “restricted models” or “guarded models”, and leave it to individuals to decide if these restrictions are beneficial or not.
I think the more interesting aspect of all this is that the confusion created by this new business model ( not sure to classify it so business model had to do ) appears to be largely intentional. The subject matter is complicated to begin with experts being niche of a niche of a niche and the assumption that the general public can even understand it ( and whether it can even dumbed down to digestible sound bites ) is, in my mind, very optimistic. Now, courts are not typically stacked with dummies, but again how many are well versed in issues of technology?
All in all, I don't disagree with the point you raised, but I worry that all this will only further muddy the water for the general population.
"Now, courts are not typically stacked with dummies, but again how many are well versed in issues of technology?"
Even if they are well versed in issues of technology that does not mean they'll make what any given one of would consider a good decision, as plenty of people well versed in issues of technology disagree with each other on these issues.
Nothing guarantees that on, on any issue, really, as you can always find people who disagree.. and if they happen to be judges, they get to decide unless another higher judge overrule them.. and that judge has the same problem as the first.
Sure. My point is that I would so much rather have a decision handed down that was considered on actual merits ( we might disagree, but at least I would be able to see some sort of real consideration and not what amounts to talking points from various lobbyists ). A judge that has zero exposure in that area is at best 50/50 and regardless of the ruling I will be annoyed that a person with zero knowledge is declaring how something he knows little to no about can be used ( just like I am more and more annoyed about political class in Washington, but I am more inclined to believe these days they know exactly what they are doing -- serve their own interests ).
To your point, it is absolutely not panacea ( new blood is inevitably ending in government and the result so far is in line with what you said ), but it would at least be a starting point.
The source seems to be a February article from Automotive News interviewing several Toyota execs off-the-record: https://archive.is/0LZYf. The relevant quotes are midway through
This now seems to be exactly what happened. The board saw Q* and decided to coup the company, to put all power in their hands and stop development. This by itself is bad enough if you care about open science or progress, but it gets worse. They didn't even want to hint at capabilities increases to avoid "advancing timelines" i.e. Open-ing knowledge about AI, so they made up some canard about Altman's "lying to the board" to hide their real reasons. This is vile libel for unscrupulous ends. When they realized this excuse wouldn't fly, they obfuscated and refused to explain their true concerns, even to their own handpicked CEO Emmett Shear. However, it turns out that destroying $80 billion and lying about why won't fly in the real world. The board had no second-order or even first-order thinking about the consequences of their actions, and were rolled up by bigger actors. These people were unprepared and unable to follow up their coup.
This failure is exactly what you'd expect from a brilliant scientist with little organizational experience (Ilya) and social-science academics and NGO organizers (Toner and McCauley). I don't care what gender they are, these people neither deserved their authority nor could use it effectively. Dismissing valid criticism of the board as "cyber bullying" or "tech bro sexism" merely underscores why most engineers hate DEI rhetoric in the first place.