> sufficiently advanced mimicry is not only indistinguishable from the real thing, but at the limit in fact is the real thing
I am continually surprised at how relevant and pervasive one of Kurt Vonnegut’s major insights is: “we are what we pretend to be, so we must be very careful about what we pretend to be”
Gall’s law: “A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.”
Your theory of premature architecture reinforces Gall’s law.
This is from the book Systemantics: How systems work and especially how they fail (1977).
Basically the answer comes down to the technical term “ensembles of agents”. All these frameworks dance around it in different ways, but ultimately everyone’s trying to recreate this basic AI structure because it’s known to work well. I mean, they even implemented it on a sub-human level with GPT’s mixture of experts!
If you haven’t had the chance yet, I highly recommend skimming Russel and Norvig’s textbook on symbolic AI. Really good stuff that is only made more useful by the advent of intuitive algorithms
Creo (based in Vancouver, BC) used to be a company that tried to address this. The concept that was used was called "unit presidency". Each employee was empowered, expected, and trained, to make decisions as if they are the president of the company. The principles behind making decisions were called "economic thinking" which the CEO used to say was everything he learnt in a Harvard (EDIT: or maybe it was Stanford) MBA distilled into the core principles. Basically looking at the ROI (Return on Investment) of the decision. Decisions were generally made by consensus though depending on the nature of the decision sometimes other methods were used. This extended to decisions that involved spending money, not just should you pick language X or language Y for your next software project.
I think it worked pretty well for quite a few years. It gradually stopped working when the company acquired a large company with a different culture and also hired people (well, managers mostly) who weren't aligned with the culture. Eventually this basically disappeared when the company was acquired by Kodak.
I've seen flavors of this in other places. Famously Andy Grove of Intel also preached that decisions need to be made by those closest to the decision and empowered people to make the right decisions. More generally this can be reflected in a servant-leadership model where leadership sees itself as facilitating the growth of the people underneath them.
Another requirement for this to work well is that management (e.g. the CEO or other leaders) are able to lay down a broad strategy for the people of the company to execute on. If the leadership has no strategy then tactical decisions can not be made properly. They also need to make sure there's coordination and structure.
I love local first, but I struggle with how to monetize truly local-first applications. I know that's not everyone's favorite topic, but I've got bills to pay and payroll to make. Our product is about 80% local-first, with live collaboration and image hosting needing a server. I plan to change that eventually, but I worry that jailbreaking the app in this way will lead to financial troubles.
Obsidian's model seems nice: base app is free, and then payment for the networked portions like sync+publish. However, there's little data available on how well this works and how big of a TAM you need to make it sustainable. Or if it's even possible without an enterprise revenue channel.
For those interested in building robust local-first + collaborative apps, I've been using Yjs for a few years now and have overall really enjoyed it. Multi-master collaboration also poses some stimulating technical and design challenges if you're looking for new frontiers beyond the traditional client-server model.
I really wanted to like Notion but it's not smooth enough for writing. The cell system makes it clumsy.
I prefer to write on markdown files because it's much faster and I can do it on my text editor of choice. I like obsidian because it's basically that with a bit of extras.
But then I lose the concurrent editing.
I want Google Docs meets obsidian type of environment. And I am yet to find it.
It sort of is fixable, though. If you think about it, the problem is a bunch of functions are all mapped to one of a small set of names: +*/, etc. That is, the operators. If we didn't try to cram all this functionality into a tiny handful of symbols because of some weak analogy they have with basic math operations[1], then the compiler would have far fewer name conflicts to try to disambiguate, and the problem goes away on its own. Like yeah, the problem still exists if we made a few dozen functions all called "a", but the solution is to not do that, not give up on an otherwise fine type system.
I'm convinced operator overloading is an anti-feature. It serves two purposes:
1) to make a small set of math operations easier to read (not write), in the case where there are no mistakes and all readers perfectly understand the role of each operator; and,
2) to make library developers feel clever.
Operator-named functions are strictly worse than properly named functions for all other uses. Yes, yes, person reading this comment, I know you like them because they make you feel smart when you write them, and you're going to reply with that one time in university that you really needed to solve a linear algebra problem in C++ for some reason. But they really are terrible for everyone who has to use that code after you. They're just badly named functions, they're un-searchable, they make error messages unreadable, and they are the cause the naming conflict that is at the root of the linked blog post. It's time to ditch operator overloading.
[1] Or because they look like the same symbol used in some entirely other context, god, please strike down everyone who has ever written an operator-/ to combine filesystem paths.
Disallowing these case/underscore variants from referring to differing values could be quite ok -- even a plus-- if (but only if) an additional rule is added to disallow any variations of the declaration-time casing for a variable within the same scope to prevent the need for bespoke grepping tools and discourage arbitrary variations. If that restriction was added one could argue that this improves code readability (& "listenability") by have fewer variable names available in the same scope which would sound the same but aren't.
I've observed another meta as well: IDEs/language servers.
I was writing a Wordpress plugin and by default in PHP all function definitions are global, e.g. is_single that WP defines is global. Naturally to avoid conflicts you would use a namespace, but since most of the examples I saw used prefixed function names, I decided to try that as well.
In my opinion, coding just feels much clearer when you prefix all your names, like my_library_my_func. When I see my_library_, I just immediately know that is my code. If you use different prefixes for different modules, you can tell from a glance which modules a piece of code interacts with. In C++, you can tell a random word is a field by the simple fact everyone prefixes them with m_. I've seen people using prefixes for pointers as well.
This is the same concept you have when designing an UI. Everyone agrees a problem with hamburger menus is that you don't know what's inside the burger until you click on it. If you want people to be aware something exists, you need to make it appear on screen, explicitly.
Instead, languages have evolved toward the very opposite. We ran away from self-documentating variable names and prefixes, and toward namespaces, "const" and other keywords, and "OOP" that feels less object-orientated and more press-the-dot-key-to-get-a-list-of-methods orientated. We depend on docstrings that show in the IDE, and then on auto-generated documentation from those docstrings. Typescript is probably the worst case of this, because it gives you a lot of tools to make the dot work the way you want, you'll spend way too much tinkering with the dot instead of just writing code that actually runs.
Thanks to this, when you look at code, you have to wait until your language server figures out what you're looking at for you. If the IDE doesn't work, it's effectively impossible to code anything in many cases. But the variable names are shorter so people think it's cleaner and better, like removing features from an UI to make it cleaner.
This is a great article! I love his description of Forth as "a weird backwards lisp with no parentheses".
Reading the source code of "WAForth" (Forth for WebAssembly) really helped me learn about how WebAssembly works deep down, from the ground up.
It demonstrates the first step of what the article says about bootstrapping a Forth system, and it has some beautiful hand written WebAssembly code implementing the primitives and even the compiler and JavaScript interop plumbing. We discussed the possibility of developing a metacompiler in the reddit discussion.
I posted this stuff about WAForth and a link to a reddit discussion with its author in the hn discussion of "Ten influential programming languages (2020)":
It's a lovingly crafted and hand written in well commented WebAssembly code, using Racket as a WebAssembly macro pre-processor.
I learned so much about WebAssembly by reading this and the supporting JavaScript plumbing.
The amazing thing is that the FORTH compiler dynamically compiles FORTH words into WebAssembly byte codes, and creates lots of tiny little WebAssembly modules dynamically that can call each other, by calling back to JavaScript to dynamically create and link modules, which it links together in the same memory and symbol address space on the fly! A real eye opener to me that it was possible to do that kind of stuff with dynamically generated WebAssembly code! It has many exciting and useful applications in other languages than FORTH, too.
Lots more discussion and links in the reddit article.
If you can't be bothered to install VS Code, you can have a look at a standalone version of the example notebook (in a 26kB self-contained page).
And if you're planning to go to FOSDEM 2023, come say hi: I'll be giving a talk there on WebAssembly and Forth in the Declarative and Minimalistic Computing devroom.
DonHopkins:
I really love your tour-de-force design and implementation of WAForth, and I have learned a lot about WebAssembly by reading it. Never before have I seen such beautiful meticulously hand written and commented WebAssembly code.
Especially the compiler and runtime plumbing you've implemented that dynamically assembles bytecode and creates WebAssembly modules for every FORTH word definition, by calling back to JavaScript code that pulls the binary bytecode of compiled FORTH words out of memory and creates a new module with it pointing to the same function table and memory.
WebAssembly is a well designed open standard that's taking over the world in a good way, and it also runs efficiently not just in most browsers and mobile smartphones and pads, but also on the desktop, servers, cloud edge nodes, and embedded devices. And those are perfect target environments for FORTH!
What you've done with FORTH and WebAssembly is original, brilliant, audacious, and eye-opening!
I'd read the WebAssembly spec before, and used and studied the Unity3D WebAssembly runtime and compiler to integrate Unity3D with JavaScript, and I also studied the AssemblyScript subset of TypeScript targeting WebAssembly and its runtime, and also Aaron Turner's awesome wasmboy WebAssembly GameBoy emulator .
I first saw your project a few years ago and linked to it in this Hacker News discussion about Thoughts on Forth Programming because I thought it was cool, but it's come a long way in three years, and I'm glad I finally took the time to read some of your code, which was well worth the investment of time.
Until reading your code, I didn't grasp that it was possible to integrate WebAssembly with JavaScript like that, and use it to dynamically generate code the way you have!
Also, the way you used Racket as a macro assembler for WebAssembly was a practical and beautiful solution to the difficult problem of writing maintainable WebAssembly code by hand.
Even for people not planning on using FORTH, WAForth is an enlightening and useful example for learning about WebAssembly and its runtime, and a solid proof of concept that it's possible to dynamically generate and run WebAssembly code on the fly, and integrate a whole bunch of tiny little WebAssembly modules together.
Playing with and reading through your well commented code has really helped me understand WebAssembly and TypeScript and the surface between them at a much deeper level. Thank you for implementing and sharing it, and continuing to improve it too!
remco:
Wow, thanks a lot, I really appreciate that! It makes me very happy that I was able to get someone to learn something about WebAssembly by reading the source code, which is exactly what I was going for.
[More links and discussion of WAForth, WebAssembly, and Forth Metacompilers:]
Parexel AI Labs | Senior Software Engineer | REMOTE (must be located in UK)
Parexel is in the business of improving the world’s health. We do this by providing a suite of biopharmaceutical services that help clients across the globe transform scientific discoveries into new treatments.
As the Senior Software Engineer, you will be responsible for the development of AI Labs’ software applications. In this role, you will work closely with other application engineers to architect, build, and optimize scalable and resilient data systems, to meet the growing needs of end users. You will also write, test, and release backend and/or frontend code according to the product roadmap and release plans. Finally, you will work collaboratively with the Product Owner and Software Quality Assurance to set the overall goals and sprint plans.
At AI Labs, we are a small ~20 person group within the larger company that operates as a start up focused on using AI to build solutions to internal company problems. We work with AWS tech, Kubernetes, Nodejs, React, and Python. Apply at https://jobs.parexel.com/en/job/united-kingdom/senior-softwa...
Sometimes, companies will interview me and then when I tell them my salary expectations, they usually pretend to be offended and refuse by arguing that it would destroy the salary structure inside their company. But that doesn't really mean anything because many of them are happy to book my consulting company at 1.5x that salary afterwards.
This is non-intuitive to me because programming languages (above the level of assembly, probably) are made by humans to represent human concepts, while construction is based on physics.
While writing that sentence I realized that programming does have a lot to do with logic, which is a construct of reality, and not humanity. So I guess it does makes sense that programming would be fiddly.
If you want inspiration, we created a (mostly) functional-first curriculum for Snap (like SCRATCH) which still has highly visual output. https://bjc.berkeley.edu/bjc-r/course/sparks.html
It's definitely trickier to bring functional programming into these sorts of environments, but I think we came up with some nice compromises there.
I started blogging more once I stopped trying to make something fancy. Instead of trying to create my own markup and create a new site every time I feel like blogging again, I just create github gists now. It's portable in that I could download all my markdown gists and just put them on a simple nginx server with a gfm renderer. And it has a built-in comments section.
The trouble with comments like this is that they make discussions shallower and more generic, which makes for worse threads [1]. Actually it's not so much the comment as the upvotes, but shallow-generic-indignant comments routinely attract upvotes, so alas it amounts to the same thing.
The most recent guideline we added says: "Please don't complain about website formatting, back-button breakage, and similar annoyances. They're too common to be interesting." I suppose that complaints about writing style fall under the same umbrella.
Not that these things don't matter (when helping people with their pieces for HN I always tell them to define jargon at point of introduction), but they matter much less than the overall specific topic and much less than the attention they end up getting. So they're basically like weeds that grow and choke out the flowers.
(This is not a personal criticism—of course you didn't mean to have this effect.)
A few strong opinions, a lot stronger than they were in my 2017 talk https://youtu.be/_gmMJwjg0Pk, surely that must be the experience speaking :)
- Creating a DSL is a commitment. It is never done, it's either in development or abandoned. Plan for this, both in terms of team size and knowledge management.
- Don't allow escape hatches or ways to access arbitrary functionality in the host language. When (not if) these are used, they ruin backwards compatibility and make maintenance of DSL code a nightmare.
- Don't use a PEG (parsing expression grammar), they are terrible for backwards compatibility. Hand-written parsers are even worse. Using an LR parser generator is worth the time investment.
- Support static analysis, or at the very least don't make it impossible to implement later. It improves productivity, and every error message is an opportunity to teach the DSL to the users.
- You need a language server. The bare minimum is: highlighting, auto-completion, hints on hover, find references, rename.
- It should be trivial to reproduce a production run on a development machine. This means the language should be deterministic, and the production context should be preserved for later reuse, and the development machine should be safe for accessing production data.
- Thousands of unit tests, thousands of regression tests.
It's fine to break these rules, if you can live with the consequences.
And here's a brand new one from this year: use RAG to have ChatGPT "read" your documentation, and ask it to produce scripts for you. When the script doesn't compile, feed the error message back to ChatGPT and try again until you get a working script. Think deeply about why ChatGPT made some mistakes, as it's often the case that humans would have the same misunderstandings. Improve your documentation, error messages or syntax to eliminate them, and then try again to see if it works. We have been doing this for 6 months now, and there have already been some significant improvements that are only obvious in hindsight. And we've now opened a PhD position to investigate whether we can have a separate "LLM-friendly" syntax for the language, dedicated to making code generation easier.
My two cents, GCP has a few excellent gems which are better than pretty much any competing cloud offering:
- Cloud Run. Best way to deploy containers hands down. All of the benefits of a serverless/containerized workload with all the ease of a traditional VPS deployment. Extremely cheap (pay $0 for side projects with little traffic).
- BigQuery. Very easy to use with immense power without having to deal with the details yourself. Billing can be either extremely cheap or rather expensive depending on what you're doing, though.
- GCS Archive Storage. 1/3 the cost of glacier for storage ($0.0012/GB/Month lol), but more for retrieval. Has none of the annoyances of Glacier (you can download your files instantly, upload using normal APIs). Perfect for extremely cheap backups.
- Cloud Spanner. I've not had a reason to use this yet myself, but there really aren't any comparable offerings that I know of.
The syntax I always want but no language I know of supports:
for 1 <= i <= 10:
do stuff
for 0 <= i < 10:
do stuff
for 1 <= i < j <= 10:
do stuff
(In the last case, the exact order in which the 45 iterations happen should maybe be left unspecified; at any rate, it would be bad style to depend on it.)
Downward iteration:
for 10 >= i >= 1:
do stuff
Obviously this doesn't cover every case of "arithmetic for loop" that would be useful: sometimes you want a stepsize that's neither 0 nor 1. I'd be quite happy with a language in which I had to do that using a more general iterate-over-an-arbitrary-sequence construction; I'm tempted by options like
for 100 <= i <= 1000 where i%3==1:
do stuff
but it's probably too clever by half; either you only support the simple "specify the value of the variable mod something that doesn't change while iterating" case, in which case users will complain that some obvious generalizations fail to work, or you support arbitrary predicates, in which case you have to choose between making the "easy" cases efficient and the "hard" cases not (in which case users will be confused when what seem like small changes have drastic effects on the runtime of their code) and making all of them inefficient (in which case users will be confused by how slowly their code runs in some cases).
> Copyright law does not protect ideas, methods, or systems. Copyright protection is therefore not available for ideas or procedures for doing, making, or building things; scientific or technical methods or discoveries; business operations or procedures; mathematical principles; formulas or algorithms; or any other concept, process, or method of operation.
> 313.3(A) Ideas, Procedures, Processes, Systems, Methods of Operation, Concepts, Principles, or Discoveries
> Section 102(b) of the Copyright Act expressly excludes copyright protection for “any idea, procedure, process, system, method of operation, concept, principle, or discovery, regardless of the form in which it is described, explained, illustrated, or embodied in such work.” 17 U.S.C. § 102(b); see also 37 C.F.R. § 202.1(b). As such, any work or portion of a work that is an idea, procedure, process, system, method of operation, concept, principle, or discovery does not constitute copyrightable subject matter and cannot be registered.
> ...
> Mathematical principles, formulas, algorithms, or equations.
Another very successful way to go about building a language is Imba.
Build a successful product with new lang https://scrimba.com, make sure the product's very hard to Jeff and take VC money.
Now you can work on the language as you please, and they can't Jeff you since nobody else can build something similar (not in a reasonable amount of time anyway)
P.S: taking VC money is optional. Scrimba is profitable so it wasn't necessary.
Scott Aaronson addresses this. Uncomputable does not mean unsolvable by humans. It just means there’s no axiomatic system powerful enough to enumerate all the busy beaver numbers. Rather we’ll need axiomatic system A for the first few, then B, then C, etc each more complicated than the last.
Ex: we already know BB(745) is independent of ZFC, any more powerful system should have its own independence limit, etc.
Since BB computing code is halting problem solving code, it goes back to my point about “one finite program to solve halting for countably infinitely many TMs and inputs” seeming intractable compression-wise. On the contrary, maybe countably many BB computing programs for each countably many BB number would be doable.
your GitHub comment about "Why CRDT didn't work out as well for collaborative editing in xi-editor" [1] was sent to me several times as an argument not to use CRDTs. I agree with you that CRDTs might be too complex for the way the xi-editor used it (although I loved the idea, and appreciate that you tried it).
But the title of the Hackernews post tells a very different story. So many people misunderstood WHY CRDTs didn't work out well for the xi-editor.
At the very least, my article shows that CRDTs are well suited for shared editing. You brought up some valid points why CRDTs are not well suited for having different editor components communicate with each other (language server, indentation, syntax highlighting, ..).
Although, I still think there is a lot of merit in using CRDTs as a data model for a code editor (or any kind of editor). Not for editor components concurrently modifying the editor model, but just as an collaboration-aware model.
• Marijn considered a CRDT as a data model for CodeMirror 6 because positions in collaborative software can be better expressed [2]. A position in the Yjs model is simply a reference to the unique id of the character.
• Even without collaboration, Yjs servers as a highly efficient selective Undo/Redo manager. Each item on the Undo/Redo stack just consumes a couple of bytes. Furthermore, most existing implementations don't support selective Undo/Redo. This is free when using Yjs as a data model.
• Some components can work in the background and annotate the CRDT model (not manipulate it). For example, a code analyzer that runs on a remote computer could annotate a function and notify the user about potential problems. The position of the annotation will still be valid if users modify the model concurrently in a distributed environment.
I am continually surprised at how relevant and pervasive one of Kurt Vonnegut’s major insights is: “we are what we pretend to be, so we must be very careful about what we pretend to be”