DeepSeek R1 just uses crappy PPO ("GRPO" is just using the sharpe ratio as a heuristic approximation to a value function) on top of distilled existing models, with tons of pipelining optimizations manually engineered in. I don't see this making leading edge research any less expensive, you won't get a "smarter" model - just a model that has a higher probability of giving an answer it could already give. If you want to try and do something interesting with the architecture the pipelining optimizations now slow down your iteration capability heavily.
The RL techniques present will only work in domains where you can guarantee an answer is right (multiple choice questions, math, etc.). It doesn't really present any convincing leap forward in terms of advancing the capability of LLMs, just a strategy for compute efficient distillation of what we know already works. The fact this shitty PPO proxy works at all is a testament to the fact that DeepSeek is bootstrapping its capability heavily off of the output of existing larger models which are much more expensive to train. What DeepSeek R1 proves is you can distill a ChatGPT et al. into a smaller model and hack certain benchmarks with RL.
If you could just do RL to predict the best next word in general this would have been done already - but the signal to noise ratio on exploration would be so bad you'd never get anything besides infinite monkeys at a typewriter. It's not a novel/complicated idea to anyone familiar with RL to try and improve probability of things you like, and whoever decided to do RLHF on an LLM surely thought of (and did) regular RL first - and found it didn't work very well with whatever pretrained model and rewards they had. it was like two weeks ago people were going crazy about O3 doing arc-agi by running the exact same kind of traces R1 is doing in "GRPO" at test time rather than train time. Doing this also isn't novel and also only helps on shitty toy problems where you can get a number to tell you good vs bad.
There is no mechanism to compute rewards for general purpose language tasks - and in fact I think people will come to see the gains in math/coding benchmark problems come at a real cost to other capabilities of models which are harder to quantify and impossible to generically assign rewards to at internet scale.
To explore the frontier of capability you will still need a massive amount of compute, in fact even more to do RL than you would need to do standard next token prediction - even if the LLM might have fewer paramters. You also can't afford to do all the optimizations as you try many different complex architectures.
Using python and constraining yourself to only use a basic subset of the standard library modules so you can run the script in pretty much any environment is almost always a better choice than trying to write even one loop, if statement, or argument parser in a bash script.
bash script is "okay" I guess if your "script" is just a series of commands with no control flow.
Hard disagree. I've written plenty in both. They both have their strengths, but bash is just more efficient if you're working with the filesystem. The UNIX philosophy of "do one thing and do it well" shines here. Python is more powerful but it's a double-edged sword. If I want to read a file containing API endpoints, send a request to them for some JSON, and do some parsing, I don't want to need or want to deal with importing modules, opening file objects, using dictionaries, methods, functions, etc.
Why do that when I can literally just
```
N=0
while read -r URL; do
curl "$URL" | jq '.data[].someProp' | grep -v "filter" > "$N.data"
N="$((N+1))"
done < ./links.txt
```
The other thing is bash makes it exceptionally easier to integrate across different system tools. Need to grab something from a remote with `rsync`, edit some exif, upload it to a CDN? That's 3 commands in a row, versus god knows what libraries, objects, and methods you need to deal with in Python.
Libraries are nice, until you have to write the glue code between the modules and functions. But sometimes you already have the features you want as programs and you just need to do some basic manipulation with their arguments and outputs. And the string model can work well in that case.
> Why do that when I can literally just ``` N=0 while read -r URL; do curl "$URL" | jq '.data[].someProp' | grep -v "filter" > "$N.data" N="$((N+1))" done < ./links.txt ```
? That code's meaning is extremely clear: it reads from a list of URLs that return JSON lists of JSON objects. For each URL, it pulls out some property and checks whether each line does not contain the string 'filter'. Those lines which clear the 'filter'-filter are written to a file whose name is the line of the original input file which contained the URL pinged suffixed by the extension '.data'.
It's very easy to read and modify if you just write it out longwise, which is what I'd always do in some actual script. (I also like to put reading data at the beginning of the pipeline, so I'd use a useless use of cat here.) To illustrate:
It's a very simple pipeline with a single loop. It's not very different from pipelines you might write to transform some data using a thread macro in Clojure, or method chaining against a collection or stream in OOP languages like Java or Scala or Ruby or whatever you like.
That's really not that hard to add above. A lot of folks act like it's impossible to handle errors etc. in bash, but it's pretty straightforward -- certainly no more difficult than in any other language. The hard part, like with all languages, is deciding how to handle errors cases. The rest is just code.
On mobile so no idea if this a) looks good or b) runs (especially considering the command substitutions, but you could also redirect to temp files instead), but it's just something like this:
You forgot "-f" flag to curl, which means it won't fail if the server returns an error. Also "jq" returns success on empty input, pretty much always. Together, this might mean that networking errors will be completely ignored, and some of your data files will mistakenly become empty when server is busy. Good luck debugging that!
And yes, you can fix those pretty easily.. as long as you aware about them. But this is a great illustration why you have to be careful with bash: it's a 6-line program which already has 2 bugs which can cause data corruption. I fully agree with the other commenter: switch to python if you have any sort of complex code!
set -euo pipefail # stop execution if anything fails
cleanup () {
rm $tmp_file
popd
# anything else you want to think of here to get back to the state of the
# world before you ran the script
}
trap cleanup EXIT # no matter how the program exits, run that cleanup function.
But why bother? The moment you start doing all that, all the arguments of "oh look how much i can solve with my cool oneliner" goes away. The python version of that code is not only safe by default, it's also shorter and actually readable. Finally it is a lot more malleable, in case you need to process the data further in the middle of it.
for N, url in enumerate(Path("links").read_text().splitlines()):
resp = requests.get(url)
resp.raise_for_status()
prop = resp.json()["data"]["someProp"]
matches = (line for line in prop.splitlines() if "filter" not in line)
Path(f"{N}.data").write_text("\n".join(matches))
I'm sure there is something about the jq [] operator i am missing but whatever. An iteration there would be a contrived use case and the difficulty to understand it on a glance just proves I'm not interested. As someone else mentioned, both curl and jq requires some extra flags to not ignore errors, i can't say if that was intentional or not. It would either way be equally easy to solve.
There's real value in 'one-liners', though. A 'one-liner' represents a single pipeline; a single sequence of transformations on a stream of data— a single uninterrupted train of thought.
Chopping a script into a series of one-liners where every command in the pipeline but the first and/or last operate only on stdin and stdout, as far as is reasonable to do, is a great way to essentially write shell in an almost 'functional' style. It makes it easy to write scripts with minimal control flow and easy-to-locate effects.
Such scripts are ime generally much easier to read than most Python code, but I don't find Python especially easy to read.
Choice is a privilege you rarely have in a day to day job. Bash most of the time is already there and you have to live with it.
Also, I've been forced to work on a huge SCons based project and I guarantee python can make your life quite miserable when used for something it's not supposed to.
I'm not suggesting you build a a whole build system with python (which is basically bazel and it seems to be good enough for google.)
A lot of originally little automation/dev scripts bloat into more complicated things as edge cases are bolted on and bash scripts become abominations in these cases almost immediately.
Bash may be native, but alot of the programs you'll want to call may not be, or will differ between platforms in subtle ways. Although this won't be a concern for small/trivial scripts, but if we're talking about python as an alternative, my point probably still applies.
This. People using bash extensions and util-linux as if they're standard are my bane.
If you can't do it in POSIX (sh and utilities) and don't want to do an extensive investigation of portability for what you need, pony up for Python/Tcl/Perl (all in MacOS base, by the way).
True, though there’s a whole world of people who will yell at you for using Bash-isms rather than pure posix precisely because Bash (at least up to date versions) isn’t everywhere either.
I agree, but I've been having a hard time even with python recently. I had a small script (50-100 lines) to format a data drive on boot I refactored, 3 or 4 obvious undeclared variables and who knows how many more I didn't notice - mypy found 0 issues.
I was looking up statically typed alternatives and stumbled upon Ammonite and Scala-CLI for scala. I haven't used them much, but Ammonite bundles some basic libraries including command line parsing, http, and json, which are probably 99% of what I used in Python too? And Scala seems like an interesting language too with decent editor integration.
> I had a small script (50-100 lines) to format a data drive on boot I refactored, 3 or 4 obvious undeclared variables and who knows how many more I didn't notice - mypy found 0 issues.
To make mypy strict enough to compare your dev experience to a typed language, you have to declare all sorts of configurations, otherwise there are huge swaths of things it’ll allow compared to most typed languages.
I use below, and only when necessary use ignore comment pragmas when third party libraries are not typed.
As a person who’s been doing shell programming for 35 years and Python for 15 years, I completely disagree.
Bash scripts and Bash control flow has been and is used in highly critical scripts all over the place, including other planets.
We’ve been writing reliable, well-designed scripts for many decades. Some of my scripts are several hundred lines long and older than the system engineers currently occupying their positions.
Python is fine too. Use the right tool for the right job.
> Used to be a viable strategy until they started to drop modules from the standard library at every single release.
That’s a bit of a ridiculous statement, there’s a small number of very-long deprecated modules removed in 3.12, and some more recently deprecated modules in 3.13. And these things are old, largely or completely unmaintained, and usually complete obsolete.
I’d be surprised if anyone has a script that’s been adversely effected by this, and if they did it’s because they stopped maintaining it years ago (and also chose to both silence warnings and upgrade interpreter versions without reading the release notes).
Consider that the python foundation absolutely has the resources to put a developer to maintain them.
If they don't is because they don't want to.
> and usually complete obsolete
The amount of modules I've had to patch to keep working on 3.12 tells me they aren't as obscure and unused as you think they are.
> I’d be surprised if anyone has a script that’s been adversely effected by this
I'd say that over 99.9999% of python users do not download python from python.org. They use whatever is on their system. Which means that updating an LTS distribution will create mayhem. And that's considering that most modules have already been patched by the distribution maintainers to fix all the brokenness introduced by the new python version.
Also, a bash script from 30 years ago still works fine. A python script from 5 years ago doesn't start.
>Consider that the python foundation absolutely has the resources to put a developer to maintain them.
The resources to pay someone doesn’t mean that someone with interest and knowledge exists, especially for modules that were formally deprecated in Python 2 and which will never be reinstated. Lots of this stuff is just cruft, most of which has an obvious replacement, and if it doesn’t there’s a decent chance it’s not been used in years by anyone and if it ever had a reason to be in the standard lib, that reason is long gone.
> The amount of modules I've had to patch to keep working on 3.12 tells me they aren't as obscure and unused as you think they are.
If that number is at all significant, where are the issues pushing back against deprecation and removal? It’s not like there hasn’t been a formal process for all these modules. What got deleted in 3.12 was well documented and easily caught just by catching DeprecationWarning… anyone getting surprised by these modules going missing isn’t doing due diligence.
> I'd say that over 99.9999% of python users do not download python from python.org. They use whatever is on their system. Which means that updating an LTS distribution will create mayhem.
And I’ll pretty much guarantee you that 99.9999% of those users haven’t heard of, much less imported, any of the modules that have been removed.
> And that's considering that most modules have already been patched by the distribution maintainers to fix all the brokenness introduced by the new python version.
But again where are the issues and hands being waved that these issues are wide-spread enough to halt or reverse the deprecation process? If distro maintainers are simply patching everything for users who are constantly advised to leave their system Python alone and they’re not reporting the issues then those distro maintainers are harming everyone.
> Also, a bash script from 30 years ago still works fine. A python script from 5 years ago doesn't start.
I’ve written plenty of Python scripts that are still running on the interpreter and stdlib they were authored for, decades later. I’m also keenly aware that most of those scripts could not be written in Bash without reimplementing a significant portion of the Python standard lib and ecosystem, none of which was materially affected by the 3.11>3.12 removals.
For instance, some fairly commonly used Linux apps like ulauncher, autokey, fail2ban and xpra depend on pyinotify which hasnt been maintained for the last 6 years or so, which is why fedora, arch and nixos now includes patches to make it 3.12 compatible. I don’t find it very unlikely that your inhouse script could be using it too.
> The resources to pay someone doesn’t mean that someone with interest and knowledge exists
That's why you can pay people. So that despite their disinterest they will read the code and acquire the knowledge needed.
> especially for modules that were formally deprecated in Python 2
??? I'm talking about modules removed now, in 2024. They were not deprecated since python2. Please don't steer the conversation to different topics.
> Lots of this stuff is just cruft, most of which has an obvious replacement
distutils? Is it cruft? The thing used to install modules? Can you tell me which stdlib replacement it has?
> it’s not been used in years by anyone
Why did I have to patch over 10 modules?
> and if it ever had a reason to be in the standard lib, that reason is long gone
Is installing modules no longer a thing then?
> those distro maintainers are harming everyone.
Aaah yes, the evil distro maintainers that keep things working instead of breaking everything. They're the real culprits here… really?
> I’ve written plenty of Python scripts that are still running on the interpreter and stdlib they were authored for, decades later.
Decades later? That's at least 20 years. If that were true they'd be written in python2 and I can promise you they wouldn't work with python 3.12. So I'll consider this statement a lie.
Please try to be more honest when you interact with me the next time. This hasn't been pleasant.
I think the point GP was making was that you restrict yourself to only the bundled standard library, which covers most of the basics needed for scripting.
This is why you force yourself to use nearly zero dependencies. The standard library sys, os, subprocess, and argparse modules should be all you need to do all the fancy stuff you might try with bash, and have extremely high compatibility with any python3.x install.
How the hell does the Seek by iNaturalist app work so well and also be small/performant enough to the job completely offline on a phone? You should really try it out for IDing animals and plants if you haven't, it's like a real life pokedex. Have they released any information (e.g. a whitepaper?) about how the model works or how it was trained? The ability to classify things incrementally and phylogenetically makes it helpful to narrow down your own search even when it doesn't know the exact species. I've been surprised by it even IDing the insects that made specific galls on random leaves or plants.
I reverse engineered their stuff a bit. I downloaded their Android APK and found a tensorflow lite model inside. I found that it accepts 299x299px RGB input and spits out probabilities/scores for about 25,000 species. The phylogenetic ranking is performed separately (outside of the model) based on thresholds (if it isn't confident enough about any species, it seems to only provide genus, family, etc.) They just have a CSV file that defines the taxonomic ranks of each species.
I use it to automatically tag pictures that I take. I took up bird photography a few years ago and it's become a very serious hobby. I just run my Python script (which wraps their TF model) and it extracts JPG thumbnails from my RAW photos, automatically crops them based on EXIF data (regarding the focus point and the focus distance) and then feeds it into the model. This cropping was critical - I can't just throw the model a downsampled 45 megapixel image straight from the camera, usually the subject is too small in the frame. I store the results in a sqlite database. So now I can quickly pull up all photos of a given species, and even sort them by other EXIF values like focus distance. I pipe the results of arbitrary sqlite queries into my own custom RAW photo viewer and I can quickly browse the photos. (e.g. "Show me all Green Heron photos sorted by focus distance.") The species identification results aren't perfect, but they are very good. And I store the score in sql too, so I can know how confident the model was.
One cool thing was that it revealed that I had photographed a Blackpoll Warbler in 2020 when I was a new and budding birder. I didn't think I had ever seen one. But I saw it listed in the program results, and was able to confirm by revisiting the photo.
I don't know if they've changed anything recently. Judging by some of their code on GitHub, it looked like they were also working on considering location when determining species, but the model I found doesn't seem to do that.
I can't tell you anything about how the model was actually trained, but this information may still be useful in understanding how the app operates.
Of course, I haven't published any of this code because the model isn't my own work.
I don't use Seek, but the iNaturalist website filters computer vision matches using a "Seen Nearby" feature:
> The “Seen Nearby” label on the computer vision suggestions indicates that there is a Research Grade observation, or an observation that would be research grade if it wasn't captive, of that taxon that is:
> - within nine 1-degree grid cells in around the observation's coordinates and
> - observed around that time of year (in a three calendar month range, in any year).
For how the model was trained, it's fairly well documented on the blog, including different platforms used as well as changes in training techniques. Previously the model was updated twice per year, as it required several months to train. For the past year they've been operating on a transfer learning method, so the model is trained on the images then updated, roughly once each month, to reflect changes in taxa. The v2.0 model was trained on 60,000 taxa and 30 million photos. There are far more taxa on iNaturalist, however there is a threshold of ~100 observations before a new species is included in the model.
>It looked like they were also working on considering location when determining species, but the model I use doesn't do that.
I do this in fish for very different work and there's a good chance the model for your species does not exist yet. For fish we have 6,000 distribution models based on sightings (aquamaps.org) but there are at least 20,000 species. These models have levels of certainty from 'expert had a look and fixed it slightly manually' to 'automatically made based on just three sightings' to 'no model as we don't have great sightings data'. So it may be that the model uses location, just not for the species you have?
that is actually surprising. surely they use location at some point in the ID process. its possible they have a secondary location based model to do sorting/ranking after the initial detection?
Merlin's bird detection system is almost non-functional without location.
yeah that's true! You can't really do that, these models are just polygons, all we do is doublecheck the other methods' predictions' overlap with these polygons as a second step.
Sounds like a real-life Pokémon Snap. You should add a digital professor who gives you points based on how good your recent photos are. (Size of subject in photo, focus, in-frame, and if the animal is doing something interesting.)
I'd never heard of this app, but your description made me want to install it. When I googled it I was surprised at the app ratings:
Apple: 4.8
Google Play: 3.4
The most common issue mentioned by negative Play store reviews is the camera not focusing on the right thing, and needing to try many different angles before something is recognized correctly. This is probably nothing to do with the underlying model, which I guess is the same on both platforms.
Camera zoom is definitely annoying, there's no way to control how zoomed in it is.
And yes, it often takes as much as a minute to identify a species, because you have to keep adjusting zoom and angle and trying to catch every important feature.
That said, once you are used to it, it becomes less noticeable and just feels like part of the game.
I tried this app for a while, and there's definitely some rough edges. My partner's phone was much quicker in recognizing plants and flowers, so after a while I gave up and we just used her phone instead.
And then there's the issue of many plants being slightly mis-identified, many really common flowers being incorrectly identified. I don't really know how people get remotely close to wild animals to identify them, I had no luck with animal life with this and after a while I started mistrusting its results and cross referencing with google lens anyway.
I wish I had the same experience as you. The vast majority of time I point at tree leaves in South East Asia it tells me it’s Dicots and it stops there. Only rarely I get the actual full classification; the last time it happened was for a common almond tree.
It's very bad at trees for some reason. Also mushrooms, but I thought that might be intentional so they don't get blamed for someone eating something poisonous that was misidentified.
Trees are generally difficult to classify well with computer vision. It's hard for the models to establish context because at a scale where you can see the whole tree you tend to include lots of background. If you include a bark photo, it's often ambiguous if there's growth/stuff/weathering on top. Flowers tend to be good inputs.
The training imagery is also really inconsistent in inaturalist and again for plants it's hard to establish context. These are mostly smartphone pictures that non experts have captured in the app. While someone might have verified that a plant is a particular species, because there isn't any foreground/background segmentation, there are often confounding objects in the frame. On top of that you only get a 300px input to classify from. With animals I'd say it's much more common for photographers to isolate the subject. There's also massive class imbalance in inat, a large number of the observations are things like mallards (ie common species in city parks).
I guess the best solution would be to heavily incorporate geographic priors and expected species in the location (which I think is partly done already).
Flowers are crucial for human IDs as well. A lot of tropical tree leaves are very similar, so without context they're virtually impossible to visually distinguish.
Yeah this is a good point. I've done some work with local experts to do tree ID from drone images over rainforest and there were several species where they would need to see the underside of the leaves to get better than the family.
My experience has been great with mushrooms, just to add another datapoint. I mean, it's often about as good as you can get by eye without breaking out the lab equipment.
For trees try to photograph the flowers, the seeds, the bark, the leaves (both sides), the trunk growth habit (especially bottom portion), and the upper branches growth habit. Often when asking it to suggest a species switching between these will create progress.
Any observations can be submitted, but the observation has to be verified by a different observer. Most identifiers are folks with more experience identifying things locally, and the data quality is high. There's very little incentive to game the system and if something is misidentified other iNatters can also add identifications correcting mistakes which happens regularly - often various scientists/specialists tend to sweep observations in their taxa of note and correct issues. There's criteria for a "high quality" observation, including being verified. Only those observations that are high quality are used for training.
Fragment shaders can only go so far. Can you do in webgpu something like this?
outputTexture2d[inputTexture2d[inputPosition]]++
In other words, if you have a large texture with (x,y) coordinates of points, can you draw another texture that shows a density cloud of these points? In webgl2 it becomes a phd level problem.
Total graphics / shaders / GPU noob here. Does that mean you'll essentially get free visualisations (albeit non-sensical ones) as a byproduct of your computations?
If you wanted to do compute in a shader before WebGPU with WebGL instead, then I think the answer is kind of yes. It wasn't "for free" without any code but it was required to do. But now WebGPU supports compute shaders properly so you don't have to do compute in a shader that produces textures.
I think it depends, but given an arbitrary compute pipeline, you should be able to write the results (or intermediary results) to the screen with minimal effort.
No one ever considers the equally likely scenario of a technological plateau instead of singularity. Complexity/Entropy always forces things to level off. There's an plausible scenario that GPT# "replaces" all knowledge work, but cannot move anything forward. All humans become comfortable and the skills/knowledge/tools required to improve anything are lost to time as systems producing capable humans erode and we gain an overreliance on GPT# to solve every knowledge problem, but the knowledge problems that both we and GPT# care to solve plateau because were all synchronized to the same crystallized state of the world that the final GPT# model was trained on and "cares" about.
Maybe at some point maybe we only act as meat-robots which shovel coal into the machine, but a lack of redundancy in GPT# due to it's own human like blind spots means it shuts down. Humans can no longer get it running again because they can't query it properly to help fix the complicated problems. The ability to even do the tasks or design systems required to keep modern world robust to unknown future disasters or breakdowns does not and will not exist in any of the training data. If we get rid of all knowledge work, we can no longer bootstrap things back to a working state should everything go wrong.
Maybe the current instantiation of GPT#/SD etc. pollute the training data with plausible but subtly flawed software, text, images etc. halting improvement around here. Maybe the ability to evaluate if the model improved becomes more noise than signal because it gets too vague what improvement even means. RLHF will already have this problem, as 100 people will have a 100 slightly different biases about what constitutes the "best" next token.
No matter how hard it tries, I think we can say GPT will not solve NP-Hard problems magically, it will not somehow find global optima in non-linear optimizations, It will not break the laws of physics, It will not make inherently serial problems embarrassingly parallel. It will probably not be more energy efficient at attempting to solving these problems, maybe just faster at setting up systems to try solving them.
Another trap, as it becomes more human like in its reasoning and problem solving capabilities, it starts to gain the same blind spots as us too, and also gains stochastic behavior which may cause it to argue with other instances of itself. I'm not convinced an AGI innovates at an unfathomable rate or even supersedes humans in all contexts. I'm especially not convinced a world filled with AGIs that is indistinguishable from a very intelligent human or corporation or what have you through imitation does any better at anything than the 9 billion embodied AGI agents that currently populate the earth.
