Human brains and experiences seem to be constrained by the laws of quantum physics, which can be simulated to arbitrary fidelity on a computer. Nit sure where Godel’s incompleteness theory would even come in here…
how are we going to deduce/measure/know the initialization and rules for consciousness? do you see any systems as not encodable/simulatable by quantum?
I think you are asking whether consciousness might be a fundamentally different “thing” from physics and thus hard or impossible to simulate.
I think there is abundant evidence that the answer is ‘no’. The main reason is that consciousness doesn’t give you new physics, it follows the same rules and restrictions. It seems to be “part of” the standard natural universe, not something distinct.
Panpsychism is actually quite reasonable in part because it changes the questions you ask. Instead of “does it think” you need to ask “in what ways can it think, and in what ways is it constrained? What types of ‘experience/qualia’ can this system have, and what can’t it have?”
When you think in these terms, it becomes clear that LLMs can’t have certain types of experiences (eg see in color) but could have others.
A “weak” panpsychism approach would just stop at ruling out experience or qualia based on physical limitations. Yet I prefer the “strong” pansychist theory that whatever is not forbidden is required, which begins to get really interesting (would imply that for example an LLM actually experiences the interaction you have with it, in some way).
I don't know that I agree that computation is a variety of thinking. It's certainly influenced by thinking, but I think of thinking as more the thing you do before, after, and in-between the computation, not the actual computation itself.
This comment will probably get buried because I’m late to the party, but I’d like to point out that while they identify a real problem, the author’s approach—using code or ASTs to validate LLM output—does not solve it.
Yes, the approach can certainly detect (some) LLM errors, but it does not provide a feasible method to generate responses that don’t have the errors. You can see at the end that the proposed solution is to automatically update the prompt with a new rule, which is precisely the kind of “vibe check” that LLMs frequently ignore. If they didn’t, you could just write a prompt that says “don’t make any mistakes” and be done with it.
You can certainly use this approach to do some RL on LLM code output, but it’s not going to guarantee correctness. The core problem is that LLMs do next-token prediction and it’s extremely challenging to enforce complex rules like “generate valid code” a priori.
As a closing comment, it seems like I’m seeing a lot of technical half-baked stuff related to LLMs these days because LLMs are good at supporting people when they have half baked ideas, and are reluctant to openly point out the obvious flaws.
A tidbit that I’m very interested in is Apple’s reports that it has not removed any apps at the request of the US government. It seems that in this case, they did; why don’t they report it transparently?
Let me second this: a baseline analysis should include papers that were published or reviewed at least 3-4 years ago.
When I was in grad school, I kept a fairly large .bib file that almost certainly had a mistake or two in it. I don’t think any of them ever made it to print, but it’s hard to be 100% sure.
For most journals, they actually partially check your citations as part of the final editing. The citation record is important for journals, and linking with DOIs is fairly common.
I don’t think that there are many things known to have as strong of an effect as HZ vaccines. The current evidence is that the vaccine eliminates like 20% of all cases, suggesting that HZ (aka chickenpox) is directly responsible for at least 20% of dementia cases, possibly much more.
The highlights are a good start. (I’m a doctor, just a nerd who likes to read papers.)
My comments in brackets.
- Herpes zoster vaccination reduced dementia diagnosis in our prior natural experiments. [Previous work. I’m familiar with the Wales experiment where they had a sharp age cutoff for getting the vaccine in their national health system. Comparing those just below and just beyond the cutoff allows for analysis similar to a randomized controlled trial (aka ‘natural experiment’). The results showed a ~20% decrease in dementia due to vaccine, so the results were already pretty strong.]
- Here, we find a lower occurrence of MCI and dementia deaths among dementia patients [MCI = ‘mild cognitive impairment’. This is a more refined result than prior work, harder to see in the data than a clear dementia diagnosis.]
- Herpes zoster vaccination appears to act along the entire clinical course of dementia. [This is not surprising given the earlier results, but the demonstration is harder, and it may lead to recommendations for earlier HZ vaccination, IIRC currently at 50 or 55 in the US.]
- This study’s approach avoids the common confounding concerns of observational data [Basically they are improving their methods and getting stronger results, classic good science.]
Months ago when this research was showing up (not on HN) there was a disclaimer that the benefit differed for Shingrix vs Zostavax (discontinued in the United States around 2017), and that Zostavax was shown to cause these benefits (Wales study for example).
It's interesting that the linked article references, in different terms, the distinction, obliquely. Zostavax is attentuated; Shingrix recombinant.
"Our findings suggest that live-attenuated HZ vaccination prevents or delays mild cognitive impairment and dementia and slows the disease course among those already living with dementia."
> Recent studies have shown convincingly that vaccines against shingles (Herpes zoster) reduce the risk of dementia. The shingles vaccine now in widespread use (Shingrix) has more of an effect than the previous one (Zostavax). A key difference between these vaccines is that Shingrix contains an ‘adjuvant’, an ingredient designed to enhance the vaccine’s effect. It is therefore possible that the adjuvant contributes to Shingrix’ greater effect than Zostavax on reducing dementia.
Sorry, I just realized the typo but I can no longer edit: I’m not a doctor, just a nerd. I blame my phone and my own crappy copy-editing. I hope this didn’t confuse too many of you.
reply