The Nonlinear Library: LessWrong The Nonlinear Fund

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing Apollo Research, published by Marius Hobbhahn on May 30, 2023 on LessWrong.
TL;DR
We are a new AI evals research organization called Apollo Research based in London.
We think that strategic AI deception – where a model outwardly seems aligned but is in fact misaligned – is a crucial step in many major catastrophic AI risk scenarios and that detecting deception in real-world models is the most important and tractable step to addressing this problem.
Our agenda is split into interpretability and behavioral evals:
On the interpretability side, we are currently working on two main research bets toward characterizing neural network cognition. We are also interested in benchmarking interpretability, e.g. testing whether given interpretability tools can meet specific requirements or solve specific challenges.
On the behavioral evals side, we plan to build up a full-stack suite of evaluations for deception and other misaligned behavior consisting of behavioral, finetuning, and interpretability-based methods.
As an evals research org, we intend to use our research insights and tools directly on frontier models by serving as an external auditor of AGI labs, thus reducing the chance that deceptively misaligned AIs are developed and deployed.
We also intend to engage with AI governance efforts, e.g. by working with policymakers and providing technical expertise to aid the drafting of auditing regulations.
We have starter funding but estimate a $1.4M funding gap in our first year. We estimate that the maximal amount we could effectively use is $4-6M in addition to current funding levels (reach out if you are interested in donating). We are currently fiscally sponsored by Rethink Priorities.
Our starting team consists of 8 researchers and engineers with strong backgrounds in technical alignment research.
We are interested in collaborating with both technical and governance researchers. Feel free to reach out at info@apolloresearch.ai.
We intend to hire once our funding gap is closed. If you’d like to stay informed about opportunities, you can fill out our expression of interest form.
Research Agenda
We believe that AI deception – where a model outwardly seems aligned but is in fact misaligned and conceals this fact from human oversight – is a crucial component of many catastrophic risk scenarios from AI (see here for more). We also think that detecting/measuring deception is causally upstream of many potential solutions. For example, having good detection tools enables higher quality and safer feedback loops for empirical alignment approaches, enables us to point to concrete failure modes for lawmakers and the wider public, and provides evidence to AGI labs whether the models they are developing or deploying are deceptively misaligned.
Ultimately, we aim to develop a holistic and far-ranging suite of deception evals that includes behavioral tests, fine-tuning, and interpretability-based approaches. Unfortunately, we think that interpretability is not yet at the stage where it can be used effectively on state-of-the-art models. Therefore, we have split the agenda into an interpretability research arm and a behavioral evals arm. We aim to eventually combine interpretability and behavioral evals into a comprehensive model evaluation suite.
On the interpretability side, we are currently working on a new unsupervised approach and continuing work on an existing approach to attack the problem of superposition. Early experiments have shown promising results, but it is too early to tell if the techniques work robustly or are scalable to larger models. Our main priority, for now, is to scale up the experiments and ‘fail fast’ so we can either double down or cut our losses. Furthermore, we are interested in benchmarking interpretability tech

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reply to a fertility doctor concerning polygenic embryo screening, published by GeneSmith on May 29, 2023 on LessWrong.
New LessWrong user grll_nrg, a fertility doctor, left a comment on my post about how to have polygenically screened children that brought up many of the common objections raised to polygenic embryo screening. I've heard these concerns brought up many times at conferences and in talks by professionals in the fertility industry. I thought other people might be interested in the discussion, so I decide to make a stand-alone post.
Here's grll_nrg's original comment:
Great post. Thank you. Fertility doctor here and a supporter of ART (assisted reproductive technologies) in general. A few thoughts (although you touched on a few of these below, worth emphasizing in my opinion):
PGT-P has not been validated yet, which may take decades to do, if ever.
The science in terms of GWAS isn't quite there yet IMHO - we don't know all the genes that are important for most traits and we may be inadvertently selecting against some desirable traits, for example.
Comparing clinic success rates using CDC data is imperfect because of different patient characteristics, patient selection, and reporting bias.
IVF pregnancies carry a significantly higher complication rate (hypertensive disorders, preterm birth, placental abnormalities, etc.) compared to spontaneous pregnancies - unclear if this is due to IVF or underlying infertility diagnosis.
The risk-benefit calculus of PGT-P is going to be different for a couple who already needs to do IVF anyway to have a baby (low additional risk/cost) compared to a couple doing IVF just so that they can do PGT-P (higher additional risk/cost).
IVF is notoriously inefficient at present. Depending on female partner age, each cycle may yield only very few embryos making the benefit and utility of PGT-P limited. It may not be practical, safe, or financially feasible to do multiple cycles of IVF to increase the cohort of transferable embryos.
IVF is expensive and often not covered by insurance which creates access disparities. PGT-P would exacerbate these disparities in access. This is not unique to IVF I realize.
Slippery-slope eugenics and discrimination are real ethical concerns that would need to be mitigated.
In-vitro gametogenesis (IVG) would be a game-changer. The utility of PGT-P would be greatly enhanced if suddenly you had thousands of eggs and hundreds of embryos to select from.
Thanks for the reply. I'm glad professionals from the ART field are reading this.
PGT-P has not been validated yet, which may take decades to do, if ever.
I think the ART field should probably reconsider what it considers acceptable evidence of "validation". In my mind, the question of "Has PGT-P been validated" should hinge on whether or not we can be confident that embryos selected via polygenic scores will display different trait values than those selected at random. For example, we want to know whether an embryo that has a low polygenic risk score for hypertension will indeed go on to develop hypertension at a lower rate.
All the people I've heard criticize PGT-P seem to think that the ONLY way to do this is with some kind of randomized control where some embryos with certain polygenic risk scores are implanted and others are not, and we then wait 20-70 years to see whether or not the polygenically screened group develops diseases at a different rate than the control group. I think this view is incorrect and is only taken because people are blindly applying traditional validation methodology to PGT-P without asking whether it is necessary.
Genes have a very special property that gives us a huge advantage over researchers trying to test whether or not a medication works; nature has already conducted a randomized control trial fo

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Sentience matters, published by So8res on May 29, 2023 on LessWrong.
Short version: Sentient lives matter; AIs can be people and people shouldn't be owned (and also the goal of alignment is not to browbeat AIs into doing stuff we like that they'd rather not do; it's to build them de-novo to care about valuable stuff).
Context: Writing up obvious points that I find myself repeating.
Stating the obvious:
All sentient lives matter.
Yes, including animals, insofar as they're sentient (which is possible in at least some cases).
Yes, including AIs, insofar as they're sentient (which is possible in at least some cases).
Yes, even including sufficiently-detailed models of sentient creatures (as I suspect could occur frequently inside future AIs). (People often forget this one.)
There's some ability-to-feel-things that humans surely have, and that cartoon drawings don't have, even if the cartoons make similar facial expressions.
Not knowing exactly what the thing is, nor exactly how to program it, doesn't undermine the fact that it matters.
If we make sentient AIs, we should consider them people in their own right, and shouldn't treat them as ownable slaves.
Old-school sci-fi was basically morally correct on this point, as far as I can tell.
Separately but relatedly:
The goal of alignment research is not to grow some sentient AIs, and then browbeat or constrain them into doing things we want them to do even as they'd rather be doing something else.
The point of alignment research (at least according to my ideals) is that when you make a mind de novo, then what it ultimately cares about is something of a free parameter, which we should set to "good stuff".
My strong guess is that AIs won't by default care about other sentient minds, and fun broadly construed, and flourishing civilizations, and love, and that it also won't care about any other stuff that's deeply-alien-and-weird-but-wonderful.
But we could build it to care about that stuff--not coerce it, not twist its arm, not constrain its actions, but just build another mind that cares about the grand project of filling the universe with lovely things, and that joins us in that good fight.
And we should.
(I consider questions of what sentience really is, or consciousness, or whether AIs can be conscious, to be off-topic for this post, whatever their merit; I hereby warn you that I might delete such comments here.)
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Wikipedia as an introduction to the alignment problem, published by SoerenMind on May 29, 2023 on LessWrong.
AI researchers and others are increasingly looking for an introduction to the alignment problem that is clearly written, credible, and supported by evidence and real examples. The Wikipedia article on AI Alignment has become such an introduction.
Link:
Aside from me, it has contributions from Mantas Mazeika, Gavin Leech, Richard Ngo, Thomas Woodside (CAIS), Sidney Hough (CAIS), other Wikipedia contributors, and copy editor Amber Ace. It also had extensive feedback from this community.
In the last month, it had ~20k unique readers and was cited by Yoshua Bengio.
We've tried hard to keep the article accessible for non-technical readers while also making sense to AI researchers.
I think Wikipedia is a useful format because it can include videos and illustrations (unlike papers) and it is more credible than blog posts. However, Wikipedia has strict rules and could be changed by anyone.
Note that we've announced this effort on the Wikipedia talk page and shared public drafts to let other editors give feedback and contribute.
I you edit the article, please keep in mind Wikipedia's rules, use reliable sources, and consider that we've worked hard to keep it concise because most Wikipedia readers spend 1 minute on the page. For the latter goal, it helps to focus on edits that reduce or don't increase length. To give feedback, feel free to post on the talk page or message me. Translations would likely be impactful.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Gemini will bring the next big timeline update, published by p.b. on May 29, 2023 on LessWrong.
There is a genre of LLM critique that criticises LLMs for being, well, LLMs.
Yann LeCun for example points to the inability of GPT-4 to visually imagine the rotation of interlocking gears as a fact that shows how far away AGI is, instead of a fact that shows how GPT-4 has not been trained on video data yet.
There are many models now that "understand" images or videos or even more modalities. However, they are not end-to-end trained on these multiple modalities. Instead they use an intermediary model like CLIP, that translates into the language domain. This is a rather big limitation, because CLIP can only represent concepts in images that are commonly described in image captions.
Why do I consider this a big limitation? Currently it looks like intelligence emerges from learning to solve a huge number of tiny problems. Language seems to contain a lot of useful tiny problems. Additionally it is the interface to our kind of intelligence, which allows us to assess and use the intelligence extracted from huge amounts of text.
This means that adding a modality with a CLIP-like embedding and than doing some fine-tuning does not add any intelligence to the system. It only adds eyes or ears or gears.
Training end-to-end on multi-modal data should allow the model to extract new problem solving circuits from the new modalities. The resulting model would not just have eyes, but visual understanding.
Deepmind did a mildly convincing proof-of-concept with Gato last year, a small transformer trained on text, images, computer games and robotics. Now it seems they will try to scale Gato to Gemini, leapfrogging GPT-4 in the process.
GPT-4 itself has image processing capabilities that are not yet available to the general public. But whether these are an add-on or result of integrated image modelling we don't know yet.
To me it seems very likely, that a world where the current AI boom fizzles is a world where multi-modality does not bring much benefits or we cannot figure out how to do it right or possibly the compute requirements of doing it right is still prohibitive.
I think Gemini will give us a good chunk of information about whether that is the world we are living in.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Kelly betting vs expectation maximization, published by MorgneticField on May 28, 2023 on LessWrong.
People talk about Kelly betting and expectation maximization as though they're alternate strategies for the same problem. Actually, they're each the best option to pick for different classes of problems. Understanding when to use Kelly betting and when to use expectation maximization is critical.
Most of the ideas for this came from Ole Peters ergodicity economics writings. Any mistakes are my own.
The parable of the casino
Alice and Bob visit a casino together. They each have $100, and they decide it'll be fun to split up, play the first game they each find, and then see who has the most money. They'll then keep doing this until their time in the casino is up in a couple days.
Alice heads left and finds a game that looks good. It's double or nothing, and there's a 60% chance of winning. That sounds good to Alice. Players buy as many tickets as they want. Each ticket resolves independently from the others at the stated odds, but all resolve at the same time. The tickets are $1 each. How many should Alice buy?
Bob heads right and finds a different game. It's a similar double or nothing game with a 60% chance of winning. He has to buy a ticket to play, but in Bob's game he's only allowed to buy one ticket. He can pay however much he wants for it, then the double or nothing is against the amount he paid for his ticket. How much should he pay for a ticket?
Alice's game is optimized by an ensemble average
Let's estimate the amount of money Alice will win, as a function how many tickets she buys. We don't know how each ticket resolves, but we can say that approximately 60% of the tickets will be winners and 40% will be losers (though we don't know which tickets will be which). This is just calculating the expected value of the bet.
If she buys x tickets, she'll make 0.6∗x∗2+0.4∗x∗0 dollars. This is a linear function that monotonically increases with x, so Alice should buy as many as she can.
Since she has $100, she can buy 100 tickets. That means she will probably come away with $120. There will be some variance here. If tickets were cheaper (say only a penny each), then she could lower her variance buy buying more tickets at the lower price.
Bob's game is optimized by a time-average
Unlike Alice's game with a result for each ticket, there's only one result to Bob's game. He either doubles his ticket price or gets nothing back.
One way people tend to approach this is to apply an ensemble average anyway via expectation maximization. If you do this, you end up with basically the same argument that Alice had and try to bet all of your money. There are two problems with this.
One problem is that Alice and Bob are going to repeat their games as long as they can. Each time they do, they'll have a different amount of money available to bet (since they won or lost on the last round). They want to know who will have the most at the end of it.
The repeated nature of these games mean that they aren't ergodic. As soon as someone goes bust, they then can't play any more games. If Bob bets the same way Alice does, and goes all in, then he can only get $0 or double out. After one round, he's 60% likely to be solvent. After 10 rounds, he's only 0.610 likely to have any money at all. That's about half a percent, and Bob is likely to spend the last few days of their trip chilling in the bar instead of gaming.
The second problem with expected value maximization here is that expected value is a terrible statistic for this problem. In Alice's game, her outcomes converge to the expected value. In Bob's game, his outcomes if he expectation maximizes are basically as far from the expected value as they can be.
This is why Bob should treat his game like a time-aver