1,515 episodes
The Nonlinear Library The Nonlinear Fund
-
- Education
-
-
4.6 • 7 Ratings
-
The Nonlinear Library allows you to easily listen to top EA and rationalist content on your podcast player. We use text-to-speech software to create an automatically updating repository of audio content from the EA Forum, Alignment Forum, LessWrong, and other EA blogs. To find out more, please visit us at nonlinear.org
-
LW - Mental Health and the Alignment Problem: A Compilation of Resources (updated April 2023) by Chris Scammell
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Mental Health and the Alignment Problem: A Compilation of Resources (updated April 2023), published by Chris Scammell on May 10, 2023 on LessWrong.
This is a post about mental health and disposition in relation to the alignment problem. It compiles a number of resources that address how to maintain wellbeing and direction when confronted with existential risk.
Many people in this community have posted their emotional strategies for facing Doom after Eliezer Yudkowsky’s “Death With Dignity” generated so much conversation on the subject. This post intends to be more touchy-feely, dealing more directly with emotional landscapes than questions of timelines or probabilities of success.
The resources section would benefit from community additions. Please suggest any resources that you would like to see added to this post.
Please note that this document is not intended to replace professional medical or psychological help in any way. Many preexisting mental health conditions can be exacerbated by these conversations. If you are concerned that you may be experiencing a mental health crisis, please consult a professional.
Preface to the 2nd Edition
This post was released in April 2022 under the same title. This April 2023 update features new resources in every section, with a particular emphasis on the Alignment Positions and People Resources sections. Within each section, resources have been thematically categorized for easier access.
Following the large capabilities leaps in the past year, these resources seem more important than ever. If you have suggestions for improving this post, for making it more accessible, or for new resources to add, please leave a comment or reach out to either Chris Scammell or DivineMango.
We hope you are all well and that you find this update helpful.
Introduction
There is no right way to emotionally respond to the reality of approaching superintelligent AI, our collective responsibility to align it with our values, or the fact that we might not succeed. As transformative AI approaches, we must ensure that we have the tools and resources to be okay. Here, the valence of “be okay” is your decision. This question could be rephrased “how can I thrive despite the alignment problem,” “how can I cope with the alignment problem,” “how can I overcome my fear of the alignment problem,” etc. Everyone needs to find their own question and their own answer.
At its foundation “being okay” is the decision to continue to live facing reality and the alignment problem directly, with internal stability and rationality intact. And as a high ideal, we’re going for some degree of inviolability, of unconditional wellbeing, the kind of wellbeing that holds onto “okayness” even if the probability of solving alignment drops to 0. It can be difficult to stand in some place of positive mental health and stability while facing the alignment problem; but it is a gift if we can do that for ourselves, and a gift if we can share it with others.
Fortunately, we don’t have to do this alone. Many community members have found ways to make sense of themselves, their work, and their lives in relation to the alignment problem, and they have kindly made their reflections and advice public.
Resources
Several resources on this subject (along with summaries) are cataloged below. While there are a number of general mental health resources on LW, the EA Forum, and elsewhere that form a great baseline, this post aims to be more specific by focusing on mental health with respect to the alignment problem. Here, we feature a wide variety of ideas and practices in the hope that you may filter through them to create and discover the approach that works for you. Human brains come in many shapes – we all have different internal subagent dynamics, motivational sy -
LW - Tools for finding information on the internet by RomanHauksson
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Tools for finding information on the internet, published by RomanHauksson on May 9, 2023 on LessWrong.
Isn't the internet such a magically useful tool? Thirty years ago, if you wanted to know how many plays Shakespeare wrote, you would have to physically walk to your local library and find a relevant book. Now, you can find the answer in less than ten seconds, at any time, wherever you are.
However, the internet is not a truthful, superintelligent oracle. Rather, it's a dangerous jungle of knowledge you must learn to navigate if you wish to find the truth. Good information is censored, hidden behind paywalls or within piles of spam, and difficult to differentiate from untrustworthy information. This post won't be a complete guide on how to navigate the world wide web of knowledge, but it will give you some tools I've discovered over the years that you can throw in your digital rucksack to aid your journey.
Search engines
The great internet sage Gwern Branwen wrote an advanced guide on finding references, papers, and books online.
The search engines Brave Search and Kagi have the features "Goggles" and "Lenses" respectively, which are presets that filter or re-rank entire categories of websites in your results.
SearXNG is a highly customizable internet metasearch engine.
Perplexity uses natural language processing to answer your query with a paragraph (with sources) and allows you to ask followup questions.
Metaphor allows you to find websites by writing creative and long-form prompts, also using NLP.
Elicit is a research assistant that helps you find relevant research papers, also using NLP.
Bypassing restrictions
Sometimes you know exactly where to find a piece of information, but it's locked behind a paywall or deleted from the internet.
Unddit displays deleted comments and posts on Reddit.
Internet Archive is a non-profit library of free books, movies, websites, et cetera. It's famous for the Wayback Machine, which displays past archived snapshots of a given URL.
Bypass Paywalls is a browser extension to help bypass paywalls on selected sites.
The subreddit r/piracy has a wiki with loads of resources on obtaining copyrighted material for free.
Anna's Archive is a shadow library metasearch engine that aggregates results from websites that host copyrighted books, academic papers, magazines, et cetera.
Trustworthy sources
It is particularly frustrating to find trustworthy knowledge about certain topics because of misaligned incentives: researching which product to buy or which supplements actually work is hard because everyone's trying to sell you something.
Consumer Reports independently tests consumer products and gives in-depth recommendations. It does not rely on affiliate commissions.
Examine is a database of research about nutrition and supplements that has no industry ties, sponsorships, or ads.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org. -
AF - A small update to the Sparse Coding interim research report by Lee Sharkey
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A small update to the Sparse Coding interim research report, published by Lee Sharkey on April 30, 2023 on The AI Alignment Forum.
This is a linkpost to a set of slides containing an update to a project that was the subject of a previous post ([Interim research report] Taking features out of superposition with sparse autoencoders).
The update is very small and scrappy. We haven't had much time to devote to this project since posting the Interim Research Report.
TL;DR for the slides:
We trained a minuscule language model (LM) (residual size = 16; 6 layers) and then trained sparse autoencoders on MLP activations (dimension = 64) from the third layer of that model.
We found that, when we compared the 'ground truth feature recovery' plots, the plots for the toy data and LM data were much more similar than in the Interim Research Report.
Very, very tentatively, we found the layer had somewhere between 512-1024 features. By labelling a subset of these features, we estimate there are roughly 600 easily labellable (monosemantic) features. For instance, we found a feature that activates for a period immediately after 'Mr', 'Mrs', or 'Dr'.
We suspect that the reason the toy data and LM data plots had previously looked different was due to severely undertrained sparse autoencoders.
We're hopeful that with more time to devote to this project we can confirm the results and apply the method to larger LMs. If it works, it would give us the ability to tell mechanistic stories about what goes on inside large LMs in terms of monosemantic features.
Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org. -
LW - Connectomics seems great from an AI x-risk perspective by Steven Byrnes
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Connectomics seems great from an AI x-risk perspective, published by Steven Byrnes on April 30, 2023 on LessWrong.
Context
Numerous people are in a position to accelerate certain areas within science or technology, whether by directing funds and resources, or by working in the area directly. But which areas are best to accelerate?
One possible consideration (among others) is the question: “Is accelerating this technology going to increase the chance that our future transition to superhuman artificial general intelligence (AGI) goes well? Or decrease it? Or make no difference?” My goal here is to try to answer that question for connectomics (the science & technology of mapping how neurons connect to each other in a brain).
This blog post is an attempt to contribute to Differential Technology Development (DTD) (part of the broader field of Differential Intellectual Progress). Successful DTD involves trying to predict complicated and deeply uncertain future trajectories and scenarios. I think the best we can hope for is to do better than chance. But I’m optimistic that we can at least exceed that low bar.
My qualifications: I’m probably as qualified as anyone to discuss AI x-risk and how it relates to neuroscience. As for connectomics, I’m not too familiar with the techniques, but I’m quite familiar with how the results are used—in the past few years I have scrutinized probably hundreds of journal articles describing neural tracer measurements. (Think of neural tracer measurements as the traditional, “artisanal”, small-scale version of connectomics.) I find such articles extremely useful; I would happily trade away 20 fMRI papers for one neural tracer paper. This post is very much “my opinions” as opposed to consensus, and I’m happy for further discussion.
TL;DR
Improved connectomics technology seems like it would be very helpful for the project of reverse-engineering circuitry in the hypothalamus and brainstem that implement the “innate drives” upstream of human motivations and morality. And that’s a good thing! We may wind up in a situation where future researchers face the problem of designing “innate drives” for an AI; knowing how they work in humans would be helpful for various reasons.
Improved connectomics technology seems like it would NOT be very helpful for the project of reverse-engineering the learning algorithms implemented by various parts of the brain, particularly the neocortex. And that’s a good thing too! I think that this reverse-engineering effort would lead directly to knowledge of how to build superhuman AGI, whereas I would like us to collectively make much more progress on AGI safety & alignment first, and to learn exactly how to build AGI second.
Improved connectomics technology might open up a path to achieving Whole Brain Emulation (WBE) earlier than non-WBE AGI. And that’s a good thing too! Generally, a WBE-first future seems difficult to pull off, because (I claim) as soon as we understand the brain well enough for WBE, then we already understand the brain well enough to make non-WBE AGI, and someone will probably do that first. But if we could pull it off, it would potentially be very useful for a safe transition to AGI. I have previously been very skeptical that WBE is a possibility at all, but when I imagine a scenario where radically improved human connectomics technology is available in the near future, then it does actually seem like a possibility to have WBE come before non-WBE AGI, at least by a year or two, given enough effort and luck.
1. Background considerations
1.1 The race between reverse-engineering the cortex versus reverse-engineering the hypothalamus & brainstem
My theory is that parts of the brain (esp. cortex, thalamus, striatum, and cerebellum) are running large-scale learning algori -
EA - Introducing Stanford’s new Humane & Sustainable Food Lab by MMathur
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing Stanford’s new Humane & Sustainable Food Lab, published by MMathur on April 30, 2023 on The Effective Altruism Forum.
We are excited to announce the new Humane & Sustainable Food Lab at Stanford University’s School of Medicine (California, USA). Our mission is to end factory farming through cutting-edge scientific research that we are uniquely positioned to conduct. I am the principal investigator of the lab, an Assistant Professor at the Stanford School of Medicine with dual appointments in the Quantitative Sciences Unit and Department of Pediatrics. Because arguments for reducing factory farming as a cause area have been detailed elsewhere, here I focus on describing:
Our approach
Our research and publications to date
Our upcoming research priorities
Why we are funding-constrained
1. Our approach
1.1. Breadth, then depth
Empirical research on how to reduce factory farming is still nascent, with many low-hanging fruit and unexplored possibilities. As such, it is critical to explore broadly to see what general directions are most promising and in what real-world contexts (e.g., educational interventions that appeal to animal welfare [1, 2, 3], choice-architecture “nudges” that subtly shift food-service environments, etc.). We are conducting studies on a range of individual- and society-level interventions (see below), ultimately aiming to find and refine the most tractable, cost-effective, and scalable interventions. As we home in on candidate interventions, we expect our research to become more deeply focused on a smaller number of interventions.
1.2. Collaborating with food service to conduct and disseminate research in real-world contexts
We have a unique collaboration with the Director and Executive Chefs at the Stanford dining halls, allowing us to conduct controlled trials in real-world settings to assess interventions to reduce consumption of meat and animal products. Some of our interventions have been as simple and scalable as reducing the size of spoons used to serve these foods. Also, Stanford Residential & Dining Enterprises is a founding member of the Menus of Change University Research Collaborative (MCURC), a nationwide research consortium of 74 colleges and universities that conduct groundbreaking, collaborative studies on healthy and sustainable food choices in food service. MCURC provides evidence-based recommendations for promoting healthier and more sustainable food choices in food service operations, providing a natural route to dissemination.
Our established research model involves conducting initial pilot studies at Stanford's dining halls to assess interventions' real-world feasibility and obtain preliminary effect-size estimates, then conducting large-scale, multisite studies by partnering with collaborating members of MCURC. We also have ongoing collaborations with restaurants and plant-based food startups in which we are studying whether adding modern plant-based analogs (e.g., Impossible Burgers or JUST Egg) to a menu reduces sales of animal-based foods.
1.3. Building a new academic field
The large majority of empirical research on reducing factory farming has been conducted by nonprofits. In contrast, academics have engaged comparatively little with this cause area (but with notable, commendable exceptions). Academics have a chick’n-and-JUST Egg problem: without a robust academic field for farmed animal welfare, academics remain largely unaware of this cause area and lack the necessary mentorship and career incentives to pursue it; conversely, without individual labs pursuing this research, a robust academic field cannot emerge. Our lab is designed as a prototype, demonstrating that it is feasible – and indeed rather joyful! – for a lab to focus on an EA-aligned, neglected cause area, while also succeeding robust -
AF - Connectomics seems great from an AI x-risk perspective by Steve Byrnes
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Connectomics seems great from an AI x-risk perspective, published by Steve Byrnes on April 30, 2023 on The AI Alignment Forum.
Context
Numerous people are in a position to accelerate certain areas within science or technology, whether by directing funds and resources, or by working in the area directly. But which areas are best to accelerate?
One possible consideration (among others) is the question: “Is accelerating this technology going to increase the chance that our future transition to superhuman artificial general intelligence (AGI) goes well? Or decrease it? Or make no difference?” My goal here is to try to answer that question for connectomics (the science & technology of mapping how neurons connect to each other in a brain).
This blog post is an attempt to contribute to Differential Technology Development (DTD) (part of the broader field of Differential Intellectual Progress). Successful DTD involves trying to predict complicated and deeply uncertain future trajectories and scenarios. I think the best we can hope for is to do better than chance. But I’m optimistic that we can at least exceed that low bar.
My qualifications: I’m probably as qualified as anyone to discuss AI x-risk and how it relates to neuroscience. As for connectomics, I’m not too familiar with the techniques, but I’m quite familiar with how the results are used—in the past few years I have scrutinized probably hundreds of journal articles describing neural tracer measurements. (Think of neural tracer measurements as the traditional, “artisanal”, small-scale version of connectomics.) I find such articles extremely useful; I would happily trade away 20 fMRI papers for one neural tracer paper. This post is very much “my opinions” as opposed to consensus, and I’m happy for further discussion.
TL;DR
Improved connectomics technology seems like it would be very helpful for the project of reverse-engineering circuitry in the hypothalamus and brainstem that implement the “innate drives” upstream of human motivations and morality. And that’s a good thing! We may wind up in a situation where future researchers face the problem of designing “innate drives” for an AI; knowing how they work in humans would be helpful for various reasons.
Improved connectomics technology seems like it would NOT be very helpful for the project of reverse-engineering the learning algorithms implemented by various parts of the brain, particularly the neocortex. And that’s a good thing too! I think that this reverse-engineering effort would lead directly to knowledge of how to build superhuman AGI, whereas I would like us to collectively make much more progress on AGI safety & alignment first, and to learn exactly how to build AGI second.
Improved connectomics technology might open up a path to achieving Whole Brain Emulation (WBE) earlier than non-WBE AGI. And that’s a good thing too! Generally, a WBE-first future seems difficult to pull off, because (I claim) as soon as we understand the brain well enough for WBE, then we already understand the brain well enough to make non-WBE AGI, and someone will probably do that first. But if we could pull it off, it would potentially be very useful for a safe transition to AGI. I have previously been very skeptical that WBE is a possibility at all, but when I imagine a scenario where radically improved human connectomics technology is available in the near future, then it does actually seem like a possibility to have WBE come before non-WBE AGI, at least by a year or two, given enough effort and luck.
1. Background considerations
1.1 The race between reverse-engineering the cortex versus reverse-engineering the hypothalamus & brainstem
My theory is that parts of the brain (esp. cortex, thalamus, striatum, and cerebellum) are running large-scale lea