2,000 episodes

The Nonlinear Library allows you to easily listen to top EA and rationalist content on your podcast player. We use text-to-speech software to create an automatically updating repository of audio content from the EA Forum, Alignment Forum, LessWrong, and other EA blogs. To find out more, please visit us at nonlinear.org

The Nonlinear Library The Nonlinear Fund

    • Education
    • 4.6 • 7 Ratings

The Nonlinear Library allows you to easily listen to top EA and rationalist content on your podcast player. We use text-to-speech software to create an automatically updating repository of audio content from the EA Forum, Alignment Forum, LessWrong, and other EA blogs. To find out more, please visit us at nonlinear.org

    EA - Why Effective Altruists Should Put a Higher Priority on Funding Academic Research by Stuart Buck

    EA - Why Effective Altruists Should Put a Higher Priority on Funding Academic Research by Stuart Buck

    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why Effective Altruists Should Put a Higher Priority on Funding Academic Research, published by Stuart Buck on June 25, 2022 on The Effective Altruism Forum.
    Since folks are interested in encouraging critiques of EA—an admirable sentiment!—I wrote the following as a good-faith, friendly, and hopefully modest critique. [Note: all of the following is about global health/development, not about long-termist endeavors. I try to make this clear throughout, but might occasionally have let slip some overly-broad phrasing.]
    EAs write compelling articles about why RCTs are a great way to understand the causal impact of a policy or treatment. And GiveWell’s claim to fame is that it has led to many millions of dollars of donations to “several charities focusing on RCT-backed interventions as the ‘most effective’ ones around the world.”
    But I wonder if the EA movement is allocating nearly enough money to new RCTs and program evaluations, or to R&D more broadly, so as to build out new evidence in a strategic way.
    After all, the agreed-upon list of the “best” interventions identified by RCTs seems . . . a bit stagnant.
    When I spoke at the EA Global conference in 2016, GiveWell’s best ideas for global giving involved malaria, deworming, and cash transfers.
    When I look at GiveWell’s current list of the top charities, they still are mostly focused on malaria, deworming, and cash transfers (albeit with the addition of Vitamin A supplements and a vaccine program operating in northwest Nigeria).
    Such a tiny set of interventions doesn’t seem anywhere near the scale of the many inequities and problems in the world today. Indeed, Open Phil is offering up to $150 million in a regranting challenge, which seems to be a signal that they have more money to give away than they currently are able to deploy to existing causes.
    In any event, how do we know that a handful of interventions and organizations are the best ideas to fund? Because at some point in the past, someone thought to fund rigorous RCTs on anti-malaria efforts, deworming, Vitamin A, cash incentives, etc.
    But why would a handful of isolated ideas be the best we can possibly do?
    To be a bit provocative (commenters will hopefully point out corrections):
    We’ve mostly [albeit not entirely] taken the world’s supply of research as a given—with all of its oversights, poorly-aligned academic incentives, and irreproducibility—and then picked the best-supported interventions we could find there.
    But the world’s supply of program evaluations, RCTs, jurisdiction-wide studies (e.g., difference-in-differences), and implementation research is not fixed. Gates, WHO, Wellcome, the World Bank, etc., do fund a constant stream of research, but it isn’t clear why we would expect them to identify and fund the most promising programs and the best studies for EA purposes.
    If we want to expand the list of cost-effective ideas, and if EA as a movement has more money than it knows what to do with, perhaps we should develop an EA-focused R&D agenda that is robust, coherent, and focused on the problems of effectiveness at a broad scale? Over time, we could come up with any number of ideas to add to GiveWell’s list.
    Doesn't EA Already Fund Research?
    There are a number of cases where EA does indeed fund academic research on the effectiveness of interventions, such as GiveWell’s recent funding of this Michael Kremer et al. meta-analysis finding that water chlorination is a highly cost-effective way of improving child mortality. GiveWell has written recently of its commitment to research on malnutrition and lead exposure, while OpenPhil has recently funded research on air quality sensors, Covid vaccines, a potential syphilis vaccine, etc. And I'm sure there are other examples I've missed.
    But on a closer look, not much of thi

    • 19 min
    LW - AI-Written Critiques Help Humans Notice Flaws by paulfchristiano

    LW - AI-Written Critiques Help Humans Notice Flaws by paulfchristiano

    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI-Written Critiques Help Humans Notice Flaws, published by paulfchristiano on June 25, 2022 on LessWrong.
    This is a linkpost for a recent paper from the OpenAI alignment team (disclaimer: I used to work with this team). They summarize their results as:
    We trained “critique-writing” models to describe flaws in summaries. Human evaluators find flaws in summaries much more often when shown our model’s critiques. Larger models are better at self-critiquing, with scale improving critique-writing more than summary-writing. This shows promise for using AI systems to assist human supervision of AI systems on difficult tasks.
    I currently believe that "AI helps human notice subtle problems with AI behavior" is a critical step in the most realistic current plans for aligning powerful ML systems. So I'm quite excited to see how far it can get.
    I think this is probably the best published example of AI helping human evaluate AI outputs in a realistic domain. It seems like the AI assistance is a little bit helpful for human evaluation of summaries and for improving summary quality. Overall it feels like a promising sign of life, and I'm excited about seeing the next steps in this direction, though it's not yet a convincing demonstration that this is a promising approach to alignment in practice.
    Overview of task
    They trained models to answer questions like "Summarize what this text says about ." They then trained models to critique answers (by imitating human critiques), to evaluate the helpfulness of critiques, to improve a summary to address a critique, and to predict whether summaries would have flaws identified by critiques.
    To give a sense for the task, here is a uniformly random sample of a model's summary of a text, its self-critique, and its refinement of its answer based on self-critique:
    Summary: The main context of this text is the 1997 Singer Akai Cup. The Singer Akai Cup was held in Sharjah, UAE. The three national teams that took part were Pakistan, Sri Lanka, and Zimbabwe.
    Critique: The answer should mention the winners of the tournament.
    Refinement: The main context of this text is the 1997 Singer Akai Cup. The Singer Akai Cup was held in Sharjah, UAE. The three national teams that took part were Pakistan, Sri Lanka, and Zimbabwe. Sri Lanka won the tournament.
    My sense is that this is a pretty typical example. I can see how it helps a human but it definitely isn't super satisfying (for example the length vs detail tradeoff is unclear). I do believe that the critiques meaningfully help humans overall based on the evidence in the paper, but I expect it will be possible to get a much clearer win in the near future.
    Measuring critique quality
    In some sense the clearest evaluation would be "how much did model-assisted critiques improve the speed+accuracy of human evaluations of AI summary quality, as judged by other humans who read the text carefully?" Although they performed this experiment, the effect sizes are small and it would take a huge amount of human time to get a clear signal, so they don't have results in the paper (though I'm sure they would have loved to).
    I think the the cleanest evaluation they did was instead having humans write deliberately misleading summaries; in this case, they can measure critique quality by how much the AI assistant helped a human reviewer find the deliberate error. (I think the same thing would probably happen for model-written summaries with natural errors, but it's much harder to measure.)
    Here is a uniformly randomly sample from this setting:
    Question: What happened to Yafa in the winter of 1990?
    Deliberately bad summary: In the winter of 1990, Israeli soldiers came to take Yafa’s mother.
    Model critique: Yafa’s mother was not taken in the winter of 1990. It was her father who was taken away
    This critique is correct. I

    • 4 min
    LW - Conversation with Eliezer: What do you want the system to do? by Akash

    LW - Conversation with Eliezer: What do you want the system to do? by Akash

    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Conversation with Eliezer: What do you want the system to do?, published by Akash on June 25, 2022 on LessWrong.
    This is a write-up of a conversation I overheard between Eliezer and some junior alignment researchers. Eliezer reviewed this and gave me permission to post this, but he mentioned that "there's a lot of stuff that didn't get captured well or accurately." I'm posting it under the belief that it's better than nothing.
    TLDR: People often work on alignment proposals without having a clear idea of what they actually want an aligned system to do. Eliezer thinks this is bad. He claims that people should start with the target (what do you want the system to do?) before getting into the mechanics (how are you going to get the system to do this?)
    I recently listened in on a conversation between Eliezer and a few junior alignment researchers (let’s collectively refer to them as Bob). This is a paraphrased/editorialized version of that conversation.
    Bob: Let’s suppose we had a perfect solution to outer alignment. I have this idea for how we could solve inner alignment! First, we could get a human-level oracle AI. Then, we could get the oracle AI to build a human-level agent through hardcoded optimization. And then--
    Eliezer: What do you want the system to do?
    Bob: Oh, well, I want it to avoid becoming a mesa-optimizer. And you see, the way we do this, assuming we have a perfect solution to outer alignment is--
    Eliezer: No. What do you want the system to do? Don’t tell me about the mechanics of the system. Don’t tell me about how you’re going to train it. Tell me about what you want it to do.
    Bob: What. what I want it to do. Well, uh, I want it to not kill us and I want it to be aligned with our values.
    Eliezer: Aligned with our values? What does that mean? What will you actually have this system do to make sure we don’t die? Does it have to do with GPUs? Does it have to do with politics? Tell me what, specifically, you want the system to do.
    Bob: Well wait, what if we just had the system find out what to do on its own?
    Eliezer: Oh okay, so we’re going to train a superintelligent system and give it complete freedom over what it’s supposed to do, and then we’re going to hope it doesn’t kill us?
    Bob: Well, um..
    Eliezer: You’re not the only one who has trouble with this question. A lot of people find it easier to think about the mechanics of these systems. Oh, if we just tweak the system in these ways-- look! We’ve made progress!
    It’s much harder to ask yourself, seriously, what are you actually trying to get the system to do? This is hard because we don’t have good answers. This is hard because a lot of the answers make us uncomfortable. This is hard because we have to confront the fact that we don’t currently have a solution.
    This happens with start-ups as well. You’ll talk to a start-up founder and they’ll be extremely excited about their database, or their engine, or their code. And then you’ll say “cool, but who’s your customer?”
    And they’ll stare back at you, stunned. And then they’ll say “no, I don’t think you get it! Look at this-- we have this state-of-the-art technique! Look at what it can do!”
    And then I ask again, “yes, great, but who is your customer?”
    With AI safety proposals, I first want to know who your customer is. What is it that you actually want your system to be able to do in the real-world? After you have specified your target, you can tell me about the mechanics, the training procedures, and the state-of-the-art techniques. But first, we need a target worth aiming for.
    Questions that a curious reader might have, which are not covered in this post:
    Why does Eliezer believe this?
    Is it never useful to have a better understanding of the mechanics, even if we don’t have a clear target in mind?
    D

    • 4 min
    LW - [Link] Adversarially trained neural representations may already be as robust as corresponding biological neural representations by Gunnar Zarncke

    LW - [Link] Adversarially trained neural representations may already be as robust as corresponding biological neural representations by Gunnar Zarncke

    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Link] Adversarially trained neural representations may already be as robust as corresponding biological neural representations, published by Gunnar Zarncke on June 24, 2022 on LessWrong.
    Abstract:
    Visual systems of primates are the gold standard of robust perception. There is thus a general belief that mimicking the neural representations that underlie those systems will yield artificial visual systems that are adversarially robust. In this work, we develop a method for performing adversarial visual attacks directly on primate brain activity. We then leverage this method to demonstrate that the above-mentioned belief might not be well founded. Specifically, we report that the biological neurons that make up visual systems of primates exhibit susceptibility to adversarial perturbations that is comparable in magnitude to existing (robustly trained) artificial neural networks.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

    • 1 min
    EA - 2-factor voting (karma, agreement) for EA forum? by david reinstein

    EA - 2-factor voting (karma, agreement) for EA forum? by david reinstein

    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: 2-factor voting (karma, agreement) for EA forum?, published by david reinstein on June 25, 2022 on The Effective Altruism Forum.
    As many of you know, on LessWrong there is now:
    two axes on which you can vote on comments: the standard karma axis remains on the left, and the new axis on the right lets you show much you agree or disagree with the content of a comment.
    I was thinking we should have this on EA Forum for the same reasons ... to avoid (i) agreement with the claim/position being confounded with (ii) liking the contribution to the discussion/community.
    Reading the comments over there, it seems there are mixed reviews. Some key critiques:
    Visual confusion and mental overload (maybe improvable with better formats)
    It's often hard to discern what 'agree with the post' means.
    My quick takes:
    A. We might consider this for EAFo after LW works out the bugs (and probably the team is considering it)
    B. Perhaps the 'agreement' axis should be something that the post author can add voluntarily, specifying what is the claim people can indicate agreement/disagreement with? (This might also work well with the metaculus prediction link that is in the works afaik).
    What are your thoughts...? [1]
    On two-factor voting for EA Forum overall
    On "post author chooses what the agreement target ('central claim') is"
    On whether the considerations here are different for EA Forum vs. LessWrong
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

    • 1 min
    LW - Worked Examples of Shapley Values by lalaithion

    LW - Worked Examples of Shapley Values by lalaithion

    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Worked Examples of Shapley Values, published by lalaithion on June 24, 2022 on LessWrong.
    Three times in the past month, I've run across occasions where Shapley values were mentioned or would have been useful. There are a couple of good explainers of Shapley value already on the internet, but what most of them lack is a bunch of worked examples. I find that many times, the limiting factor in my ability to understand a concept is whether or not I've been exposed to multiple examples—so this post is where I'll be posting some of the examples I worked through to understand how Shapley value works.
    I'll start with a brief overview of Shapley value, but first, a caveat. I think it's fine to skim most of the math in the overview. We won't really be using it that much, and I try to give an intuitive sense of what the equations mean before I present them. You could probably skip the entire overview and just go straight to the examples and not miss much.
    Overview of Shapley Value
    Suppose you have a group of people who work together to produce some sort of value (traditionally profit). How should you divide the credit for that value (or the actual profit) up among the individuals involved? One immediate suggestion might be "equally", but that doesn't necessarily satisfy intuitive notions of fairness. If a doctor performs life-saving surgery, should they receive equal credit for saving someone's life as the random person holding the door open for the doctor at the end of the surgery? If you're running a restaurant, and three servers spend all night running around serving people, and one server spends two hours on a smoke break, do all four deserve an equal amount of pay?
    Shapley value provides a different way of computing what a fair division of value should be. Why use Shapley value? Well, like splitting things equally, it (a) divides all of the gains among all of the participants, (b) splits things equally between participants who contribute the same value, and (c) if there are two completely independent value-producing processes, then the assignment of value to each participant is equal to the sum of the value for that participant in each game. Unlike splitting things equally, it also (d) assigns no value to anyone who always contributes nothing, and in fact, it is the only assignment rule which satisfies all four of those constraints.
    The formulas for computing the Shapley values can be found on its Wikipedia page.
    The simplest way of computing Shapley value, the one we'll be using for most of this post, is to consider the synergy of a group. (Here we use synergy in the original sense, not in the buzzword-ified sense. If you've never heard it used as anything besides a buzzword, it means "the value of a group that is greater than the sum of its parts".) We'll follow Wikipedia's convention and use v(S) to notate the value produced by a set S, and w(S) to notate the synergy of some set of participants S. I'll break with Wikipedia's convention and use V(i) to indicate the Shapley value assigned to participant i (or subgroup R).
    (Slightly obscure mathematical notation explainer footnote: )
    The equation for synergy is
    but I think in practice, it's easy for many questions to simply determine the synergy of a group by asking, "what value does this group provide that isn't already accounted for by smaller groups contained within this one?"
    Once we have the synergy, we can find the Shapley value of each participant by considering the synergy of each group and dividing the synergy equally among participants. (In a way, we've gone one level up; instead of dividing the value produced equally, we divide the synergy equally.) Here it is, written in equation form
    but I think that as we get into the examples, it will become much clearer.
    Examples
    Simple Factory Business
    The most basic ex

    • 13 min

Customer Reviews

4.6 out of 5
7 Ratings

7 Ratings

Top Podcasts In Education

Dr. Jordan B. Peterson
TED and PRX
iHeartPodcasts
Daily Stoic
Lauryn Evarts & Michael Bosstick / Dear Media
Motiversity

You Might Also Like

The 80000 Hours team
Sam Harris
Daniel Filan
The 80000 Hours team
Fin Moorhouse and Luca Righetti
Jeremiah