[2024-04-17 Wed]
Research pedagogy thoughts
Inspired by reading this quote in An Introduction to Logical Decision Theory for Everyone Else - Arbital viewer
Q: Pause just a second. Before we get technical, is there anything you can tell me about why the Prisoner’s Dilemma matters? Like, I do remember hearing about this a while back, and I’ve seen it mentioned here and there like it’s a big deal, but I don’t know why it’s a big deal.
The ideal response here would be one that understands and accepts the pretending behavior and then tries to help the other person understand things
Mentorship and menteeship seems hard
Worrying about whether or not your worth in someone else’s eyes is dependent on how fast you learn something, or how much you already know, seems bad in the long term
Reminds me of the idea of building a ‘safe container’
Some people may have surprisingly little understanding of foundational pieces of knowledge, usually not because they are unskilled at learning, but because they historically had circumstances that get in the way of actually learning the things they need to know now
eg.
Theory of Computation for me (JL)
Linear algebra (when I worked with CL)
Insane bursts of imposter syndrome
Only reason I didn’t get consumed by it, afaict, is a constant social interaction in a community of people who were and are willing to explain things to me
Even if it seems to cost them more than you think
Even if I feel like overall they seem to be wasting their time
Even if I feel indebted somewhat
It still meant that I had evidence that I wasn’t fundamentally incapable of doing the research that I was doing or wanted to do (or learn the thing I wanted to learn)
While it may not be the ‘ideal’ way of doing things, I would in general optimize for pointing people to high quality content like textbooks or articles, instead of trying to explain things to them myself
While explaining something by myself to someone does have its benefits – for one, I understand the topic better; and it also helps me connect stuff I’m doing with more fundamental things; also spaced repetition
It does seem like most of the time when people have tried explaining things to me, they have usually repeatedly failed in doing so (maybe it is also my responsibility to help steer them?), and in general I’ve learned this aversion to people trying to explain things to me, and usually just ask people to recommend me pedagogical sources to wrap my head around things
I think that the illusion of transparency plays a huge role here as an issue in making it difficult for people to help someone else understand what you have in mind
Another big obstacle is status differentials. Especially in intellectual communities, or if you are being mentored by someone you feel like you cannot afford to look dumb in front of, there’s a massive anxiety thing that comes up in me that gets in the way of providing honest in-the-moment feedback as someone is trying to explain something to me
A third thing is kind of trauma, where if you have had bad experiences where people would be frustrated with you if you don’t grok some complex concept fast enough as they are trying to explain it to you.
This has usually resulted in me pretending to grok things as well as possible in the moment, and once the interaction ends, resolving to never ask this person a question again
Another bottleneck to in-the-moment explanation attempts is that there’s a qualitative difference between chunked concepts and non-chunked concepts, such that if you are trying to build and understanding from foundational enough concepts to get to the concept you are actually interested in (for the purposes of the conversation), it usually is better to chop the explanation attempt into parts, and distribute it across time
Potential solutions?
As a learner, or mentee
Safe strategies
It seems like most of my intellectual progress has involved asking questions and talking to people who I both have a social relationship with (so there’s a non-trivial amount of casual conversation) and also no strong status differential
eg. [redacted 1]
I think I will stick with this
I have historically had neutral to bad outcomes trying to learn one-on-one via social interactions with teachers or research leads or whatever, and my current strategy is to make that clear and ask them to recommend me pedagogical sources they may have in mind
Essentially ask for strategic help / directional verification, not object level advice
sidenote: there’s a line of thought that implies that one ‘should’ value the time of senior people / researchers / whatever a lot more than your own
See this comment by plex, for example
There’s something extra which is closely connected with this: The time and motivation of central nodes is extremely valuable. Burning the resources of people who are in positions of power and aligned with your values is much more costly than burning other people’s resources[1], and saving or giving them extra resources is proportionately more valuable.
I find this sort of thinking rather distasteful, mostly because it usually is downstream of utilitarianism and egalitarianism, both of which are lovely concepts in theory, and I’d believe an AI system would be able to do things I like with it, but humans in general have not evolved to a point where they can roll with such thinking and… well, do things I think work
A more sensible way of orienting to such people, in my opinion, is as equals / political rivals / colleagues. The epistemic corruption resulting from the sort of thinking that plex communicates, is not something I would recommend.
General strategies
Try multiple high quality sources. Most pedagogy is abyssmal.
Ask multiple people for their recommendations to learn something
Self: Create your own recommendation list for textbooks to learn things
[2024-04-23 Tue]
A regular conversation with someone in a mentorship situation is very valuable
LessWrong forums gives you some of this but you still fail to get a lot of valuable information you’d get from actual back-and-forth informal conversations
The social atmosphere and milieu of [redacted 1] and [redacted 2], for example, have been quite influential in my intellectual direction
The importance of informal social atmosphere seems quite important, and also a willingness for people in the community to answer questions
Even if you lurk and don’t engage or ask questions, you’ll probably come across a high density of information that is not spoken out loud or found in formal places (especially academic papers), and this stuff is extremely valuable
Also, this ‘social filter’ for your attention has extremely high value in dealing with the deluge of research out there, and the deluge of news out there
The signal/noise ratio matters a lot here, and the smarter your community / cohort, the better the signal/noise ratio
This, of course, has its own costs – the higher the signal/noise ratio of a community, the more valuable it is for an average person to join, the more the community is incentivized to prevent an ‘Eternal September’
The default strategy most such communities leverage is a sort of costly entrance mechanism
Joining OpenAI or Anthropic or a MATS cohort, for example, all require you to engage in a process of legibly signalling your value to allow you acess to their community
Lower threshold paths involve working at an academic research lab – conversations with your professor / supervisor and fellow PhD students can be invaluable
Note: this never worked for me, but I hear this is the standard way people learn the unspoken informal tacit knowledge related to interfacing with the research community for their field.
Another alternate strategy some communities engage in is a ‘dark forest strategy’, which essentially involves private forums or chats between members
Usually these forums are invite-only, so you need to know someone to access such places, which again turns this into an entrance mechanism, except this one is informal, and in general, your every action and social interaction has an effect on this, which is somewhat stifling
The better kinds of such forums are not invite-only, but mostly hidden from the broader internet (please don’t link them when discussing this – in general the people of such forums want to continue to stay unknown).
A third strategy you see people use on Twitter, for example, sometimes involves building a coded language or jargon, such that only insiders can understand what the person is saying.
I expect that the effectiveness of this one is significantly less, but this is mainly because I find Twitter rather daunting and low signal/noise. I expect that getting into Twitter is a process
It seems likely that having one-off conversations with individuals in general doesn’t scale.
A one-on-one mentorship can be valuable, but in general it seems limited to specific situations (for both the mentor and the mentee), and it isn’t a long term strategy
You could have a sequence of mentors
I have found myself feeling guilt about this, partially because I worried that implicitly my movement from one mentor to another implies that my previous mentor was not skilled enough to help me in the specific path I had in mind – but that seems confused from my current standpoint
People change, their goals change, and therefore the actions that are relevant for their current goals will change
No seriously, ask questions! Step one is to Google, if that doesn’t work, you could ask people at your university, people online, and of course, ideally in this community you inhabit
[2024-05-01 Wed]
This is turning into a rough cluster of learning / upskilling and social stuff related to that, and that’s fine
Reading The Alignment Community Is Culturally Broken - LessWrong 2.0 viewer
I agree with this quote
So, I think pushing too many EAs into AI safety will lead to those EAs suffering much more, which happened to me, so I don’t want that to happen and I don’t want the AI Alignment community to stop saying “You should stay if and only if you’re better than Y”.
On one hand, I think that it is counterproductive for both people involved for one person to persuade the other to do alignment research – shouldness in general gets in the way of functioning great (due to internal conflict destroying epistemics and wasting resources), and is particularly toxic when it involves high intellectual difficulty quests like alignment research
On the other hand, it is true that if you aren’t even inviting people, and enabling them to try their hand at this, and building a holding environment for them to try and fail (and not feel scared of losing valuable things if they fail), your nascent research field will continue to remain nascent
One could uncharitably think of them as tests as to whether someone has the ‘ability’ to do alignment research. In my perspective, it is less a test and more a way to enable good outcomes for all parties involved
Again, the research output of someone who isn’t in a state where they choose to trek the intellectual mountain by themselves, will be less than someone who chooses to do so of their own volition
Second, it could just be damaging to the other person: primarily due to shouldness and its consequences given the difficulty of the thing and the possibility of failure, and secondarily due to the opportunity cost of the other things they could have done instead
Also in general the alignment research community selects for certain kinds of people. These people usually are significantly more autistic / literalist than the demographic average, which correlates with neuroticism (due to social isolation).
On top of that, the fear of existential risk and the world ending is a sort of toxic motivation source that also motivates a massive amount of “shouldness” and internal conflict
One cannot really function like this for extended periods of time, really
This means that selecting for high agency autistic people who are strongly motivated to prevent x-risk leads to a pretty high stakes for the people involved, and it could be a rough time
The notion that you may not be good enough to do the thing you think is the most important, can be devastating, and people can and will flinch away from that
Similarly, communities that demand very highly of you, and cause you to feel like you’ve transgressed if you aren’t able to achieve those norms (EA norms specifically) exacerbate such situations
And people who get into alignment research because they started out as EAs are particularly prone to this
Overall, I don’t really have a solution for this phenomenon, aside from the things I wrote about how to orient to new people attempting to do things in this community
This is one example of just how ridiculous this can be
in my first CFAR workshop in 2015, I was told by a senior person in the community that I was probably not smart enough to work on AI Alignment (in particular, it was suggested that I was not good enough at math, as I avoided math in high school and never did any contest math). This was despite me having studied math and CS for ~2 hours a day for almost 7 months at that point, and was incredibly disheartening.Thankfully, other senior people in the community encouraged me to keep trying, which is why I stuck around at all. (In 2017, said senior person told me that they were wrong. Though, like, I haven’t actually done that much impressive alignment research yet, so we’ll see!)
Telling someone that they cannot ever do what they really really want to do is pretty much hurting them incredibly badly
An alternate strategy would be to try to say something like “It seems like you’ll have to take a while to get yourself up to speed on the skills I think you’ll want, and I think there may be better ways for you to achieve the things you want instead of going down this path.”
[2024-05-07 Tue]
Issa Rice on disillusionment with academia
The trend continues in which I become even more disillusioned with UW and academics. It feels incredible to me how everyone else can wake up, attend class each day, dutifully complete homework each week, etc. I suppose it’s easier for people who believe more in the human capital narrative of education, but even then, it seems so incredibly exhausting to do this each year. It’s difficult for me to explain this precisely, and sometimes I feel that one either intuitively understands the mundaneness, repetitiveness, and stressfulness of being at UW or else one is completely incapable of understanding it5. Indeed, I think UW is no better than being in high school6. I am reminded of days in high school when I would search online for reasons others had come up with against school, because it seemed so obvious and intuitive to me that I shouldn’t be at school, yet difficult to verbalize why exactly that was.
I empathize with this intensely – it has always seemed to me that academia has gotten in the way of my desire to learn the things I think are important (even if they are the things that I am ‘expected’ to learn as part of my undergrad or masters course!)
It has been my experience that doing academic courses has resulted in me disliking the subject and having some level of learned distaste or aversion to it
A part of it seems to come from the friction between taking the time to learn things at one’s pace, and having to ‘learn’ things to pass exams
This pressure is significantly lessened in most universities, and with enough organizational skill, in my opinion, when exams are only and clearly scheduled at the end of the semester
Another part of it is suboptimal pedagogy
In my undergrad I had to deal with rather abyssmal teaching and explanation, mainly due to the illusion of transparency hampering people who weren’t able to teach, and people (particularly TAs) who didn’t seem to actually care about the things they were teaching (and oriented to these things more as irrelevant things they were ‘supposed’ to do instead of things that matter
A third killer was just the intense time crunch or the lack of slack
Academic life, even as a student, can impose a wide variety of slack consumers – regular assignments, mandatory attendance (I had to deal with this during my undergrad), academic projects
Most of these things consume slack that would have been useful for you to be able to take your time to learn things at your pace, regardless of how they were taught
On top of that, my experience as an undergrad involved a lot of social interaction that was not really avoidable, and while I learned some facets of basic social skills, I expect that the opportunity cost was somewhat ridiculous
On top of that, you also have to manage your own life as an adult
If you have to deal with physical or mental health issues on top of that, you are running on less than the usual amount of capacity the university system expects of you as a full-time student
I don’t think you have the slack to learn things ‘properly’, as a full-time student. At least, I didn’t.
Issa even mentions this explicitly:
So I don’t experience the sort of pressure to be creative under a deadline that he mentions. However, I still experience the continual series of deadlines that slowly numbs me each quarter from everything interesting in the world. (3) With the noted caveats, I still relate strongly to the quote, because my intense frustration at UW is really just the result of apathy and lack of awareness of a system that hands me deadline after deadline after deadline, not something maleficent.
Issa also mentions explicitly that he noticed his interest in math fading due to his experience
I should also note that my interest in math as a whole has been going down steadily, which could contribute to part of why I find the problems uninteresting. In fact I am often amazed at how much the other students seem to be working on the problems, and often wonder if it is idiosyncratic to me that I am hopelessly bored by the problems.