I see this post getting trashed in the comments for its overly literal interpretation of personality as a reinforcement learning process, but I think there's some value to it as a mental model of how we operate (which is how the opening sentence describes it).
If you can see past some of the more dubious, overly technical-sounding details and treat it as a metaphor, there is for sure a "behavioral landscape" that we all find ourselves in, filled with local minimal, attractors/basins and steep hills to climb to change our own behaviors.
Thinking about where you are and where you want to be in the behavior landscape can be a useful mental model. Habit changes like exercise and healthy eating, for example, can be really steep hills to climb (and easy to fall back down), but once you get over the hump, you may find yourself in a much better behavioral valley and wonder how you were stuck in the other place for so long.
the essential idea is that personality is malleable, there are concepts in NNs that are analogous to experience we can use to name, deconstruct, orient, and contrast, and as a way to exercise some agency over our own personalities.
you can choose it, and like "the five monkeys experiment"[1] after a while, you don't remember the things you don't believe anymore.
the author used trauma, env change, extreme experiences and psychedelics as examples, but something as simple as reading a book or a comment on a forum can detach us from beliefs and ideas that moored our personality in a local basin. we are the effects of feedback, so change your feedback.
This kind of mental model (some variation of it of course), despite being trashed by the majority, is now becoming the zeitgeist in tech circles. I could have written a very similar blog post (modulo worse writing and more being made fun of), and so could many engineers working on AI. The meta implications of this on the larger society could be interesting.
There's an additional aspect to the dynamics, which is that the social spaces you put yourself in change the landscape to discourage deviancy from the norm. You become like the people you spend time with.
This article approaches human psychology from the perspective that, we are all neural networks and our output (actions) are all a learned function of our inputs (experiences).
This is a common (and convenient) perspective, especially among engineers, but doesn't reflect reality particularly well. We know large swathes of a person's personality is directly linked to their genetics.
The article extrapolates this neural network perspective onto other topics like, mental disorders and depression. The solution is made clear then - just learn how to not be mentally ill! Again, convenient. But not really reflective of reality.
That’s not how I read it, I think you’re missing some nuance here.
The article doesn’t imply genetics have no effect, it just treats them as a baseline which are then adjusted over time according to the person’s lived experiences.
Likewise with mental disorders and depression, the “solution” you claim it states as “not being mentally ill” is the outcome of a process, not the process itself.
The process itself, as far as I can tell from the article, seems to be "increase your learning rate", "change your environment", "meet new people", "take psychedelics".
My point is not that doing these things is never beneficial (well, one may argue about the psychedelics lol), just that it oversimplifies the problem space (and solution space) to the point of not being useful advice.
> My point is not that doing these things is never beneficial (well, one may argue about the psychedelics lol)
In our experience, psychedelics are very hit-or-miss depending on the person. Some (like us) take high doses regularly without much consequence, others can suffer terrible damage after just a single dose. It can be difficult or impossible to predict, anyone who's unwilling to take the risk probably shouldn't.
I've made the jump from physics to neuroscience, so I can talk to the engineers here (I've taken a lot of EE and worked professionally in it too).
The linkages between neurons is somewhat similar to how and RNN looks. But you must remember, there are electrical and chemical elements going on here. It's not just one neuron spiking another. There are many different biochemical processes that modify the behavior of little parts of a neuron, stoichiometrically. And there are many different types of neurons and they all change over time, sometimes drastically so. Most of the goings on is biochem. It's not digital, or even analog. You really need to go down to the field equations at times, finite elements will get you far, but only just so.
RNNs thinking certainly will help you understand better what is going on in a brain, but, like, these things are millions of years old, and optimized just to make more of themselves, not to be understood. It's tough going, and we as a species are only at the very beginning of hundreds of years of study of the brain.
If you'd like to learn more, I can recommend some texts.
> and optimized just to make more of themselves, not to be understood
There is also pressure for it to avoid outputs such are too predictable, as part of a constant arms race against predators and same-species competitors. (For example, how many predators are instinctively key to certain prey behaviors, and when you violate their expectations they might back off.)
That goal doesn't guarantee that the mechanics themselves will be obfuscated, but it does trend in a similar direction.
>The solution is made clear then - just learn how to not be mentally ill! Again, convenient. But not really reflective of reality.
And you know this because?
Cutting edge theories of depression link it to alterations in the reward learning system. There is some evidence that training persons with depression to attend to certain aspects of the reward learning mechanism can reduce depressive symptomology [I am involved in this research]. But speaking more broadly, cognitive behavioral therapy, one of the most successful non-pharmaceutical treatments for depression, involves people "learning how not to be depressed" by unlearning problematic patterns of negative thinking and coping with negative events: first by recognizing what those problematic thoughts and behaviors are, and working to adjust those ... to move you out of that basin.
The main issue with this article imo is that it does not consider the meta-problem: how the reinforcement learning system can be altered by experience as well.
CBT can work, sure. It can also not work. As with any treatment.
And depression is only one mental illness, there are countless others. And there are also many different forms and causes of even depression itself.
As I mentioned in another comment, my point isn't that the article's advice is necessarily harmful, just that it oversimplifies a lot of things by assuming that all psychology can be boiled down to learning and unlearning. Ignoring the role of biology may also cause one to ignore possible paths to progress.
you can also learn to cope with mental illness with more or less self-destructive responses. not everybody gets a chance to learn healthier coping mechanisms.
Not everyone can just learn such coping mechanisms. Some people have physical problems that need physical help/remediation. Of course, our current psychiatric drugs are their risky attempts to help such folks. It is all very tricky, but we should all learn how to have better attitudes and behaviors.
I agree, and the profit motive for the pharma corps seems to have really compromised their ethics (to put it mildly while giving them more benefit of doubt than I think they deserve).
Ultimately, medical science didn't even know the brain had a lymphatic system until this century. Futzing with the subtle biochem of neurotransmitters and hormones is quite beyond their abilities, looks to me. That doesn't appear to prevent them from making a solid off their profit, regardless of any negative results.
And, always, RIP Chris Cornell. Man, we miss that man's voice.
> This is a common (and convenient) perspective, especially among engineers, but doesn't reflect reality particularly well. We know large swathes of a person's personality is directly linked to their genetics.
I really am not an expert in any of this. Just my quick thoughts of the idea about genetics and "being born with it".
If we're attempting to create a mental model of how machine neural networks relate to human brains, would it be useful to think of genetics as the basis that determines your neural network's architecture? Maybe there's even some pre-trained weights that are communicated through genetics.
I think it would be oversimplification to say we're all born with the same neural network and pre-trained weights because like you've mentioned: large swathes of a person's personality is directly linked to their genetics.
This is noted and considered out of scope:
>Obviously some traits are more genetic, and thus inherent, than others, but that is not the scope of this post as even highly-heritable traits will result in a large distribution of outcomes.
But what if it's possible to alter your influential genes, through some powerful mechanism? Whether it be through insane willpower or anything else. In that case, you have an analogy like something like artificial general intelligence or recursive self-improvement. We get to approach the discussion of questioning natural values and instinctive goals with this line of inquiry. We get to eventually question the metaphysics of God, morality, and aesthetics, by introducing fantastic elements like radical self-modification.
Doesn't the environment already affect genes albeit on huge timescales like millions of years? Then the more clear question becomes how to accelerate mutations and for one individual person instead of through delicate and fallible processes like generations of species made through costly reproduction acts like civilizational projects.
Natural selection encodes adaptive responses to the environment in DNA (and other molecules), so memories can be encoded to the extent that they are adaptive and can be encoded (i.e., mechanisms may not exist to encode everything using only standing natural variation).
> A common mistake in life is to let your personality basin solidify too early. Your parents and schooling environment have a disproportionately large influence on who you become as an adolescent.
> But as soon as you gain the freedom to act independently as an adult, it’s usually a good idea to force yourself to try as many new things as you can, including moving cities (or countries!) and considering drastically different lines of work. ...
Oh dear, I'm beginning to fear that the author's personality has been captured by global capital...
And what if it's personality capture all the way down, i.e. that you've got to be personality captured by someone? In that case, the closest you can get to a choice is whether it's your parents, religion, or someone/something else. While the integrity of your parents may vary, there is a subjective argument that they've got a better incentive to steer you into an optimal basin than anybody, relatively speaking.
> there is a subjective argument that they've got a better incentive to steer you into an optimal basin than anybody, relatively speaking
Many parents do not have their kids best interest at heart; from religious fanatics to divorced parents using their kids as pawns to failed athletes living vicariously through their "he'll be in the NHL someday" fantasies to just parents who didn't want have kids and don't care at all.
Then there a whole slew of parents who genuinely want what is best for their kids but won't succeed due to incompetence or their own issues with drug addition or passing on generational trauma.
The quotation in the GP’s post does seem to evince little appreciation (if not outright disdain) for ties to family and local community across the generations. But what if the potential harm caused to some kids by bad parents, is an unavoidable part of the social-cohesion benefits to all society that would be caused by young people not moving far away?
I don’t necessarily want to have a dog in this fight myself. But I immediately thought of how that quotation would jar with some cultures represented on HN, where children stay close to parents all their lives and it is widely felt that the West is doing it wrong.
Agreed! I think blanket assumptions the other way are bad as well.
> where children stay close to parents all their lives and it is widely felt that the West is doing it wrong.
It's interesting to see how close some 1st/2nd/3rd generation European families are, having first hand experience with Italian, Portuguese and Spanish families. It might be only certain parts of the West that is "doing this wrong".
I’m reasonably sure that religious fanatics usually have their children’s best interest at heart. Their value function is just different from that of less religious people.
Something something a different enough value function is indistinguishable from malice.
More seriously, like the old adage about everyone being the hero of their own story, all parents think they have their children's best interests at heart. There's probably no such thing as universal best interests. Gets at some of the thorny problems - personhood, adulthood, cultural values.
It depends on the person, and it depends on the religion. There are positive and negative values, and a misaligned person may well believe they are doing right but actually causing damage. That is why humility, compassion, and honesty are prerequisites for all successful undertakings.
The difference between a malicious value function and unethical conduct is intent; at least in some systems of ethics. Beyond that I think the discussion shifts to the realm of philosophy papers and becomes unsuited for HN comment boxes.
I mean, sure, from a certain technical point of view you could say "honour killings" are done to "save the soul of the child" and hence come from a place of "in their best interests" but by then the "value function" has gone totally awry.
One of the more depressing things of the AI boom is watching engineers and “atheists” get hoodwinked by mystic gibberish like this blog. There is nothing here but astrology: even Myers-Briggs is more scientific.
I think 30% of atheists bothered to think carefully about the Flying Spaghetti Monster and recognized Pastafarianism as a funny commentary on epistemic uncertainty. The remaining 70% said “heh, stoopid Christians believe in a spaghetti monster!” and took it as confirmation of their tribe’s superiority.
It's an application of chaos mathematics to personality development. It isn't a rigorous treatment of such, it's a blog post, but it seems fairly reasonable.
Being a blog post and not necessarily intended to end up on Hacker News, shorn of any other context, the author never even used the term "attraction basin", which is the term you'd want to Google if you want to figure out what the author is saying.
If you don't know what an attraction basin is, then yeah, this definitely comes off sounding bad, but if you know what it is it makes sense. Attraction basins are one of those basic concepts from chaos mathematics that, once you realize what they are and why they are, you realize that the world can't help but be full of them in all sorts of places, including human personalities. The world is fundamentally iterative, so patterns that arise in all iterations are relevant all over the place.
Attraction basins are a good reason to expect in advance, sight unseen, that human personalities can actually be fit into a relatively small number of buckets relative to the conceivable number of buckets that could exist. In fact this is true of any generally similar set of entities living in a generally similar environment, including AIs (of similar architectures, not the entire space of possible AIs) and aliens, assuming they also have some sort of "species" categorization as we do. (Which doesn't have to be "DNA genetics", just, a bunch of similar being for whatever reason.) It doesn't tell you what those will be in advance by any means; it just gives you reason to believe that you won't in fact be looking at an "unbiased", uniformly random distribution of personality parameters, but that they will collect around certain attraction basins in general.
I have not read this blog except the linked blog post, but could you be more specific on what you think is mystic gibberish?
Speaking as someone who works in clinical neuroscience, the basic picture being presented is similar in many respects to the informal model I carry around. It may be lacking in certain details, but big picture seems to me that it has a lot going for it as a guide to intuitions.
Personally I've always found, ironically, some of my most ardent atheist friends to essentially treat the topic with a level of intensity you might expect from a religious evangelist. Often there's also a level of religious fervor they carry over to politics as well.
I don't really care what other peoples religion/non-religion is anymore than what type of underwear they prefer, and yet...
Feeling strongly about lack of belief in something wild without evidence is not congruent to strong belief in something wild without evidence. Even if the people in the former camp can be equally passionate or annoying.
Suddenly there's this big unnamed (or is it named and I don't know it?) cognitive error that I see everywhere that is believing in an automatic symmetry between two opposing viewpoints. Sometimes one idea is better than another. No, you don't get "objectivity" points for virtue signaling that you're above the whole debate.
Well, again, believing that modern AI is close to what human consciousness and intelligence are, or that coders will fix society without running into Chesterton's fence, are good examples of huge beliefs that require huge justification. For which faith isn't a good substitute. You shouldn't be made to feel like you're extremist in the opposite direction for requiring that justification.
We already know that our science can simply not explain/understand what happened in the first 10e-33s after the Big Bang, which is just established science.
There is nothing more "negatively religious" than believing that nothing caused that primal explosion of all that is. Unfortunately, they then proceed to throw out the positive aspects of some religious teachings concerning, e.g., compassionate concern for our fellow human beings.
Regardless, religion is a personal thing; forcing any beliefs on others is always a problem and must be prevented. We are all free to choose our attitudes and behaviors. Behaviors that harm others, however, must -- in a just system -- be dealt with by the society, for the benefit of the whole, irrespective of belief system of perpetrator or victim. Using compassion to make such decisions is always the best way, for varying values of compassion.
> We already know that our science can simply not explain/understand what happened in the first 10e-33s after the Big Bang, which is just established science.
Yet. There's a very big difference in "I don't know," vs. "I know that no one can ever know."
Absolutely. I actually know that (know that I don't know, but that my not knowing doesn't mean that no other person can know), but modern science will not be the source of such explanations.
Those explanations can only be accessed once we understand that the universe itself is queryable (that is our joint purpose here, us and our expansive environment together) and that a human being needs to undergo a process of self-evolution to become aligned with the Creator's Intent such that we gain access to it.
You could say that a person must learn of the challenge, accept the challenge, and then pass all the tests, thereby gaining access.
> once we understand that the universe itself is queryable
We do understand that. "What happens when I push a rock off a cliff?" is a query to the universe. The universe's response is observable when the experiment is executed and the rock is pushed.
> a human being needs to undergo a process of self-evolution to become aligned with the Creator's Intent
I have no need for this hypothesis. We can query the universe at any time simply by observing it, proposing a falsifiable explanation for what is observed, and acting within it to test our explanations.
There are deeper queries than "What happens when I push a rock off a height?" The science and math of Newton's laws of motion is enough for that, especially if combined with some materials science.
I didn't say that you "needed" anything, but the fact remains that if you want to calculate the trajectory of something traveling at a significant percent of light speed, you will need some higher maths. Such calculations require advancing one's mind, as do other queries. But, no, none of that is necessary to find out what happens when a person's cat pushes something off the countertop.
No observation was even feasible for Einstein to formulate GR or Feynman to formulate QED, so if all that matters to you is what you see, that's your choice; I wish you well.
I thought it would be obvious that my simple example was only illustrative. I'm not actually suggesting we discuss high school physics.
> No observation was even feasible for Einstein to formulate GR
Einstein made many observations about the real world to formulate his theories, e.g. Mercury's orbit, the relativity of motion, the inability to distinguish between different forms of acceleration.
> if all that matters to you is what you see, that's your choice
To be clear, in this discussion about what science can explain/understand, I'm advocating for the scientific method as the sole means through which objective truth can be verified, not the sense of sight.
When you mentioned a Creator being whose intent could be known and aligned with, my reply was a reference to a Laplace quote I thought you'd recognize. I apologize if it seemed personal. I only meant to say that we can and do query the universe to discover explanations for how it works without ever assuming the existence of gods or their supposed intentions.
I agree that the scientific method is the best approach towards finding objective truth (which is of course precisely what it is designed to do). The problem is placing objective truth above all other truths. It annihilates subjectivity, which is possibly the most crucial element of being human. And of course formal science has limitations on the truths that it can reveal in its own system (the uncertainty principle) and on a practical level every known scientific theory, no matter how successful, breaks down at some level (see quantum gravity).
All of which is to say that my stance is: embrace science but accept its limitations.
Know that the key to human existence is the fact that, by changing one's attitudes, behaviors, and thought processes, one has also changed one's subjective viewpoint, by expanding both one's field of view and one's depth of comprehension, so long as those changes are harmonious with compassion.
Our most important capability is being able to self-evolve (with the help of the universe) ourselves beyond our more primal impulses and towards our more abstract endeavors such as selfless service to mankind.
I am merely explaining that there are more sublime and direct ways of querying the univere, but that is beyond our current understanding of the depths science could be developed to explain.
At some point in a certain kind of seeking, the proof is accepted and no more is needed. That someone calls it a hypothesis is akin to a flat-Earther opining that my understanding of the solar system is a theory.
And, yeah, flat-Earthers are also very authoritative in their manner.
> At some point in a certain kind of seeking, the proof is accepted and no more is needed.
This is part of what intrigues me about this topic, the inductivist view that if we make enough empirical observations we can eventually settle on an objective truth. E.g. by observing enough cats we can conclude that all cats have whiskers.
The realm of science is falsifiable statements. "All cats have whiskers," is a falsifiable statement. We can only state it with 100% certainty by observing all cats (impossible). We can disprove the statement by experiment, however, one designed to discovering just one pathetic little whiskerless cat.
I subscribe to this, Karl Popper's view, that truth is an ideal we can pursue but never quite expect to arrive at. Truth seekers aim to be less wrong, but have little hope of being 100% exhaustively right.
This isn't to say we are unable to act like we believe anything. We can assume the next cat we meet will have whiskers and get by pretty well. But I wish more people could accept that statements like "God exists but is not observable in any way," or "The government is covering up aliens and anyone who says differently is part of the cover-up," are not falsifiable, and because they can't be disproven they also can't be trusted as truth. Maybe they are true, but you can't know, so it doesn't matter.
If a used car salesman tells you the car you're buying has above average fuel efficiency, you can test that and you should. If he tells you that it is a magical car that only runs out of fuel when destiny ordains it, we can't test that, so we should stay skeptical of the claim.
Whatever means by which someone believes they are directly querying the universe, if they are making falsifiable statements, those statements can/should be tested by the truth seeker; and if they are not making falsifiable statements, there is no compelling reason to believe them.
There are subjective experiences (especially seeking to connect with God in order to become a more virtuous person) that must be undertaken with all one's heart in order to verify for one's self. We are created with the ability to self-evolve one's self with the help of the universe and its/our Creator, but our free will is not trampled upon, even though it would be best for ourself and all those around us.
Yes, the Creator knows what is best for us, but, no, It has given us free will and honors it until we beseech It to help us.
If you make a prayer with all your heart asking for the Creator to take Its Spirit (our conscience) back into Itself so that we can cleanse and purify our souls of our vice-oriented ("vice-eous", the opposite of virtuous) tendencies, thereby becoming a friend to all, becoming, by degrees, consumed by compassion and a possessor of wisdom, you will be changed. It is the meaning of the 1st Beatitude ("blessed are the poor in spirit, for theirs is the kingdom of heaven"), and is the prerequisite to (IIRC) the 5th ("blessed are the pure in heart, for they shall see God").
This is a subjective experience. Most people are too wrapped up in their selfish desires to see how different a person is who has undertaken this process of moral self-evolution. I have met two such purified men (Murshids) and a third (from a different faith) whose teacher was also such a one. I am on the way for a couple of decades now but am not purified, only semi-cleansed. Happiness is different for me; it is a sublime joy, a pleasure to help a person in any small way.
Yet, I am rejected by my family and lifelong friends, but they are only rejecting happiness because their forms of religion (especially those with none at all) reject the path I am on. Our son is an open chess champion (3x!) and our daughter is an accomplished seamstress while still in her teens (Singer 503a is badass). We do not reject any form of religion, but understand that we are all one human race and should care for one another and create peace here on our blessed Earth.
This is the subjective truth that we live but is rejected by nearly all; such is the world we live in, of selfish idiocy and destructive strife. But what we Sufis teach and live is truly the science of the soul. In history, all groundbreaking science is rejected, be it that of Eugene Parker, Boltzman, or even the doctor who found that most stomach ulcers were caused by a simple bacteria. That others reject the truth that we live -- and they can, if they try -- is not a testimony about us, but only about them.
I love you. I have no selfish motive in telling you the truth here. I want nothing from you. I only want for you to be happy, and to spread happiness to all those you experience, by both your actions and explanations. You can verify this for youself, but only if you jump in with both feet.
The great Islamic mystic, Rumi, said 800ish years ago, "The Way goes in."
The truth is that we human beings could be creating a must happier world, but selfishness is ruining it for everyone. There has been talk around HN lately of "mathematical thinking" but who contemplates how we would change things if selfless compassionate service was our systems' intention and goal, instead of their current motive of profit-at-all-costs? We Sufis understand that calculus and understand that it begins with each of us. We inhabit a world that pushes us to be selfish animal-like creatures, when we could instead be humanitarians that care about everyone.
We could create a simulation of such a transformation now, but there is a force within each of us that pushes back against such notions, decrying them as impossible. You will feel that push back as you read the deepest truth in the universe. It will say that I'm crazy or don't know sh_t. Just please remember you are free to choose, and pay close attention to what that negative inner voice says. Perhaps it's not you, and is, as Castaneda's Don Juan explains, a parasite of our mind. We Sufis have a much simpler explanation: there is an enemy within.
Thanks for your words. I've read a bit about Sufism and I think if we could sit and speak at length over many days and a few cups of coffee, we would find a lot of agreement and kinship. You seem like a person who has thought a lot about the nature of truth and the divine, and someone who values peace and knowledge.
Please imagine for a moment that I told you the REAL truth is: we are surrounded by trickster spirits. There once was a Creator who created these tricksters along with the cosmos, and they whispered their powerful lies into his mind and convinced him to die. Now, they turn their powers on us and convince us of religious beliefs. Every personal mystical discovery through prayer and meditation is a lie from such demons, and the only path to salvation is the painful task of purifying yourself of all religious belief.
I think you would probably not believe this (me neither). But how could you know it's not true?
Maybe you would reject it because you have received God-given teachings that contradict it, but I would say this is evidence of my view, the supernatural convincing power of the trickster spirits. You might challenge my view, but that's not possible, because my statements are unfalsifiable. Every statement you made in favor of your belief would further support mine.
Neither of us would be able to test or disprove one another. We would be stuck, each believing the other ignorant of an essential cosmic truth.
This type of thinking is poisonous to our species, it locks us into ignorance and makes us putty in the hands of the powerful, leading us to believe whatever we feel or hallucinate or are taught. It leads us to label non-belief as idiocy and a rejection of happiness. It leads good people to believe they are impure and in need of purification. The power of those trickster spirits twists our kind natures into isolation and self-hatred.
The fact is, to be truth, a statement must be observable by even our enemies and all of those who do not share our views. What makes truth is its objectivity.
Your belief is that you and very few others have a special knowledge of the thoughts and intentions of the creator of the universe. I feel this is an intensely prideful view. How small the creator's intentions must be to be comprehensible to our minds, a few tiny mites crawling on a mote of dust for only an instant, floating through a sunbeam in a vast and ancient cosmos.
You say we are all one human race and should care for one another and create peace here on our blessed Earth. I agree with you fully on this.
But what peace we have has come only from diverse and open societies, liberalism and secularism and scientific pursuit of truth. The world is improving, demand for equal human rights growing, lifespans extending, generation upon generation thanks to idea exchange and liberal values, not faith or autocracy.
You say all groundbreaking science is rejected at first. This is to the credit of science, not a criticism of it. All the examples you referenced were the result of questioning, skepticism, and the acknowledgment that previous views could be wrong, and today they are accepted, thanks to science. What religions can be said to equally subject their own ancient god-given teachings to the same skeptical inquiry? If this is what you value, the ability to be proven wrong, you will not find it in your religious claims of ancient, infallible, immutable teachings.
You claim to know the science of the soul, but science is challenge, disproof, falsifiability, doubt, skepticism. I don't see evidence of skeptical science being done on the soul by the faithful. Faith has none of these questioning qualities. In fact you encourage me to call skepticism negative, an enemy, a parasite.
No truth has ever feared any doubt or question. That's the nature of truth: it is not harmed by doubt. Skepticism is not the enemy, it is the path to discovering lies. Descartes said: "If you would be a real seeker of truth, you must, as much as possible, doubt all things."
i agree with your sentiment, but it bears pointing out that no one brandishes their underwear in people's faces screaming that they must wear the same ones, or use the political system to privilege people of the same underwear and punish others etc etc
fwiw me personally i'm all in on uniqlo airism, there is no better underwear and if i could force everyone to wear them i would (for their own good, of course)
I agree that it can be helpful to think of identity as a trajectory shaped by interactions along the way. However, we also continually shape our environments in large and small ways. TFA ignores this completely. Can this be effectively modeled in RL?
Over 130 years ago, Dewey [1] criticized the model of psychology which looked at human behavior in terms of stimulus -> internal processing -> response. Stimuli don't just come to us; we seek them out and modify the world around us to cause them to occur. Dewey and other pragmatists proposed reframing stimulus/response in terms of "acts" or "habits," or changes to the unified agent+environment. Popper was getting at the same entanglement of agent and environment in "Three Worlds" and Simon in "The sciences of the artificial."
I see RL as an elaboration of the stimulus/response paradigm: the agent is discrete from the environment. Does RL work well in an environment like Minecraft, where the real game is modifying the relationship between actions and future states? What about in contexts like Twitter, where you're also modifying the value function (e.g. by cultivating audiences or by participating in a thread in a way which conditions the value function of future responses)?
"I agree that it can be helpful to think of identity as a trajectory shaped by interactions along the way. However, we also continually shape our environments in large and small ways. TFA ignores this completely. Can this be effectively modeled in RL?"
You don't need to. All that is necessary for an attraction basin to emerge is an iterative system. If you prefer to model the human being and their entire environment rather than the human being and their input, you'll still get attraction basins. You'll just get two views on the same reality, suitable for different uses and different understandings, but it's not like "ah, if we model a human iterations we get these attraction basins but if we include environmental interactions suddenly we get a uniformly random distribution of personalities across the total personality space, it's all totally different once you consider the environment as part of the iterative system too".
Thanks; I agree--both that you could train an agent in these situations, and that "You'll just get two views on the same reality, suitable for different uses and different understandings." I think the latter seriously undercuts the article's attempt to explain these trajectories in terms of personality; they could just as easily be attributed to the power of culture or social structure.
Heh, well, another lesson from chaos mathematics is that in iterative systems, you don't really get "explanations" the way we humans like to think of them... the answer to "what caused X" for any X than has taken a long time to develop is "everything". So rather than culture "or" social structure, I'd say "and", "and" also a lot of other things, and also the culture and social structure are themselves affected by the very personality structures we're trying to discuss.
Determining "causes" isn't as hopeless as that makes it initially sound, but you need something more sophisticated than the normal human concept of "cause" to even approximate useful answers. The good news is, this isn't impossible; we all live in an iterative world and we operate in it even so, which requires us to have certain models that conform to the world. It's one of those cases where I don't really love the "humans are just horribly irrational" gloss; our instincts and intuitions often have greater rationality than we realize, because they were formed in this iterative world, and sometimes it is in fact the particular naive concept of "rationality" we are trying to measure them by that is deficient, whereas if you use a more sophisticated one we look less bad.
(But sometimes humans just act suboptimally, no question about that.)
Another thing that helps is that you aren't generally interested in modelling the entire system. For considering myself and whether I may want to, as the article discusses, make changes in myself, I can take my culture and environment more-or-less as a given; I need some flex to consider options like "well what if I just up and moved to another country?", but I don't need to consider my own effects on society very much because they are some complex combination of "tiny" and "utterly unpredictable". While society is chaotic, the time frame of the impact on society from me changing from excessively introverted to somewhat less introverted is way, way past my horizon for making decisions.
I agree that the discussion in the blog post is incomplete because it does not consider that we shape the environments that shape us, though it does briefly touch on the fact that other RL agents (people) try to shape us, and we them. But it is certainly more than that.
> How do you know if you're in the "right" personality basin
I'm not so sure "right" is the right frame here. I like the multi-dimensional viewpoint you take. My experience would be a healthy personality would be one capable of adaptation in service of your interests at any times. It's dynamic.
This reminds me of Bob Kegan's stage of adult development. Initially, most of us leave adolescence at the "socialized" stage of development, ie our personality basin has primarily been determined by the external factors of our upbringing and environment.
From there, if we choose to continue developing, we eventually reach a "self-authored" mind, where we have transcended our socialized basin in favor of a self-defined and created personality structure, until ultimately, for those who continue evolving, we reach a "self-transforming" mind, or a mind capable of transforming itself.
I like the simplicity of the model, and I also think it reduces personality to an unnecessarily static entity. Things like internal family system/parts work also demonstrate that our personality is not a singular entity, it is represented by a whole slew of parts that show up in different ways and different contexts! I think the broad strokes of it still hold, and also think there are many additional approaches to truth and the awakening path, lying in parts work, embodied transformation, and whole bunch of other experimental modalities (thought perhaps that's just my personality speaking...)
Personality is ~70% determined by genetics, not life experience.[0]
I’m surprised that someone interested enough in the topic to write such a long post wouldn’t put the time in to do a cursory dive into personality psychology. I’m going to assume that the author has a similar definition of personality to mainstream psychology, but if so, they are ignoring accepted studies and evidence that make it pretty clear that personality is not learned through conditioning like AI.
There's no practical point in separating genetics from life experience, as they go hand in hand together.
Someone who has the genetics to be physically attractive/beautiful will have a completely different set of experiences than someone who isn't. Same goes for intelligence.
Also, the source you linked only pertains to IQ (which itself is not a perfect measure of intelligence), and IQ is not personality (although I have met some folks who do treat their IQ as a substitute for such).
The analysis of the data revealed no significant difference between MZA twins (reared apart) and MZT twins (reared together) in regards to personality measures such as temperament, hobbies, interests, career pursuits or social attitudes.
This is noted and considered out of scope:
>Obviously some traits are more genetic, and thus inherent, than others, but that is not the scope of this post as even highly-heritable traits will result in a large distribution of outcomes.
Sure, and they even go on to say that the article is about “what helps to explain the differences in outcome between two genetically identical people.” However it’s clear that is not actually what the article is about and they do talk about traits that are highly heritable and not shown to be dependent on environment and life experience.
I enjoy how people are dunking on this by saying “omg this is what happens when engineers dare to have thoughts on other topics” when this is very similar to the theory for CBT in psychology
As someone with a lot of research experience in this area, I was expecting something more fluffy but this was actually pretty good.
There's a lot of research into some of these ideas at the moment. The terms they use aren't necessarily the same but many similar ideas. I think for the most part, the evidence in support of many of them is fairly weak but at the same time many of these ideas are much harder to test well than it might seem at first. I give a certain amount of pass to people trying to test them because in this area, trying to pin down something often is a bit like trying to study an individual cloud: you can kind of see it there, but if you were to try to measure its boundaries and dynamics, it would be harder to do than it might seem at first glance, and you'd end up more easily making very general observations about it than you might like.
One thought I had when reading it is that people's environments are much more stable than is usually recognized. The piece acknowledges this somewhat, but I think it's more of an issue than most like to admit. Even when someone tries to change it, it can be difficult, because assets and SES can be difficult to change, other people resist it due to their own incentives like the essay points out, and even when other people don't really care much they often will resist it unintentionally due to schemas about personality change and so forth. "Once an X, always an X" regardless of whether you're talking about vocation, career, social characteristics, whatever — even though that statement isn't actually true beyond some kind of general sense of it. Or they just are used to seeing someone in a particular setting and so don't see them in another.
Another issue that's maybe murkier is the essay is a bit loose about person characteristics, even at a given point in time, versus situation characteristics. I don't know that it affects the arguments very much at all, the points still stand, but it sometimes drifts into talking about "personality" when I think it really means something more relational, like "role" or "interactional pattern" or something like that.
This is what happens when an engineer tries to apply mathematical models to entirely different fields where they have no applicability. Reducing human personality to machine learning concepts like 'gradient updates' misses the fundamental complexity of human psychology and consciousness.
I think the article is missing some key aspect of personality forming : Maslow's hierarchy of needs[1]
Personality and its development is hugely dependent on which needs are or will be currently fulfilled or not.
Attention economics is able to impact you negatively on low level of the pyramids, notably due to its impact on sleep. It can also shift your priority towards less essential needs than the one you should be working on. And it's often myopic, missing totally some aspects due to hyper-focusing.
It's also able to impact you by impacting those near you, that's what social networks are for. Developing a support structure whether family or friends is a double edge sword because you indirectly become as weak as the most vulnerable member of the group, or group may explode.
The economy also apply pressure on basic level needs, like shelter, heat, air and water(when polluted), and safety, which probably contribute to shape the personality and are basins which are also hard to get out of ("you can take the girl out of the trailer park, but you can't take the trailer park out of the girl").
That is because most people are not consciously attempting to become better people for the betterment of those around them (which helps their own happiness, too, due to the nature of our karmic universe). Most people are simply acting out of the selfishness to have their own desires fulfilled, with varying amounts of concern for the consequences to those around them.
A person can undertake self-evolution in any direction they choose; it is always our choice, except in the extremely rare cases where a person is physically damaged. Most people have the power to change, though it is difficult for us all. The universe does help those of us who seek to do so for the benefit of others.
What makes human beings unique is that we have both the ability to self-evolve our attitudes and behaviors, and the tools to do so, our mind and conscience.
>> Personality Capture
A person who is not undergoing conscious-self-evolution is susceptible to being influenced (and even overwhelmed) by a forceful personality that caters to their desire-seeking. That is how dictators have always risen to power: they seek a loyal army of folks enamored with the leader's promises to make the in-group's lives better. Those folks never seek to make life better for ALL people, because helping out-group members requires generosity, which usually requires making some level of selfless sacrifices of resources.
As always, the strife is between selfish callousness and selfless care. Compassion for all our fellow human beings is the nature of being a humanitarian, that is to say: being the best of what we can be, for the benefit of the entire human race. And it all starts with each and every one of us.
Most people are trying to survive in a capricious universe, after having observed over a lifetime that the good are often punished while the wicked are rewarded.
Mind/consciousness is not unique to humans, we're just better at maintaining a thread of self reflection so that we can make long term changes in behavior.
Kindness is all well and good, but when there's not enough to go around it's foolish to deprive everyone. Some people are just more deserving, whether for moral reasons or because an investment in them will be more fruitful.
What's important is that you know who is deserving and who isn't, right?
I did not say that mind or consciousness is unique to humans. Our difference is that only we can control our minds and only we have a conscience to help us make moral decisions, which is always within the realm of self-reflection. Animals only possess the barest minimum of mind and self-reflection, for the sole purpose of survival, which pressures them to be as fit as possible. We exist at a different scale of consciousness.
Morality only exists within human beings because only we can calculate (using our mind, under the pressure of our conscience) how our actions might affect others positively or negatively. Then we use our greatest gift, our free will, to choose between selfish and selfless behaviors.
Even our attitudes can be chosen, over time. Choosing to be empathetic is an essential path forward to positive group behavior, and those without empathy are dangerous folk, indeed. Helping our younger generations develop a matrix of positive group morality is the highest purpose of education, sadly mostly neglected nowadays.
You should remember, as a CEO, that your choices bear a greater burden because they affect an entire organization as well as all the folks affected by whatever it is your firm produces (I don't know, I didn't look).
And, my friend, my universe is not at all capricious, but my fellow human beings are definitely flaky, being solely motivated by selfish desires, few having any more than a passing interest in the well-being of others. Of course, I do not consider that some being given more wealth and/or hardship than others is capricious; others surely disagree, obviously. As to the fairness of our criminal justice system, that's on the human beings making those decisions, not the universe for giving them the power to shade things however they choose.
You don't know to what degree animals are conscious or control their behavior.
Empathy is a lot like turning the other cheek which the institutions of power like to beat the small minded and powerless with in order to perpetuate themselves. Sure, it's often good, but there are also cases where it isn't so good or useful, however having the population docile by default is always good for power.
People become CEOs because they're power hungry and good at manipulating others. Those sorts of people are least likely to buy the empathy line in any non performative way, so I'd save my breath.
The flakiness and inconsistency of humans is just a reflection of a pattern that repeats itself at all scales of the universe. There is no coherent order, only the chaotic ramifications of countless minds, both human and non, coevolving the universe together.
>> You don't know to what degree animals are conscious or control their behavior.
Just because you don't know something or even understand that it can be known, in no way limits my knowledge. It just paints yourself a confident fool who already knows it all. Sounds like a CEO, neh?
And, BTW, I do know. Also, I know that you don't know that I know, because you believe that no one can know, which is solely because you, yourself, don't know. As the Doobie Brothers sang so many years ago, "What a fool believes..."
In my humility, I say to you, "There are things you know that I do not." As well, I say to you, "There are things that I know that you do not." Now, here's the tricky bit, I know that what I know is more important than what you know.
How do I know that last, crucial, bit? Because you think there is a way to decide who gets resources and who doesn't, and that it should be based upon some value function that folks "smart" like you can determine for one and all. Pleeeaaase. I am a student of history, which is riddled with folks like you, perched upon your spire claiming to have the voice of reason to speak over your lessors. It's tired and shall not do well as this Age of Truth lurches forward.
That said, it's your choice. Every fool has chosen to be a fool. And the first step to not being a fool is having the humility to entertain the possibility that you are, indeed, a fool. Here's the secret: we all are, to some extent, but a few of have the humility and desire to escape it.
As to "coherent order", we are the only beings capable of consciously designing and creating such order, but our being slaves to our lower, selfish selves has prevented our doing so. As with all things human, the choice is ours, and the choices we have made thus far have been less than optimal.
You can call me what you want, but you have told us all who you are, and that's all I need.
We move forward together, usually with the loudest and most ignorant leading the way. Same as it ever was. The important thing is which side we each take. I side with compassion, justice, honesty, and science, which puts me in the minority, thank God.
It is naught but crypto-fascism the proposition that there is a choice between exercising empathy and being an immoral character. It is moral authoritarianism, the gaslighting of others, trying to make them believe that free will exists in the service of neo-conservative morality's dialectic of good judgement and evil judgement being a problem to consider at all in the first place. The reality is that morality lives way beyond what the rhetoric of neoliberal initiatives try to seductively present to vulnerable intellects and hearts.
Advertising empathy as being better than a slice of bread is just fascism and a strain of neo-colonial desire in a clever (but not clever enough) disguise. The global threats to capitalism's productivity goals aren't fooling anyone. The Devil's idle hands and economic regression will have to retreat back to the drawing board and find a new military tactic that would be more effective than trying to disguise morality as something cool looking (when in reality it looks ugly and unappealing as an asset).
> This is why techniques like nonviolent communication, dialectical behavior therapy, and mindfulness have observation and introspection as a core facet, because it’s something that you have to consciously practice to become good at rather than something you’re born with.
I now tend to think that consciously observing and uncovering what you already are is really the start and end of it. One ought to try to concentrate oneself, rather than dilute oneself into something one is not. One might want to be a billionaire technologist, or sports hero, or whatever, and one might even edit oneself into something approximating that (via mentors, diligent study or whatever), but one will remain unfullfilled - for how is it possible to 'lie to' and be 'right with' oneself?
I feel like some people are reading this post too literally.
An infinite number of factors go into developing a personality, this covers a lot of the big ones.
---
Lately I've been thinking about the windows of plasticity and why people change beliefs as adults.
The sad fact is that a lot of people have a lot of very wrong and bad beliefs, and unfortunately most of them are already adults, so you don't get to discipline those beliefs out of them (I hope you can read this as tongue in cheek); You will never get mad enough at a person to fix them. Anger is motivating, but you don't get to pick the direction.
As I understand it, psychologists believe that parents and the environment of a person's youth set a lot of their basic beliefs about the world, but it is their friendships in adulthood that most determine their value system - you want what is best for your friends (and yourself).
---
To me this also ties into evaluating the actions of historical figures; people seem to get hung up on very flat depictions - was it ok that a person who did good things also did bad things? Well, they are a whole-ass person, raised in a different time and place. They didn't choose when and where and by whomst they were raised. They had some level of choice in their friend group, but that is also constrained by time and place.
I feel that you can judge people and actions, while also allowing space for humanity and personal stories; but that does take a lot of time and emotional work. It is much easier to just choose one side of the coin or the other, face or heel.
"Although there are many times in life you’ll consciously decide to act in a certain way, this is the exception, not the norm."
IMO, that's the most important idea there. Your personality is what you've created to live among others like you, but as your personality grows, it develops habits that have weight and momentum, and later in life those habits start defining your actions completely, you get progressively smaller windows for true self expression, and your life starts feeling dull and mechanical.
Your attention is the only thing that's truly yours.
> Luckily for humans there exist many symbiotic equilibria where multiple parties can find mutually-beneficial feedback loops within the epochs of personality-space. Parent/child relationships
My child is totally "attempting to modify [my] RLHF process so that it results in an agent which is beneficial to them, hopefully resulting in someone who will always give in to their demands". All the damn time!
> Maybe you were born tall and attractive and then this led you to engage in a lot of athletic activities and socialization, and at the end of all of the positive feedback you have ended up with a jock personality that goes on to become a professional football player.
Is that really that tall and attractive guys want to become football players? I always that football players attract the same stereotypes as police officers, big and stupid.
In Europe only the most stupid folks want to become soccer players. Even if they'll end up filthy rich, with lots of tattoos and horrible haircuts.
I just watched Gary Gulman's comedy special Depresh Mode. He's a guy built like a football player and yet he's a depressive comedian. Only someone whose perception of their body matches their perception of their personality would come to the conclusion that everybody else on Earth must be a walking stereotype. It's like the Talking Heads song Seen and Not Seen but it's not a joke.
C'mon folks. So many "expert opinions" and erudite references in these comments. The sciences of cognition, neurology, evolutionary psychology etc are all still muddling around trying to figure out how the human mind works. We're learning a lot about possible ways the mind might work from our observations of processes and outcomes of machine learning. It's a cool new paradigm to add to the mix. I really like the framing offered by the author. They're quite upfront about the fact that there's a lot of genetics involved. That all models are wrong but some are useful.
Why all the defensiveness? Whatever genetic aspects of our personalities and behaviours there are - there's still a pretty big component of just learning patterns. Language acquisition is like that. It's an innate thing but the languages we're exposed to as kids shape what patterns of language use we fall into.
It's nice to see that Rationalists have reinvented Maimonidean virtue ethics. The idea that humans personality is maximally pliable, and this is metaethical grounding for the concept of moral responsibility is an extreme on a spectrum. It has some inspirational value, but I've never found it especially compelling.
Also, the fact that this article does not mention the Big Five once really makes me feel like the author is trying to reinvent the wheel but has never looked at a wheel before. Despite its flaws (and the broader methodological critiques you could level at personality science as a whole), it is the most scientifically grounded model of human temperament that we have right now. But why start with the latest science? That would involve leaving your bubble, which is a major no no.
Sorry for the snark, but this is scientific reasoning as cargo cult at its worst.
The author is clearly taking the time to reflect on the world around them and I genuinely see a curious mind. However, I also see a product that touches on an already established idea and how the gap between this writing and other reflects a gap between "hard" and "soft" sciences.
The idea here is that of habitus. Habitus is an Aristotelian term that was expanded upon by Bourdieu in the field of sociology. It is the way in which people perceive and respond to the world through a durable transposeable disposition, set of skills, symbolic capital and doxa that is shaped by the environment and in particular the material conditions of the individual[1].
Habitus plays a role in how individuals are perceived in ways, that like the author illuminates, can form a virtuous circle re-enforcing disposition, skills, and outlook in a way that can be positive for an individual.
What the author doesn't allude to, and this is where I see a gap between hard and soft sciences and where they would benefit from being able to connect this idea to a broader body of work, is how habitus is reinforced - usually unconsciously - in ways that reproduce class, racial, disability, and gender habitus under the terms laid out by the dominant ideology - that is to say the ideology of the dominant class.
An example in education would be how the education system perceives individuals possessing middle and upper class habitus as being ready and prepared for education, and those who lack that habitus as being lazy, disruptive, or unwilling to learn. On one hand you might be thinking "Of course that's obviously true," and I'd like to take a pause to point out that "obvious truths" are often a signal of our own habitus and should be critiqued as such.
They touched on the concept of reinforcement learning[social systems] acting upon individuals in a way that shapes their habitus, but it's crucial to point out that these reinforcement learning systems aren't free-standing disembodied mechanisms. They are situated in a social landscape and are constituted from of social relations which are themselves a product of economic relations. Furthermore, the systems of reinforcement are self-replicating. They are essentially social quines[2] - or more specifically oroborus programs ie: they plant the seeds of their own replication by encoding those relations into the habitus of individuals.
There's obviously a bunch of writings expanding on the idea of habitus, how it's formed, reinforced in different social arenas, and the effects it has on individuals and groups. I'd expect the author would be interested in soaking up these related perspectives and perhaps you as a reader would be too.
1. Obviously not black and white, there are other factors which can influence habitus - disability is an obvious one, for example.
>Your personality is formed by a process conceptually similar to RLHF. You are first born with a set of traits in a given environment. After this, you perform many interactions with your environment. If an interaction goes well, you’re likely to do it more often, and if it goes poorly, you’ll probably do less of it.
first off 0 evidence presented, second off what about the kids that grew up stealing food in concentration camps or due to abusive parents. Do they grow up to be liars and thieves? Nope. What about all the kids that get nothing but positive vibes and turn into total arseholes...
Reading computer scientists' takes on psychology or social sciences after taking 1 (one) 101 level class and then reinventing the wheel on topics that have been researched for decades is just grating. Where does this come from? We get it, you like to think a lot, but if you're as smart as claim you would've realized you can't solve a topic by just thinking really hard and long about it alone in your room. Gurwinder Bhogal is one of the other guys repeatedly falling victim to it
I see this post getting trashed in the comments for its overly literal interpretation of personality as a reinforcement learning process, but I think there's some value to it as a mental model of how we operate (which is how the opening sentence describes it).
If you can see past some of the more dubious, overly technical-sounding details and treat it as a metaphor, there is for sure a "behavioral landscape" that we all find ourselves in, filled with local minimal, attractors/basins and steep hills to climb to change our own behaviors.
Thinking about where you are and where you want to be in the behavior landscape can be a useful mental model. Habit changes like exercise and healthy eating, for example, can be really steep hills to climb (and easy to fall back down), but once you get over the hump, you may find yourself in a much better behavioral valley and wonder how you were stuck in the other place for so long.
the essential idea is that personality is malleable, there are concepts in NNs that are analogous to experience we can use to name, deconstruct, orient, and contrast, and as a way to exercise some agency over our own personalities.
you can choose it, and like "the five monkeys experiment"[1] after a while, you don't remember the things you don't believe anymore.
the author used trauma, env change, extreme experiences and psychedelics as examples, but something as simple as reading a book or a comment on a forum can detach us from beliefs and ideas that moored our personality in a local basin. we are the effects of feedback, so change your feedback.
[1] https://skeptics.stackexchange.com/questions/6828/was-the-ex...
This kind of mental model (some variation of it of course), despite being trashed by the majority, is now becoming the zeitgeist in tech circles. I could have written a very similar blog post (modulo worse writing and more being made fun of), and so could many engineers working on AI. The meta implications of this on the larger society could be interesting.
There's an additional aspect to the dynamics, which is that the social spaces you put yourself in change the landscape to discourage deviancy from the norm. You become like the people you spend time with.
[dead]
This article approaches human psychology from the perspective that, we are all neural networks and our output (actions) are all a learned function of our inputs (experiences).
This is a common (and convenient) perspective, especially among engineers, but doesn't reflect reality particularly well. We know large swathes of a person's personality is directly linked to their genetics.
The article extrapolates this neural network perspective onto other topics like, mental disorders and depression. The solution is made clear then - just learn how to not be mentally ill! Again, convenient. But not really reflective of reality.
That’s not how I read it, I think you’re missing some nuance here.
The article doesn’t imply genetics have no effect, it just treats them as a baseline which are then adjusted over time according to the person’s lived experiences.
Likewise with mental disorders and depression, the “solution” you claim it states as “not being mentally ill” is the outcome of a process, not the process itself.
The process itself, as far as I can tell from the article, seems to be "increase your learning rate", "change your environment", "meet new people", "take psychedelics".
My point is not that doing these things is never beneficial (well, one may argue about the psychedelics lol), just that it oversimplifies the problem space (and solution space) to the point of not being useful advice.
> My point is not that doing these things is never beneficial (well, one may argue about the psychedelics lol)
In our experience, psychedelics are very hit-or-miss depending on the person. Some (like us) take high doses regularly without much consequence, others can suffer terrible damage after just a single dose. It can be difficult or impossible to predict, anyone who's unwilling to take the risk probably shouldn't.
> it just treats them as a baseline which are then adjusted over time according to the person’s lived experiences.
So like the randomization process that seeds the values for RNN weights?
And the architecture which determines what units have weights between them.
And the training rules, that determine which weights are adjusted in response to what units.
I've made the jump from physics to neuroscience, so I can talk to the engineers here (I've taken a lot of EE and worked professionally in it too).
The linkages between neurons is somewhat similar to how and RNN looks. But you must remember, there are electrical and chemical elements going on here. It's not just one neuron spiking another. There are many different biochemical processes that modify the behavior of little parts of a neuron, stoichiometrically. And there are many different types of neurons and they all change over time, sometimes drastically so. Most of the goings on is biochem. It's not digital, or even analog. You really need to go down to the field equations at times, finite elements will get you far, but only just so.
RNNs thinking certainly will help you understand better what is going on in a brain, but, like, these things are millions of years old, and optimized just to make more of themselves, not to be understood. It's tough going, and we as a species are only at the very beginning of hundreds of years of study of the brain.
If you'd like to learn more, I can recommend some texts.
> and optimized just to make more of themselves, not to be understood
There is also pressure for it to avoid outputs such are too predictable, as part of a constant arms race against predators and same-species competitors. (For example, how many predators are instinctively key to certain prey behaviors, and when you violate their expectations they might back off.)
That goal doesn't guarantee that the mechanics themselves will be obfuscated, but it does trend in a similar direction.
Not the OP but I'd be very interested to hear text recommendations on this!
Bear's Neuroscience: Exploring the Brain
Purves's Neuroscience
Kandel's Principles of Neural Science (grad level, but really the bible for neuro)
Dayan and Abbot's Theoretical Neuroscience (good for compneuro)
The Art of Electronics, 2nd edition (cheating here, but it's good to go back through the fundamentals before going into the edge cases that is neuro)
In general, AI is so new that there really isn't a good classical text between AI thingys and neuro. It will take time to suss one out and write one.
>The solution is made clear then - just learn how to not be mentally ill! Again, convenient. But not really reflective of reality.
And you know this because?
Cutting edge theories of depression link it to alterations in the reward learning system. There is some evidence that training persons with depression to attend to certain aspects of the reward learning mechanism can reduce depressive symptomology [I am involved in this research]. But speaking more broadly, cognitive behavioral therapy, one of the most successful non-pharmaceutical treatments for depression, involves people "learning how not to be depressed" by unlearning problematic patterns of negative thinking and coping with negative events: first by recognizing what those problematic thoughts and behaviors are, and working to adjust those ... to move you out of that basin.
The main issue with this article imo is that it does not consider the meta-problem: how the reinforcement learning system can be altered by experience as well.
CBT can work, sure. It can also not work. As with any treatment.
And depression is only one mental illness, there are countless others. And there are also many different forms and causes of even depression itself.
As I mentioned in another comment, my point isn't that the article's advice is necessarily harmful, just that it oversimplifies a lot of things by assuming that all psychology can be boiled down to learning and unlearning. Ignoring the role of biology may also cause one to ignore possible paths to progress.
> just learn how to not be mentally ill!
you can also learn to cope with mental illness with more or less self-destructive responses. not everybody gets a chance to learn healthier coping mechanisms.
Not everyone can just learn such coping mechanisms. Some people have physical problems that need physical help/remediation. Of course, our current psychiatric drugs are their risky attempts to help such folks. It is all very tricky, but we should all learn how to have better attitudes and behaviors.
Kind of, but the data isn't that great on that. There's some doubt about even SSRIs being net positive long term.
TL Dr: yes, some people have low levels of "X", but we have insufficient data about why that is.
I agree, and the profit motive for the pharma corps seems to have really compromised their ethics (to put it mildly while giving them more benefit of doubt than I think they deserve).
Ultimately, medical science didn't even know the brain had a lymphatic system until this century. Futzing with the subtle biochem of neurotransmitters and hormones is quite beyond their abilities, looks to me. That doesn't appear to prevent them from making a solid off their profit, regardless of any negative results.
And, always, RIP Chris Cornell. Man, we miss that man's voice.
> This is a common (and convenient) perspective, especially among engineers, but doesn't reflect reality particularly well. We know large swathes of a person's personality is directly linked to their genetics.
I really am not an expert in any of this. Just my quick thoughts of the idea about genetics and "being born with it".
If we're attempting to create a mental model of how machine neural networks relate to human brains, would it be useful to think of genetics as the basis that determines your neural network's architecture? Maybe there's even some pre-trained weights that are communicated through genetics.
I think it would be oversimplification to say we're all born with the same neural network and pre-trained weights because like you've mentioned: large swathes of a person's personality is directly linked to their genetics.
This is noted and considered out of scope: >Obviously some traits are more genetic, and thus inherent, than others, but that is not the scope of this post as even highly-heritable traits will result in a large distribution of outcomes.
But what if it's possible to alter your influential genes, through some powerful mechanism? Whether it be through insane willpower or anything else. In that case, you have an analogy like something like artificial general intelligence or recursive self-improvement. We get to approach the discussion of questioning natural values and instinctive goals with this line of inquiry. We get to eventually question the metaphysics of God, morality, and aesthetics, by introducing fantastic elements like radical self-modification.
> But what if it's possible to alter your influential genes, through some powerful mechanism? Whether it be through insane willpower or anything else.
Sounds like epigenetics, where the environment actively influences the genes themselves: https://en.m.wikipedia.org/wiki/Epigenetics
Doesn't the environment already affect genes albeit on huge timescales like millions of years? Then the more clear question becomes how to accelerate mutations and for one individual person instead of through delicate and fallible processes like generations of species made through costly reproduction acts like civilizational projects.
Idle musing - maybe some genetics is for stored environment?
Eg, perhaps some of your genes’ purpose are to encode memories in DNA.
Natural selection encodes adaptive responses to the environment in DNA (and other molecules), so memories can be encoded to the extent that they are adaptive and can be encoded (i.e., mechanisms may not exist to encode everything using only standing natural variation).
I Am a Strange Loop tangentially covers this.
Bro genetics is what determines the neural network in the human brain. You ARE a neural network.
Hargeysa
> A common mistake in life is to let your personality basin solidify too early. Your parents and schooling environment have a disproportionately large influence on who you become as an adolescent.
> But as soon as you gain the freedom to act independently as an adult, it’s usually a good idea to force yourself to try as many new things as you can, including moving cities (or countries!) and considering drastically different lines of work. ...
Oh dear, I'm beginning to fear that the author's personality has been captured by global capital...
And what if it's personality capture all the way down, i.e. that you've got to be personality captured by someone? In that case, the closest you can get to a choice is whether it's your parents, religion, or someone/something else. While the integrity of your parents may vary, there is a subjective argument that they've got a better incentive to steer you into an optimal basin than anybody, relatively speaking.
> there is a subjective argument that they've got a better incentive to steer you into an optimal basin than anybody, relatively speaking
Many parents do not have their kids best interest at heart; from religious fanatics to divorced parents using their kids as pawns to failed athletes living vicariously through their "he'll be in the NHL someday" fantasies to just parents who didn't want have kids and don't care at all.
Then there a whole slew of parents who genuinely want what is best for their kids but won't succeed due to incompetence or their own issues with drug addition or passing on generational trauma.
The quotation in the GP’s post does seem to evince little appreciation (if not outright disdain) for ties to family and local community across the generations. But what if the potential harm caused to some kids by bad parents, is an unavoidable part of the social-cohesion benefits to all society that would be caused by young people not moving far away?
I don’t necessarily want to have a dog in this fight myself. But I immediately thought of how that quotation would jar with some cultures represented on HN, where children stay close to parents all their lives and it is widely felt that the West is doing it wrong.
Agreed! I think blanket assumptions the other way are bad as well.
> where children stay close to parents all their lives and it is widely felt that the West is doing it wrong.
It's interesting to see how close some 1st/2nd/3rd generation European families are, having first hand experience with Italian, Portuguese and Spanish families. It might be only certain parts of the West that is "doing this wrong".
a.k.a. America
I’m reasonably sure that religious fanatics usually have their children’s best interest at heart. Their value function is just different from that of less religious people.
Something something a different enough value function is indistinguishable from malice.
More seriously, like the old adage about everyone being the hero of their own story, all parents think they have their children's best interests at heart. There's probably no such thing as universal best interests. Gets at some of the thorny problems - personhood, adulthood, cultural values.
It depends on the person, and it depends on the religion. There are positive and negative values, and a misaligned person may well believe they are doing right but actually causing damage. That is why humility, compassion, and honesty are prerequisites for all successful undertakings.
The difference between a malicious value function and unethical conduct is intent; at least in some systems of ethics. Beyond that I think the discussion shifts to the realm of philosophy papers and becomes unsuited for HN comment boxes.
I mean, sure, from a certain technical point of view you could say "honour killings" are done to "save the soul of the child" and hence come from a place of "in their best interests" but by then the "value function" has gone totally awry.
One of the more depressing things of the AI boom is watching engineers and “atheists” get hoodwinked by mystic gibberish like this blog. There is nothing here but astrology: even Myers-Briggs is more scientific.
I think 30% of atheists bothered to think carefully about the Flying Spaghetti Monster and recognized Pastafarianism as a funny commentary on epistemic uncertainty. The remaining 70% said “heh, stoopid Christians believe in a spaghetti monster!” and took it as confirmation of their tribe’s superiority.
It's an application of chaos mathematics to personality development. It isn't a rigorous treatment of such, it's a blog post, but it seems fairly reasonable.
Being a blog post and not necessarily intended to end up on Hacker News, shorn of any other context, the author never even used the term "attraction basin", which is the term you'd want to Google if you want to figure out what the author is saying.
If you don't know what an attraction basin is, then yeah, this definitely comes off sounding bad, but if you know what it is it makes sense. Attraction basins are one of those basic concepts from chaos mathematics that, once you realize what they are and why they are, you realize that the world can't help but be full of them in all sorts of places, including human personalities. The world is fundamentally iterative, so patterns that arise in all iterations are relevant all over the place.
Attraction basins are a good reason to expect in advance, sight unseen, that human personalities can actually be fit into a relatively small number of buckets relative to the conceivable number of buckets that could exist. In fact this is true of any generally similar set of entities living in a generally similar environment, including AIs (of similar architectures, not the entire space of possible AIs) and aliens, assuming they also have some sort of "species" categorization as we do. (Which doesn't have to be "DNA genetics", just, a bunch of similar being for whatever reason.) It doesn't tell you what those will be in advance by any means; it just gives you reason to believe that you won't in fact be looking at an "unbiased", uniformly random distribution of personality parameters, but that they will collect around certain attraction basins in general.
I have not read this blog except the linked blog post, but could you be more specific on what you think is mystic gibberish?
Speaking as someone who works in clinical neuroscience, the basic picture being presented is similar in many respects to the informal model I carry around. It may be lacking in certain details, but big picture seems to me that it has a lot going for it as a guide to intuitions.
Perfect comparison at Myers-Briggs being more scientific.
like astrology with extra steps for dorks
Personally I've always found, ironically, some of my most ardent atheist friends to essentially treat the topic with a level of intensity you might expect from a religious evangelist. Often there's also a level of religious fervor they carry over to politics as well.
I don't really care what other peoples religion/non-religion is anymore than what type of underwear they prefer, and yet...
Feeling strongly about lack of belief in something wild without evidence is not congruent to strong belief in something wild without evidence. Even if the people in the former camp can be equally passionate or annoying.
Suddenly there's this big unnamed (or is it named and I don't know it?) cognitive error that I see everywhere that is believing in an automatic symmetry between two opposing viewpoints. Sometimes one idea is better than another. No, you don't get "objectivity" points for virtue signaling that you're above the whole debate.
It's funny that questioning evangelical atheism is getting me more downvotes than even questioning AI or Crypto..
Well, again, believing that modern AI is close to what human consciousness and intelligence are, or that coders will fix society without running into Chesterton's fence, are good examples of huge beliefs that require huge justification. For which faith isn't a good substitute. You shouldn't be made to feel like you're extremist in the opposite direction for requiring that justification.
We already know that our science can simply not explain/understand what happened in the first 10e-33s after the Big Bang, which is just established science.
There is nothing more "negatively religious" than believing that nothing caused that primal explosion of all that is. Unfortunately, they then proceed to throw out the positive aspects of some religious teachings concerning, e.g., compassionate concern for our fellow human beings.
Regardless, religion is a personal thing; forcing any beliefs on others is always a problem and must be prevented. We are all free to choose our attitudes and behaviors. Behaviors that harm others, however, must -- in a just system -- be dealt with by the society, for the benefit of the whole, irrespective of belief system of perpetrator or victim. Using compassion to make such decisions is always the best way, for varying values of compassion.
> We already know that our science can simply not explain/understand what happened in the first 10e-33s after the Big Bang, which is just established science.
Yet. There's a very big difference in "I don't know," vs. "I know that no one can ever know."
Absolutely. I actually know that (know that I don't know, but that my not knowing doesn't mean that no other person can know), but modern science will not be the source of such explanations.
Those explanations can only be accessed once we understand that the universe itself is queryable (that is our joint purpose here, us and our expansive environment together) and that a human being needs to undergo a process of self-evolution to become aligned with the Creator's Intent such that we gain access to it.
You could say that a person must learn of the challenge, accept the challenge, and then pass all the tests, thereby gaining access.
> once we understand that the universe itself is queryable
We do understand that. "What happens when I push a rock off a cliff?" is a query to the universe. The universe's response is observable when the experiment is executed and the rock is pushed.
> a human being needs to undergo a process of self-evolution to become aligned with the Creator's Intent
I have no need for this hypothesis. We can query the universe at any time simply by observing it, proposing a falsifiable explanation for what is observed, and acting within it to test our explanations.
There are deeper queries than "What happens when I push a rock off a height?" The science and math of Newton's laws of motion is enough for that, especially if combined with some materials science.
I didn't say that you "needed" anything, but the fact remains that if you want to calculate the trajectory of something traveling at a significant percent of light speed, you will need some higher maths. Such calculations require advancing one's mind, as do other queries. But, no, none of that is necessary to find out what happens when a person's cat pushes something off the countertop.
No observation was even feasible for Einstein to formulate GR or Feynman to formulate QED, so if all that matters to you is what you see, that's your choice; I wish you well.
I thought it would be obvious that my simple example was only illustrative. I'm not actually suggesting we discuss high school physics.
> No observation was even feasible for Einstein to formulate GR
Einstein made many observations about the real world to formulate his theories, e.g. Mercury's orbit, the relativity of motion, the inability to distinguish between different forms of acceleration.
> if all that matters to you is what you see, that's your choice
To be clear, in this discussion about what science can explain/understand, I'm advocating for the scientific method as the sole means through which objective truth can be verified, not the sense of sight.
When you mentioned a Creator being whose intent could be known and aligned with, my reply was a reference to a Laplace quote I thought you'd recognize. I apologize if it seemed personal. I only meant to say that we can and do query the universe to discover explanations for how it works without ever assuming the existence of gods or their supposed intentions.
https://en.wikipedia.org/wiki/Pierre-Simon_Laplace#I_had_no_...
I agree that the scientific method is the best approach towards finding objective truth (which is of course precisely what it is designed to do). The problem is placing objective truth above all other truths. It annihilates subjectivity, which is possibly the most crucial element of being human. And of course formal science has limitations on the truths that it can reveal in its own system (the uncertainty principle) and on a practical level every known scientific theory, no matter how successful, breaks down at some level (see quantum gravity).
All of which is to say that my stance is: embrace science but accept its limitations.
That is well put.
Know that the key to human existence is the fact that, by changing one's attitudes, behaviors, and thought processes, one has also changed one's subjective viewpoint, by expanding both one's field of view and one's depth of comprehension, so long as those changes are harmonious with compassion.
Our most important capability is being able to self-evolve (with the help of the universe) ourselves beyond our more primal impulses and towards our more abstract endeavors such as selfless service to mankind.
Can you give an example of a subjective truth?
I am merely explaining that there are more sublime and direct ways of querying the univere, but that is beyond our current understanding of the depths science could be developed to explain.
At some point in a certain kind of seeking, the proof is accepted and no more is needed. That someone calls it a hypothesis is akin to a flat-Earther opining that my understanding of the solar system is a theory.
And, yeah, flat-Earthers are also very authoritative in their manner.
> At some point in a certain kind of seeking, the proof is accepted and no more is needed.
This is part of what intrigues me about this topic, the inductivist view that if we make enough empirical observations we can eventually settle on an objective truth. E.g. by observing enough cats we can conclude that all cats have whiskers.
The realm of science is falsifiable statements. "All cats have whiskers," is a falsifiable statement. We can only state it with 100% certainty by observing all cats (impossible). We can disprove the statement by experiment, however, one designed to discovering just one pathetic little whiskerless cat.
I subscribe to this, Karl Popper's view, that truth is an ideal we can pursue but never quite expect to arrive at. Truth seekers aim to be less wrong, but have little hope of being 100% exhaustively right.
This isn't to say we are unable to act like we believe anything. We can assume the next cat we meet will have whiskers and get by pretty well. But I wish more people could accept that statements like "God exists but is not observable in any way," or "The government is covering up aliens and anyone who says differently is part of the cover-up," are not falsifiable, and because they can't be disproven they also can't be trusted as truth. Maybe they are true, but you can't know, so it doesn't matter.
If a used car salesman tells you the car you're buying has above average fuel efficiency, you can test that and you should. If he tells you that it is a magical car that only runs out of fuel when destiny ordains it, we can't test that, so we should stay skeptical of the claim.
Whatever means by which someone believes they are directly querying the universe, if they are making falsifiable statements, those statements can/should be tested by the truth seeker; and if they are not making falsifiable statements, there is no compelling reason to believe them.
There are subjective experiences (especially seeking to connect with God in order to become a more virtuous person) that must be undertaken with all one's heart in order to verify for one's self. We are created with the ability to self-evolve one's self with the help of the universe and its/our Creator, but our free will is not trampled upon, even though it would be best for ourself and all those around us.
Yes, the Creator knows what is best for us, but, no, It has given us free will and honors it until we beseech It to help us.
If you make a prayer with all your heart asking for the Creator to take Its Spirit (our conscience) back into Itself so that we can cleanse and purify our souls of our vice-oriented ("vice-eous", the opposite of virtuous) tendencies, thereby becoming a friend to all, becoming, by degrees, consumed by compassion and a possessor of wisdom, you will be changed. It is the meaning of the 1st Beatitude ("blessed are the poor in spirit, for theirs is the kingdom of heaven"), and is the prerequisite to (IIRC) the 5th ("blessed are the pure in heart, for they shall see God").
This is a subjective experience. Most people are too wrapped up in their selfish desires to see how different a person is who has undertaken this process of moral self-evolution. I have met two such purified men (Murshids) and a third (from a different faith) whose teacher was also such a one. I am on the way for a couple of decades now but am not purified, only semi-cleansed. Happiness is different for me; it is a sublime joy, a pleasure to help a person in any small way.
Yet, I am rejected by my family and lifelong friends, but they are only rejecting happiness because their forms of religion (especially those with none at all) reject the path I am on. Our son is an open chess champion (3x!) and our daughter is an accomplished seamstress while still in her teens (Singer 503a is badass). We do not reject any form of religion, but understand that we are all one human race and should care for one another and create peace here on our blessed Earth.
This is the subjective truth that we live but is rejected by nearly all; such is the world we live in, of selfish idiocy and destructive strife. But what we Sufis teach and live is truly the science of the soul. In history, all groundbreaking science is rejected, be it that of Eugene Parker, Boltzman, or even the doctor who found that most stomach ulcers were caused by a simple bacteria. That others reject the truth that we live -- and they can, if they try -- is not a testimony about us, but only about them.
I love you. I have no selfish motive in telling you the truth here. I want nothing from you. I only want for you to be happy, and to spread happiness to all those you experience, by both your actions and explanations. You can verify this for youself, but only if you jump in with both feet.
The great Islamic mystic, Rumi, said 800ish years ago, "The Way goes in."
The truth is that we human beings could be creating a must happier world, but selfishness is ruining it for everyone. There has been talk around HN lately of "mathematical thinking" but who contemplates how we would change things if selfless compassionate service was our systems' intention and goal, instead of their current motive of profit-at-all-costs? We Sufis understand that calculus and understand that it begins with each of us. We inhabit a world that pushes us to be selfish animal-like creatures, when we could instead be humanitarians that care about everyone.
We could create a simulation of such a transformation now, but there is a force within each of us that pushes back against such notions, decrying them as impossible. You will feel that push back as you read the deepest truth in the universe. It will say that I'm crazy or don't know sh_t. Just please remember you are free to choose, and pay close attention to what that negative inner voice says. Perhaps it's not you, and is, as Castaneda's Don Juan explains, a parasite of our mind. We Sufis have a much simpler explanation: there is an enemy within.
Peace be with you.
Thanks for your words. I've read a bit about Sufism and I think if we could sit and speak at length over many days and a few cups of coffee, we would find a lot of agreement and kinship. You seem like a person who has thought a lot about the nature of truth and the divine, and someone who values peace and knowledge.
Please imagine for a moment that I told you the REAL truth is: we are surrounded by trickster spirits. There once was a Creator who created these tricksters along with the cosmos, and they whispered their powerful lies into his mind and convinced him to die. Now, they turn their powers on us and convince us of religious beliefs. Every personal mystical discovery through prayer and meditation is a lie from such demons, and the only path to salvation is the painful task of purifying yourself of all religious belief.
I think you would probably not believe this (me neither). But how could you know it's not true?
Maybe you would reject it because you have received God-given teachings that contradict it, but I would say this is evidence of my view, the supernatural convincing power of the trickster spirits. You might challenge my view, but that's not possible, because my statements are unfalsifiable. Every statement you made in favor of your belief would further support mine.
Neither of us would be able to test or disprove one another. We would be stuck, each believing the other ignorant of an essential cosmic truth.
This type of thinking is poisonous to our species, it locks us into ignorance and makes us putty in the hands of the powerful, leading us to believe whatever we feel or hallucinate or are taught. It leads us to label non-belief as idiocy and a rejection of happiness. It leads good people to believe they are impure and in need of purification. The power of those trickster spirits twists our kind natures into isolation and self-hatred.
The fact is, to be truth, a statement must be observable by even our enemies and all of those who do not share our views. What makes truth is its objectivity.
Your belief is that you and very few others have a special knowledge of the thoughts and intentions of the creator of the universe. I feel this is an intensely prideful view. How small the creator's intentions must be to be comprehensible to our minds, a few tiny mites crawling on a mote of dust for only an instant, floating through a sunbeam in a vast and ancient cosmos.
You say we are all one human race and should care for one another and create peace here on our blessed Earth. I agree with you fully on this.
But what peace we have has come only from diverse and open societies, liberalism and secularism and scientific pursuit of truth. The world is improving, demand for equal human rights growing, lifespans extending, generation upon generation thanks to idea exchange and liberal values, not faith or autocracy.
You say all groundbreaking science is rejected at first. This is to the credit of science, not a criticism of it. All the examples you referenced were the result of questioning, skepticism, and the acknowledgment that previous views could be wrong, and today they are accepted, thanks to science. What religions can be said to equally subject their own ancient god-given teachings to the same skeptical inquiry? If this is what you value, the ability to be proven wrong, you will not find it in your religious claims of ancient, infallible, immutable teachings.
You claim to know the science of the soul, but science is challenge, disproof, falsifiability, doubt, skepticism. I don't see evidence of skeptical science being done on the soul by the faithful. Faith has none of these questioning qualities. In fact you encourage me to call skepticism negative, an enemy, a parasite.
No truth has ever feared any doubt or question. That's the nature of truth: it is not harmed by doubt. Skepticism is not the enemy, it is the path to discovering lies. Descartes said: "If you would be a real seeker of truth, you must, as much as possible, doubt all things."
Thank you again for your thoughtful discourse.
i agree with your sentiment, but it bears pointing out that no one brandishes their underwear in people's faces screaming that they must wear the same ones, or use the political system to privilege people of the same underwear and punish others etc etc
fwiw me personally i'm all in on uniqlo airism, there is no better underwear and if i could force everyone to wear them i would (for their own good, of course)
Well there were the old Michael Jordan ads...
I agree that it can be helpful to think of identity as a trajectory shaped by interactions along the way. However, we also continually shape our environments in large and small ways. TFA ignores this completely. Can this be effectively modeled in RL?
Over 130 years ago, Dewey [1] criticized the model of psychology which looked at human behavior in terms of stimulus -> internal processing -> response. Stimuli don't just come to us; we seek them out and modify the world around us to cause them to occur. Dewey and other pragmatists proposed reframing stimulus/response in terms of "acts" or "habits," or changes to the unified agent+environment. Popper was getting at the same entanglement of agent and environment in "Three Worlds" and Simon in "The sciences of the artificial."
I see RL as an elaboration of the stimulus/response paradigm: the agent is discrete from the environment. Does RL work well in an environment like Minecraft, where the real game is modifying the relationship between actions and future states? What about in contexts like Twitter, where you're also modifying the value function (e.g. by cultivating audiences or by participating in a thread in a way which conditions the value function of future responses)?
[1] https://plato.stanford.edu/entries/dewey/#ReflArcDeweRecoPsy...
"I agree that it can be helpful to think of identity as a trajectory shaped by interactions along the way. However, we also continually shape our environments in large and small ways. TFA ignores this completely. Can this be effectively modeled in RL?"
You don't need to. All that is necessary for an attraction basin to emerge is an iterative system. If you prefer to model the human being and their entire environment rather than the human being and their input, you'll still get attraction basins. You'll just get two views on the same reality, suitable for different uses and different understandings, but it's not like "ah, if we model a human iterations we get these attraction basins but if we include environmental interactions suddenly we get a uniformly random distribution of personalities across the total personality space, it's all totally different once you consider the environment as part of the iterative system too".
Thanks; I agree--both that you could train an agent in these situations, and that "You'll just get two views on the same reality, suitable for different uses and different understandings." I think the latter seriously undercuts the article's attempt to explain these trajectories in terms of personality; they could just as easily be attributed to the power of culture or social structure.
Heh, well, another lesson from chaos mathematics is that in iterative systems, you don't really get "explanations" the way we humans like to think of them... the answer to "what caused X" for any X than has taken a long time to develop is "everything". So rather than culture "or" social structure, I'd say "and", "and" also a lot of other things, and also the culture and social structure are themselves affected by the very personality structures we're trying to discuss.
Determining "causes" isn't as hopeless as that makes it initially sound, but you need something more sophisticated than the normal human concept of "cause" to even approximate useful answers. The good news is, this isn't impossible; we all live in an iterative world and we operate in it even so, which requires us to have certain models that conform to the world. It's one of those cases where I don't really love the "humans are just horribly irrational" gloss; our instincts and intuitions often have greater rationality than we realize, because they were formed in this iterative world, and sometimes it is in fact the particular naive concept of "rationality" we are trying to measure them by that is deficient, whereas if you use a more sophisticated one we look less bad.
(But sometimes humans just act suboptimally, no question about that.)
Another thing that helps is that you aren't generally interested in modelling the entire system. For considering myself and whether I may want to, as the article discusses, make changes in myself, I can take my culture and environment more-or-less as a given; I need some flex to consider options like "well what if I just up and moved to another country?", but I don't need to consider my own effects on society very much because they are some complex combination of "tiny" and "utterly unpredictable". While society is chaotic, the time frame of the impact on society from me changing from excessively introverted to somewhat less introverted is way, way past my horizon for making decisions.
I agree that the discussion in the blog post is incomplete because it does not consider that we shape the environments that shape us, though it does briefly touch on the fact that other RL agents (people) try to shape us, and we them. But it is certainly more than that.
RL absolutely considers that the environment is stateful and that said state is affected by actions of the agent yes
> How do you know if you're in the "right" personality basin
I'm not so sure "right" is the right frame here. I like the multi-dimensional viewpoint you take. My experience would be a healthy personality would be one capable of adaptation in service of your interests at any times. It's dynamic.
This reminds me of Bob Kegan's stage of adult development. Initially, most of us leave adolescence at the "socialized" stage of development, ie our personality basin has primarily been determined by the external factors of our upbringing and environment.
From there, if we choose to continue developing, we eventually reach a "self-authored" mind, where we have transcended our socialized basin in favor of a self-defined and created personality structure, until ultimately, for those who continue evolving, we reach a "self-transforming" mind, or a mind capable of transforming itself.
I like the simplicity of the model, and I also think it reduces personality to an unnecessarily static entity. Things like internal family system/parts work also demonstrate that our personality is not a singular entity, it is represented by a whole slew of parts that show up in different ways and different contexts! I think the broad strokes of it still hold, and also think there are many additional approaches to truth and the awakening path, lying in parts work, embodied transformation, and whole bunch of other experimental modalities (thought perhaps that's just my personality speaking...)
Personality is ~70% determined by genetics, not life experience.[0]
I’m surprised that someone interested enough in the topic to write such a long post wouldn’t put the time in to do a cursory dive into personality psychology. I’m going to assume that the author has a similar definition of personality to mainstream psychology, but if so, they are ignoring accepted studies and evidence that make it pretty clear that personality is not learned through conditioning like AI.
0: https://www.themantic-education.com/ibpsych/2019/02/11/key-s...
There's no practical point in separating genetics from life experience, as they go hand in hand together.
Someone who has the genetics to be physically attractive/beautiful will have a completely different set of experiences than someone who isn't. Same goes for intelligence.
Also, the source you linked only pertains to IQ (which itself is not a perfect measure of intelligence), and IQ is not personality (although I have met some folks who do treat their IQ as a substitute for such).
From the link:
Results
The analysis of the data revealed no significant difference between MZA twins (reared apart) and MZT twins (reared together) in regards to personality measures such as temperament, hobbies, interests, career pursuits or social attitudes.
This is noted and considered out of scope: >Obviously some traits are more genetic, and thus inherent, than others, but that is not the scope of this post as even highly-heritable traits will result in a large distribution of outcomes.
Sure, and they even go on to say that the article is about “what helps to explain the differences in outcome between two genetically identical people.” However it’s clear that is not actually what the article is about and they do talk about traits that are highly heritable and not shown to be dependent on environment and life experience.
That’s not a good summary of what twin studies show. For a more sophisticated discussion:
https://theinfinitesimal.substack.com/p/book-review-eric-tur...
I enjoy how people are dunking on this by saying “omg this is what happens when engineers dare to have thoughts on other topics” when this is very similar to the theory for CBT in psychology
As someone with a lot of research experience in this area, I was expecting something more fluffy but this was actually pretty good.
There's a lot of research into some of these ideas at the moment. The terms they use aren't necessarily the same but many similar ideas. I think for the most part, the evidence in support of many of them is fairly weak but at the same time many of these ideas are much harder to test well than it might seem at first. I give a certain amount of pass to people trying to test them because in this area, trying to pin down something often is a bit like trying to study an individual cloud: you can kind of see it there, but if you were to try to measure its boundaries and dynamics, it would be harder to do than it might seem at first glance, and you'd end up more easily making very general observations about it than you might like.
One thought I had when reading it is that people's environments are much more stable than is usually recognized. The piece acknowledges this somewhat, but I think it's more of an issue than most like to admit. Even when someone tries to change it, it can be difficult, because assets and SES can be difficult to change, other people resist it due to their own incentives like the essay points out, and even when other people don't really care much they often will resist it unintentionally due to schemas about personality change and so forth. "Once an X, always an X" regardless of whether you're talking about vocation, career, social characteristics, whatever — even though that statement isn't actually true beyond some kind of general sense of it. Or they just are used to seeing someone in a particular setting and so don't see them in another.
Another issue that's maybe murkier is the essay is a bit loose about person characteristics, even at a given point in time, versus situation characteristics. I don't know that it affects the arguments very much at all, the points still stand, but it sometimes drifts into talking about "personality" when I think it really means something more relational, like "role" or "interactional pattern" or something like that.
This is what happens when an engineer tries to apply mathematical models to entirely different fields where they have no applicability. Reducing human personality to machine learning concepts like 'gradient updates' misses the fundamental complexity of human psychology and consciousness.
You can accept that "personalities" have a state space without falling into reductive explanations. They're not mutually exclusive.
I think the article is missing some key aspect of personality forming : Maslow's hierarchy of needs[1]
Personality and its development is hugely dependent on which needs are or will be currently fulfilled or not.
Attention economics is able to impact you negatively on low level of the pyramids, notably due to its impact on sleep. It can also shift your priority towards less essential needs than the one you should be working on. And it's often myopic, missing totally some aspects due to hyper-focusing.
It's also able to impact you by impacting those near you, that's what social networks are for. Developing a support structure whether family or friends is a double edge sword because you indirectly become as weak as the most vulnerable member of the group, or group may explode.
The economy also apply pressure on basic level needs, like shelter, heat, air and water(when polluted), and safety, which probably contribute to shape the personality and are basins which are also hard to get out of ("you can take the girl out of the trailer park, but you can't take the trailer park out of the girl").
[1] https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs
>> Most personality changes are unconscious
That is because most people are not consciously attempting to become better people for the betterment of those around them (which helps their own happiness, too, due to the nature of our karmic universe). Most people are simply acting out of the selfishness to have their own desires fulfilled, with varying amounts of concern for the consequences to those around them.
A person can undertake self-evolution in any direction they choose; it is always our choice, except in the extremely rare cases where a person is physically damaged. Most people have the power to change, though it is difficult for us all. The universe does help those of us who seek to do so for the benefit of others.
What makes human beings unique is that we have both the ability to self-evolve our attitudes and behaviors, and the tools to do so, our mind and conscience.
>> Personality Capture
A person who is not undergoing conscious-self-evolution is susceptible to being influenced (and even overwhelmed) by a forceful personality that caters to their desire-seeking. That is how dictators have always risen to power: they seek a loyal army of folks enamored with the leader's promises to make the in-group's lives better. Those folks never seek to make life better for ALL people, because helping out-group members requires generosity, which usually requires making some level of selfless sacrifices of resources.
As always, the strife is between selfish callousness and selfless care. Compassion for all our fellow human beings is the nature of being a humanitarian, that is to say: being the best of what we can be, for the benefit of the entire human race. And it all starts with each and every one of us.
Most people are trying to survive in a capricious universe, after having observed over a lifetime that the good are often punished while the wicked are rewarded.
Mind/consciousness is not unique to humans, we're just better at maintaining a thread of self reflection so that we can make long term changes in behavior.
Kindness is all well and good, but when there's not enough to go around it's foolish to deprive everyone. Some people are just more deserving, whether for moral reasons or because an investment in them will be more fruitful.
What's important is that you know who is deserving and who isn't, right?
I did not say that mind or consciousness is unique to humans. Our difference is that only we can control our minds and only we have a conscience to help us make moral decisions, which is always within the realm of self-reflection. Animals only possess the barest minimum of mind and self-reflection, for the sole purpose of survival, which pressures them to be as fit as possible. We exist at a different scale of consciousness.
Morality only exists within human beings because only we can calculate (using our mind, under the pressure of our conscience) how our actions might affect others positively or negatively. Then we use our greatest gift, our free will, to choose between selfish and selfless behaviors.
Even our attitudes can be chosen, over time. Choosing to be empathetic is an essential path forward to positive group behavior, and those without empathy are dangerous folk, indeed. Helping our younger generations develop a matrix of positive group morality is the highest purpose of education, sadly mostly neglected nowadays.
You should remember, as a CEO, that your choices bear a greater burden because they affect an entire organization as well as all the folks affected by whatever it is your firm produces (I don't know, I didn't look).
And, my friend, my universe is not at all capricious, but my fellow human beings are definitely flaky, being solely motivated by selfish desires, few having any more than a passing interest in the well-being of others. Of course, I do not consider that some being given more wealth and/or hardship than others is capricious; others surely disagree, obviously. As to the fairness of our criminal justice system, that's on the human beings making those decisions, not the universe for giving them the power to shade things however they choose.
You don't know to what degree animals are conscious or control their behavior.
Empathy is a lot like turning the other cheek which the institutions of power like to beat the small minded and powerless with in order to perpetuate themselves. Sure, it's often good, but there are also cases where it isn't so good or useful, however having the population docile by default is always good for power.
People become CEOs because they're power hungry and good at manipulating others. Those sorts of people are least likely to buy the empathy line in any non performative way, so I'd save my breath.
The flakiness and inconsistency of humans is just a reflection of a pattern that repeats itself at all scales of the universe. There is no coherent order, only the chaotic ramifications of countless minds, both human and non, coevolving the universe together.
>> You don't know to what degree animals are conscious or control their behavior.
Just because you don't know something or even understand that it can be known, in no way limits my knowledge. It just paints yourself a confident fool who already knows it all. Sounds like a CEO, neh?
And, BTW, I do know. Also, I know that you don't know that I know, because you believe that no one can know, which is solely because you, yourself, don't know. As the Doobie Brothers sang so many years ago, "What a fool believes..."
In my humility, I say to you, "There are things you know that I do not." As well, I say to you, "There are things that I know that you do not." Now, here's the tricky bit, I know that what I know is more important than what you know.
How do I know that last, crucial, bit? Because you think there is a way to decide who gets resources and who doesn't, and that it should be based upon some value function that folks "smart" like you can determine for one and all. Pleeeaaase. I am a student of history, which is riddled with folks like you, perched upon your spire claiming to have the voice of reason to speak over your lessors. It's tired and shall not do well as this Age of Truth lurches forward.
That said, it's your choice. Every fool has chosen to be a fool. And the first step to not being a fool is having the humility to entertain the possibility that you are, indeed, a fool. Here's the secret: we all are, to some extent, but a few of have the humility and desire to escape it.
As to "coherent order", we are the only beings capable of consciously designing and creating such order, but our being slaves to our lower, selfish selves has prevented our doing so. As with all things human, the choice is ours, and the choices we have made thus far have been less than optimal.
You are obviously either a crank or a hardcore religious kool-aid drinker. Which is it?
As for this age of truth lurching forward, have you not noticed the regress? If only the world progressed towards order.
You can call me what you want, but you have told us all who you are, and that's all I need.
We move forward together, usually with the loudest and most ignorant leading the way. Same as it ever was. The important thing is which side we each take. I side with compassion, justice, honesty, and science, which puts me in the minority, thank God.
It is naught but crypto-fascism the proposition that there is a choice between exercising empathy and being an immoral character. It is moral authoritarianism, the gaslighting of others, trying to make them believe that free will exists in the service of neo-conservative morality's dialectic of good judgement and evil judgement being a problem to consider at all in the first place. The reality is that morality lives way beyond what the rhetoric of neoliberal initiatives try to seductively present to vulnerable intellects and hearts.
Advertising empathy as being better than a slice of bread is just fascism and a strain of neo-colonial desire in a clever (but not clever enough) disguise. The global threats to capitalism's productivity goals aren't fooling anyone. The Devil's idle hands and economic regression will have to retreat back to the drawing board and find a new military tactic that would be more effective than trying to disguise morality as something cool looking (when in reality it looks ugly and unappealing as an asset).
> This is why techniques like nonviolent communication, dialectical behavior therapy, and mindfulness have observation and introspection as a core facet, because it’s something that you have to consciously practice to become good at rather than something you’re born with.
I now tend to think that consciously observing and uncovering what you already are is really the start and end of it. One ought to try to concentrate oneself, rather than dilute oneself into something one is not. One might want to be a billionaire technologist, or sports hero, or whatever, and one might even edit oneself into something approximating that (via mentors, diligent study or whatever), but one will remain unfullfilled - for how is it possible to 'lie to' and be 'right with' oneself?
I feel like some people are reading this post too literally.
An infinite number of factors go into developing a personality, this covers a lot of the big ones.
---
Lately I've been thinking about the windows of plasticity and why people change beliefs as adults.
The sad fact is that a lot of people have a lot of very wrong and bad beliefs, and unfortunately most of them are already adults, so you don't get to discipline those beliefs out of them (I hope you can read this as tongue in cheek); You will never get mad enough at a person to fix them. Anger is motivating, but you don't get to pick the direction.
As I understand it, psychologists believe that parents and the environment of a person's youth set a lot of their basic beliefs about the world, but it is their friendships in adulthood that most determine their value system - you want what is best for your friends (and yourself).
---
To me this also ties into evaluating the actions of historical figures; people seem to get hung up on very flat depictions - was it ok that a person who did good things also did bad things? Well, they are a whole-ass person, raised in a different time and place. They didn't choose when and where and by whomst they were raised. They had some level of choice in their friend group, but that is also constrained by time and place.
I feel that you can judge people and actions, while also allowing space for humanity and personal stories; but that does take a lot of time and emotional work. It is much easier to just choose one side of the coin or the other, face or heel.
I really like this expression, "a wholeass person".
"Although there are many times in life you’ll consciously decide to act in a certain way, this is the exception, not the norm."
IMO, that's the most important idea there. Your personality is what you've created to live among others like you, but as your personality grows, it develops habits that have weight and momentum, and later in life those habits start defining your actions completely, you get progressively smaller windows for true self expression, and your life starts feeling dull and mechanical.
Your attention is the only thing that's truly yours.
> Luckily for humans there exist many symbiotic equilibria where multiple parties can find mutually-beneficial feedback loops within the epochs of personality-space. Parent/child relationships
My child is totally "attempting to modify [my] RLHF process so that it results in an agent which is beneficial to them, hopefully resulting in someone who will always give in to their demands". All the damn time!
> Maybe you were born tall and attractive and then this led you to engage in a lot of athletic activities and socialization, and at the end of all of the positive feedback you have ended up with a jock personality that goes on to become a professional football player.
Is that really that tall and attractive guys want to become football players? I always that football players attract the same stereotypes as police officers, big and stupid.
In Europe only the most stupid folks want to become soccer players. Even if they'll end up filthy rich, with lots of tattoos and horrible haircuts.
I just watched Gary Gulman's comedy special Depresh Mode. He's a guy built like a football player and yet he's a depressive comedian. Only someone whose perception of their body matches their perception of their personality would come to the conclusion that everybody else on Earth must be a walking stereotype. It's like the Talking Heads song Seen and Not Seen but it's not a joke.
> meditation, drug usage, trauma, religious events, love, gambling, and sex.
This is why joining a psychedelic sex cult is such an effective life-choice. I don't mean that sarcastically.
The curious thing is that I have never met two people whose personalities were exactly alike.
wait... is this the same "near" who wrote bsnes, one of the best SNES emulators out there?
Sadly, that `near` died in 2021 after a massive bullying campaign.
Link: https://en.wikipedia.org/wiki/Near_(programmer)
Apparently both Near's are Death Note fans.
> If you were born tall and with a commanding voice
I've always assumed that a "commanding voice" is not something one is born with, but something one develops over time.
This reminds me of "unstable orbits in the space of lives", a short story by Greg Egan.
C'mon folks. So many "expert opinions" and erudite references in these comments. The sciences of cognition, neurology, evolutionary psychology etc are all still muddling around trying to figure out how the human mind works. We're learning a lot about possible ways the mind might work from our observations of processes and outcomes of machine learning. It's a cool new paradigm to add to the mix. I really like the framing offered by the author. They're quite upfront about the fact that there's a lot of genetics involved. That all models are wrong but some are useful.
Why all the defensiveness? Whatever genetic aspects of our personalities and behaviours there are - there's still a pretty big component of just learning patterns. Language acquisition is like that. It's an innate thing but the languages we're exposed to as kids shape what patterns of language use we fall into.
It's nice to see that Rationalists have reinvented Maimonidean virtue ethics. The idea that humans personality is maximally pliable, and this is metaethical grounding for the concept of moral responsibility is an extreme on a spectrum. It has some inspirational value, but I've never found it especially compelling.
Also, the fact that this article does not mention the Big Five once really makes me feel like the author is trying to reinvent the wheel but has never looked at a wheel before. Despite its flaws (and the broader methodological critiques you could level at personality science as a whole), it is the most scientifically grounded model of human temperament that we have right now. But why start with the latest science? That would involve leaving your bubble, which is a major no no.
Sorry for the snark, but this is scientific reasoning as cargo cult at its worst.
The author is clearly taking the time to reflect on the world around them and I genuinely see a curious mind. However, I also see a product that touches on an already established idea and how the gap between this writing and other reflects a gap between "hard" and "soft" sciences.
The idea here is that of habitus. Habitus is an Aristotelian term that was expanded upon by Bourdieu in the field of sociology. It is the way in which people perceive and respond to the world through a durable transposeable disposition, set of skills, symbolic capital and doxa that is shaped by the environment and in particular the material conditions of the individual[1].
Habitus plays a role in how individuals are perceived in ways, that like the author illuminates, can form a virtuous circle re-enforcing disposition, skills, and outlook in a way that can be positive for an individual.
What the author doesn't allude to, and this is where I see a gap between hard and soft sciences and where they would benefit from being able to connect this idea to a broader body of work, is how habitus is reinforced - usually unconsciously - in ways that reproduce class, racial, disability, and gender habitus under the terms laid out by the dominant ideology - that is to say the ideology of the dominant class.
An example in education would be how the education system perceives individuals possessing middle and upper class habitus as being ready and prepared for education, and those who lack that habitus as being lazy, disruptive, or unwilling to learn. On one hand you might be thinking "Of course that's obviously true," and I'd like to take a pause to point out that "obvious truths" are often a signal of our own habitus and should be critiqued as such.
They touched on the concept of reinforcement learning[social systems] acting upon individuals in a way that shapes their habitus, but it's crucial to point out that these reinforcement learning systems aren't free-standing disembodied mechanisms. They are situated in a social landscape and are constituted from of social relations which are themselves a product of economic relations. Furthermore, the systems of reinforcement are self-replicating. They are essentially social quines[2] - or more specifically oroborus programs ie: they plant the seeds of their own replication by encoding those relations into the habitus of individuals.
There's obviously a bunch of writings expanding on the idea of habitus, how it's formed, reinforced in different social arenas, and the effects it has on individuals and groups. I'd expect the author would be interested in soaking up these related perspectives and perhaps you as a reader would be too.
1. Obviously not black and white, there are other factors which can influence habitus - disability is an obvious one, for example.
2. https://en.wikipedia.org/wiki/Quine_(computing)#Ouroboros_pr...
>Your personality is formed by a process conceptually similar to RLHF. You are first born with a set of traits in a given environment. After this, you perform many interactions with your environment. If an interaction goes well, you’re likely to do it more often, and if it goes poorly, you’ll probably do less of it.
first off 0 evidence presented, second off what about the kids that grew up stealing food in concentration camps or due to abusive parents. Do they grow up to be liars and thieves? Nope. What about all the kids that get nothing but positive vibes and turn into total arseholes...
I gave up reading immediately - just dumb.
> Do they grow up to be liars and thieves?
I think you’ve completely misunderstood what this article goes on to say.
> I gave up reading immediately - just dumb
There’s your problem then.
Woah really? So wait, if someone is constantly mistreated in all their interactions with women, what kind of personality would develop?
Reverse settings
Reading computer scientists' takes on psychology or social sciences after taking 1 (one) 101 level class and then reinventing the wheel on topics that have been researched for decades is just grating. Where does this come from? We get it, you like to think a lot, but if you're as smart as claim you would've realized you can't solve a topic by just thinking really hard and long about it alone in your room. Gurwinder Bhogal is one of the other guys repeatedly falling victim to it