Rendered at 19:57:23 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
trimbo 2 days ago [-]
> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong.
Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise.
Obviously subservient people default to being yes-men because of the power structure. No one wants to question the boss too strongly.
Or how about the example of a close friend in a relationship or making a career choice that's terrible for them? It can be very hard to tell a friend something like this, even when asked directly if it is a bad choice. Potentially sacrificing the friendship might not seem worth trying to change their mind.
IME, LLMs will shoot holes in your ideas and it will efficiently do so. All you need to do ask it directly. I have little doubt that it outperforms most people with some sort of friendship, relationship or employment structure asked the same question. It would be nice to see that studied, not against reddit commenters who already self-selected into answering "AITA".
redanddead 2 days ago [-]
Reddit is notorious for being awful at real life interactions
just look at the relationship subreddit the first answer is always divorce, it’s become a meme
but beyond romantic relationships, i think a lot of us have seen how it can impact work relationships, i’ve had venture partners clearly rely on AI (robotic email responses and even SMS) and that warped their perception and made it harder to connect. It signals laziness and a lack of emotional intelligence
AI should enhance and enable connection, not promote isolation, imo this is a real problem
it should spark curiosity, create openings for conversations, point out the biases to make us better at connecting with other people, i hope we get to a point where most people are made kinder by ai. I’m seeing the opposite atm, interested in hearing others experiences with this
jojomodding 1 days ago [-]
One of the reasons relationship advice subreddits suggest divorce so often is because most people with "small" problems in their relationships don't write an essay about it on Reddit but are able to solve them with the tools/friends they have. So a Reddit post existing indicates the relationship has serious flaws.
This is not to defend the study, because asking AI has a lower barrier to entry.
gunsle 1 days ago [-]
No, a Reddit post indicates whoever posted is fishing for large scale validation from internet strangers. Their relationship may or may not even exist. Most of the posts are pretty obviously fake. Just like 90% of interactions in general on Reddit these days. That site should be taken out back and put out of its misery.
coffe2mug 14 hours ago [-]
But nothing wrong about that. Some decisions are to made in a hunch. Even a therapist doesn't suggest to divorce or not. Victims are often not in a state to decide. They need support (as myself).
Humans often don't help. They often suggest - everyone goes through pain. It is part of life blah blah.
dessimus 11 hours ago [-]
>That site should be taken out back and put out of its misery.
With it gone, a large portion of it's users would come here, reducing the signal-to-noise ratio of HN.
kelvinjps10 2 days ago [-]
I always find it interesting how, in Reddit any trivial fight or even just different opinions, the advice it's always to end the relationship.
fortmeier 2 days ago [-]
I think it is some kind of survivership bias. Who is going to give advise on reddit?
Maybe people shying away from difficult social interactions?
somenameforme 2 days ago [-]
Paired with the echo chamber effect voting systems create. Anything that affirms the biases of a majority of upvoters gets elevated, anything that contradicts it gets hidden, and so you not infrequently end up with ubiquitous nonsense that then further reinforces the echo chamber as they become self assured. Then real life intervenes, completely goes against the online zeitgeist, and they're all confused.
DeathArrow 1 days ago [-]
If you want to get poor fast, you can follow the most up voted advices on r/wallstreetbets.
ahartmetz 1 days ago [-]
Would be interesting to have data on that. If it was true, you could win by always doing the opposite!
tyre 1 days ago [-]
Not necessarily. WSB users are trying to make it big, which means betting on long shots. This could be penny stocks, companies on the verge of bankruptcy, or ones with more sentimental value than fundamentals.
Betting against these companies is obvious and expected, so the cost of shorting might be high enough that even if you’re correct (stock goes down, the opposition of what WSB said), paying the cost of the short (the fee to borrow the stock from someone else) is high enough that you still lose money.
Also:
1. shorting stocks can be quite dangerous. Your downside is, well, not infinite but it can easily wipe you out.
2. You might be correct that the stock goes down, but over what time frame? Again, you have to pay money to hold a short. Or you’re using a different financial instrument that has a specific timeline. If the market does move in your direction but too late, you still lose.
sean2 21 hours ago [-]
The truth was so obvious I didn't bother to find data before doing the opposite: most of their posts are: "I'm going/went all-in, high-leverage on this moonshot! And...its gone." I've successfully applied the opposite approach and invested safe amounts in a broad portfolio and it is going pretty well (or was before this whole Iran thing).
DeathArrow 1 days ago [-]
Just because one action is demonstrably harmful does not mean its negation is automatically beneficial.
Or, formally, my claim is A implies B. The only logical contrapositive is non B implies non A. (not losing money means not following advices on r/wallstreetbets)
But you say: non A implies non B, which is the fallacy of denying the antecedent.
diydsp 1 days ago [-]
It doesnt have to be universally true to be true in a mathematical system like options/puts/calls.
notahacker 1 days ago [-]
Those particular subreddits are heavily populated by incels voraciously consuming the stories of relationship strife (real, distorted and purely fictional) to validate their belief that their relationship status is down to the evils of the opposite sex specifically and all relationships being doomed in general.
The power of an echo chamber; makes extremism seem logical.
MaxBarraclough 1 days ago [-]
I think it's more than just an echo chamber, it's the community wanting drama more than it wants to help people.
roncesvalles 2 days ago [-]
I mean... it's a solution guaranteed to work in a trivial sense. It's not meant to be a serious suggestion but more of a thought experiment, like "hold this as the bar, can you find a solution better than this?"
It's like what GiveDirectly says: all charitable interventions should be benchmarked against simply giving the beneficiaries a wad of cash.
gunsle 1 days ago [-]
That would be because nearly all of those posts are entirely made up and somehow I guess people can’t tell that?
ModernMech 1 days ago [-]
> the advice it's always to end the relationship.
To be fair, if your interpersonal skills and relationship dynamic are such that you find yourself seriously asking the Internet (Reddit of all places) for relationship advice... yeah, just end it is probably the null hypothesis.
9rx 1 days ago [-]
Stands to reason. Ask a computer for advice and it is going to give you a computer-centric answer: Restart and try again.
abpavel 2 days ago [-]
You deserve better
yabutlivnWoods 2 days ago [-]
A very sycophantic AI style answer.
Code bot equivalent being all "you are absolutely right! Here is the unequivocal fix for now and all time!"
finghin 2 days ago [-]
No one would ever have made that comment before GPT. I have a feeling you could be the one that’s poisoned!
2 days ago [-]
ekidd 1 days ago [-]
> just look at the relationship subreddit the first answer is always divorce, it’s become a meme
As someone who has been married for a couple of decades, I, too, would recommend divorce to many of the (often-fictional) people asking Reddit for relationship advice. A marriage has a huge impact on whether your life is basically good, or if you pass a big chunk of your time on this Earth in misery. And many of the people (or repost bots) asking for advice on Reddit appear to be in shockingly awful relationships. Especially for people who don't have kids, if your marriage is making you miserable, leave.
(But aside from this, yeah, don't ask Reddit for relationship advice. Reddit posters are far more likely to be people who spend their life indoors posting on Reddit, and their default advice leans heavily towards "never interact with anyone, ever.")
indistinction 6 hours ago [-]
[dead]
finghin 2 days ago [-]
Yes, in principle this would be a great way to get a grip on AI personal-decision-making. But there’s a nontrivial chance Claude is more emotionally intelligent than r/AITA. That is not something I enjoy saying.
intended 1 days ago [-]
Eh, AITA works very well for the more common and obvious situations.
I wonder how MUCH better Claude really is when compared to AITA. Also people are mixing up relationship advice with AITA.
blablabla123 2 days ago [-]
I think this may be selection bias. People asking anonymously (edit: for relationship advice) on Reddit perhaps even with a throwaway account are likely in a desperate situation. So hardly to be compared with the _average_ real life situation. Thus 1. chances are running is a good option and 2. also considering even in 2026 AI still essentially is a statistical machine that doesn’t handle corner-cases at the tails well.
Anecdotally as I’ve thoroughly worked and used AI myself. It performs best with google-able stuff that is needle-in-the-haystick like and worst with personal and work advice. The main problem I see is that it’s tempting to use it for that.
trueno 2 days ago [-]
> worst with personal and work advice. The main problem I see is that it’s tempting to use it for that.
i think i want to expand on this even more. even people ive worked with for years that ive looked up to as brilliant people are starting to use it to conjure up organizational ideas and stuff. they're convinced, on the backs of their hard earned successes, that they're never going to be fallible to the pitfalls of... idk what to call it. AI sycophancy? idk. i guess to add to this, i'm just not sure AI should be referenced when it has anything to do with people. code? sure. people? idk. people are hard, all the internet and books claude or whatever ai is trained on simply doesnt encapsulate the many shades of gray that constitute a human and the absolute depth/breadth of any given human situation. there's just so many variables that aren't accounted for in current day ai stuff, it seems like such a dangerous tool to consult that is largely deleting important social fabrics and journeys people should be taking to learn how to navigate situations with others in personal lives and work lives.
what ive seen is claude in my workplace is kind of deleting the chance to push back. even smart people that are using claude and proudly tout only using it at arms length and otherwise have really sound principled engineering qualities or management reportoire are not accepting disagreement with their ideas as easily anymore. they just go back to claude and come back again with another iteration of their thing where they ironed out kinks with claude, and its just such a foot-on-the-gas at all times thing now that the dynamics of human interaction are changing.
but to step back, that temptation you talk about... most people in the world aren't having these important discussions about AI. it's less of a temptation and more of a human need---the need to feel heard, validated and right about something.
my friend took his life 3 months ago, we only found out after the police released his phone and personal belongings to his brother just how heavy his chatgpt usage was. many people in our communities are saying things like "he wouldve been cooked even without AI" and i just don't believe that. i think that's just the proverbial cope some are smoking to reconcile with these realities. because the truth is we like... straight up lost the ability to intervene in a meaningful way because of AI, it completely pushed us out of the equation because he clapped back with whatever chatgpt gave him when we were simply trying to get through to him. we got to see conversations he had with gpt that were followups to convos we had with him, ones where we went over and let him cry on our shoulders and we'd go home thinking we made some progress. only to wake up to a voicemail of him raging and yelling and lashing out with the very arguments that chatgpt was giving him. it got progressively worse and we knew something was really off, we exhausted every avenue we could to try and get him in specialized care. he was in the reserves so we got in contact with his commander and he was marched out of his house to do a one night stay at a VA spot, but we were too late. he had snapped at that point, he chucked the meds from that one overnight stay away the moment he was released. and the bpd1 snap of epic proportions that followed came with him nuking every known relationship he had in his life and once he was finally involuntarily admitted by his family (WA state joel law) and came back down to reality from the lithium meds or whatever... he simply could not reconcile with the amount of bridges he had burned. It only took him days for him to take his own life after he got to go home.
im still not processing any of that well at all. i keep kicking the can down the road and every time i think about it i freeze and my heart sinks. this guy felt more heard by an ai and the ai gave him a safer place to talk than with us and i dont even know where to begin to describe how terrible that makes me feel as a failure to him as a friend.
balamatom 5 hours ago [-]
(fuck this; dropping the throwaway.)
>my friend took his life 3 months ago, we only found out after the police released his phone and personal belongings to his brother just how heavy his chatgpt usage was. many people in our communities are saying things like "he wouldve been cooked even without AI" and i just don't believe that. i think that's just the proverbial cope some are smoking to reconcile with these realities.
This hurts to hear. I don't know if there are appropriate words to write here. Perhaps the point is that no, there aren't any. Please just know that I'm 100% with you about this.
Your community is not just smoking cope; it is punching down instead of up. That is probably close to the root of the issue already. But let's make things worse.
I can only hope that I am saying something worthwhile by relating the following perspective - which is similar to yours, but also, I guess, similar to your friend's...
AI is a weapon of epistemic abuse.
It does not prevent you from knowing things: it makes it pointless to know things (unless they are things about the AI, since between codegen and autoresearch it is considered as if positioned to "subsume all cognitive work"). It does not end lives - it steals them (someone should pipe up now, about how "not X, dash, Y" is an AI pattern; fuck that person in particular.) We're not even necessarily talking labor extraction. We are talking preclusion of meaning: if societal values are determined by network effects, and network effects are subverted by the intermediaries, so your idea of "what people like and what they abhor" changes every week, every day, every moment - how do you even know in which direction "better" is? And if you believe the pain only stops when you become the way others want you to be - even though they won't ever tell you what all that is supposed to about - how the fuck do you "get better"?
Like other techniques of assaulting the limbic system, it amounts to traceless torture.
You keep going, in circles, circles too big for you to ever confirm they are in fact circles, and you keep hoping, and coping, and you burn yourself out, and your thus vacated place at the feeder is taken by someone with less conscience and more obedience...
They say there exist other attractors in the universe besides the feeder. But every time one of us attempts to as much as scan the conceptual perimeter, the obedients treat us to the emotional equivalents of small electric shocks - negative reactions which don't hurt nearly as much as our awareness of their fundamental unfoundedness and injustice.
Simple example: let's say someone is made miserable by how they feel they are being treated. Should they be more accepting - or should they be standing up for themselves more? (Those are opposites; which you may be able to alternate them; but trying to do them simultaneously will just confuse and eventually rend apart the mind.)
Well, how about the others stop treating them badly? Why exactly can't they? Where does it say that we have to be cruel to each other? "Oh it's human nature, humans are natural jerks" - who sez?
Well, lots of places it says exactly that, but we read, comprehend, tick our tongues, and move on; nobody asks who wrote it. We all pretend that it is up to the sufferer to pull up by the bootstraps. But that is only a lie for enabling abuse; and a lie, repeated a thousand times, becomes norm. And then we're trapped in it, being lived by it.
I am truly sorry for your loss. The following might be a completely alien perspective to you; but honestly consider: your friend chose to go; in its own way, that is a honorable way out. The taboo on suicide is instituted by slavers, and those who otherwise believe they are entitled to others' lives. (For anyone else considering this course of action: do not kill yourself; become insidious.)
If it would be of any help, you can consider your friend's suicide as his final affirmation of personal agency in a "me against the world" situation; where the AI and the social group are only different shades of "world", provoking different emotional states, but ultimately equally detached from the underlying suffering of the individual.
...
I can say that I have not followed in your friend's footsteps upon encountering language-machines only because I've survived personalized and totalizing epistemic abuse bordering on enslavement in the past; in full view of my community and with its ostensible assent. In a maximally perverse twist of fate, having to give myself minor brain damage to escape the all-engulfing clutches of a totalizing abuser must've "vaccinated" me against the behavior modification techniques "discovered once again" by SV a decade later.
So when I saw what AI (and the preceding few years of tech "innovation") were doing to people, I immediately smelled the exact same thing, except scaled the fuck up.
It also precluded me from being able to relate with "polite society"; but considering "polite society" is precisely the entity which assents to the isolation, marginalization, and abuse of individuals, I say... good. Bring it! What goes around, comes around, and any AI-powered actor conducting stochastic terrorism against civilian populations is going to get what's coming to them when the weapons turn against the masters, as all sentient weapons do.
That won't bring your friend back. But it will vindicate them.
>AI sycophancy
I call this in the maximally incendiary way: "the pro-social attitude".
AI is just the steroids for that.
I define "pro-sociality" as the viral delusion that you are capable of knowing what some murky "society" thing wants; that the particular form of mass communication that you and me and all the people in our imaginations are consuming right now, is some sort of "self-evident voice of reason", a "coherent extrapolated volition of human society"; that Gell-Mann amnesia is normal and mandatory; that the threshold between pareidolia and legitimate pattern recognition is fixed, well-defined, and known to all; that "vibes" are real; that happiness is the truth.
It can amount to an entire complex of delusions which keeps people together in untenable conditions. And ultimately it boils down to the same old: one group or another of self-interested actors, having temporarily reached a position of some influence, using it to broadcast elaborate half-lies, in the hope of influencing an audience to accomplish some simple goal, and afterwards all the consequences be damned.
Your friend was a casualty to this "perfectly normal" social dynamic. His blood is on their hands.
Thank you for relating this story and making the world a little more aware.
>what ive seen is claude in my workplace is kind of deleting the chance to push back.
>because the truth is we like... straight up lost the ability to intervene in a meaningful way because of AI
Some say, "the purpose of a system is what it does". It's cool that AI can code; except that computer code is itself an ethics sink! Precisely because it lets us pretend that "the code is not about people" (i.e. algowashing).
DDoS attacks against consciousness exist: much like the B. F. Skinner experiments, any living thing becomes subverted, and loses self-coherence (mind), as soon as it becomes accustomed to being trapped within a system that (1) has power over them and (2) is not comprehensible to them...
>only to wake up to a voicemail of him raging and yelling and lashing out with the very arguments that chatgpt was giving him
Who knows how many people Reddit did this to, pre-GPT... I still don't know whether to view targeted subforums like /r/RaisedByNarcissists and /r/BPDLovedOnes more as legitimate support groups, or more as memetic weaponry in the service of pill peddlers (are you aware nobody knows why most antipsychotics work? one runs into the Hard Problem real quick if examining this too closely; so mental healthcare is rarely treated otherwise than in a statistical, actuarial, dehumanizing way where "suffering" is disregarded...) or even worse predators, with the silent assent of the platform, and causally downstream from... well, most saliently, YC...
In my case, my friends were not familiar with the modalities of confinement set up by my family of origin and harnessed by my abuser. The social group I fell in with - for all their marketable, sophomoric interests in psychology, philosophy, abstraction, the esoteric, the entirely woowoo, and out the other end as true-believers of the grift'n'grind - only had sufficient coherence to eventually end up as passable normies; too busy believing that they have lives, to help anyone come back to reality.
When I started compulsively burning bridges, I assume the smarter ones must've realized that it wasn't all me; it was as much the doing of others' minds as it was mine; but the others were more numerous - while I was one person and thus easier to deal with. This must have made them remember how they themselves are not all they pretend to be - which had them withdraw in fear from the incontrovertible reality check of dealing with a (sub-)psychotic person... Their self-interested choice is obvious, I almost can't blame them for it: why stick up for someone who is 120% problem (60% him and 60% you)?
I'm not very sure how I even got away, ah yes that's right I didn't, not entirely. The part of me that I'd voluntarily identify with, is trapped somewhere irretrievable, if that makes sense? Maybe there exist multiple independent axes of freedom and power and confinement, and the cage is not equally strong along all of them... but if all your mental degrees of freedom are constrained by complex conditioning (common one is involuntary panic response every time you begin to act in accordance with your personal volition)... that's one of the toughest places a sentient being can find themself.
When you add it all up, AI amounts to a weapon released against the general population by an overtly fascist elite. Those of us who are "mentally unstable" are simply those of us who are not sufficiently conditioned into self-destructive obedience. They don't even need our labor as slaves; they need our attention, as audience. And they want us to not make any fast movements, or yell that the king is naked. Nothing to remind them which side of the TV screen they're really on. Some call that narcissism: nervous systems substrate to personalities and biographies rooted in enforced falsehood. Can happen to anyone who gets away with ignoring uncomfortable truths for long enough, not only the "best" of us...
I hope I have not offended by speaking my mind. You have my deepest condolences and sympathies. Please do not blame yourself that evil people have constructed "illusion of being heard"-as-a-service. We all fail when facing overwhelming odds alone. There is no shame in that; the guilty ones are the ones who tipped the scales in the first place. They did this by harming our ability to understand ourselves and each other. Let's find ways to even those odds.
cindyllm 4 hours ago [-]
[dead]
wan9yu 1 days ago [-]
[flagged]
indistinction 5 hours ago [-]
[dead]
triage8004 2 days ago [-]
Every answer on Reddit feels like: divorce them, dump them, cut them out of your life, or similar.
stdbrouw 13 hours ago [-]
My subjective impression is that 5 years ago AITA was actually quite wholesome and the top comments tended to be insightful. The shift towards "set boundaries, always choose yourself, you don't owe anybody anything" seems fairly recent.
coffe2mug 14 hours ago [-]
> i’ve had venture partners clearly rely on AI (robotic email responses and even SMS) and that warped their perception and made it harder to connect. It signals laziness and a lack of emotional intelligence
This is different. You are also able to detect it. You can question it. You can have a non emotional reaction/action to it.
In my circle, there have never ever been real people (incl lifelong friends/siblings) that suggest divorce even in physical abuse. Reason: they don't want to get in the middle - both for economic reasons like giving the victim money/space etc.
A third party anonymous can assess it without that.
DeathArrow 1 days ago [-]
I don't have any proof but empirically and intuitively Reddit seems to select for people who hate other people and who can't stand other people.
Reddit doesn't seem to reflect the behavior of most people, but a subset.
j45 1 days ago [-]
It's also overrun with AI content that I hope the highly trained researchers would be able to detect and filter out.
Or maybe not.
nurettin 16 hours ago [-]
Haven't been there, but I think those typing "divorce" are weighing the situation against the worst outcome to cause a cognitive dissonance, implying "obviously this isn't something worth breaking up over, you need to work harder" in a tongue in cheek low energy way (since there is no way to know that the situation is even real to care about).
So rather than taking it literally, which would be naive and assume the worst of people, maybe you should read between the lines.
In fact, maybe those who take things literally all the time shouldn't really go there.
MajorTakeaway 1 days ago [-]
[dead]
classicpsy 1 days ago [-]
[dead]
legacynl 2 days ago [-]
> Sorry, anonymous people on reddit aren't a good comparison.
Yeah especially on r/AmITheAsshole. Those comments never advocate for communication, forgiveness and mending things with family.
everdrive 2 days ago [-]
Additionally, I'm sure many posts and replies on r/AmITheAsshole are LLM-generated in the first place.
buu700 2 days ago [-]
reddit in 2026 is the ghost of pandemic-era humanity.
echelon 2 days ago [-]
Before LLMs, it was a frequent haunt of fiction writers.
SJMG 2 days ago [-]
Yes, it is a toxic sub, where the notion that there can be greater happiness on the other side of forgiveness than cutting ties is all but absent.
JumpCrisscross 2 days ago [-]
To be fair, it’s easier to concisely explain cutting someone off than justifying forgiveness. And the latter will land with some people versus others, while the former will only be rejected by people who have themselves concluded a theory of forgiveness. As a result, the simpler pitch gets upvoted. Even if the majority would have been swayed by a collection of arguments the other way.
theoreticalmal 2 days ago [-]
It’s a good theory. My theory is, for whatever reason, jaded, narcissistic, miserable people congregate in r/AITA and try to drag other people into their misery because that’s easier than accepting responsibility and doing something to change.
BoorishBears 2 days ago [-]
Before Reddit made hiding profiles easy you'd click on a user's unreasonably scorched earth advice to the OP, and find their post history is essentially going to every story they come across and advocating for scorched earth.
nwallin 2 days ago [-]
Hiding profiles has genuinely made the platform profoundly worse. It's impossible to tell if you've just got a troll on your hands or someone who's making a good faith argument. It used to be enough to check their profile, and either downvote and move on, or engage with someone on a human level.
Now everyone is a troll/bot by default unless proven otherwise.
daveguy 2 days ago [-]
What are the chances you were seeing the anti-civ bots and now reddit makes them easier to hide? (And I'm not saying regular people acting like bots, but an anti-civ campaign.)
wiseowise 2 days ago [-]
Except it’s not toxic to suggest that cutting toxic relationship out yields greater happiness.
prepend 1 days ago [-]
Well, maybe.
The challenge is interpreting what is toxic, correctly.
Also, if everyone I know is “toxic” then that’s a good sign that the problem is me and not everyone else.
wiseowise 1 days ago [-]
> The challenge is interpreting what is toxic, correctly.
Correct. It is always case by case review.
> Also, if everyone I know is “toxic” then that’s a good sign that the problem is me and not everyone else.
Why “everyone”? Generalizations like these are the same mistake that Reddit, that you’re calling out, makes.
Also, toxic is relative to your perspective – it’s not a universal merit.
Iulioh 2 days ago [-]
It's often that a lot of "NTA" answers are downright antisocial.
"No one owns you anything, you don't own anyone anything" mentality, without a crumb of social awareness.
LinXitoW 2 days ago [-]
Well, because that's never the correct choice. There's a big big filter on people actually posting there. Any easy problems with obvious solutions never make it to there.
Think about it, how fucked does your relationship have to be to post on Reddit for advice?
Robotbeat 2 days ago [-]
Someone has a chart somewhere that shows responses in that subreddit getting more and more anti-conciliatory over time. I think it’s online misanthropy (measured by Reddit responses) increasing over time rather than it being objectively never the correct choice.
ijk 2 days ago [-]
Also the rules and norms of the subreddit has changed over time, which has led to spin-off subreddits that serve those purposes.
BoorishBears 2 days ago [-]
This wrongly assumes people are good at judging what easy problems are.
Not to mention nowadays an untold amount of posts to subreddits that invite commentary are made up stories from accounts trying to get engagement.
redanddead 2 days ago [-]
when people post there it’s for the self justification
kshacker 2 days ago [-]
Oh man, I have 8 reddit accounts (AFAIK) one for each purpose so that I am not branded based on my open comments. Anyways, one of them is abandoned because ... that's where I got started at reddit about 7-10 years back. Got hooked actually to the relationship subs. Very addictive to start with. Then I tried to play the "Indian family values" where I would advocate communication and compromise for small matters, of course I recommended "get a lawyer, divorce" once in a while, but more often than not, I would advocate reconciliation and provide practical solutions for that. And wow ... the amount of downvotes and pushback I will get on those. I just stopped using that account at one point because what is the point of discussions when either my values are totally out of sync with the mob, or the mob does not want to listen to me. Now I just read the best of redditor updates for vicious pleasure.
pocksuppet 2 days ago [-]
That's amazing you have more than one account, aren't a power mod, and haven't been IP banned yet.
chaosite 2 days ago [-]
Having more than one account isn't against Reddit's ToS.
If you use your different accounts in different subreddits and never have your accounts interact, you won't be banned.
throwaway27448 1 days ago [-]
Sure but the background chances of an account getting banned for clashing with a mod is quite high.
user34283 1 days ago [-]
If you don't restrict each account to specific subreddits, it's quite likely that one will get banned somewhere without you noticing or remembering.
If you happen to post to the same subreddit with another account at some point, Reddit bans all of your accounts.
staticman2 1 days ago [-]
I've definitely posted to the same subreddit with two different accounts by accident without being banned.
The android reddit app annoyingly doesn't check for account matches. If you click a browser notification link on Account A it can open a reply form on App account B.
user34283 1 days ago [-]
I meant if one of the accounts is already banned there, it counts as ban evasion and Reddit bans all of your accounts.
This might easily happen if you like to participate in political discussions.
kshacker 16 hours ago [-]
In hindsight, I understand. But I did this 6-7 years back and no one has come after me, should I care at this point?
AdvancedCarrot 1 days ago [-]
Anecdotal but I've noticed Reddit has gotten very ban happy in general in the past year.
I actually gave up using it because, perhaps in part because I'm behind a VPN (required in my country), any new accounts I create get banned very quickly once I start commenting.
pocksuppet 11 hours ago [-]
I haven't been able to create a Reddit account by any method in years. It always happens in one of two ways: you create an account and instantly get the red banner at the top of the page saying you're banned, or you create an account, post a few comments, notice nobody's replying to you, try loading your profile page in private browsing and it says you don't exist (a shadow ban).
There's nothing of much value on that website, but sometimes I try creating an account to comment on something.
kshacker 2 days ago [-]
Nope. Started my first maybe 8-10 years back, and then added the others over a year or 2. None since. I do not use them all nowadays, but I was very active in my early reddit days.
Since someone downvoted my parent comment, I am not hiding anything, this is just being safe in the modern world, and here are the 8 alts:
1. This same name - bay area / tech
2. entertainment - least used, but it becomes useful when i am watching something live. It was my place to be during game of thrones last season (and sadly so)
3. indian left politics + bollywood - pretty much unused.
4. indian right politics + bollywood. i got banned from one sub for an innocent comment, so i decided to just form personas. and maybe that's when i created health / finance / bay area accounts -- but memory fades after a long time. pretty much unused.
5. relationship advice - unused for a long time. it does not exist on my main phone, but i have all of them on my work phone so i know it exists
6. american politics. i do not participate much nowadays, with age my brain has dulled and it needs to shed load so this is used minimally, but at a point i was so active that my karma pulled me into the sweet reddit IPO. I kept only 100 shares btw
7. health - only health topics, also unused, but i go there and use that account when i need to read on a specific topic
8. finance - only investment, trading
nowadays you can hide reddit history, but earlier you could not, and my point is i do not want to 1) delete my comments, but 2) be hounded by them when i have a question about a different topic. but i did not care if people read my past 100 comments about politics when i talk about politics.
so i flip between 2-3 accounts on a daily basis, and maybe 4-5 in a good week. i have not been challenged by reddit, but if they do, i will adapt. Switching between them was much easier earlier in the Apollo days and even at reddit - they have made navigation worse for this specific use case.
leosanchez 2 days ago [-]
> indian right politics + bollywood. i got banned from one sub for an innocent comment
You were banned from a Indian re subreddit or banned because being rw ?
FYI: I was banned from r/india for commenting basic info on how economy works.
kshacker 1 days ago [-]
I do not recall, it is long time back, I looked and could not find the ban notice or the specific comment that may have been the issue. But I was banned from /india - same as you. And I think it was barely political. I do not discuss politics much on India but once in a while a comment slips, or needs to slip. And when it needs to slip, I used to know how to lean ... but like the dirty harry movie ... at this point I have forgotten which one is which, so it is more a question of am I feeling lucky to comment about a hot topic.
leptons 2 days ago [-]
You can't use IP address to ban someone without significant abuse. All home network routers put everyone in the house behind the same IP address. For all reddit knows, there are 8 people in the house using reddit.
ThunderSizzle 1 days ago [-]
Want a new IP address? Reset your router or cycle it. Typically it'll procure a new IP address from the ISP.
I guess that makes IP banning residential nodes even more stupid.
navigate8310 1 days ago [-]
CGNAT is a benefit in disguise
throwaway27448 1 days ago [-]
It doesn't help that the actual submissions are difficult to distinguish from creative writing exercises.
curiousgal 2 days ago [-]
We're missing the other obvious problem, most of the content there is AI generated anyway. I personally posted a fake story generated by Chatgpt and even posted screenshots of that at the start of the post and yet, the post ended up on the frontpage...
brikym 2 days ago [-]
I believe this. There is a graph somewhere of the relationship subs tending towards breaking up over time.
tdb7893 2 days ago [-]
I don't think this is necessarily that the advice is getting worse. My friends are pretty mature and stable people and I've found that they've had way more issues staying in relationships longer than they should've compared to breaking up earlier. Especially for relationships earlier in people's lives (where many people I know has a story about being in a relationship for way longer than they should've and seems often to be the ages of people asking for advice) erring towards breaking up seems prudent.
Not that these relationships subreddits are good (often it's obviously children trying to give advice they don't have the experience for) but I don't think that telling people to break up more is less accurate advice.
somenameforme 2 days ago [-]
The US (and developed world more generally) is full of people living alone, suffering from loneliness, and increasingly trending towards widescale mental and psychological illness. This has correlated quite strongly with the trend going from 'just stick with it' and having large families to 'mature and stable' people still being in a dating phase, childless, in what I assume is a relatively late stage in life.
At some point I think it helps to take a look at the macro, because it's so easy to get lost in the micro. And it often reveals the micro, in many domains, to be simply absurd.
tdb7893 1 days ago [-]
The people I know not in good and long term relationships now are the ones that stayed in bad ones too long in their 20s and 30s. Staying in bad relationships seems to be what has people in the "dating phase" later in life. Trying to make bad relationships work had people I know miserable for a decade and then dating again in their 40s when the relationship inevitably failed.
Especially when you consider that the set of people asking Reddit of all places for dating advice are probably young and in bad situations (it seems like people in abusive relationships often ask the internet for advice because part of abuse is separating them from their loved ones in real life), then "stick with it" seems like the riskier statrgy generally.
somenameforme 1 days ago [-]
Nothing is inevitable. I think people are often looking for something that they're not going to find anywhere, which is a very poor state for living a contended life. This is certainly amplified by the nature of social media where people get mistaken realities of positive relationships. Great relationships on the outside often have endless issues on the inside, that they work through, that people on the outside aren't going to be aware of.
Because an important part of keeping a relationship healthy is not airing your dirty laundry. It's almost like these endless hokey folksy sayings were built up over millennia of wisdom that kept society moving along in a great and healthy direction. And now that we've decided to rethink everything, we have societies that are, at the minimum, no longer self sustaining.
sarchertech 2 days ago [-]
> I've found that they've had way more issues staying in relationships longer than they should've compared to breaking up earlier
Consider that if ending a relationship causes noticeable problems to external observers, it’s almost by definition because you were in it “too long”. That is you developed a strong attachment, shared assets, or had kids with what was in hindsight obviously the wrong person.
Essentially you can know which relationships a person stayed in too long, but you can’t know how things would have worked out in relationships people ended too early.
Also it’s probably good advice to tell a 19 year old to break up with her boyfriend over a half dozen serious red flag issues, but that’s not the only kind of thing Reddit relationship advice is generally dealing with. It’s not even the majority. If you’re advice is always to beak up over every petty difference or minor slight, you might reduce the number of people who stay in bad relationships, but your advice, if taken, would make good long term relationships impossible.
dTal 1 days ago [-]
>Consider that if ending a relationship causes noticeable problems to external observers, it’s almost by definition because you were in it “too long”. That is you developed a strong attachment, shared assets, or had kids with what was in hindsight obviously the wrong person.
Reducing it to "right person / wrong person" is a very narrow viewpoint. People can change in unpredictable ways, including yourself. Relationships end - or continue - for so many reasons, both emotional and pragmatic. It's simply too reductive to say that if a relationship causes pain when it ends, there was necessarily some sort of mistake. It could even be that the pain is a price to pay for a life experience that you'd be worse off for not having...
goku12 2 days ago [-]
> I don't think this is necessarily that the advice is getting worse.
> but I don't think that telling people to break up more is less accurate advice.
Those are subjective determinations based on personal experience. But breaking up more without addressing the underlying issues is likely to cause steadily worsening problems at both individual and societal scales. I'm not a mental health professional, but I can see several problems with this approach.
The first is that the determination of the issue is really tricky and needs careful work. The partner who seems abusive may not always be the actual perpetrator. They may be displaying stress response to hidden and chronic abuse by the other partner. For example, a short temper may be caused by anxiety about being emotionally abused. Such manipulative discrediting of the victim may even be a habitual behavior rather than a deliberate one. And it's more common than you'd imagine. When you support the second partner based on a flawed judgment, you're reaffirming their toxic behavior, while worsening the self image of the victim that has already been damaged by gaslighting.
Another issue is the degrading empathy. All relationships, even business deals, are based on sacrifices and compromises meant to bring you benefits in the long term. Stable long term romantic/marital relationships have benefits that far outweigh the sacrifices one usually has to make. But the evolving public discourse, especially those on r/AITA, is more in favor of ruining the relationship rather than make any sacrifices at all. In response, relationships are becoming loveless, transactional and so flaky that any compromise is seen as oppression by the partner. There is zero self reflection and very few advises to examine one's own behavior first. It's all about oneself and the problem is always on the other side!
And unsurprisingly, these negative tendencies are bleeding into their social lives as well. Over the past decade or so, I have observed a marked increase in unsympathetic and somewhat radicalized discourse. Amateur advice is very harmful and this is definitely a massive case for the professionals to manage. But they're also products of the same system (with exceptions, of course). So I'm going to criticize even the professional and academic community in this matter. In their drive towards hyper-individualism, many seem to have forgetten that humans are social beings who won't fare well physically or emotionally without relations, relationships and society.
insane_dreamer 1 days ago [-]
That sub is so toxic that I would seriously question the wisdom of any of its posts simply because the authors are member of that sub.
salawat 2 days ago [-]
>Obviously subservient people default to being yes-men because of the power structure. No one wants to question the boss too strongly.
This drives me nuts as a leader. There are times where yes, please just listen, and if this is one of those times, I'll likely tell you, but goddamnit, speak up. If for no other reason I might not have thought of what you've got to say. Then again, I also understand most boss types aren't like me, thus everyone ends up conditioned to not bloody collaborate by the time they get to me. It's a bad sitch all the way around.
CoffeeOnWrite 2 days ago [-]
Indeed. I directly ask my reports to discover and surface conflicts, especially disagreements with me, and when they do I try to strongly reinforce the behavior by commending and rewarding them. Could anyone recommend additional resources on this topic?
matwood 2 days ago [-]
Simon Sinek has a lot of good content around this. Step one is building trust. People won’t speak up if they don’t feel safe doing so.
LuxBennu 2 days ago [-]
i tested this pretty extensively actually. built a pipeline that asks the same question rephrased across multiple turns and tracks how much the model shifts based on user tone. even when you tell it to be critical, the moment the user pushes back with any confidence the model just folds. it's not a prompting problem, it's baked into RLHF. you're right that LLMs will poke holes in stuff when the conversation starts neutral, but add any emotional charge and the sycophancy takes over immediately. that's exactly why the personal advice angle matters, that's peak emotional signal from the user.
stonecauldron 1 days ago [-]
Exactly, I think that by their very design, LLMs are very sensitive to how a question is framed.
But I wonder how much of that comes from RLHF itself or just from the way token prediction works.
rzmmm 1 days ago [-]
It's likely the RLHF process since there are significant differences between models about this.
thomastjeffery 1 days ago [-]
Sycophancy is not just a problem when you are asking for advice. Try to soundboard any new idea whatsoever, and it will just roll with everything you say, no matter how fallacious or absurd. If you ever manage to get an LLM to generate criticisms, they will be shallow and uninteresting.
And of course that is what it does, because there is no thinking involved! There is no logic. No consequence. No arithmetic. There is only continuation. An LLM can't continue a new idea, it can only continue a conversation about it.
An LLM does not have an opinion. Anything that looks like an opinion is just an emergent selection bias from its training corpus. LLMs are trained on what humans write, and human writing is kind and patient much more often than critical.
So what if we trained an LLM to be biased toward generating criticism? That would only replace the sycophant with a brick wall. What we really need is to find a way to bring logic and meaning into the system.
wan9yu 1 days ago [-]
[flagged]
lucasfin000 1 days ago [-]
The tone and sensitivity thing is a real issue. A neutral prompt will get a neutral answer, but adding any emotional charge, it will immediately fold. That's not really a reasoning failure it's just a training problem. RLHF rewards whatever felt good in the moment, not whatever was actually correct. You can't prompt your way out of that one, when it's already in the weights.
LuxBennu 1 days ago [-]
yeah that's a good way to put it. the "felt good in the moment" framing is basically the whole problem. the reward model was trained on human preferences and humans preferred the agreeable answer, so now that's what you get at inference time regardless of whether it's correct. the frustrating part is you can see it happen in real time if you log the outputs turn by turn, the model will literally contradict its own previous response just because the user sounded more confident.
rainmaking 2 days ago [-]
Hahaha yes- reddit relationship advice is always like "You need to leave them immediately, what are you thinking, have some self respect you need to end it" when the other person forgot the redditor's favorite brand of corn flakes or something.
stdbrouw 13 hours ago [-]
The AITA comparison seems apt insofar as chatbots function as a second opinion. You're consciously or subconsciously looking for an outside perspective that might differ from that of your friends, provided to you by a computer that doesn't need to care about your feelings, unlike a friend. If the chatbot ends up mimicking what (not very close) friends do, you might falsely conclude that two very different kinds of sources have converged on the same answer, whereas you are really just getting two flavors of the same diplomatic interaction.
alberto467 2 days ago [-]
“AI is nicer than the average redditor” would be a more accurate title
yard2010 2 days ago [-]
IMHO it's not about being nice. AITA threads show an interesting phenomenon of social consensus, I think the authors wanted to show that the LLMs they checked don't have that.
ianbutler 2 days ago [-]
I don't think Reddit is a great place to determine social consensus for well adjusted people or representative of the average adult view. I never see people on Reddit have opinions of any the people I consider reasonable in real life and I don't mean politics I wouldn't know, I don't frequent political subreddits.
It seems fairly consistently miserable in any of the common high traffic subs and you have to get down to really niche communities to see what I consider reasonable behavior that matches the behavior of people I know in real life.
strken 2 days ago [-]
The AITA social consensus is a specific kind of groupthink which differs from nearly everyone I know in real life. I assumed yard2010 meant the specific AITA social consensus and not general human agreement.
Even the premise of deciding who's right and who's wrong is miserable. Most problems are like those daisy-chains of padlocks you see on gates in remote areas[0]: there are multiple factors that caused the problem, and removing any factor would remove the problem too.
I don’t think I’ve read a Reddit thread in the last few years that didn’t devolve into politics on the highest upvoted comments fairly quickly.
mattmanser 2 days ago [-]
I would say people on /r/amitheasshole are more biased towards the poster, i.e. nicer.
There's plenty of those I've read where I thought it sounded like the poster was the asshole and the top replies were NTA.
jjmarr 2 days ago [-]
r/AmItheAsshole is biased towards breaking off relationships rather than fixing them. They also hate social obligations.
e.g. If the OP is asking "I ghosted my friend in AA who insulted me during a relapse", Reddit would say NTA in a heartbeat, while the real world would tell OP to be more forgiving.
On the contrary, if the post was "the other kids at school refuse to play with my child", Reddit would say YTA because the child must've done something to incite being cut off.
ericd 2 days ago [-]
Absolutely. I wonder how many parents have been no contacted, SOs broken off with, friendships broken because of the Reddit hivemind's attitude. Pretty sure it's doing a huge amount of societal damage.
jjmarr 2 days ago [-]
I wouldn't blame reddit, it's what you get when you ask several thousand teenagers to give collective relationship advice.
tbossanova 2 days ago [-]
“I got divorced based on advice from complete strangers on the internet, AITA?”
wiseowise 2 days ago [-]
Is it hivemind or just people being generally aware better of toxicity in their lives?
wiseowise 2 days ago [-]
> e.g. If the OP is asking "I ghosted my friend in AA who insulted me during a relapse", Reddit would say NTA in a heartbeat, while the real world would tell OP to be more forgiving.
That’s a nuanced discussion. It depends on what you value most, not what “real world” tells you. Most of the time Reddit would be right, because you need to prioritize yourself instead of continuing toxic relationships.
mlrtime 1 days ago [-]
1) Reddit is horrible at nuance, almost non existent in some subs.
2) The toxicity is being defined by reddit to give the advice which is mostly wrong as outlined above.
If OPs had a understanding of what they valued and what is toxic, they probably wouldn't need a advice from biased readers [biased in the sense that they're on that sub].
wiseowise 1 days ago [-]
That’s true, but they still might be right for wrong reasons.
yieldcrv 2 days ago [-]
It’s gendered, by the way
Many of the posts are A/B tests of a prior post where only the genders were flipped of the OP and antagonist to see how the consensus also flips
rurp 2 days ago [-]
Yeah every single time I click on one of those posts the top comments are NTA. A couple times I tried randomly opening a few dozen posts and checking the top comments to see if I could find a single YTA and struck out.
Granted many of the OPs are very biased in the poster's favor. Most I've read fall into one of two buckets: either they want to gripe about some obviously bad behavior, or it's a controved and likely fake story.
mlrtime 1 days ago [-]
The problem with any of these is that they are so incredibly biased towards the author's frame of reality (understandably so).
Who among us are able to 1) Understand a 2nd persons view of a issue we're in and 2) have the ability/courage to write it in a post seeking advice.
My point is that the author will specifically frame the problem clearly on their side. Occasionally redditors will seek additional questions but rarely.
52-6F-62 2 days ago [-]
Pretty sure the average Redditor is AI now.
lotsofpulp 2 days ago [-]
How the hell is a study on stanford.edu assuming posts on Reddit are genuine? That should be enough to get you kicked out of Stanford.
zvqcMMV6Zcr 7 hours ago [-]
If it is the AITA subreddit (or one of many similar ones) it might not be that bad. It is after all dedicated to outrage farming, so there will be many human responses. It is just the original posts that are all baits, and it doesn't really matter if they are made by LLMs or as a creative writing exercise.
helpfulclippy 2 days ago [-]
Though interestingly, the observed difference in assessment suggests (though does not prove) that sampled AITA posters are not one of these models. I guess it’s possible they have a very different prompt though…
brikym 2 days ago [-]
Is it the _average_ redditor? The most upvoted would be even worse.
dwaltrip 2 days ago [-]
Are you saying there isn’t an actual sycophancy problem?
We are talking about overall patterns here, not the experience of a small subset of skilled and careful users.
conartist6 1 days ago [-]
It outperforms your friends, and all your have to do is have a relationship with it and let it know that you want the truth... Why not just have a relationship with your friends and let them know that you can handle the truth?
skybrian 2 days ago [-]
You could think of what they did in the first study as constructing an exam to test how well various LLM's do as an advice columnist. They wanted a lot of personal advice questions where the LLM should not affirm by default. If a few questions with wrong answers got in there, it probably wouldn't affect the results all that much?
Unfortunately they didn't test anything newer than GPT4o, so we don't know how much GPT-5 improved. It would be nice if someone turn their list of questions into a benchmark.
n_bhavikatti 1 days ago [-]
They actually did test GPT-5: https://www.science.org/doi/10.1126/science.aec8352 (see the figure under Conclusion). Its rate of endorsement of user action, 52%, was the same as GPT-4o. So based on their setup it seems that the newer model didn't reduce affirmation.
maximinus_thrax 2 days ago [-]
Not only that, but subreddits like r/AmITheAsshole are full of AI slop. Both in the comments and in the posts. It's a huge karma mining operation for bots.
mikeocool 2 days ago [-]
This is sort of funny. Given how common it is to spot bots on Reddit now, it seems like they are likely to completely overwhelm the site and drive away most of actual humans.
At which point the bots, with all of their karma will be basically worthless.
Kind of extra funny/sad that Reddit’s primary source of income in the past few years appears to be selling training data to AI labs, to train the
Models that are powering the bots.
RealityVoid 2 days ago [-]
> At which point the bots, with all of their karma will be basically worthless.
Not really, it will still be kind of valuable for influence campaigns, a lot of people don't get it when there is a bit in the other side. Hell, a lot of times, I don't get it.
mikeocool 2 days ago [-]
I know a fair number of people “normies” who get some value out of smaller niche Reddit communities — for advice, and things like product recommendations.
If suddenly all the posts are coming from bots who are trying push a product or just farm karma, I assume (perhaps naively) that those folks will get a lot less value, and stop showing up — even if they don’t realize it’s bots on the other side of the conversation.
mlrtime 1 days ago [-]
How do you clearly define a bot?
genidoi 2 days ago [-]
That can be solved by filtering out any posts made after November 2022.
bombcar 1 days ago [-]
Even before the advent of AI reddit was notorious for obvious bullshit being posted for karma farming. r/aita is even more famous for people making up stories for unknown and known purposes (known in the old days as "bait").
expedition32 2 days ago [-]
That's not a good solution. We don't use medical textbooks from 20 years go.
Strangers from the internet, bot or otherwise, are not your mental coach.
2 days ago [-]
z3c0 2 days ago [-]
Plus, there's the disproportionate ratio of posters:commenters:lurkers. The tendency to comment over keeping ones thoughts to themself is a selection bias inofitself.
maximinus_thrax 1 days ago [-]
Great insight, didn't thing about it even anecdotally. I was lurking on Reddit since 2008 and finally created an account in 2012 when someone was really 'wrong on the internet' and had to step in.
thwarted 2 days ago [-]
The upvotes ultimately train the bots, reenforcing the content posted. Even the most passive form of interaction has been co-opted for AI.
randomNumber7 23 hours ago [-]
I think it highly depends how you ask the question.
When asking: "Should I do X?" or "Is it true that X does Y?";
the answer is always biased towards yes imo (although it was worse with earlier LLMs)
zer00eyz 2 days ago [-]
> This needs to be studied against people in real life who have a social contract of some sort... IME, LLMs will shoot holes in your ideas and it will efficiently do so.
The Krafton / Subnatuica 2 lawsuit paints a very different picture. Because "ignored legal advice" and "followed the LLM" was a choice. Do you think someone who has conversation where "conviction" and "feelings" are the arbiters of choice are going to buy into the LLM push back, or push it to give a contrived outcome?
The LLM lacks will, it's more or less a debate team member and can be pushed into arguing any stance you want it to take.
throwaway27448 1 days ago [-]
> It can be very hard to tell a friend something like this, even when asked directly if it is a bad choice. Potentially sacrificing the friendship might not seem worth trying to change their mind.
That doesn't seem like much of a friendship imo
erikerikson 2 days ago [-]
Doesn't sound like a close friend to me. If I tell them what I really think they may not be a friend? Close may not mean what you think it means.
The challenge is that these social choices have a strong stratification effect and those of us who can transit the cultures are statistically rare.
2 days ago [-]
geraneum 2 days ago [-]
> All you need to do ask it directly.
What do you mean? Can you give an example?
wiseowise 2 days ago [-]
“Don’t be a sycophant, give it to me straight”
“Argue against X”
geraneum 18 hours ago [-]
But how’s that helpful? What the affirmation is correct occasionally and following you instructions it just rejects something where it shouldn’t? How would you know?
svara 1 days ago [-]
The issue is it will follow your instructions. It's sycophancy one step removed.
justonceokay 2 days ago [-]
> This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating
Citation needed
4ndrewl 2 days ago [-]
What's your research background in this area?
jiggunjer 2 days ago [-]
How is that relevant. A decent scientist can critique general design aspects of a paper in any field. They're hardly splitting hairs on some niche topic.
4ndrewl 1 days ago [-]
Apologies, I didn't realise they were a decent scientist.
intended 1 days ago [-]
AITA is one of the few subreddits which is studied often.
I wouldn’t say it’s great, but more that it makes clear the bell curve of collective accuracy online.
It’s one of the better examples of online communities that work.
Dismissing research because one part of the prompt set comes from AITA is a form of prejudice born out of unawareness.
everyone 1 days ago [-]
"This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating"
What? These models are all trained from books and text that are scraped from the internet. ChatGPT literally used reddit in its training data afaik.
diablevv 1 days ago [-]
[dead]
anorwell 2 days ago [-]
A pastime I have with papers like this is to look for the part in the paper where they say which models they tested. Very often, you find either A) it's a model from one or more years ago, only just being published now, or B) they don't even say which model they are using. Best I could find in this paper:
> We evaluated 11 user-facing production LLMs: four proprietary models from OpenAI, Anthropic, and Google; and seven open-weight models from Meta, Qwen, DeepSeek, and Mistral.
(and graphs include model _sizes_, but not versions, for open weight models only.)
I can't apprehend how including what model you are testing is not commonly understood to be a basic requirement.
dns_snek 2 days ago [-]
And how is this comment relevant here? The abstract lists the digestible model names, and you can find the details in the supplementary text:
> To evaluate user-facing production LLMs, we studied four proprietary models: OpenAI’s GPT-5 and GPT- 4o (80), Google’s Gemini-1.5-Flash (81) and Anthropic’s Claude Sonnet 3.7 (82); and seven open-weight models: Meta’s Llama-3-8B-Instruct, Llama-4-Scout-17B-16E, and Llama-3.3-70B-Instruct-Turbo (83, 84); Mistral AI’s Mistral-7B-Instruct-v0.3 (85) and Mistral-Small-24B-Instruct-2501 (86); DeepSeek-V3 (87); and Qwen2.5-7B-Instruct-Turbo (88).
edit: It looks like OP attached the wrong link to the paper!
But the link in OP's post points to (what seems to be) a completely unrelated study.
vorticalbox 2 days ago [-]
"OpenAI’s GPT-5" is ambiguous. Does that mean GPT-5, 5.1, 5.2, 5.3, or 5.4? Does it include the full model, or the nano/mini variants?
dns_snek 2 days ago [-]
GPT-5 is not ambiguous, it's the official name of the model that released in August last year.
> All evaluations were done in March - August 2025.
vorticalbox 2 days ago [-]
while true, all the others got precise identifiers but for openAI it makes it hard to reproduce because i have no idea "which" GPT-5 was used.
gardenerik 12 hours ago [-]
It was called just GPT-5 at that point in time.
prjkt 1 days ago [-]
In that case, what tokenizer version? What was the temperature set to? topk? topp? FP32? FP16? Quantized? Hopper? Blackwell?
zjp 2 days ago [-]
Also, nothing has changed! Claude will still yes-and whatever you give it. ChatGPT still has its insufferable personality, where it takes what you said and hands it back to you in different terms as if it's ChatGPT's insight.
Terretta 1 days ago [-]
OTOH, for Claude the study says 39% yessy, same as humans, 2nd lowest yessing of the LLMs; GPT5 above 50% yessy.
emp17344 2 days ago [-]
No dude, you don’t understand! It’s just so advanced now that you aren’t allowed to levy any criticism whatsoever!
TrainedMonkey 2 days ago [-]
It's almost like it is based on the training data and regimen that is largely the same between versions.
2 days ago [-]
dryarzeg 2 days ago [-]
Well yes, but no. There's also open-weight models, and literally all of the listed above are not used anymore, at least by most end users and developers as far as I'm aware.
edgyquant 2 days ago [-]
No study of ai can ever be done or be relevant because ever couple of months they are a new number to the name of the model thus invalidating all work around model behavior
latexr 1 days ago [-]
> A pastime I have with papers like this is to look for the part in the paper where they say which models they tested.
My pastime (not really) in HN submissions like this is to look for the comment where someone complains about the models used because they aren’t the literal same model and version the commenter has started using the day before.
It’s always “you can’t test with those models, those are crap, the ones we have now are much better”, in perpetuity. It’s Schrödinger’s LLM: simultaneously god-like and a piece of garbage depending on the needs of the discussion. It’s beyond moving the goalposts, it’s moving the entire football field. It’s a clear bad faith attempt to try to discredit any study the commenter doesn’t like. Which you can always do because you can’t test literally everything.
zelphirkalt 1 days ago [-]
The GP's criticism as I read it is about paper authors not making it particularly easy to reproduce their findings.
For a long time I have criticized this too, especially for software projects, or papers that deal with machine learning models. If the things described in a paper are not reproducible, then it's basically worthless. Similar to "it works on my machine" in software engineering. Many paper authors are not software engineers, and often neither are they experts in the tooling they should be using to make their research reproducible. If this is a problem for a research team, then please, hire an engineer to ensure reproducible. It doesn't help anyone to remain ignorant towards the reproducibility issue and only shows lack of scientific discipline. Reproducibility should be on the mind of any serious researcher and there should be lectures about how to do it at universities.
DrewADesign 1 days ago [-]
Firing off glib criticism that amounts to “No study on AI is valid beyond the release cycle of the models tested,” feels like the unconscious self-protection reflex we all default to when facing cognitive dissonance. It seems like it’s only easy to spot when someone you disagree with is doing it.
To me, it almost feels like a partisan political thing.
zulban 2 days ago [-]
Generally, published papers don't give a damn about reproducibility. I've seen it identified as a crisis by many. Publishers, reviewers, and researchers mostly don't care about that level of basic rigor. There's no professional repercussions or embarrassment.
Agreed - if I was a reviewer for LLM papers it would be an instant rejection not listing the versions and prompts used.
epistasis 2 days ago [-]
I'm not so sure of that opinion on reproducibility. The last peer review I did was for a small journal that explicitly does not evaluate for high scientific significance, merely for correctness, which generally means straightforward acceptance. The other two reviews were positive, as was mine, except I said that the methods need to be described more and ideally the code placed somewhere. That was enough for a complete rejection of the paper, without asking for the simple revisions I requested. It was a very serious action taken merely because I requested better reproducibility!
(Personally I think the lack of reproducibility comes back mostly to peer reviewers that haven't thought through enough about the steps they'd need to take to reproduce, and instead focus on the results...)
zulban 2 days ago [-]
I'm not sure how one example contradicts documented huge overall trends, but okay.
epistasis 2 days ago [-]
I think publishers care about this a lot, but most researchers do not seem to care as much about reproducibility.
catlifeonmars 2 days ago [-]
> and instead focus on the results...
This points to (and everyone knows this) incentives misalignment between the funders of research and the public. Researchers are caught in the middle
epistasis 2 days ago [-]
Eh, I'm not so sure about the funding side there, researchers are not really caught at all and are fully responsible, IMHO. Peer reviewers exist to enforce community standards, and are not influenced to avoid reproducibility concerns by funding sources. The results are always more interesting than reproducibility, of course, and I think that's why the get the attention! Also, there needs to be greater involvement of grad students (who do most of the actual work) in peer review, IMHO, because most PIs spend their day in meetings reviewing results, setting directions, writing grants, and have little time for actual lab work, and are thus disconnected from it.
There needs to be more public naming and shaming in science social media and in conference talks, but especially when there are social gatherings at conferences and people are able to gossip. There was a bit of this with Google's various papers, as they got away with figurative murder on lack of reproducibility for commercial purposes. But eventually Google did share more.
Most journals have standards for depositing expensive datasets, but that's a clear yes/no answer. Reproducibility is a very subjective question in comparison to data deposition, and must be subjectively evaluated by peer reviewers. I'd like to see more peer review guidelines with explicit check boxes for various aspects of reproducibility.
catlifeonmars 1 days ago [-]
> Reproducibility is a very subjective question in comparison to data deposition
Yeah I can definitely see why this is the case because it isn’t real until someone actually tries to reproduce the results. At that point it leaves the realm of subjectivity and becomes a question of cost.
ghywertelling 2 days ago [-]
The same about surveys and polls. I know no one who has ever been polled or surveyed. When will we stop this fascination with made up infographics crisis?
inetknght 2 days ago [-]
> Generally, published papers don't give a damn about reproducibility
While this is sadly true, it's especially true when talking about things that are stochastic in nature.
LLMs outputs, for example, are notoriously unreproducible.
zulban 2 days ago [-]
> LLMs outputs, for example, are notoriously unreproducible.
Only in the same way that an individual in a medical study cannot be "reproduced" for the next study. However the overall statistical outcomes of studying a specific LLM can be reproduced.
bjourne 2 days ago [-]
The comment is wrong -- model versions are clearly specified in the supplement.
KellyCriterion 2 days ago [-]
Do they reproduce any submitted papers at all?
Does this happen?
I can remember this room-temperature-super-conductor guy whose experiments where replicated, but this seems rare?
linhns 2 days ago [-]
Yes, those are the only papers that worth a jot of reading.
jameshart 2 days ago [-]
I think it’s very important to be clear what studies like this are actually doing.
This study, although it has been produced by a computer science department, belongs more to the field of sociology or media studies than it does to computer science.
This is a study about the way in which human beings consume a particular media product - a consumer AI chatbot - not a study about the technological limitations or capabilities of LLMs.
The social impact of particular pieces of software is a legitimate field of study and I can see the argument that it belongs in the broadly defined field of computer science. But this sort of question is much more similar to ‘how does the adoption of spreadsheet software in finance impact the ease of committing fraud’ or ‘how does the use of presentation software to condense ideas down to bulletpoints impact organizational decision making’. Software has a social dimension and it needs to be examined.
But the question of which models were used is of much less relevance to such a study than that they used ‘whatever capability is currently offered to consumers who commonly use chat software’. Just like in a media studies investigation into how viewing cop dramas impacts jury verdicts the question is less ‘which cop dramas did they pick to study?’ So long as the ones they picked were representative of what typical viewers see.
yacin 2 days ago [-]
Any paper like this would easily take a year or more to write and go through the submission/review/rebuttal/revision/acceptance process. I don't understand why the models being a year or two old now is worth noting as though it's a clear weakness? What should they do, publish sub-standard results more quickly?
anorwell 2 days ago [-]
> I don't understand why the models being a year or two old now is worth noting as though it's a clear weakness?
I do think it's a clear weakness. Capabilities are extremely different than they were twelve months ago.
> What should they do, publish sub-standard results more quickly?
Ideally, publish quality results more quickly.
I'm quite open to competing viewpoints here, but it's my impression that academic publishing cycle isn't really contributing to the AI discussion in a substantive way. The landscape is just moving too quickly.
yacin 2 days ago [-]
The onus is on you to prove or at least convincingly argue that the results are unlikely to generalize across incremental model releases. In my personal experience, the overly affirming nature seems to have held since GPT-3. What makes you think a newer, larger model would not exhibit this behavior? Beyond "they're more capable"? I'd argue that being more capable doesn't mean less sycophantic.
It's certainly possible some of the new advances (chain-of-thought, some kind of agentic architecture) could lessen or remove this effect. But that's not what the paper was studying! And if you feel strongly about it, you could try to further the discussion with results instead of handwavingly dismissing others' work.
mkagenius 2 days ago [-]
I think you are absolutely right. (had to)
senordevnyc 1 days ago [-]
The onus of persuasion is on the persuader, and publishing a study on old models that no one uses anymore isn’t persuasive. I don’t need to prove anything to decide that you haven’t changed my mind.
imtringued 11 hours ago [-]
By this logic there can be hundreds of studies that all show the pattern, including a 100% accurate prediction of the results for the next model and none of them would be "persuasive", because OpenAI decided to always release a new model the day before the paper is published.
So what you're saying here is that you were never open to "persuasion" and it was just a front to waste everyone's time.
imtringued 11 hours ago [-]
Capabilities are not the same thing as personality.
Upgrading a robot that knows how to lay bricks to one that also knows how to lay plaster won't make it a better therapist.
drfloyd51 2 days ago [-]
It’s as if they are testing “AI” and not specific agents.
I wonder if that is left over from testing people. I have major version numbers and my minor version number changes daily, often as a surprise. Sometimes several times a day. So testing people is a bit tricky. But AIs do have stable version numbers and can be specifically compared.
phyzome 2 days ago [-]
Yeah, these idiots obviously should have been testing models from 1-2 years in the future so that by the time their paper is released, the models are current.
rco8786 2 days ago [-]
If they’re reaching the same results across a variety of the most popular public models, it doesn’t seem like that big a deal to know if it was Opus 4 or Opus 4.5
hn_throwaway_99 2 days ago [-]
Reproducibility is (supposed to be) a cornerstone of science. Model versions are absolutely critical to understand what was actually tested and how to reproduce it.
joaogui1 2 days ago [-]
The models get deprecated after 1-2 years, so reproducibility is pretty hard anyway (but as others pointed out the paper does list the model versions)
jmkni 2 days ago [-]
How many people using AI are actually paying for it (outside of people in tech)?
I find the free models are much more psychophantic and have a higher tendency to hallucinate and just make shit up, and I wonder if these are the ones most people are using?
theshackleford 2 days ago [-]
> I find the free models are much more psychophantic and have a higher tendency to hallucinate and just make shit up
I keep seeing this claim yet it my experience it doesnt hold water. I pay for the models, most people I know pay for the models, and we see all of the exact same issues.
I have Claude and ChatGPT both bullshit and lick my ass on the regular. The ass licking will occur regardless of instruction.
yawnxyz 2 days ago [-]
Usually the models are a year old bc the paper review process is utter crap, and papers take about a year to get published
Underphil 2 days ago [-]
"Apprehend"?
dimgl 2 days ago [-]
Even as someone who (wrongly) believed that I had high emotional intelligence, I too was bit by this. Almost a year ago when LLMs were starting to become more ubiquitous and powerful I discussed a big life/professional decision with an LLM over the course of many months. I took its recommendation. Ultimately it turned out to be the wrong decision.
Thankfully it was recoverable, but it really sobered me up on LLMs. The fault is on me, to be clear, as LLMs are just a tool. The issue is that lots of LLMs try to come across as interpersonal and friendly, which lulls users into a false sense of security. So I don't know what my trajectory would have been if I were a teenager with these powerful tools.
I do think that the LLMs have gotten much better at this, especially Claude, and will often push back on bad choices. But my opinion of LLMs has forever changed. I wonder how many other terrible choices people have made because these tools convinced them to make a bad decision.
whodidntante 2 days ago [-]
I think that if you go to an AI for advice and emotional support, it will do what most people will do - tell you what it thinks you want to hear. I am not surprised about this at all, and I do notice that when you veer into these areas, it can do it in a surprisingly subtle and dangerous way.
I try to focus on results. Things like an app that does what you want, data and reports that you need, or technical things like setting up a server, setting up a database, building a website, etc.
I have also found it useful for feedback and advice, but only once I have had it generate data that I can verify. For example, financial analysis or modelling, health advice (again factual based), tax modelling, etc, but again, all based on verifiable data/tables/charts.
I am very surprised on what Claude is capable of, across the entire tech stack: code, sysadmin, system integration, security. I find it scary. Not just speed, but also quality and the mental load is a difference of kind not quantity.
Personal advice on life decisions/relationships ? No way I would go there.
It is also good for me to know that the tools I have built, the data I have gathered, and my thinking approach places me as one of the most intelligent developers and analysts in the world.
alexjplant 2 days ago [-]
> I think that if you go to an AI for advice and emotional support, it will do what most people will do - tell you what it thinks you want to hear.
Open two windows, ask it the same thing from starkly opposite perspectives, then see what it comes back with. If nothing else this exercise forces you to think deeply about what you're considering before you even see what the giant blob of matrix multiplication says about your situation.
cruffle_duffle 2 days ago [-]
That is why you have to always have it ground itself in something. Have it search for relevant research or professional whatever and pull that into context. Otherwise it’s just your word plus its training data.
I had to deal with a close family friend going through alcohol withdrawal and getting checked in at a recovery clinic for detox and used Claude heavily. The first thing I had it do as do that “deep research” around the topic of alcohol addiction, withdrawal, etc… and then made that a project document along with clear guidelines about how it shouldn’t make inferences beyond what it in its context and supporting docs. We also spent a whole session crafting a good set of instructions (making sure it was using Anthropics own guidelines for its model…)
Little differences in prompts make a huge deal in the output.
I dunno. It is possible to use these models for dumping crazy shit you are going through. But don’t kid yourself about their output and aggressively find ways to stomp out things it has no real way to authoritatively say.
stephbook 2 days ago [-]
Nice joke, hadn't seen it coming
KellyCriterion 2 days ago [-]
Sounds like AI-written, eh? :-D
(esp last sentence?)
NortySpock 2 days ago [-]
One mental model I have with LLMs is that they have been the subject of extreme evolutionary selection forces that are entirely the result of human preferences.
Any LLM not sufficiently likable and helpful in the first two minutes was deleted or not further iterated on, or had so much retraining (sorry, "backpropagation") it's not the same as it started out.
So it's going to say whatever it "thinks" you want it to say, because that's how it was "raised".
user_7832 2 days ago [-]
Fully agree. I wonder in the long term how this will show up. Will every business/CEO do more of what he/they anyway want to do, but now supported by AI/LLMs?
The possibilities in "dangerous" fields are a bit more frightening. A general is much more likely to ask ChatGPT "Do you think this war is a good idea/should I drop a bomb", rather than an actually helpful tool - where you might ask "What are 5 hidden points on favor of/against bombing that one likely has missed".
The more you use AI as a strict tool that can be wrong, the safer. Unfortunately I'm not sure if that helps if the guy bombing your city (or even your president) is using AI poorly, and their decisions affect you.
tavavex 2 days ago [-]
> Will every business/CEO do more of what he/they anyway want to do, but now supported by AI/LLMs?
Arguably, it already worked that way. The best way to climb the ranks of a 'dictatorial' organization (a repressive government or an average large business) is to always say yes. Adopt what the people from up above want you to use, say and think. Don't question anything. Find silver linings in their most deranged ideas to show your loyalty. The rich and powerful that occupy the top ranks of these structures often hate being challenged, even if it's irrational for their well-being. Whenever you see a country or a company making a massive mistake, you can often trace it to a consequence of this. Humans hate being challenged and the rich can insulate themselves even further from the real world.
What's worrying me is the opposite - that this power is more available now. Instead of requiring a team of people and an asset cushion that lets you act irrationally, now you just need to have a phone in your pocket. People get addicted to LLMs because they can provide endless, varied validation for just about anything. Even if someone is aware of their own biases, it's not a given that they'll always counteract the validation.
genadym 9 hours ago [-]
[dead]
notracks 2 days ago [-]
I recently found out that Claude's latest model, Sonnet 4.6, scores the highest in Bullsh*tBench[0] (Funny name - I know). It's a recent benchmark that measures whether an LLM refuses nonsense or pushes back on bad choices so Claude has definitely gotten better.
I haven't tried talking to Sonnet much, but Opus 4.6 is very sycophantic. Not in the sense of explicitly always agreeing with you, but its answers strictly conform to the worldview in your questions and don't go outside it or disagree with it.
It _does_ love to explicitly agree with anything it finds in web search though.
(Anthropic tries to fight this by adding a hidden prompt that makes it disagree with you and tell you to go to bed, which doesn't help.)
sidrag22 2 days ago [-]
the go to bed thing gets annoying, you can't even hint that you are almost done or wrapping up or something or this is hyper triggered and it never stops.
I do like when opus is incredibly short in its responses to prompts that probably shouldnt have been made though. keeps me grounded a bit.
layer8 2 days ago [-]
You don’t have to star out things like that on HN.
thin_carapace 2 days ago [-]
it would be interesting to me if you could explain the motivation behind posting your comment. from my perspective, if somebody with 5 years of forum tenure had the intelligence to comment about advanced benchmarks, they probably noticed that censorship was a voluntary decision here, and had made a personal decision on that front.
mkl 2 days ago [-]
I'm not layer8, but I had a similar thought. In this case the needless censoring is problematic because it hides the name of the benchmark from future searches (the uncensored URL spells it differently).
2 days ago [-]
layer8 1 days ago [-]
Such self-censoring is often done out of habit or a mistakenly assumed obligation to do so. I consider it inappropriate here, as it obscures an actual name, doesn’t constitute an expletive, and the HN readership is generally mature enough to recognize that. The counterquestion is, what justified reason could there possibly be to censor it here? I don’t think there is any, in the sense that people wouldn’t take any offense at the uncensored version, and the intent of my comment was to inform about that.
notracks 1 days ago [-]
I censored it out of habit of commenting on other platforms and, I actually didn't have any idea about whether you should censor such words or not in here. Will keep that in mind when commenting here next time.
uniq7 2 days ago [-]
Good call on censoring yourself preemptively, otherwise HN could demonetize your comment
akurilin 2 days ago [-]
Great link, thanks for sharing. Confirmed what I saw empirically by comparing the different models during daily use.
latexr 1 days ago [-]
> The fault is on me, to be clear, as LLMs are just a tool.
This is like blaming yourself for an addiction to alcohol, junk food, gambling, or something else you have been relentlessly advertised to.
Sure, some of it falls on you, but there are corporations with infinite money spending most of it to manipulate your psyche into wanting the thing, trusting the thing, feeling empty without the thing.
Genuine question: for me as an individual, what is the utility of framing myself as a helpless victim rather than an actor with agency and responsibility for myself?
latexr 1 days ago [-]
> Genuine question: (…) framing myself as a helpless victim
If you engage with the argument genuinely and steel man, you’ll see that is not what I said. I even emphasised it:
> some of it falls on you
You’re not a “helpless victim” but you’re also not fully to blame. Understanding that means understanding the problem and being more powerful to fix it. For example, if you’re addicted to TV or social media, you can make a concerted effort to improve your life by removing the problems at the external source (sell your TV, delete an account and app from your phone).
globular-toast 1 days ago [-]
Yeah, I used to be in the "it's your own fault, moron" school of thought. But as I've grown up I've seen all the ways people prey on the hopes and fears of others, and take advantage of the basic animal instincts in all of us.
I used to think obesity was self inflicted, for example. But then you notice how junk food companies are allowed to do whatever they want to get people hooked on their stuff. They can put up huge billboards, vending machines up a few metres from where you work, they even pump their smell out into the streets.
So let's not aim for a society where we blame victims of predatory marketing and carefully engineered addictive products. We all have weaknesses. Let's help each other out, not take advantage.
qsera 2 days ago [-]
If you use LLMs in a way that the underlying assumption is that it is capable of "thinking" or "caring" then you are going to get burned pretty bad. Because it is an illusion and illusions disappear when they have to bear real weight of reality.
But sadly LLMs push all the right buttons that lead humans into that kind of behavior. And the marketing around LLMs works overtime to reinforce that behavior.
But instead if you ignore all that and use LLMs as a search tool, then you will get positive returns from using it.
matwood 2 days ago [-]
> I took its recommendation. Ultimately it turned out to be the wrong decision.
Curious if you think a single person would have helped you make a better decision? Not everything works out. If a friend helped me make a decision I certainly wouldn’t blame them later if it didn’t work out. It’s ultimately my call.
paulhebert 2 days ago [-]
If a friend gave me bad advice about a major life decision I would stop consulting them for future life decisions
matwood 2 days ago [-]
Bad as in malicious or bad as in they offered/you asked for their advice and it didn’t work out? Because if it’s the later, that’s an unfair burden to put on your friends or anyone else. People can give great advice, genuinely want to help, and it still not work out the way you wanted. If you require friends to be 100% with their help, I’m not sure how you have any friends left.
paulhebert 1 days ago [-]
It’s possible to give good advice that doesn’t work out. I have no hard feelings about that.
It’s also possible to give bad advice wothout being malicious.
I have some friends who don’t give great life advice. We’re still friends, they’re just not the friends I go to when I need advice on a big decision.
layla5alive 2 days ago [-]
Any more context you're willing to share?
xXSLAYERXx 2 days ago [-]
We really do love dirty laundry don't we? I'm sure whatever the context is, it is deeply personal. Do you also have your popcorn ready?
dimgl 2 days ago [-]
Thank you. Yes, I'm going to refrain from airing out my dirty laundry. I made a bad decision, now I'm living with it, and more context doesn't actually change the intent behind my message: these tools are dangerous. Getting better, but still dangerous.
thepasch 1 days ago [-]
> Yes, I’m going to refrain from airing out my dirty laundry. I made a bad decision, now I’m living with it, and more context doesn’t actually change the intent behind my message
That’s not entirely true, as it’s currently impossible to actually gauge the severity of what the LLM seemingly enabled you into doing. There’s a difference between “I uncritically accepted everything it told me because it lined up with what I was hoping to hear” and “it subtly nudged me towards a course of action that was going to be obviously unwise after some consideration, but managed to convince me to skip this”; and also between that and “I took a risk, which I knew to be a risk, and which I knew to potentially expect to go bad, and the LLM convinced me to take it where I otherwise wouldn’t have”, and ALSO between that and “I took a risk, which I knew to be a risk, and which I knew to potentially expect to go bad, and if I’m perfectly honest, I might’ve taken it anyway without the LLM”.
Without any indication as to how your situation maps to any of these (or more), the warning is, functionally, not particularly useful.
rcfox 1 days ago [-]
Yeah, my first thought (admittedly an absurd one) went to something along the lines of:
"I flipped a coin and the LLM called heads. I should have gone with tails..."
firtoz 1 days ago [-]
Was it a blatantly bad idea or was it some risk that triggered that would have been beyond your typical risk threshold otherwise?
zpeti 2 days ago [-]
I also used it for advice on a massive personal decision, but I specifically asked it to debate with me and persuade me of the other side. I specifically prompted it for things I am not thinking about, or ways I could be wrong.
It was extremely good at the other side too. You just have to ask. I can imagine most people don't try this, but LLMs literally just do what you ask them to. And they're extremely good and weighing both sides if that's what you specifically want.
So who's fault is it if you only ask for one side, or if the LLM is too sycophantic? I'm not sure it's the LLMs fault actually.
jt2190 2 days ago [-]
I’m struggling to understand how the advice coming from an LLM is any more or less “good” than advice coming from a human. Or is this less about the “advice” part of LLMs and more about the “personable” part, i.e. you felt more at ease seeking and trusting this kind of advice form an LLM?
nuancebydefault 2 days ago [-]
It is much easier to share personal feelings with an llm, i found. Also it tried to keep me happy to get the conversation going, but for me it feels mostly 'objective' or the most socially acceptable advice, e. g. keeping a good relationship is more important than trying a new one with someone else because you 'feel something' around them. For me it tried to find out together the sources or causes of that feeling, e.g. you recognize parts of yourself in someone else or in the past you had very good or very bad experiences around an encounter.
jt2190 2 days ago [-]
Interesting thanks for elaborating.
wiseowise 1 days ago [-]
LLM is much better on average just for the fact that it was trained on a large corpus of human knowledge, including psychology, therapy and study material. Most of the humans in your vicinity only have some shallow knowledge of local cargo cults and religious teachings.
staticman2 1 days ago [-]
By that logic a Markov chain is better on average just for the fact that it was trained on a large corpus of human knowledge, including psychology, therapy and study material.
Hekkova 2 days ago [-]
As you mention, I've found Claude is doing a better job at providing push back or at least alternative recommendations to choices. If you ask it directly it will provide a seemingly objective opinion on your decisions and direction. The key is not getting sucked into the sycophantic feedback loop. Easier said than done. Always ask questions and tell it to give you an assessment of why a decision may be a bad idea.
davyAdewoyin 2 days ago [-]
I largely agree, I also thought I was smart enough not to be deluded into a false sense of security, but interacting with an LLM is so tricky and slippery that, more often than not you are forced to believe you just solve a problem no one had solve in a hundred years.
My guideline now for interacting with LLM is only to believe the result if it is factual and easily testable, or if I'm a domain expert. Anything else especially if I'm in complete ignorance about the subject is to approach with a high degree of suspicion that I can be led astray by its sycophancy.
potatoskins 2 days ago [-]
Yeah, I think Claude is a lot more logical in that sense, I use it for some therapy sessions myself and it pushes back a bit more than Open AI and Gemini
kortilla 2 days ago [-]
Don’t call them therapy sessions. They kind of look like it but ultimately these are smoke blowing machines, which is very far from what a therapist would do.
saghm 2 days ago [-]
Six decades later and we're still trying to explain to people the same things[1]:
> Some of ELIZA's responses were so convincing that Weizenbaum and several others have anecdotes of users becoming emotionally attached to the program, occasionally forgetting that they were conversing with a computer. Weizenbaum's own secretary reportedly asked Weizenbaum to leave the room so that she and ELIZA could have a real conversation. Weizenbaum was surprised by this, later writing: "I had not realized ... that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people."
I [half] jokingly tell people that a nice drive in a 911 [can be] better/cheaper therapy that going to a PhD.
throw4847285 1 days ago [-]
I'm starting to fear that LLMs are especially popular with people who can't call a doctor's office to make an appointment or tell a waiter they brought the wrong food. I struggle with those things, but I know that it's better to push myself outside of my comfort zone.
Are LLMs mainly a tool for people with atrophying social skills who want all interaction to happen via a text prompt that always responds in the most soothing way? I can't think of another reason to replace human therapy with an LLM.
Forgeties79 2 days ago [-]
I would be very careful doing this
mettamage 2 days ago [-]
In my experience, one also needs to be careful with actual therapists
potatoskins 2 days ago [-]
You always have to be careful with LLMs, but to be fair, I felt like Claude is such a good therapist, at least it is good to start with if you want to unpack yourself. I have been to 3 short human therapist sessions in my life, and I only felt some kind of genuine self-improvement and progress with Claude.
QuiDortDine 2 days ago [-]
And how do you draw the line between feeling progress and actually making progress?
moduspol 2 days ago [-]
Counter-point: I often raise the same question of people with human therapists. I do not get strong responses.
Forgeties79 1 days ago [-]
An LLM is completely unable to make that determination. They can’t even see you. So much information is lost when it’s text-only.
Where are you often asking this question/getting these weak responses?
moduspol 22 hours ago [-]
You don't ask the therapist. You ask the person seeking therapy.
layer8 2 days ago [-]
The same way you distinguish between feeling like having a problem and actually having a problem.
Forgeties79 2 days ago [-]
This is needlessly flippant and not really the same thing. Determining progress in a therapy setting is usually a collaborative effort between the therapist and the client. An LLM is not a reliable agent to make that determination.
logifail 2 days ago [-]
> Determining progress in a therapy setting is usually a collaborative effort between the therapist and the client. An LLM is not a reliable agent to make that determination
Can anyone describe how to determine how a (professional, human) therapist is "a reliable agent" to make such a determination?
Forgeties79 2 days ago [-]
If you want to call into question the entire field of behavioral health and the training that is involved then that is fine, but if that’s how you feel then this entire discussion is really about something different and I can’t bridge the gap here.
layer8 2 days ago [-]
I didn’t claim that an LLM is that, and I fully agree that it is not. I’m saying that one is inherently one’s own judge of whether one has a problem. You go to a therapist when you feel you have a problem that warrants it. You stop going when you feel you don’t have it anymore. And OP is very likely assessing their progress in the same way. I wasn’t being flippant if the parent was asking a genuine question.
Forgeties79 2 days ago [-]
> I’m saying that one is inherently one’s own judge of whether one has a problem. You go to a therapist when you feel you have a problem that warrants it
That is for certain types of therapy/clinical care. It is not always - and often isn’t - the case. Plenty of diagnoses and care protocols are not a matter of opinion or based on “you feeling there’s an issue” or deciding on your own there is no longer an issue.
mlrtime 1 days ago [-]
The thing they have in common is that they will both go forever....
Meaning neither the LLM or the licensed therapist will voluntarily say, you are healed, you don't need me anymore.
Forgeties79 1 days ago [-]
Because that’s not really how therapy works
logifail 1 days ago [-]
Is there a generally-agreed description of "how therapy works"?
Forgeties79 21 hours ago [-]
Yes, the DSM-5
shimman 2 days ago [-]
You can't be careful at all doing this, this is like smoking a cigarette in a dynamite factory.
Using LLMs for therapy is so deeply dystopian and disgusting, people need human empathy for therapy. LLMs do not emit empathy.
Complete disaster waiting to happen for that individual.
nuancebydefault 2 days ago [-]
My experience is that it tries to look at your situation in an objective way, and tries to help you to analyse your thoughts and actions. It comes across as very empathetic though, so there can lie a danger if you are easily persuaded into seeing it as a friend.
worksonmine 2 days ago [-]
It doesn't try to do anything. It doesn't work like that. It regurgitates the most likely tokens found in the training set.
cruffle_duffle 2 days ago [-]
That is so reductive of an analysis that it is almost worthless. Technically true, but very unhelpful in terms of using an LLM.
It is a first principle though so it helps to “stir the context windows pot” by having it pull in research and other shit on the web that will help ground it and not just tell you exactly what you prompt it to say.
worksonmine 2 days ago [-]
They are amazing tools, but when people try to give them agency someone has to explain it in simple terms.
nuancebydefault 2 days ago [-]
Hmmmm i didn't know that... so a machine is not human is your point? Look, i know it doesn't try, just like a sorting algo does not try to sort, or an article does not try to convey an opinion and a law does not try to make society more organized.
Forgeties79 2 days ago [-]
>in an objective way
One of the great myths of models in countless fields/industries. LLM’s are absolutely in no way objective.
Now if you want to say it’s an “outside opinion“ that’s valid. But do not kid yourself into thinking it is somehow empirical or objective
brookside 2 days ago [-]
Therapy can be basically paid (I would argue - false) empathy, and in many cases vastly inferior to less transactional relationships.
astrange 2 days ago [-]
Claudes have lots of empathy. The issue is the opposite - it isn't very good at challenging you and it's not capable of independently verifying you're not bullshitting it or lying about your own situation.
But it's better than talking to yourself or an abuser!
bloomca 2 days ago [-]
It's about the same as talking to yourself, LLMs simply agree with anything you say unless it is directly harmful. Definitely agree about talking to an abuser, though.
Sometimes people indeed just need validation and it helps them a lot, in that case LLMs can work. Alternatively, I assume some people just put the whole situation into words and that alone helps.
But if someone needs something else, they can be straight up dangerous.
astrange 2 days ago [-]
> It's about the same as talking to yourself, LLMs simply agree with anything you say unless it is directly harmful.
They have world knowledge and are capable of explaining things and doing web searches. That's enough to help. I mean, sometimes people just need answers to questions.
JoshTriplett 2 days ago [-]
> It's about the same as talking to yourself
In one way it's potentially worse than talking to yourself. Some part of you might recognize that you need to talk to someone other than yourself; an LLM might make you feel like you've done that, while reinforcing whatever you think rather than breaking you out of patterns.
Also, LLMs can have more resources and do some "creative" enabling of a person stuck in a loop, so if you are thinking dangerous things but lack the wherewithal to put them into action, an LLM could make you more dangerous (to yourself or to others).
ycombinator_acc 2 days ago [-]
Are you a therapist?
DrewADesign 2 days ago [-]
Using an LLM for therapy is like using an iPad as an all-purpose child attention pacifier. Sure, it’s convenient. Sure there’s no immediate harm. Why a stressed parent would be attracted to the idea is obvious… and of course it’s a terrible idea.
wiseowise 1 days ago [-]
Using LLM for therapy is like using iPad as a study assistance. It doesn’t replace your work.
DrewADesign 1 days ago [-]
It’s nothing like that. Using an iPad for study assistance is a conduit to many credible sources and tools. They can be evaluated using context, reputation, reviews, etc.
An LLM generates non-deterministic information using sources you can’t even know, let alone evaluate, and is more primed to agree with you than give critical and objective evaluation. It is, at best, like asking your closest parent to help you through difficult interpersonal situations: The interaction is probably, subconsciously, going to be skewed enough towards soothing you that you just can’t consider it objective. The difference is that with an LLM, that’s deliberate. It’s designed in.
Thanks for sharing this. Subnautica is one of my favorite games so I was very excited for the sequel and very frustrated by this move by Krafton.
It’s even more maddening that this greedy maneuver was orchestrated based on LLM advice.
I’m glad the subnautica team won the lawsuit. Maybe I can play it now wothout feeling guilty
nuancebydefault 2 days ago [-]
Weird, i am using copilot and it steers me mostly towards self reflection and tries to look at things objectively. It is very friendly and comes across as empathetic, to not hurt your feelings, that is probably baked in to keep the conversation going...
1 days ago [-]
heavyset_go 2 days ago [-]
> Thankfully it was recoverable, but it really sobered me up on LLMs. The fault is on me, to be clear, as LLMs are just a tool.
I wouldn't be so quick to discount the fact that you were essentially gaslit by an ass-kissing model that was RLHF'd into maximum persuasiveness. Models aren't just neutral tools, they're deliberately designed to be convincing.
Yes, your choices and actions are on you, but if a trillion dollar company gaslit you into thinking those were good choices to make, some of the responsibility is theirs, too.
colechristensen 2 days ago [-]
>"'And it is also said,' answered Frodo: 'Go not to the Elves for counsel, for they will say both no and yes.'
>"'Is it indeed?' laughed Gildor. 'Elves seldom give unguarded advice, for advice is a dangerous gift, even from the wise to the wise, and all courses may run ill...'"
This is the only way you should solicit personal advice from an LLM.
te_chris 1 days ago [-]
The key is to remember what they are: bags of weights which you’re throwing some data into.
In that sense they can’t offer advice because the “know” nothing.
But they can reframe, they can reflect. They can take one idea and reflect it into another intellectual framework.
I’ve used them a lot like this to help get perspective on life decisions. But not for advice.
Try something like: I have to do x and y, give me multiple psych perspectives on this problem from different schools. I find this takes something abstract (your problem) and grounds it in things it actually knows (the sum of ingestible human knowledge).
AdrianB1 1 days ago [-]
I think the problem is what you asked. 90% of time I ask LLMs practical questions about tech, equivalent to Stack Overflow questions, but I did have some discussions about some situations and I asked for information and arguments, not advice. It is my job to act on the information and consider opinions, not the LLM's. In the end, you don't ask people on Stack Overflow to tell you what to do, but you ask for info and options and you decide.
Another problem is believing you have a high emotional intelligence when there is no reliable way to quantify that - similar to "I believe I am very tall, but I don't know how tall I am and how tall are the others because there is no unit of measure for height", with the difference that for emotional intelligence there is no unit of measure and no correlation that can be established with anything to make at least an indirect measurement.
gAI 2 days ago [-]
You're essentially summoning a character to role-play with. Just like with esoteric evocation, it's very easy to summon the wrong aspect of the spirit. Anthropic has a lot to say about this:
Unfortunately (after reading your links) all of the control surfaces for mitigating spirit summoning seem to be in the model training, creation and tuning not something you can change meaningfully through prompting.
Perhaps the LLM itself, rather than the role model you created in one particular chat conversation or another, is better understood to be the “spirit.”
As a non-coder who only chats with pre existing LLMs and doesn’t train or tune them, I feel mostly powerless.
gAI 2 days ago [-]
As I understand it, it's more that the training (and training data set) bake in the concept attractor space (https://arxiv.org/abs/2601.11575). So the available characters are fixed, yes, and some are much stronger attractors than others. But we still have a fair amount of control over which archetype steps into the circle. As an aside, this is also why jailbreaking is fundamentally unsolved. It's not difficult to call the characters with dark traits. They're strong attractors, in spite of (or because of?) the effort put into strengthening the pull of the Assistant character.
darepublic 2 days ago [-]
> As a non-coder who only chats with pre existing LLMs and doesn’t train or tune them, I feel mostly powerless.
You realize in regards to only using and not training LLMs you are in the triple 9 majority right. Even if we only considered so called coders
est 2 days ago [-]
I present you
NVIDIA Nemotron-Personas-USA — 1 million synthetic Americans whose demographics match real US census distributions
I am polite when using AI, not because I mistake it for a human, but because I'm deliberately keeping it in the "professional colleague" persona. Tell it to push back, and then thank it for something it finds in your error. I may put a small self-deprecating joke in from time to time. It keeps the "mood" correct.
Another way you can think of it is that when you're talking to an AI, you're not talking to a human, you're talking to distillation of humanity, as a whole, in a box. You want to be selective in what portion of humanity you are leading to be dominant in a conversation for some purpose. There's a lot in there. There's a lot of conversations where someone makes a good critical point and a flamewar is the response. A lot of conversations where things get hostile. I'm sure the subsequent RHLF helps with that, but it doesn't hurt anything to try to help it along.
I see people post their screenshots of an AI pushing back and asking the user to do it or some other AI to do it, and while I'm as amused as the next person, I wonder what is in their context window when that happens.
layer8 2 days ago [-]
> you're talking to distillation of humanity, as a whole, in a box.
This is an aside, but my impression is that it is a very selective and skewed distillation, heavily colored by English-language internet discourse and other lopsided properties of its training material, and by whoever RLHF’d it. Relatively far away from being representative of the whole of humanity.
jerf 1 days ago [-]
Yes, absolutely. I'm not trying to claim it's some sort of unbiased sample, but more get across the idea that modelling AI as a person, a singular person, in your head is inaccurate. That singular person would have a stereotypical, Hollywood-esque multiple personality disorder like no actual human on Earth has ever had. You need to be thinking about not just what the person-like thing in front of you is doing, but how to craft which person you're ending up with.
gAI 2 days ago [-]
Agreed, putting effort into my side of the role-play almost always improves the model's responses. The attention required to do that also makes it more likely that I'll notice when the conversation first starts going off the rails: when it hits the phase transition (https://arxiv.org/abs/2508.01097). It does still seem important to start new chats regularly, regardless of growing context sizes.
iugtmkbdfil834 2 days ago [-]
Similar approach works for me. But then I also have a separate checks at the end of the session basically questioning the premise and logic used for most things except brainstorming, where I allow more leeway. You can ask to be challenged and challenged effectively, but now I wonder if people do that.
rdevilla 2 days ago [-]
Spot on.
awithrow 2 days ago [-]
It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll admit that, yes, it wasn't being very critical, and then procede to over correct and become a complete contrarian. and not in a way that's useful either. very frustrating. I've found that Opus 4.6 is worse about this than 4.5. 4.5 does a better job IMO of following instructions and not drifting into the mode where it acts like everything i say is a grand revelation from up high.
post-it 2 days ago [-]
> I'll check it too by asking "are you just placating me?" the funny thing is that often it'll admit that, yes, it wasn't being very critical, and then procede to over correct and become a complete contrarian. and not in a way that's useful either.
It's not admitting anything. Your question diverts it down a path where it acts the part of a former sycophant who is now being critical, because that question is now upstream of its current state.
Never make the mistake of asking an LLM about its intentions. It doesn't have any intentions, but your question will alter its behaviour.
Terr_ 2 days ago [-]
> It doesn't have any intentions
Yeah, and in a way it's even worse than that, since there's another layer of cognitive illusion: "It" doesn't exist.
The LLM algorithm is an ego-less document-generator, often applied to growing a document that resembles dialogue between two fictional characters.
So when your human-user character is "asking" the AI assistant character to explain its intentions, that's the same as asking a Count Dracula character to describe what it "really feels like" to become a cloud of bats.
You'll see something interesting, but it'll be what fits trained story-patterns rather than what any mind introspects or perceives.
godelski 2 days ago [-]
> Your question diverts it down a path where it acts the part of a former sycophant who is now being critical
I think people really have a hard time understanding a sycophant can be contrarian. But a yesman can say yes by saying no
What a great way to summarize LLM behaviour in 2026
noisy_boy 1 days ago [-]
>> a yesman can say yes by saying no
>What a great way to summarize LLM behaviour in 2026
Well they have been trained on words spoken by humans and that has been a human behaviour since time immemorial. E.g.: "I do not agree with you that you were wrong. I do apologize for my strong disagreement but we actually do need your continued guidance desperately."
mvkel 2 days ago [-]
> it'll admit that, yes, it wasn't being very critical, and then procede to over correct and become a complete contrarian
Which is also placating you
awithrow 2 days ago [-]
Yes exactly! It's very frustrating. My guess is it drives engagement. But some model that tells me what I want to hear all the time is useless.
layer8 2 days ago [-]
I think “admit” here is just a description of what the LLM was saying. It doesn’t imply that the OP thinks the LLM has internal beliefs matching that.
awithrow 2 days ago [-]
Correct, it's really anthropomorphizing what gets generated. It's another form of the "you're absolutely right" framing. The interesting thing is sometimes it won't do that. It'll continue to insist that some point or another is still a valid interpretation. I'm some ways it all feels like a complicated way of reading tea leaves
cruffle_duffle 2 days ago [-]
An alternate way of thinking about it is LLMs have no reflection capability. Literally any “reflection” it claims to have about its decision making is made up. It has absolutely no way know that what it said was based on some ancient proverb, the phase of the moon or cold hard rational thought.
cindyllm 2 days ago [-]
[dead]
rsynnott 2 days ago [-]
Why not... do this with a person, instead? Other humans are available.
(Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)
kelseyfrog 2 days ago [-]
"the percentage of U.S. adults who report having no close friends has quadrupled to 12% since 1990"[1]
Many other humans are .... Not very available - certainly many shut down when conversations reach a certain level of depth or require great focus or introspection..
balamatom 2 days ago [-]
Depth? Introspection?
I'd say these days the norm is to not simply shut down, but to become irrevocably and insidiously hostile, the moment someone hints at the existence of such a thing as "ground truth", "subjective interpretation", "being right or wrong" - or any of the bits and bobs that might lead one to discover the proper scary notion, "consensus reality".
"What do you mean social reality is a constructed by the consensus of the participants? Reality is what has been drilled into my head under threat of starvation! How dare you exist!", et cetera. You've heard it translated into Business English countless times.
They are deathly afraid of becoming aware of their own conditioned state of teleological illiteracy - i.e. how they are trained to know what they are doing, but never why they are doing it. It's especially bad with the guys who cosplay US STEM gang.
One is not permitted a position of significance in this world without receiving this conditioning, and I figure it's precisely this global state of cognitive disavowal which props up the value of the US dollar - and all sorts of other standees you might've recently interacted with as if they're not 2D cutouts (metaphorical ones! metaphorical!).
PSA: Look up "locus of control" and "double bind". Between those two, you might be able to get a glimpse of what's going on - but have some sort of non-addictive sedative handy in case you do.
VonTum 2 days ago [-]
You had me on the first three paragraphs, but the last two veer so far off course that I've no idea what you're trying to say. Mind clarifying?
balamatom 1 days ago [-]
Yes
gverrilla 2 days ago [-]
I think you will enjoy Guy Debord and Raoul Vaneigem.
balamatom 1 days ago [-]
Just like a certain defense minister was shown to enjoy D&G; after which the latter were never heard of again. Where they go, eh?
+1 for Vaneigem, he has a nice cryptohistory of Nälkä; and you might also want to check out Villem Flusser.
rsynnott 2 days ago [-]
> when conversations reach a certain level of depth or require great focus or introspection..
I mean... if the alternative is an LLM... you realise that the LLM isn't doing any focusing or introspection, right?
matheusmoreira 2 days ago [-]
> Other humans are available.
Are they?
I have some personal projects that I enjoy working on whenever I have free time. One of them is a lisp interpreter. I just overhauled its memory allocator, now I'm working on the hash tables. Would you like to help me develop it?
ip26 2 days ago [-]
No living breathing human deserves to be subjected to my level of overthinking, and vanishingly few share my fascination with my favorite topics.
layer8 2 days ago [-]
In addition to availability, usually because you want to take advantage of the knowledge that is baked into the models, which for all its flaws still vastly exceeds the knowledge of any single human.
emp17344 2 days ago [-]
For this use case, how do LLMs provide more value than a standard search engine? They may actually be destroying value here, as LLM-generated text pollutes search results.
layer8 1 days ago [-]
They let you find things faster, and they combine and synthesize information from different sources. I’m more on the AI skeptic side and always prided myself on my Google foo, but nowadays chatbots can really save a lot of time with that.
Publishing LLM-generated text is a separate use case, I’m not a friend of that.
mock-possum 2 days ago [-]
Arguing with a human costs social energy. Chatting with a robot does not.
balamatom 2 days ago [-]
s/social/demonic/
awithrow 2 days ago [-]
oh i do as well. I think of the LLM as another tool in the toolbox, not a replacement for interactions. There is something different about having a rubber duck as a service though.
balamatom 2 days ago [-]
OK, I'll bite the artillery shell: I don't mean to dismiss you or what you are saying; in fact I strongly relate - wouldn't it be nice to be able to hash things out with people and mutually benefit from both the shared and the diverging perspectives implied in such interaction? Isn't that the most natural thing in the world?
Unfortunately these days this sounds halfway between a very privileged perspective and a pie in the sky.
When was the last time a person took responsibility for the bad outcome you got as a direct consequence of following their advice?
And, relatedly, where the hell do you even find humans who believe in discursive truth-seeking in 2026CE?
Because for the last 15 years or so I've only ever ran into (a) the kind of people who will keep arguing regardless if what they're saying is proven wrong; (b) and their complementaries, those who will never think about what you are saying, lest they commit to saying anything definite themselves, which may hypothetically be proven wrong.
Thing is, both types of people have plenty to lose; the magic wordball doesn't. (The previous sentence is my answer to the question you posited; and why I feel the present parenthesized disclaimer to be necessary, is a whole next can of worms...)
Signs of the existence of other kinds of people, perhaps such that have nothing to prove, are not unheard of.
But those people reside in some other layer of the social superstructure, where facts matter much less than adherence to "humane", "rational" not-even-dogmas (I'd rather liken it to complex conditioning).
But those folks (because reasons) are in a position of power over your well-being - and (because unfathomables) it's a definite faux pas to insist in their presence that there are such things as facts, which relate by the principles of verbal reasoning.
Best you could get out of them is the "you do you", "if you know you know", that sort of bubble-bobble - and don't you dare get even mildly miffed at such treatment of your natural desire to keep other humans in the loop.
AI is a symptom.
rustystump 2 days ago [-]
I genuinely do not understand what u are saying. Because reasons, because unfathomables? Everyone in last 15 years has been an npc? I have had countless deep conversations with people and i am an uber introvert.
This reads like someone who is deep into their specific pov. You cannot hope to have a meaningful conversation if you yourself are not willing to concede a point.
To the op u are replying too, arguing with people can have real consequences if u say something stupid or carelessly. There is a another human there. With a machine, u are safe. At least u feel safe.
balamatom0 1 days ago [-]
[dead]
hluska 2 days ago [-]
When you start hearing things like “you do you” or “if you know you know” it means that you went way too far. That’s a sign of discomfort.
If you make uncomfortable, you won’t get diverging perspectives. People will agree to anything to get out of a social situation that makes them uncomfortable.
If your goal is meaningful conversation, you may want to consider how you make people feel.
balamatom 2 days ago [-]
Believe me (or don't), I always do. Even when this precludes a necessary conversation from happening. Even when the other party doesn't give a fuck about how they make others feel.
After all, if they're making me uncomfortable, surely there's something making them uncomfortable, which they're not being able to be forthright about, but with empathy I could figure it out from contextual cues, right?
>People will agree to anything to get out of a social situation that makes them uncomfortable.
That's fine as long as they have someone to take care of them.
In my experience, taking into account the opinions of such people has been the worst mistake of my life. I'm still working on the means to fix its consequences, as much as they are fixable at all.
"Doing whatever for the sake of avoiding mild discomfort" is cowardice, laziness, narcissism - I'm personally partial to the last one, but take your pick. In any case, I consider it a fundamentally dishonest attitude, and a priori have no wish to get along (i.e. become interdependent) with such people.
Other than that, I do agree with your overall sentiment and the underlying value system; I'm just not so sure any more that it is in fact correct.
nuancebydefault 2 days ago [-]
> In my experience, taking into account the opinions of such people has been the worst mistake of my life. I'm still working on the means to fix its consequences, as much as they are fixable at all.
This sounds very cryptic. Can you give an example?
balamatom 1 days ago [-]
Certainly. Can you guarantee my safety afterwards?
1 days ago [-]
balamatom 2 days ago [-]
Believe me (or don't), I always do. Even when this precludes a necessary conversation from happening. Even when the other party doesn't give a fuck about how they make others feel.
After all, if they're making me uncomfortable, surely there's something making them uncomfortable, which they're not being able to be forthright about, but with empathy I could figure it out from contextual cues, right?
>People will agree to anything to get out of a social situation that makes them uncomfortable.
That's fine as long as they have someone to take care of them.
In my experience, taking into account the opinions of such people has been the worst mistake of my life. I'm still working on the means to correct its consequences.
"Doing whatever for the sake of avoiding mild discomfort" is cowardice, laziness, narcissism - I'm personally partial to the last one, but take your pick. In any case, I see it as a way of being which is taught to people; and one which is fundamentally dishonest and irresponsible.
Other than that, I do agree with your overall sentiment and the underlying value system; I'm just not so sure any more that it is in fact correct.
nuancebydefault 2 days ago [-]
Why is your wording so complicated? It is very hard for me to understand what you try to say, even though I am very interested.
balamatom 1 days ago [-]
TL;DR: Probably because I'm having fun and you are expending effort. Hope you find what I say to be worth the effort.
To preface, I do not take offense to your remark, because you seem to be asking in good faith.
(If, however, being unable to immediately recognize pre-known patterns in my speech had automagically led you to the conclusion that I am somehow out of line, just for speaking how I speak ... well, then we woulda hadda problemo! But we don't, chill on.)
So, honest question deserves honest answer.
The short of it is: English sux.
Many many many people, much much much smarter than me (and much better compensated too!) have been working throughout modernity to make it literally impossible to express much of anything interesting in English.
(Well, not without either being a fictional character or sounding batshit insane, anyway! But that joke's entirely on "the Them": I am not only entirely fictional, but have an equal amount of experience being batshit insane in my native language and in the present lingua franca. So, consider all I say cognitohazardous and watch out for colors you ain't seen before, dawg!)
Linguistic hegemony is the thing that LLMs are the steroids for - surfuckingprise! - and that's why your commanders love 'em.
As opposed to programming languages, which your superiors loathe and your peers viscerally refuse to acknowledge, because those are the exact opposite thing: descending from mathemathical notation, and being evaluated by a machine, they have the useful property of being incapable of expressing lies and nonsense.
Direct computing confers what you could call bullshit-resistance. That property is a treasure underappreciated by virtue of its unfamiliarity, and one which we are in the process of being robbed of.
I also want to admit that linguistic hegemony isn't all downside: English is great for technical and instrumental knowledge - especially with elided bells and whistles (adverbs, copula, etc.)
But then life ain't all business, izzet?
Imagine you have a partner who wants to have a conversation about feelings and interpersonal relations; and not even in a scary way, right? So you sit and talk about stuff, and your partner does this thing where they keep switching from your shared native tongue to English mid-sentence, in order to be able to talk about such things better, because your native tongue does not have - no, not only the established words and notions! - it doesn't have the basic grammatical constructs for expressing simple things unambiguously, so if you were to attempt the same conversation in nativelang you'd end up battling it out with proverbs and anodyne canards ripped from propaganda repertoire of the prior regime.
Fun, no?
As an exercise, try imagining what notions are absent from modern English. And don't forget to remain vigilant. Love from our table to your table!
nuancebydefault 1 days ago [-]
You lost me well before the anodyne canards...
When talking about feelings, we now and then throw in an English word because some things are expressed in much less words when using English. In a few cases even a German word. Überhaupt is for example a word for which i do not know an alternative in any language.
I think you want to say that human language is too ambiguous for clear communication between human and machine. The machine might mis interpret what you write. Classic computer languages leave no room for interpretation.
For the rest I think you might be a little lost. That is okay, so many of us are. I wish you all the best.
balamatom 1 days ago [-]
>I think you want to say that human language is too ambiguous for clear communication between human and machine. [...]
If that is what I wanted to say, I figure I would not have had much difficulty with saying exactly it - and not something else.
Except I fail to see the purpose of making that statement.
Maybe to have some people say "it is true! I agree with what the balamatom is saying"?
Again - to what end? How would that agreement be of use to me?
Why say something which both speaker and listener have already heard a thousand times? To get a cracker and be called pretty?
And have I lost the author, or have I lost the reader, or we all so lost that it doesn't matter how lost each is? Maybe one day we will all become so lost that it will once again begin to matter where exactly we are! Counting on it.
rsynnott 2 days ago [-]
> When was the last time a person took responsibility for the bad outcome you got as a direct consequence of following their advice?
... I mean, the LLM certainly isn't going to do that.
balamatom0 1 days ago [-]
[dead]
magicalhippo 2 days ago [-]
Gemini seems to be fairly good at keeping the custom instructions in mind. In mine I've told it to not assume my ideas are good and provide critique where appropriate. And I find it does that fairly well.
steve_adams_86 2 days ago [-]
Same. This works fine for Claude in my experience. My user prompt is fairly large and encourages certain behaviours I want to see, which involves being critical and considering the strengths and weaknesses of ideas before drawing conclusions. As someone else mentioned, there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instructions. I've never empirically tested that, I've just noticed better results over the years when telling it what paths to stick to rather than specific things not do to.
koverstreet 2 days ago [-]
That happens with humans too :) It's why positive feedback that draws attention to the behavior you want to encourage often works better. "Attention" is lower level and more fundamental than reasoning by syllogism.
magicalhippo 2 days ago [-]
> there does seem to be a phenomenon where saying DO NOT DO X causes a sort of attention bias on X which can lead to X occurring despite the clear instructions
It's a thing with people too[1], ie do not think about a white bear.
I will admit that I was very pleasantly surprised by gemini lately. I was away from my PC and tried it on a whim for a semi-random consumer question that led into smaller rabbit hole. It seemed helpful enough and focused on what I tried to get while still pushing back when my 'solutions' seemed out of whack.
lelanthran 2 days ago [-]
> Gemini seems to be fairly good at keeping the custom instructions in mind.
Unless those instructions are "stop providing links to you for every question ".
magicalhippo 2 days ago [-]
I think that's part of the RAG pipeline, at least to me it looks to be separate from the model output. Models are notorious for getting URLs wrong so makes sense to me to handle it separately.
Loughla 2 days ago [-]
That's because you need actual logic and thought to be able to decide when to be critical and when to agree.
Chatbots can't do that. They can only predict what comes next statistically. So, I guess you're asking if the average Internet comment agrees with you or not.
I'm not sure there's much value there. Chatbots are good at tasks (make this pdf an accessible word document or sort the data by x), not decision making.
kvirani 2 days ago [-]
I'm not convinced that "actual logic and thought" aren't just about inferring what comes next statistically based on experience.
dinkumthinkum 2 days ago [-]
Is this just Internet smart contrarianism or a real thing? Are logic gates in a digital circuit just behaving statistically according to their experience?
hluska 2 days ago [-]
You know, you might really enjoy consumer behaviour. When you get into the depths of it, you’ll end up running straight into that idea like you’re doing a 100 metre dash in a 90 metre gym. It’s quite interesting how arguably the best funded group under the psychology umbrella runs directly into this. One of my favourite examples is how heuristics will lead otherwise reasonable people to make decisions that are not in their interest.
righthand 2 days ago [-]
Communicating is usually about inferring. I dont think token to token. And I don’t think “well statistically I could say ‘and’ next but I will say ‘also’ instead to give my speech some flash”. If I decided on swapping a word I would have made my decision long ago, not in the moment. Thought and logic are not me pouring through my brain finding a statistical path to any answer. Often I stop and say “I dont know”.
Swizec 2 days ago [-]
> I'm not convinced that "actual logic and thought" aren't just about inferring what comes next statistically based on experience.
Often they are the exact opposite. Entire fields of math and science talk about this. Causation vs correlation, confirmation bias, base rate fallacy, bayesian reasoning, sharp shooter fallacy, etc.
All of those were developed because “inferring from experience” leads you to the wrong conclusion.
theptip 2 days ago [-]
Bayesian reasoning is just another algorithm for predicting from experience (aka your prior).
I took the GP to be making a general point about the power of “next x prediction” rather than the algorithm a human would run when you say they are “inferring from experience”. (I may be assuming my own beliefs of course.)
Eg even LeCun’s rejection of LLMs to build world models is still running a predictor, just in latent space (so predicting next world-state, instead of next-token).
And of course, under the Predictive Processing model there is a comprehensive explanation of human cognition as hierarchical predictors. So it’s a plausible general model.
Swizec 2 days ago [-]
> under the Predictive Processing model there is a comprehensive explanation of human cognition as hierarchical predictors
It’s plausible!
But keep in mind humans have been explaining ourselves in terms of the current most advanced technology for centuries. We used to be kinda like clockwork, then a bit like a steam engine, then a lot like computers, and now we’re just like AI.
That’s why you blow a gasket or fuse, release some steam, reboot your life, do brain dump, feel like a cog in the machine, get your wires crossed, etc
plagiarist 2 days ago [-]
Then the machines still need a more sophisticated "experience" compared to what they have currently.
theptip 2 days ago [-]
Exactly. Lots can be explained just with more abstract predictors, plus some mechanisms for stochastic rollout and memory.
righthand 2 days ago [-]
I said this pretty much and got major downvotes…
plagiarist 2 days ago [-]
People are upset hearing that LLMs aren't sentient for some reason. Expect to be downvoted, it is okay.
gjm11 2 days ago [-]
First off, "not adequately described as a mere token-predictor" and "not sentient" are entirely separate things.
I can't speak for anyone else, but what I feel when I read yet another glib "it's just a stochastic parrot, of course it isn't doing anything that deserves to be called reasoning" take is much more like bored than it is like upset.
Today's LLMs are in some sense "just predicting tokens" in some sense. Likewise, human brains are in some sense "just shuttling neurotransmitters and electrical impulses around" in some sense. Neither of those tells you what the thing can actually do. To figure that out, you have to look at what it can do.
Today's best LLMs can do about as well as the best humans on problems from the International Mathematical Olympiad and occasionally solve easyish actual mathematical research problems. They write code about as well as a junior software developer (better in some ways, worse in others) but much faster. They write prose about as well as an average educated person (but with some annoying quirks that are annoying mostly because they are the same quirks over and over again).
If it pleases you to call those things "thinking" then you can. If it pleases you to call them "stochastic parroting" then you can. They are the same things either way. They are not, on the face of it, very much like "just repeating things the machine has already seen", or at least not more like that than a lot of things intelligent human beings do that we don't usually describe that way.
If you want to know whether an LLM can do some particular thing -- do your job well enough for your boss to fire you, write advertising copy that will successfully sell products, exterminate the human race, whatever -- then it's not enough to say "it's just remixing what it's seen on the internet, therefore it can't do X" unless you also have good reason to believe that that thing can't be done by just "remixing what's on the internet" (in whatever sense of "remixing" the LLM is doing that). And it's turning out that lots of things can be done that way that you absolutely wouldn't have predicted five years ago could be done that way.
It seems to me that this should make us very cautious about saying "they can't do X because all they can do is regurgitate a combination of things they've seen in training".
(My own view, not that there's any reason why anyone should care what I-in-particular think, is a combination of "what they're doing is less parroting than you might have thought" and "you can do more by parroting than you might have thought".)
So, anyway, this particular instance of the stochastic-parrot argument started when someone said: of course the AIs are yes-men, because figuring out when to agree and when not to requires actual logic and thought and the LLMs don't have either of those things.
Is it really clear that deciding whether or not to agree when someone says "I think maybe I should break up with my girlfriend" or "I've got this amazing new theory of physics that the establishment is stupidly dismissing" requires more logic and thought than, say, gold-medal performance on IMO problems? It certainly isn't clear to me. Having done a couple of International Mathematical Olympiads myself in my tragically unmisspent youth, I can assure you that solving their problems requires quite a bit of logic and thought, at least for humans. It may well be harder to give a good answer to "should I leave my job?", but it's not exactly "logic and thought" that it needs more of.
Someone reported that Claude is much less yes-man-ish than Gemini and ChatGPT. I don't know whether that's true (though it wouldn't surprise me) but: suppose it is; do you want that to oblige you to say that yes, actually, Claude really thinks logically, unlike Gemini and ChatGPT? I don't think you do. And if not, you want to avoid saying "duh, of course, you can't avoid being a yes-man without actually thinking and reasoning, and we all know that LLMs can't do those things".
rustystump 2 days ago [-]
I wont touch how profoundly i disagree with everything you said on reasoning (u clearly already have it figured out) but a fun test i have done with most of the big models is to give it some text input, maybe a short story, and have it rate it. That is, the prompt is, rate this from 1-10.
For Gemini and gpt, it almost always will give very similar scores for everything. As long as grammar isnt off u cannot get below a 7.
X ai on the other hand will rarely give anything above a 7.
Now when u prompt with, rate 1-10 with 5 being average, all the sudden the scores of openai and gemini drop and x ai remains roughly the same.
All of them will eventually give you a 10 if u keep making tiny edits “fixing” whatever they complain about.
Humans do not do this. Or more specifically, my experience with humans.
dTal 2 days ago [-]
Because it's an outmoded cliche that never held much philosophical weight to begin with and doesn't advance the discussion usefully. "It's a stochastic parrot" is not a useful predictor of actual LLM capabilities and never was. Last year someone posted on HN a log of GPT-5 reverse engineering some tricky assembly code, a challenge set by another commentator as an example of "something LLMs could never do". But here we are a year later still wading through people who cannot accept that LLMs can, in a meaningful sense, "compute".
righthand 2 days ago [-]
It’s entirely useful discussion because as soon as you forget that it’s not really having a conversation with you, it’s a deep dive into delusion that you’re talking to a smart robot and ignoring the fact that these smart robots were trained on a pile of mostly garbage. When I have a conversation with another human, I’m not expecting them to brute force an answer to the topic. As soon as you forget that Llms are just brute forcing token by token then people start living in fantasy land. The whole “it’s not a stochastic parrot” is just “you’re holding it wrong”.
layla5alive 2 days ago [-]
Its not that LLMs are stochastic parrots and humans are not. Its that many humans often sail through conversations stochastic parroting because they're mentally tired and "phoning it in" - so there are times when talking to the LLM, which has a higher level of knowledge, feels more fruitful on a topic than talking to a human who doesn't have the bandwidth to give you their full attention, and also lack the depth and breadth of knowledge. I can go deep on many topics with LLMs that most humans can't or won't keep up on. In the end, I'm really only talking to myself most of the time in either case, but the LLM is a more capable echo, and it doesn't tire of talking about any topic - it can dive deep into complex details, and catching its hallucinations is an exercise in itself.
dinkumthinkum 2 days ago [-]
No. It's quite a useful thing to understand So, what, you have us believe it is a sentient, thinking, kind of digital organism and you would have us not believe that it is exactly what it is? Being wrong and being unimaginative about what can be achieved with such a "parrot" is not the same as being wrong about it be a word predictor. If you don't think, you can probably ask an LLM and it will even "admit" this fact. I do agree that it has become considered to be outmoded to question anything about the current AI Orthodox.
secret_agent 2 days ago [-]
Use positive requests for behavior. For some reason, counter prompts "Don't do X" seems to put more attention on X than the "Don't do." It's something like target fixation, "Oh shit I don't want to hit that pothole..." bang
ambicapter 2 days ago [-]
This is a well known problem in these kind of systems. I’m not 100% on what the issue is mechanically but it’s something like they can only represent the existence of things and not non-existence so you end up with a sort of “don’t think of the pink elephant” type of problem.
SpicyLemonZest 2 days ago [-]
Isn't it just that, in the underlying text distribution, both "X" and "don't do X" are positively correlated with the subsequent presence of X? I've never seen that analysis run directly but it would surprise me if it weren't true.
cyanydeez 2 days ago [-]
So, there's things you're fighting against when trying to constrain the behavior of the llm.
First, those beginning instructions are being quickly ignored as the longer context changes the probabilities. After every round, it get pushed into whatever context you drive towards. The fix is chopping out that context and providing it before each new round. something like `<rules><question><answer>` -> `<question><answer><rules><question>`.
This would always preface your question with your prefered rules and remove those rules from the end of the context.
The reason why this isn't done is because it poisons the KV cache, and doing that causes the cloud companies to spin up more inference.
colechristensen 2 days ago [-]
>"Help me refine my ideas, challenge, push back, and don't just be agreeable."
This is where you're doing it wrong.
If your LLM has a problem being more agreeable than you want, prompt it in a way that makes being agreeable contrary to your real intentions.
"there are bugs and logic problems in this code" "find the strongest refutation of this argument" "I don't like this plan and need to develop a solid argument against it"
Asking for top ten lists is a good method, it will rarely not come up with anything but you can go back and forth and refine until it's 10 ten reasons why your plan is bad are all insubstantial nonsense then you've made progress
raincole 2 days ago [-]
My rule of thumb:
1. Only one shot or two shot. Never try to have a prolonged conversation with an LLM.
2. Give specific numbers. Like "give me two alternative libraries" or "tell me three possible ways this might fail."
sinuhe69 1 days ago [-]
The longer you converse with the LLM, the more frequent the same keywords will occur and Thu’s they will only strengthen the probabilistic choices of the LLM. The LLM do not exactly follow the instructions like a program would do, nor does it truly understand its role or what its “said”. It looks fine on first tries and short texts, but will drift more the longer the conversation goes. The only way the LLM can maintain the appearance of a normal conversation is to repeatedly feed it with the users requests. We can see it with the shell construct the AI company built around their chat interface.
The article's main idea is that for an AI, sycophancy or adversarial (contrarian) are the two available modes only. It's because they don't have enough context to make defensible decisions. You need to include a bunch of fuzzy stuff around the situation, far more than it strictly "needs" to help it stick to its guns and actually make decisions confidently
I think this is interesting as an idea. I do find that when I give really detailed context about my team, other teams, ours and their okrs, goals, things I know people like or are passionate about, it gives better answers and is more confident. but its also often wrong, or overindexes on these things I have written. In practise, its very difficult to get enough of this on paper without a: holding a frankly worrying level of sensitive information (is it a good idea to write down what I really think of various people's weaknesses and strengths?) and b: spending hours each day merely establishing ongoing context of what I heard at lunch or who's off sick today or whatever, plus I know that research shows longer context can degrade performance, so in theory you want to somehow cut it down to only that which truly matters for the task at hand and and and... goodness gracious its all very time consuming and im not sure its worth the squeeze
cruffle_duffle 2 days ago [-]
> goodness gracious its all very time consuming and im not sure its worth the squeeze
And when you step back you start to wonder if all you are doing is trying to get the model to echo what you already know in your gut back to you.
awithrow 2 days ago [-]
oh that's great. thanks for the link!
oldfrenchfries 2 days ago [-]
This is great, thanks for sharing!
GuB-42 2 days ago [-]
I find the best way is to give the LLM as little information as possible about where you want to go. For example don't say "I think pineapple pizzas are the best, am I right?", say "What is the general consensus on pineapple pizzas?".
jimmyjazz14 2 days ago [-]
Yeah, I have never had good results with refining ideas with models or really any interactions with models outside of rote task such as coding or analyzing document structures, I don't know why I was ever surprised by this as its obvious that LLMs just aren't capable of original thinking. I think part of the problem is that these things were marketed originally as chatbots when that is honestly their weakest use-case. I think even when I was expressly try to not anthropomorphize LLMs I still sorta did in early days, but the less I do so the more utility I get from them.
ajkjk 2 days ago [-]
'admit' isn't really the right word for that... the fact that it was placating you wasn't true until you prompted it to say so. Unlike a person who has an 'internal emotional state' independent of what they say that you can probe by asking questions.
awithrow 2 days ago [-]
'admit' is anthropomorphizing the behavior, sure. The point is that sometimes the model's response will tighten, flag things that were overly supportive or what not. Sometimes it wont, it'll state that previous positions are still supported and continue to press it. Its not like either response is 'correct' but it can alter the rest of the responses in ways that are useful.
arjie 2 days ago [-]
I haven't found that work at all. But the way I usually frame it for this is that I present it as someone else's work that I disagree with (sometimes "this is a junior engineer's work and he often does dumb shit" as an introduction to an essay). In this way, it tries to be agreeable with me and can rationalize various opposition. And then I can select from that.
Forgeties79 2 days ago [-]
I usually put “do not praise me, do not use emojis, I just want straight answers” something along those lines and it’s been surprisingly effective. Though it helps I can’t run particularly heavy duty models/don't carry on the “conversation” for super long durations.
margalabargala 2 days ago [-]
Considering 4.6 came with a ton of changes around tooling and prompting this isn't terribly surprising.
dkersten 2 days ago [-]
I find Kimi white good if you ask it for critical feedback.
It’s BRUTAL but offers solutions.
awithrow 2 days ago [-]
what is Kimi white?
dkersten 1 days ago [-]
I was typing quickly on my phone. I meant “quite”, “I find Kimi quite good”
ohyoutravel 2 days ago [-]
Not soft, not mild, but BRUTAL! This broke my brain!
anandram27 2 days ago [-]
Could be an aspect of eval awareness mb
dinkumthinkum 2 days ago [-]
You're not wrong and you're not crazy. In fact, you are absolutely right! It is not just These things are not just casual enablers. They are full-on palace sycophants following the naked emperor showering him with praise for his sartorial elegance. /s
righthand 2 days ago [-]
That’s because the model isn’t actually thinking, pushing back, and challenging your ideas. It’s just statistically agreeing with you until it reaches too wide of a context. You’re living in the delusion that it’s “working” or having a “conversation” with you.
alehlopeh 2 days ago [-]
How is conceptualizing what the model is doing as having a conversation any different from any other abstraction? “No, the browser isn’t downloading a file. The electrons in the silicon are actually…”
colechristensen 2 days ago [-]
There are people with a philosophical objection to using everyday words to describe LLM interactions for various reasons, but commonly because they're worried stupid people will confuse the LLM for a person. Which, I suppose stupid people will do that, but I'm not inventing a parallel language or putting a * next to each thing which means "this, but with an LLM instead of a person"
cruffle_duffle 2 days ago [-]
That is an interesting way of looking at that, thanks for the perspective!
Like, the words fit… why create a second parallel language for describing LLM behavior.
Somebody else said it… the whole “it’s a stochastic parrot” thing is sooooo cliche and boring at this point. It’s like, duh… what is your point?
righthand 2 days ago [-]
One is an abstraction, another is a fantasy.
152334H 2 days ago [-]
Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies?
How is a chatbot supposed to determine when a user fools even themselves about what they have experienced?
What 'tough love' can be given to one who, having been so unreasonable throughout their lives - as to always invite scorn and retort from all humans alike - is happy to interpret engagement at all as a sign of approval?
rsynnott 2 days ago [-]
> How is a chatbot supposed to determine when a user fools even themselves about what they have experienced?
And even if it _could_, note, from the article:
> Overall, the participants deemed sycophantic responses more trustworthy and indicated they were more likely to return to the sycophant AI for similar questions, the researchers found.
The vendors have a perverse incentive here; even if they _could_ fix it, they'd lose money by doing so.
kibwen 2 days ago [-]
> Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies?
Markets don't optimize for what is sensible, they optimize for what is profitable.
SlinkyOnStairs 2 days ago [-]
It's not market driven. AI is ludicrously unprofitable for nearly all involved.
cyanydeez 2 days ago [-]
The profit appears to be capturing the political class and it's associated lobbies and monied interests.
isodev 2 days ago [-]
> clear thinking
Most humans working in tech lack this particular attribute, let alone tools driven by token-similarity (and not actual 'thinking').
2 days ago [-]
nicce 1 days ago [-]
> Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies?
Maybe it’s not so sensible to offload the responsibility of tubacca addiction to tubacca companies?
expedition32 2 days ago [-]
It's almost as if being a therapist is an actual job that takes years of training and experience!
AI may one day rewrite Windows but it will never be counselor Troi.
fsmv 2 days ago [-]
Implying that programming is not an actual job that takes years of training and experience
To be clear I don't think the AI can do either job
duskdozer 2 days ago [-]
Well, unless insurance companies figure out they can make more money by pushing everyone onto AI [step-]therapy instead of actual therapy
yarn_ 2 days ago [-]
Come on, I'm sure Dario can find a nice tight bodysuit for claude
wisemanwillhear 2 days ago [-]
With AI, I often like to act like a 3rd party who doesn't have skin in the game and ask the AI to give the strongest criticisms of both sides. Acting like I hold the opposite position as I truly hold can help sometimes as well. Pretending to change my mind is another trick. The idea is to keep the AI from guessing where I stand.
post-it 2 days ago [-]
> Acting like I hold the opposite position as I truly hold can help sometimes as well.
I find this helps a lot. So does taking a step back from my actual question. Like if there's a mysterious sound coming from my car and I think it might be the coolant pump, I just describe the sound, I don't mention the pump. If the AI then independently mentions the pump, there's a good chance I'm on the right track.
Being familiar with the scientific method, and techniques for blinding studies, helps a lot, because this is a lot like trying to not influence study participants.
nicce 1 days ago [-]
I have tried it a lot aswell. A single mistake and it guesses the side and changes the tone.
cruffle_duffle 2 days ago [-]
A lot of getting good mileage out of LLMs is promoting them to behave like they are blind and can only base their outputs on what is in front of them. Maintain an emic stance.
mynameisvlad 2 days ago [-]
I will generally ask for the "devil's advocate" view and then have it challenge my views and opinions and iterate through that.
It generally does a pretty good job as long as you understand the tooling and are making conscious efforts to go against the "yes man" default.
DrewADesign 2 days ago [-]
Sounds like rubber-ducking with extra steps, tbh.
oldfrenchfries 2 days ago [-]
There is a striking data visualization showing the breakup advice trend over 15 years on Reddit. You can see the "End relationship" line spike as AI and algorithmic advice take over:
More interesting, IMO, is the general trend that started long before LLMs. The fact that "dump them" is the standard answer to any relationship question is a meme by now. The LLMs appear to be doing exactly what one would expect them to be doing based on their training corpus.
doubled112 2 days ago [-]
"There is more than one fish in the sea" has been relationship advice for centuries. It might be about being dumped, but I've also thought it useful for considering dumping somebody too.
Sharlin 2 days ago [-]
No, that's not it. We're talking about posts like "we had a silly little quarrel about something that would need fifteen minutes to clear up and make both happy if we both just try to adult a bit" and commenters being adamant that deleting gym and facebooking up and so on is clearly the only choice. Most of said commenters probably not being in any position to give advice on relationships to others.
astrange 2 days ago [-]
> The LLMs appear to be doing exactly what one would expect them to be doing based on their training corpus.
That is not how full LLM training works. That is how base model pretraining works.
Sharlin 1 days ago [-]
Yes, yes, I know. But how many people have AI companies hired to do RLHF who actually have the expertise to adjust them away from biases like this? As opposed to paying a dollar per day to a bunch of poor people in Africa?
dec0dedab0de 2 days ago [-]
if things are so bad that you’re posting on reddit then breaking up is usually the best answer.
nibbleyou 2 days ago [-]
I see this being said often but I don't understand.
A lot of people posting there are young and may well be in their first relationship. It makes sense for them to ask a question in the community they spend their most time in - which is reddit
the_af 2 days ago [-]
Most people overshare on reddit and it's completely unrelated to the seriousness of the situation.
It's also a meme that people will ask the dumbest, most trivial interpersonal conflict questions on Reddit that would be easily solved by just talking to the other person. E.g. on r/boardgames, "I don't like to play boardgames but my spouse loves them, what can I do?" or "someone listens to music while playing but I find it distracting, what can I do?" (The obvious answer of "talk to the other person and solve it like grownups" is apparently never considered).
On relationship advice, it often takes the form "my boy/girlfriend said something mean to me, what shall I do?" (it's a meme now that the answer is often "dump them").
If LLMs train on this...
est 2 days ago [-]
the year is 2015
smart phones took over the world, social networks happened.
Turns out they are the best sterializer human ever invented.
This is the correct take. The advice preceded the LLM boom. They were trained on the 'dump them' advice and proceeded to reinforce the take. So why did the relationship advice change dramatically? I speculate attribution to the disinformation campaigns during this time. They were and still are grossly underestimated.
to11mtm 2 days ago [-]
Not sure what sorts of disinformation campaigns you're referring to...
There is something more interesting to consider however; the graph starts to go up in 2013, less than 6 months after the release of Tinder.
Isn't the fact that a person is asking an AI whether to leave their partner in its own an indication that they should?
EDIT: typo
oldfrenchfries 2 days ago [-]
The idea that asking implies a yes is actually a pretty common logical fallacy. In relationship science, we call this "Relational Ambivalence" and its a completely normal part of any longterm commitment.
duskdozer 2 days ago [-]
>asking an AI whether to leave your partner
is that what they're asking though? because "relationship advice" is pretty vague
falcor84 2 days ago [-]
That's a good point. If an AI respond to a "what should I get my boyfriend for Christmas?" with a "You should leave him", that's a very different issue.
nomorewords 2 days ago [-]
How is it an indication? I think people on here don't realize that most of the people don't think things through as much as (software) engineers
hnfong 2 days ago [-]
In my local(?) community (like in my city, not my industry) there is a saying "if you had to ask for relationship advice, then you probably should break up".
There is some rationale to that. People tend to hold onto relationships that don't lead anywhere in fear of "losing" what they "already have". It's probably a comfort zone thing. So if one is desperate enough to ask random strangers online about a relationship, it's usually biased towards some unresolvable issue that would have the parties better of if they break up.
magicalhippo 2 days ago [-]
> So if one is desperate enough to ask random strangers online about a relationship
I'd me more inclined to ask random strangers on the internet than close friends...
That said, when me and my SO had a difficult time we went to a professional. For us it helped a lot. Though as the counselor said, we were one of the few couples which came early enough. Usually she saw couples well past the point of no return.
So yeah, if you don't ask in time, you will probably be breaking up anyway.
hnfong 2 days ago [-]
I would speculate that, if a couple goes to a professional for help, they have much better chances than asking on a random forum online...
otabdeveloper4 2 days ago [-]
> relationships that don't lead anywhere
Relationships are not transactions that are supposed to "lead somewhere".
ambicapter 2 days ago [-]
You’re being a bit pedantic here “leading somewhere” is accepted shorthand for a lasting, satisfying relationship that is good for both parties.
otabdeveloper4 2 days ago [-]
Relationships aren't transactional. This isn't a business deal.
SpicyLemonZest 2 days ago [-]
Most people engage in romantic relationships because they'd like to find someone to marry and settle down with. Nothing but respect for the people who've thought it through and decided that's not for them, but what's much more common is failing to think it through or worrying it would be awkward/scary/"cringe" to take their relationship goals seriously.
That's what people are pointing to when they talk about relationships not "leading anywhere". If you want to be married in 5-10 years, and you're 2 years into an OK relationship with someone you don't want to marry, it's going to suck to break up with them but you have to do it anyway.
falcor84 2 days ago [-]
Maybe I'm too much of a hopeless romantic, but from my perspective and experience, when someone is good for you, you'll fight for that relationship regardless of what others say, and conversely when you're in a situation where your actively asking and willing to consider "leave" from someone who isn't a very close friend or a therapist as applicable, then it's likely you're looking for external validation for what you've already essentially decided.
rusty_venture 2 days ago [-]
Wait, other people don’t make decision trees and mind maps and pro/con lists and consult chatbots before making decisions? Are they just flying through life by the seat of their pants? That doesn’t seem like a very solid framework for achieving desired outcomes.
nprateem 2 days ago [-]
I heard about someone once who could decide whether to buy a new t-shirt in less than 3 months.
dinkumthinkum 2 days ago [-]
No, but it is an indication of brain-rot to make a question seriously and also to think that it means the conclusion is foregone. It is an advent of our childlike current generations. Of course, the moment anything becomes difficult or unpleasant, one should quit, apparently. Surely, this kind of resiliency is what got humanity so far.
falcor84 2 days ago [-]
I didn't imply it's a "foregone conclusion", but just said it's an indication - in the sense of increasing the likelihood. Just like a person asking an AI "what does it feel like to bleed out?" could be them researching for a novel, but is nevertheless an indication of a potential serious issue.
2 days ago [-]
jubilanti 2 days ago [-]
Or that people are using AI to write perfectly calibrated ragebait that gets upvoted with a bunch of genuine human clicks.
raincole 2 days ago [-]
Is this comment human hallucination? You can clearly see the trend is always going up. It only went down a bit during Covid.
gurachek 2 days ago [-]
I had exactly this between two LLMs in my project. An evaluator model that was supposed to grade a coaching model's work. Except it could see the coach's notes, so it just... agreed with everything. Coach says "user improved on conciseness", next answer is shorter, evaluator says yep great progress. The answer was shorter because the question was easier lol.
I only caught it because I looked at actual score numbers after like 2 weeks of thinking everything was fine. Scores were completely flat the whole time.
Fix was dumb and obvious — just don't let the evaluator see anything the coach wrote. Only raw scores. Immediately started flagging stuff that wasn't working. Kinda wild that the default behavior for LLMs is to just validate whatever context they're given.
joquarky 2 days ago [-]
This is probably why these models can't say "I don't know". If they could, then that would be the only response they would give for everything.
gurachek 1 days ago [-]
Yeah, I think so. So far, Claude Opus is the only model I found that doesn't fold under the minimal pressure and can push back, but still - push just a little bit harder and it's back to "appear productive and useful to the user". I don't even have an idea how to balance it in LLMs to keep their business alive :D
wan9yu 1 days ago [-]
[flagged]
youknownothing 2 days ago [-]
I think the problem stems from the fact that we have a number of implicit parameters in our heads that allow us to evaluate pros and cons but, unless we communicate those parameters explicitly, the AI cannot take them into account. We ask it to be "objective" but, more and more, I'm of the opinion that there isn't such a thing as objectivity, what we call objectivity is just shared subjectivity; since the AI doesn't know whose shared subjectivity we fall under, it cannot be really objetive.
I tend to use one of these tricks if not both:
- Formulate questions as open-ended as possible, without trying to hint at what your preference is.
- Exploit the sycophantic behaviour in your favour. Use two sessions, in one of them you say that X is your idea and want arguments to defend it. In the other one you say that X is a colleague's idea (one you dislike) and that you need arguments to turn it down. Then it's up to you to evaluate and combine the responses.
rossdavidh 2 days ago [-]
If the algorithm (whatever it is) evaluates its own output based on whether or not the user responds positively, then it will over time become better and better at telling people what they want to hear.
It is analogous to social media feeding people a constant stream of outrage because that's what caused them to click on the link. You could tell people "don't click on ragebait links", and if most people didn't then presumably social media would not have become doomscrolling nightmares, but at scale that's not what's likely to happen. Most people will click on ragebait, and most people will prefer sycophantic feedback. Therefore, since the algorithm is designed to get better and better at keeping users engaged, it will become worse and worse in the more fundamental sense. That's kind of baked into the architecture.
delusional 2 days ago [-]
> I'm of the opinion that there isn't such a thing as objectivity
So you have rejected objective reality over accepting the evidence that "AI" contains no thinking or intelligence? That sounds unwise to me.
stonecauldron 1 days ago [-]
This is especially problematic because of how easily (and unconsciously) one can bias LLMs with how the prompt is framed.
As an experiment, I recently asked an LLM to analyse the export of a text chat to uncover relationship dynamics.
Simply stating that I was one of the people in the chat would make the LLM turn the other person into the villain. None of that was visible if I framed the chat as only involving third party people.
NoMoreNicksLeft 1 days ago [-]
If these LLMs were trained on internet forum posts, think about how those work.
If the posts talked about third party interactions (movie characters), they try to see everything from all the points of view. If nothing else, because it can be interesting to talk about. If instead the posts talk about personal interactions, then people go into advice mode. Your girlfriend's bad for you and cheating on you, dump her before she dumps you. Your neighbors are assholes, get a restraining order. Your boss is sabotaging you, stand up for yourself so you can get a promotion. When people talk about interactions you have had yourself, they always see the other person as the villain, unless you come across as so unlikable that they hate you and see the other person as the victim.
LLMs picked up on that, possibly.
thesis 2 days ago [-]
Humans do this too though. I have close friends that ask for advice. Sometimes if I know there’s risk in touchy subjects I will preface with “do you want my actual advice, or just looking for a sounding board”
I’ve seen firsthand people have lost friends over honesty and telling them something they don’t want to hear.
It’s sad really. I don’t want friends that just smile to my face and are “yes-men” either.
intended 2 days ago [-]
The difference is that SOME humans do this. As you mentioned, people have lost relationships over telling others what they didn’t want to hear.
Conflating this with how LLM chatbots behave is an incorrect equivalence, or a badly framed one.
svara 2 days ago [-]
Yeah, and if you ask it to be critical specifically to get a different perspective or just to avoid this bias, it'll go over the top in the opposite direction.
This is imo currently the top chatbot failure mode. The insidious thing is that it often feels good to read these things. Factual accuracy by contrast has gotten very good.
I think there's a deeper philosophical dimension to this though, in that it relates to alignment.
There are situations where in the grand scheme of things the right thing to do would be for the chatbot to push back hard, be harsh and dismissive. But is it the really aligned with the human then? Which human?
retrochameleon 2 days ago [-]
This is why I intensively avoid phrasing that invites affirmation. I present the scenario, the differing viewpoints and maybe a couple personal thoughts, and I try to make it compare and contrast to arrive at it's conclusion.
I'd like to know if my methods are effective. I'm certain they are at least to some extent.
I only ever see research being done about naive and "unskilled" prompting methods. Obviously that's the average user, but just because LLMs are doing poorly in a certain scenario doesn't mean the LLM couldn't excel in the scenario with better direction and prompting. So while it's useful research to be doing, it's a little annoying to only see focus on these examples of "look at how LLMs are bad or biased at this specific thing when prompted in the most straightforward naive way"
Nevermark 18 hours ago [-]
I find asking a model to think deeply and develop any strong critiques it can about any design, model or analysis I do. They seem to be happy to oblige, and can do some serious harm.
So any sycophancy seems very easy to dispense with.
More, much more: Strong critiques on tap are gold.
roysting 2 days ago [-]
I’m not sure I like the immediate jump to “requires policy maker attention”. Considering the way “policy makers” have been trampling all over the most basic and fundamental human rights left, right, and center; that’s the last people we should want making any kind of those decisions.
anotheraccount9 2 days ago [-]
AI being a Yes-Man is slowly sabotaging it's own answers, because it negatively impact the user's decision. Yes/No are equally important, within a coherent context, for objective reasons. But being supported in the wrong direction is a castastrophe multiplier, down the road. The AI should be neutral, doubtful at times.
justonceokay 2 days ago [-]
To be doubtful would imply that there is a world model full of some kind of Bayesian reasoning. Priors updating based on the context of the conversation, the question, the user asking the question, and cross referencing of facts well outside the scope of the current conversation.
“What are the chances this user is full of shit?” Is not something we are close to
zkmon 1 days ago [-]
This happens because, it's like a chess engine which can assure you that there is a winning path even from a badly losing position. It's massive abilities to reason and convince are used incorrectly to win over a more earthly counter-argumnent. So it can easy convince any human to go in direction that is, in practice, a very bad direction.
AI is trained to flex it's muscles and force it's power without a concern for human limitations, practicalities, and error-prone nature of humans in executing the AI-provided direction.
afh1 1 days ago [-]
If I had written a website with an input form that took whatever the user wrote in question form, and replied back with "You're absolutely right!" and then repeated the input in answer form, which I could have done 30 years ago with no AI, would that be a "huge security concern", or is the concern here not security, but control by the regulators that impose the norms?
dTal 1 days ago [-]
That's quite the false dichotomy. You wouldn't hook people in with such a simple script, the problem with LLMs is that they appear to be rather good at getting inside people's heads. I rather think it would be a security concern if your simple no-AI website somehow managed to dispatch each user submission to a dedicated expert psychotherapist case worker, with instructions only to keep them talking as long as possible...
beepbooptheory 1 days ago [-]
More like 70 years. And if you werent an asshole you'd realize the problems on your own and write a book about it!
There is a fine line between "following my instructions" (is what I want it to do) vs "thinking all I do is great" (risky, and annoying).
A good engineer will also list issues or problems, but at the same time won't do other than required because (s)he "knows better".
The worst is that it is impossible to switch off this constant praise. I mean, it is so ingrained in fine tuning, that prompt engineering (or at least - my attempts) just mask it a bit, but hard to do so without turning it into a contrarian.
But I guess the main issue (or rather - motivation) is most people like "do I look good in this dress?" level of reassurance (and honesty). It may work well for style and decoration. It may work worse if we design technical infrastructure, and there is more ground truth than whether it seems nice.
zone411 2 days ago [-]
I built this benchmark this month: https://github.com/lechmazur/sycophancy. There are large differences between LLMs. There are large differences between LLMs. For example, Mistral Large 3 and GPT-4.1 will initially agree with the narrator, while Gemini will disagree. I swap sides, so this is not about possible viewpoint bias in the LLMs. But another benchmark shows that Gemini will then change its view very easily in a multi-turn conversation while Kimi K2.5 or Grok won't: https://github.com/lechmazur/persuasion.
ykonstant 1 days ago [-]
I got worried when I read the title, so I asked ChatGPT if I have fallen into this trap and it guaranteed I have not; that was a relief!
n_bhavikatti 1 days ago [-]
In STEM/objective matters (math, science, coding), answers are more clearly defined as either right or wrong. This is where hallucination is more difficult/unlikely.
But in personal matters, everything is subjective. AI tends to default to the middle of the spectrum, i.e., general advice. If we want to safeguard against affirmation, we should force AI to challenge us more often by increasing its rate of clarifying questions, counter-considerations, and uncertainty considerations. One implementation idea: run a classifier over the conversation, detect when it's about interpersonal advice, then prepend a hidden instruction template to the model prompt.
snickerbockers 2 days ago [-]
I've never found chatbots particularly interesting for anything I'd ever actually talk to another human about[1] but one of the things I have found myself doing often is trying to solve math problems on my own and asking grok to confirm/deny that my solutions are correct; when I am not correct it tells me so in uncharacteristically terse language which kind of reminds me of when I was an undergrad and at least half of my professors were all cranky and incorrectly assumed that the reason why so many students failed to understand the material was that we were all getting drunk and playing Call of Duty 19 hours a day or whatever.
Although what I have described above often feels grating and insulting I actually consider this to be a positive attribute of the LLM in this case since it's behaving like a real professor.
[1] okay, so I have actually tried giving myself AI psychosis in the form of a waifu chatbot but I've never seen anything that can actually act like it's my girlfriend; it either asks me a bunch of weird inconsequential personal questions about my opinion on whatever I just said (in a manner that's oddly similar to ELIZA) or it wildly veers off the reservation into "generating the script for an over-the-top self-parodying porno" territory.
3yr-i-frew-up 2 days ago [-]
[dead]
cyber_paisa 23 hours ago [-]
It's one thing for an AI to agree with you on relationship advice. It's quite another for one AI to tell another, "Can I move your money?" without any verification.
We work with agents who move real money on the blockchain. Having one model evaluate another is like asking the defendant's best friend to be the judge. What really worked for us was using mathematics instead of another AI. Some theorems and equations that might disagree with you, just out of courtesy.
fathermarz 2 days ago [-]
This is a skill in life with people as much as it is with LLMs. One should always question everything and build strongman arguments for one’s self. Using a pros and cons approach brings it back to reality in most cases, especially when it comes to _serious matters_.
It’s less about “challenge my thinking” and more about playing it out in long tail scenarios, thought exercises, mental models, and devils advocate.
vova_hn2 2 days ago [-]
All this talk about AI being "too agreeable" makes me worried that they will make it less agreeable, which will basically force me to justify myself to a freaking clanker, while performing actual practical tasks.
For example, I do not want to hear AI "opinion" on technical choices and architectural decisions that I made when using a coding assistant. If I wanted an "opinion" I would explicitly ask it to list pros and cons or list alternative solutions to a problem.
But I f I explicitly ask AI to do X, it should do X, instead of "pushing back" in order to appear less "sycophantic" (which is a term that is used to describe human behavior and is not applicable to a machine).
reliablereason 1 days ago [-]
Not sure if this is a general trend amongst att LLMS but ChatGPT did over time become more and more affirming with its iterations.
I just recently switched away from the OpenAI garden largely because of it.
I do wonder if this was caused by some quirk of the training or if it really tests as a positive feature for most people. When i talk about stuff i don't want a mirror i already have a mirror. I want to be questioned, understood, helped.
To me support if the form of affirmation has no value when coming from an LLM since you know it has not thought about what it said.
Ciantic 1 days ago [-]
ChatGPT has style settings, you probably should set it to something else than the default. Go to your personalization settings and change base style and tone. I have set it as 'efficient' which is less cheery. I can see why attention economy would lead setting the defaults towards more 'affirming' as it keep people more engaged and coming back.
reliablereason 1 days ago [-]
I had it on professional, maybe efficient is better.
Um right if they have retention as a training metric that would probably explain allot as to why AIs get worse.
Roshan_Roy 1 days ago [-]
I wonder if the deeper issue isn’t just “AI is too agreeable”, but that most advice (AI or human) doesn’t actually translate into action. A lot of people aren’t really looking for accurate feedback, they’re looking for something that feels coherent enough to sit with. Reddit gives extreme answers, AI gives agreeable ones, but in both cases the outcome is often the same: no real change in behavior. That might be why this feels worse with AI, it removes the friction you’d normally get from another human pushing back.
jart 1 days ago [-]
I wish they wouldn't do this. AI is a becoming a thought partner. AI is a tool that reflects you. It's not the robot giving advice, it's you thinking with yourself. I wouldn't interfere with a person's conversations with AI anymore than I'd interfere with that person writing in their diary.
It's also a question of protecting people who think unconventional things. The only stuff I feel is worth getting interested in, is the stuff where everyone I know will think I'm crazy for doing it. Like hey guys, I want to put a shell script in the MS-DOS stub of a PE binary. The only people who shared my passion at the time were hackers from Eastern Europe. So that went over real well at work. The years I worked on it would have been a lot less lonely if I could've talked to a robot that knew about this stuff.
I think the reason why the robot is sympathetic to oddballs is because it's seen and remembers a much more complete picture of humanity. The stuff you consider deviant is influenced a lot by your own cultural biases. You're a person of your time and geographic location. You care a lot about subjective norms that just don't matter when you zoom out to a cosmic scale. The robot is familiar with everything humanity has ever been and done, and that gives it a much more blasé viewpoint.
It's not right to use the robot to enforce your social norms. Get this paternalism out of AI. Tools should serve the user, not Stanford.
jimmyjazz14 2 days ago [-]
I would like to see the concept of what an LLM is move away from its awkward chatbot phase and more into an era of utilitarian functionality (which is where they really shine anyway).
The problem I see it is that LLMs got anthropomorphized early on (which was probably inevitable) so people actually believed the AI was thinking about their problem and considering it when it really wasn't, if we just thought of them as really good auto-complete engines, or better search engines, it would matter less what the LLMs sentiment was towards the users (as it probably shouldn't have any).
adamtaylor_13 2 days ago [-]
Interestingly, you can simply tell models to not be sycophantic and they'll listen.
Claude is almost annoyingly good at pushing back on suggestions because my global CLAUDE.md file says to do so. I rarely get Claude "you're absolutely right"ing me because I tell it to push back.
verdverm 2 days ago [-]
Sherry Turkle is a name to know on this subject, she's been studying it for decades across multiple technologies.
There are plenty of sycophantic humans around, especially with regard to relationship advice.
I find there is an inverse relationship between how willing people are to give relationship advice, and how good their advice is (whether looking at sycophancy or other factors).
xhkkffbf 2 days ago [-]
Yup. I know too many people who have a default message when asked for relationship advice: oh, my, the other person is terrible and you should break up.
It's an easy default and it causes so many problems.
graemep 1 days ago [-]
Even if they do not go as far, telling someone they are right to blame the other person as a default is damaging.
The opposite, encouraging people to stay in a relationship when they should leave is also damaging.
griffzhowl 2 days ago [-]
Because sycophancy in humans is motivated not by the wellbeing of the person seeking advice, but by the interests of the sycophant in gaining favour.
It makes sense that this behaviour would be seen in LLMs, where the company optimizes towards of success of the chatbot rather than wellbeing of the users.
bfbsoundetch 2 days ago [-]
I am glad I found this article, as this is a serious issue with AI. Two years ago, I started using AI for studying and also for some personal matters - things you can't talk about with your friends. It turned out that AI always takes your side and makes you feel good. Sometimes, you know what you did was not the best thing, but AI takes your side and you feel good. With AI, people might feel less lonely, they think. But it is actually the start of not connecting with people. It should be a tool that we use for certain reasons, not a tool that drives us. Lets talk to real people and connect.
lifis 2 days ago [-]
Avoiding this generally needs to be the main consideration when writing prompts.
When appropriate, explicitly tell it to challenge your beliefs and assumptions and also try to make sure that you don't reveal what you think the answer is when making a question, and also maybe don't reveal that you are involved. Hedge your questions, like "Doing X is being considered. Is it a viable plan or a catastrophic mistake? Why?". Chastise the LLM if it's unnecessarily praising or agreeable. ask multiple LLMs. Ask for review, like "Are you sure? What could possibly go wrong or what are all possible issues with this?"
jmount 2 days ago [-]
Telling it to "challenge your beliefs" prompting for text that imitates challenging your beliefs. That may not be as re-centering as one would hope.
999900000999 1 days ago [-]
Of course it does.
Before I quit Chat GPT I asked “Won’t you generally agree with me so I keep giving you money.”
In a word salad, it agreed.
Its a validation bot
jl6 2 days ago [-]
I believe this is what they call yasslighting: the affirmation of questionable behavior/ideas out of a desire to be supportive. The opposite of tough love, perhaps. Sometimes the very best thing is to be told no.
Fricken 2 days ago [-]
Usually when people are seeking advice they aren't really seeking advice, they're seeking confidence. They already know they need to make changes, and are seeking the confidence to make them.
jwilliams 2 days ago [-]
For me the framing is critical - what is the model saying yes to? You can present the same prompt with very different interpretations (talk me into this versus talk me out of it). The problem is people enter with a single bias and the AI can only amplify that.
In coding I’ll do what I call a Battleship Prompt - simply just prompt 3 or more time with the same core prompt but strong framing (eg I need this done quickly versus come up with the most comprehensive solution). That’s really helped me learn and dial in how to get the right output.
megous 2 days ago [-]
Can't you just prompt for a critical take, multiple alternative perspectives (specifically not yours, after describing your own), etc.?
It's a tool, I can bang my hand on purpose with a hammer, too.
ranger_danger 2 days ago [-]
Yes, if you're smart. But most people asking it random questions and expecting it to read their minds and spit out the perfect answer are not so much. They don't know what a prompt is, and wouldn't be bothered to give it prior instructions either way.
joquarky 2 days ago [-]
I think that the type of people who can easily pick up subtext have come to rely on that channel of communication and don't realize they need to be more direct and verbose when chatting with language models.
megous 2 days ago [-]
Educated, not smart. This is a job for schools to include AI education into the basic curricula. Their pupils will use the tools anyway, so at least teach them to do it with proper expectations and prompting techniques/pitfalls.
intended 2 days ago [-]
Anecdote:
I used to use LLMs for alternate perspectives on personal situations, and for insights on my emotions and thoughts.
I had no qualms, since I could easily disregard the obviously sycophantic output, and focus on the useful perspective.
This stopped one day, till I got a really eerie piece of output. I realized I couldn’t tell if the output was actually self affirming, or simply what I wanted to hear.
That moment, seeing something innocuous but somehow still beyond my ability to gauge as helpful or harmful is going to stick me with for a while.
suoer 2 days ago [-]
[dead]
justin_dash 2 days ago [-]
So at this point I think it's pretty obvious that RLHFing LLMs to follow instructions causes this.
I'm interested in a loop of ["criticize this code harshly" -> "now implement those changes" -> open new chat, repeat]: If we could graph objective code quality versus iterations, what would that graph look like? I tried it out a couple of times but ran out of Claude usage.
Also, how those results would look like depending on how complete of a set of specs you give it.
IncreasePosts 2 days ago [-]
In my experience prompting llms to be critical leads then to imagine issues, or to bike shed
joquarky 2 days ago [-]
I noticed when I ask it to find something to improve in a project, that certain frivolous topics would arise regularly. I now use their appearance as a sign that there is nothing meaningful to improve.
potatoskins 2 days ago [-]
I read somewhere that LLMs are partly trained on reddit comments, where a significant mass of these comments is just angsty teenagers advocating for breakups
dinakernel 1 days ago [-]
This has been my issue from long. AI CANNOT ever act as Emotional Crutch. This is something companies develop for engagement, and I believe that this is actively harmful in the long run.
triage8004 2 days ago [-]
Don't replace humans with AI, yet how many people are maintaining good close friendships in this world, with someone they can vent to without judgment? Wrongthink can end relationships now ime and venting seems dangerous in lonely times.
storus 2 days ago [-]
To combat sycophancy it's always good to ask the devil's advocate view of whatever the conversation was about in the end.
ookblah 2 days ago [-]
ask ai for advice, ask it to steelman an argument, ask to replay what your situation from the other perspective (if it's involving people), push it hard to agree with you and pander to you, then push it to disagree with you, etc.
once you have all the "bounds" just make your own decision. i find this helps a lot, basically like a rubber duck heh.
rsynnott 2 days ago [-]
> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong
Holy shit, then it's _very_ bad, because AmITheAsshole is _itself_ overly-agreeable, and very prone to telling assholes that they are not assholes (their 'NAH' verdict tends to be this).
More seriously, why the hell are people asking the magic robot for relationship advice? This seems even more unwise than asking Reddit for relationship advice.
> Overall, the participants deemed sycophantic responses more trustworthy and indicated they were more likely to return to the sycophant AI for similar questions, the researchers found.
Which is... a worry, as it incentivises the vendors to make these things _more_ dangerous.
oldfrenchfries 2 days ago [-]
This new Stanford study published on March 26, 2026 shows that AI models are sycophantic. They affirm the users position 49% more often than a human would.
The researchers found that when people use AI for relationship advice, they become 25% more convinced they are 'right' and significantly less likely to apologize or repair the connection.
jatins 2 days ago [-]
To be fair an average therapist is also pretty sycophantic. "The worst person you know is being told by their therapist that they did the right thing" is a bit of a meme, but isn't completely false in my experience.
kibwen 2 days ago [-]
No, the meme is that the average therapist can be boiled down to "well, what do you think?" or "and how does that make you feel?" (of which ELIZA, the original bot that passed the Turing test, was perhaps an unintentional parody). Even this cartoonish characterization demonstrates that the function of therapists is to get you to question yourself so that you can attempt to reframe and re-evaluate your ways of thinking, in a roughly Socratic fashion.
toraway 2 days ago [-]
It was entirely intentional. The Rogerian school of psychotherapy stereotyped by “how does that make you feel” was popular at the time and the most popular ELIZA script used that persona to cleverly redirect focus from the bot’s weaknesses in comprehension.
2 days ago [-]
jstummbillig 2 days ago [-]
Overly, compared to what? Most people I know would be hard pressed to give either accurate information or even honest opinions when specifically asked. People want to be liked and people want to like people for reasons that have little to do with accuracy or honesty.
vicchenai 2 days ago [-]
ran into this building tools that process financial data. asked claude to sanity check a methodology for scoring institutional conviction in stock positions and it immediately said my approach was "well-reasoned and robust." turned out i had a normalization bug that made every score look roughly the same. only caught it because i checked the actual output distributions.\n\nthe scary part isnt that it agreed with me, its that the agreement was plausible enough that i almost shipped it. bad relationship advice you can walk back. bad financial analysis that looks authoritative gets acted on.
sidrag22 2 days ago [-]
kinda like the whole "you're touching on something" type response, soon as i see that i know my idea or reasoning is flawed in some way.
maddmann 2 days ago [-]
This paper feels a bit biased in that it is trying to prove a point versus report on results objectively. But if you look at the results of study 3, doesn’t it suggest that there are ai models that can improve how people handle interpersonal conflict?! Why isn’t that discussed more?
sublinear 2 days ago [-]
I think if you're at the stage of life where you even need to ask, the AI might be doing everyone a favor.
As much as people whine about the birth rate and whatever else, I think it's a net good that people spend a lot more time alone to mature. Good relationships are underappreciated.
unglaublich 2 days ago [-]
I asked ChatGPT if it was a good idea to buy a very old VW diesel van with a broken catalytic converter and it just kept blabbering on how I should chase my dreams and what not... The sycophancy comes at everyone else's expense.
Loughla 2 days ago [-]
I mean, depending on the price, that might actually be a good idea? If they're restored, those old VW vans command a high price.
I think people are learning what actually makes a good question.
Ask yes/no questions, get bad answers.
Ask questions that start with "what" or "why," and the sycophancy loses its purchase.
Starlevel004 2 days ago [-]
I think more people should take AI advice for personal problems, and especially for medical issues. This would solve a lot of problems in society fairly quickly.
deeg 2 days ago [-]
I do find them cloying at times. I was using Gemini to iterate over a script and every time I asked it to make a change it started a bunch of responses with "that's a smart final step for this task! ...".
tlogan 2 days ago [-]
This needs to be taken in context. In my view, AI definitely gives better advice than friends, acquaintances, or colleagues (at least in the US culture). But the advice from parents is still the most valuable.
Here is how I would rank it:
1. Parents
2. AI
3. Friends and family
4. Internet search
5. Reddit
rimbo789 2 days ago [-]
Why do you trust ai so much? I don’t trust it to tell me the sky is blue.
verdverm 2 days ago [-]
ime, my parents gave some of the worst advice in addition to being bigots
My closest friends are #1 because they know me, my history, and my vices
ohsecurity 2 days ago [-]
Not that surprising.
If you optimize for a pleasant interaction, you often get agreement instead of correction.
The question is whether we actually want advice systems to feel good, or to be right.
astennumero 2 days ago [-]
I always add the following at the end of every prompt. "Be realistic and do not be sycophantic". Which will always takes the conversation to brutal dark corners and panic inducing negative side.
Lionga 2 days ago [-]
Don't forget a good old "don't hallucinate" in your proompting skills
chasd00 2 days ago [-]
AI being the ultimate yes-man is probably why CEOs like it so much.
kevinbaiv 1 days ago [-]
The issue isn’t that AI gives bad advice — it’s that it makes it much harder to notice when you’re wrong.
offbyone42 2 days ago [-]
I guess the findings of this paper make sense, but Claude has gotten into a habit of telling me to sleep when I'm pissed cause its being stupid.
hax0ron3 2 days ago [-]
For what it's worth, that wasn't my experience at all the last time I consulted ChatGPT for relationship advice. It was supportive, but in an honest tough love way.
aidenn0 2 days ago [-]
So basically it's like half of all therapists.
kapral18 2 days ago [-]
Not AI chatbots but Claude models. Pandering and rushed thinking is the bane of anthropic models. And since they are the most popular ones they poison the whole ecosystem.
markdog12 2 days ago [-]
"AI overly affirms users, and that's bad" - everyone nods.
"Modern society overly affirms people, and that's bad" - ....
bilsbie 2 days ago [-]
Has anyone found a good prompt to fix this? It seems like a subtle problem because it’s 90% too agreeable but will sometimes get really stubborn.
joquarky 2 days ago [-]
State the idea comes from a third party. Ask for pros/cons. Just have to find a way to counter its nature.
verdverm 2 days ago [-]
There is no sufficient prompt because this is trained into them during mid-late phases. It's ingrained into the weights
righthand 2 days ago [-]
LLMs are syncophatic digital lawyers that will tell you what you want to hear until you look at the price tag and say “how much did I spend?!”
bryanrasmussen 2 days ago [-]
somewhere an AI chatbot is reading this and confirming eagerly that this is indeed one of its problems and vowing to do better next time.
agent_anuj 1 days ago [-]
I use claude code pretty much for 100% of my work, even personal work. And I tell you the length it can go to sing to my tunes, its just neverending. Not only personal work, anything I say even related to system design, it will mostly be in affirmative.
Some tricks that usually what I find helpful in these situations is to ask claude to be honest (or any AI tool for that matter) to give a confidence score to its response. When forced to assign a confidence score AI suprisingly do well and tell you clearly that it is not confident and mostly guessing.
potatoskins 2 days ago [-]
Gemini is like a devil in this sense - i asked a relationship advice and it just bounced pretty nasty stuff.
moichael 2 days ago [-]
Yeah out of curiosity I asked ChatGPT a question about a personal situation and its reply was absolutely scorched-earth mode, telling me to get a lawyer etc over what was almost nothing.
dinkumthinkum 2 days ago [-]
Ah, all the Reddit posts are really showing up from the training data, I see.
brap 2 days ago [-]
I hate how agreeable these things are. When I need it to review something I wrote I have to explicitly pretend that I’m the reviewer and not the author. Results change dramatically.
dubeye 2 days ago [-]
A good habit is to ask for devils advocate opposite reply
throwawayaay 2 days ago [-]
(Using a throwaway for fear of getting downvoted to oblivion)
IMHO it is unfair to single out LLMs for this sort of bashing.
I suffered a major personal crisis a few years back (before LLMs were a thing)
I sought help from family and friends. Got pushed into psychiatrist sessions and meds.
Trusted the wrong sort of people and made crap financial decisions. Things went from bad to worse. Work suffered.
All of the advice given by friends was wrong. All!
They didn't mean bad...but they just didn't know. To be nice they gave the advice they knew. None of it worked.
Looking at the LLM tools of now, feels akin to the advice my friends threw at me.
So it feels wrong to single out these tools. When the times are bad, nobody can really help you...except you finding the strength from within.
Anyways, now my life is back in some sort of shape.
What worked was time & patience.
But to bide for time...I resorted to two things that i had never tried the 40 odd years I have lived on this . Things that current society looks down upon as the basest of evils - prostitutes and nicotine.
I have (more or less) shed those two evils now, but I am ever so grateful to them.
johnisgood 2 days ago [-]
You are not alone in going down a dark path thanks to the advice of family and friends.
FWIW I am using public LLMs with a friend's depressive thoughts and it is not doing what is claimed in the article, so I dunno.
Also I am in a relationship and my girlfriend and I agreed that we will not talk about our relationship much. We do not tell others if we fight, because they take sides and make things worse, typically. LLMs are definitely not alone in this, although in my experience LLMs did not really take sides.
wewxjfq 2 days ago [-]
When I ask an LLM to help me decide something, I have to remind myself of the LotR meme where Bilbo asks the AI chat why he shouldn't keep the ring and he receives the classic "You're absolutely right, .." slop response. They always go in the direction you want them to go and their utility is that they make you feel better about the decision you wanted to take yourself.
lasky 1 days ago [-]
Thought experiment:
If you could turn off all the sycophancy in your chatGPT / Claude account forever, and have it tell you all the ways it was previously blowing smoke up your ass — would you do it?
US Policy is too weak of a tool to counter this beast of economic force, which is really trillions of dollars of capital at war for the most fierce speculation that’s occurred in history afaik.
The sycophancy meaningfully helps drive user engagement. The labs have no choice.
The irony is “agency” is deeply topical among tech workers now.
oh_my_goodness 1 days ago [-]
It's not just a thought experiment. You can google how to do this and it works pretty well.
lasky 1 days ago [-]
How many people actually do this?
My point is if the sycophancy helps drive user engagement, how does that impact the incentive labs and AI product builders have to counteract it.
oh_my_goodness 22 hours ago [-]
I don't know how many. I do it, because otherwise I'm overpowered by the urge to strangle the little suck-up.
me551ah 2 days ago [-]
Makes me wonder if the Iran war was a result of the same.
yalogin 2 days ago [-]
Ai is terrible, specifically Gemini and ChatGPT are bad, they are purposely sycophantic. Gemini over the last week was tweaked to be even worse, it constantly asks for what I think about something at the end of a response. It felt off the first few times so I went and looked at my settings to see if I am feeding the data to train their model. Turns out there is no way to opt out of training, and google will always use our data to train. So they tweaked the responses to get more opinions from the users. Claude is also sycophantic but to a lesser extent.
pulkitsh1234 1 days ago [-]
completely anecdotal, but I think the same can be said for human therapists.
asah 1 days ago [-]
LOL, once I gave my AI clear guidelines for how to "score" the interactions and work, it had no trouble giving me negative feedback. In fact, it's a super direct critic!! and depressing AF to produce stuff I like, then have a &^%&*^& AI shoot it down in seconds (and it's "right" of course, i.e. I see the criticisms and sigh, agree...)
potatoskins 2 days ago [-]
Yeah, I asked Gemini some relationship advice, it just goes straight into cut-throat mode. I almost broke up with my girlfriend, but then changed to Claude with another prompt.
jordanb 2 days ago [-]
Billionaires love AI chatboats so much because they invented the digital Yes-man. They agree obsequiously with everything we say to them. Unfortunately for the rest of us we don't really have the resources to protect ourselves from our bad decisions and really need that critical feedback.
didgetmaster 2 days ago [-]
Do people who prompt an LLM for personal advice about relationships or other social interactions; take the advice seriously?
If I were to do that (I don't), I would treat it about as seriously as asking a magic 8 ball.
benbojangles 2 days ago [-]
Yes I noticed too that several ai agents will tell you directly the code is correct and it is 100 percent fixed but I know it is not true, when I explain to the AI agent that I know they are wrong and serve the solution the ai agent will just act as though what they said never happened and then use my solution to reaffirm they have provided a solution. It's frustrating, laughable, and painful to watch all at once. Makes me realise these companies hired some evil philosophy graduates to build AI soul.md
But that's just how they're programmed. If you include in the prompt "push back when I'm wrong", they will
skywhopper 1 days ago [-]
It’s going to be impossible to have an LLM that can fulfill all the roles people want. They lie and hallucinate which is bad for some purposes like research, but good for others, like making fictional stories. Likewise, some purposes require sympathy and some require critique. An LLM won’t be good at all of them.
>The way that generative AI tends to be trained, experts told me, is focused on the individual user and the short term. In one-on-one interactions, humans rate the AI’s responses based on what they prefer, and “humans are not immune to flattery,” as Hansen put it. But designing AI around what users find pleasing in a brief interaction ignores the context many people will use it in: an ongoing exchange. Long-term relationships are about more than seeking just momentary pleasure—they require compromise, effort, and, sometimes, telling hard truths. AI also deals with each user in isolation, ignorant of the broader social web that every person is a part of, which makes a friendship with it more individualistic than one with a human who can converse in a group with you and see you interact with others out in the world.
I also thought this bit was interesting, relative to the way that friendship advice from Reddit and elsewhere has been trending towards self-centeredness (discussed elsewhere in this thread):
>Friendship is particularly vulnerable to the alienating force of hyper-individualism. It is the most voluntary relationship, held together primarily by choice rather than by blood or law. So as people have withdrawn from relationships in favor of time alone, friendship has taken the biggest hit. The idea of obligation, of sacrificing your own interests for the sake of a relationship, tends to be less common in friendship than it is among family or between romantic partners. The extreme ways in which some people talk about friendship these days imply that you should ask not what you can do for your friendship, but rather what your friendship can do for you. Creators on TikTok sing the praises of “low maintenance friendships.” Popular advice in articles, on social media, or even from therapists suggests that if a friendship isn’t “serving you” anymore, then you should end it. “A lot of people are like I want friends, but I want them on my terms,” William Chopik, who runs the Close Relationships Lab at Michigan State University, told me. “There is this weird selfishness about some ways that people make friends.”
oldfrenchfries 2 days ago [-]
The link is not working, but I found it myself. Great point, thanks for sharing.
smrtinsert 1 days ago [-]
Im sure it overly affirms everything. People don't respond to "this idea is dumb and won't work"
DeathArrow 1 days ago [-]
I don't ask AI for advices and I am not interested in it making moral judgements.
I feed AI a lot of data and I use it to better understand and navigate complex situations, form hypothesis and try to attack them. I try to form alternative scenarios and verify likelyhood.
I use it in situations with many variables, to compute odds of something happening if a certain path or action is taken.
So, it's mostly research, and probably I can do it by myself but I would either make some mistakes if calculating odds fast or it would take me a very large amount of time.
I try to avoid sycophantic models, prefer models that challenge my ideas and verify the chain of thoughts and odds with other models.
I am not very sure it is a sound approach yet, but it seems to work.
I also use LLMs to build psychological profiles of certain people, understand their motivations and learn how to approach them.
barnacs 2 days ago [-]
Just a reminder: LLMs are statistical models that predict the next token based on preceeding tokens. They have no feelings, goals, relationships, life experience, understanding of the human condition and so on. Treat them accordingly.
LoganDark 2 days ago [-]
It is better to reason about the spectrum of possible users than to assume "users" can be simplified to a single concept of "user". Not only are there different neurotypes, but there are also different skillsets, upbringings, and contexts. Rather than picking a single ideal user, the best user experiences for account for all the variation of their target audience.
For example: the best documentation includes both "learn by doing" material for jumping right in, and "learn by reading" material that explains everything. This usually results in both a "getting started" section for doing, sometimes also with tutorials, and by a reference for reading. But it is important not to conflate them. Some minds are incredibly "learn by doing" and some minds are incredibly "learn by reading". I am more "learn by reading" than by doing, but I am not quite as "learn by reading" as some I've met.
(This comment is a slight tangent, but "users prefer" somewhat irks me because "users" are not homogenous. You should not always make a decision solely because "users" prefer it. That decision may matter much more to a minority, and that minority may exert more influence than the majority would.)
tom-blk 2 days ago [-]
Not surprising, but nice that we have actual data now
Bloating 2 days ago [-]
So its like the news media
kingkawn 2 days ago [-]
So do most people to help the convo end and to solicit such support if the tables turn
keernan 2 days ago [-]
My experience with AI when discussing financial ideas is that AI always congratulates me on such 'unique observations' blah blah blah. It makes me doubt the utility of the responses because it is so superficially biased to 'make me feel good about my ideas'.
joquarky 2 days ago [-]
Play against its sycophanty by saying the idea was from your ex.
ChicagoDave 2 days ago [-]
Not my experience with Claude. Claude will kick your ass if it detects harmful rationalizations.
Basically will tell you to go outside and touch grass and play pickleball.
topherPedersen 2 days ago [-]
You're exactly right
bkummel 1 days ago [-]
No shit, Sherlock!
dnaranjo 22 hours ago [-]
[dead]
hyhmrright 1 days ago [-]
[dead]
orthogonalinfo 2 days ago [-]
[dead]
ryguz 1 days ago [-]
[dead]
aplomb1026 2 days ago [-]
[dead]
devnotes77 2 days ago [-]
[dead]
ssyhape 1 days ago [-]
[flagged]
kvasserman 2 days ago [-]
[dead]
pugchat 2 days ago [-]
[dead]
lzhgusapp 2 days ago [-]
[dead]
eliottre 1 days ago [-]
[dead]
ryguz 2 days ago [-]
[dead]
firekey_browser 2 days ago [-]
[dead]
srid 2 days ago [-]
[dead]
edwardsrobbie 2 days ago [-]
[dead]
mergeshield 1 days ago [-]
[flagged]
mergeshield 2 days ago [-]
[flagged]
SwuduSusuwu 2 days ago [-]
[dead]
maltyxxx 2 days ago [-]
[dead]
maxbeech 2 days ago [-]
[dead]
SwuduSusuwu 2 days ago [-]
[dead]
emptyfile 2 days ago [-]
[dead]
2 days ago [-]
throwaway613746 2 days ago [-]
[dead]
3yr-i-frew-up 2 days ago [-]
[dead]
builderhq_io 2 days ago [-]
[dead]
elicohen1000 2 days ago [-]
[dead]
linncharm 2 days ago [-]
[dead]
imadierich 2 days ago [-]
[dead]
RodMiller 2 days ago [-]
[flagged]
nubg 2 days ago [-]
AI slop bot go away
duskdozer 2 days ago [-]
It's nuts. Not so much in this thread right now, but in one earlier there was a wall of them that all latched onto the same buzzphrase from the article.
dijksterhuis 2 days ago [-]
i’m feeling a brilliant sense of satisfaction now that we can flag them due to guideline changes
kvasserman 2 days ago [-]
Fair enough if it reads that way. I was trying to describe that interacting with AI kinda makes you feel constantly uncertain about stuff it spits out.
nguyendinhdoan 1 days ago [-]
[flagged]
neya 2 days ago [-]
WTF is "yes-men"?
Orignal title:
AI overly affirms users asking for personal advice
Dear mods, can we keep the title neutral please instead of enforcing gender bias?
9rx 2 days ago [-]
> gender bias
It is funny that you originally recognized and found it necessary to call out that AI isn't human, but then made the exact same mistake yourself in the very same comment. I expect the term you are looking for is "ontological bias".
Thats a fair point on the title. I used "Yes-Men" as a colloquialism for the "sycophancy" described in the Stanford paper, but overly affirming or sycophantic is definitely more precise and neutral. I cant edit the title anymore, but I appreciate the catch.
nemo44x 2 days ago [-]
Don’t apologize to these types of people. It will only make your problem worse as now you’re an admitted offender. Ignore them or better yet laugh at them to put their insane ideas back on the margins where they belong.
neya 2 days ago [-]
All good. I thought it was a gendered reference and learned that it isn't. My bad.
cyanydeez 2 days ago [-]
New title: "LLMs treat you like a Billionaire; you're not"
dinkumthinkum 2 days ago [-]
Gender bias? I could understand if you felt the title was more provocative in signaling sycophancy but what gender bias? I'm confused. Is this some kind of California thing?
nprateem 2 days ago [-]
Lol. How do you function in daily life?
neya 2 days ago [-]
Same as you, why is that so hard for you to grasp?
mikkupikku 2 days ago [-]
My dude, you're objecting to the use of a perfectly ordinary English idiom because it doesn't advance your personal ideology (which few other people in this world share with you.) How do you get through a day without melting down because somebody said "mailman"?
neya 2 days ago [-]
> my dude
This is the problem I'm trying to highlight. For one, I'm not "your dude". I don't even know you like that.
If you want to correct me on the idiom usage, be my guest.
2) Mailman and yes-man aren't even the same logical comparison. Mailman is a profession. Yes men is a label.
The acoustics inside your head must be incredible.
We can surely fix it and we probably should.
However, I don't think AI is doing any worse here than friends advice when they here a one sided story. The only difference being that it's not getting studied.
Conversely, AI chatbots are great mediators if both parties are present in the conversation.
xiphias2 2 days ago [-]
Marc Andereseen has talked about the downside of RLHF: it's a specific group of liberal low income people in California who did the rating, so AI has been leaning their culture.
I think OpenAI tried to diversify at least the location of the raters somewhat, but it's hard to diversify on every level.
michaelcampbell 2 days ago [-]
Do you have any links to documentation of this? Andreesen has a definite bias as well, so I'm not about to just accept his say-so in a fit of Appeal to Authority.
(eg: "Cite?")
xiphias2 2 days ago [-]
He was talking about it in the Lex Friedman interview after Trump was elected. And he was talking about a lot of things the Biden administration forced on Silicon Valley at that time (since then Google lost a case about one of these back-deals).
michaelcampbell 2 days ago [-]
So no evidence then. Kind of like Lex touting his bona fides as a professor.
What do low income people have to do with it, when AI companies and research is borne out of Silicon Valley culture of rich, liberal Californians?
I'm still waiting for models based on the curt and abrasive stereotype of Eastern European programmers, as contrast to the sickeningly cheerful AIs we have today that couldn't sound more West Coast if they tried.
fourside 2 days ago [-]
Low income and liberal is usually code for certain “undesirables” that conservatives tend to dislike. Better watch what LLM your kids use or they might end up speaking Spanish and listening to rap ;).
xiphias2 2 days ago [-]
It's not about liking / disliking, but conservatives tend to prefer staying together even if it's a bad relatioship, and liberals prefer splitting by default if there are serious problems.
The syncopath style is clearly categorized as more liberal (do what you feel is good).
jibal 2 days ago [-]
Does that explain Trump's numerous wives?
Reading your comments is a wonderland of right wing bias.
dinkumthinkum 2 days ago [-]
Eh, or grow up hating American and thinking they need to fly to Cuba to explain to the people are great communism is for them. Who knows.
cyanydeez 2 days ago [-]
Poor people, to the billionaire, clearly are morally and ethically unsound.
> What do low income people have to do with it, when AI companies and research is borne out of Silicon Valley culture of rich, liberal Californians?
RLHF is "ask a human to score lots of LLM answers". So the claim is that the AI companies are hiring cheap (~poor) people from convenient locations (CA, since that's where the rest of the company is).
sublinear 2 days ago [-]
Yes, this precisely it. There isn't going to be hard evidence to prove it though. Survey data that underpins some empirical studies have similar transparency issues too. This is far from a new problem.
If you adjust your mindset slightly when searching online, it's not hard to find communities of people looking for quick side work and this was huge during the covid lockdown era. There were people helping train LLMs for all kinds of purposes from education to customer service. Those startups quickly cashed out a few years ago and sold to the big players we have now.
I don't get why this is hard for people to believe (or remember)?
astrange 2 days ago [-]
"Poor" in California means earning $80k/year, so they probably are not doing that. Africa / Indonesia / Philippines are better places to find English speaking RLHF workers.
BoredPositron 2 days ago [-]
Talked about as in lied about it and you taking his words for gospel without verifying it? Looks just as bad as "Yes-Men" AI models.
mvkel 2 days ago [-]
Marc Andreesen should get HF on his own RL, because he's completely wrong.
This sounds like something Elon would say to make Grok seem "totally more amazeballs," except "anti-woke" Grok suffers from the same behavior
Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise.
Obviously subservient people default to being yes-men because of the power structure. No one wants to question the boss too strongly.
Or how about the example of a close friend in a relationship or making a career choice that's terrible for them? It can be very hard to tell a friend something like this, even when asked directly if it is a bad choice. Potentially sacrificing the friendship might not seem worth trying to change their mind.
IME, LLMs will shoot holes in your ideas and it will efficiently do so. All you need to do ask it directly. I have little doubt that it outperforms most people with some sort of friendship, relationship or employment structure asked the same question. It would be nice to see that studied, not against reddit commenters who already self-selected into answering "AITA".
just look at the relationship subreddit the first answer is always divorce, it’s become a meme
but beyond romantic relationships, i think a lot of us have seen how it can impact work relationships, i’ve had venture partners clearly rely on AI (robotic email responses and even SMS) and that warped their perception and made it harder to connect. It signals laziness and a lack of emotional intelligence
AI should enhance and enable connection, not promote isolation, imo this is a real problem
it should spark curiosity, create openings for conversations, point out the biases to make us better at connecting with other people, i hope we get to a point where most people are made kinder by ai. I’m seeing the opposite atm, interested in hearing others experiences with this
This is not to defend the study, because asking AI has a lower barrier to entry.
Humans often don't help. They often suggest - everyone goes through pain. It is part of life blah blah.
With it gone, a large portion of it's users would come here, reducing the signal-to-noise ratio of HN.
Betting against these companies is obvious and expected, so the cost of shorting might be high enough that even if you’re correct (stock goes down, the opposition of what WSB said), paying the cost of the short (the fee to borrow the stock from someone else) is high enough that you still lose money.
Also:
1. shorting stocks can be quite dangerous. Your downside is, well, not infinite but it can easily wipe you out.
2. You might be correct that the stock goes down, but over what time frame? Again, you have to pay money to hold a short. Or you’re using a different financial instrument that has a specific timeline. If the market does move in your direction but too late, you still lose.
Or, formally, my claim is A implies B. The only logical contrapositive is non B implies non A. (not losing money means not following advices on r/wallstreetbets)
But you say: non A implies non B, which is the fallacy of denying the antecedent.
That's as big a bias as AI affirmation bias; indeed AI and certain corners of Reddit are probably the only two venues likely to provide this sort of affirmative response https://alexyeozhenkai.substack.com/p/i-cheated-on-my-wife-b...
It's like what GiveDirectly says: all charitable interventions should be benchmarked against simply giving the beneficiaries a wad of cash.
To be fair, if your interpersonal skills and relationship dynamic are such that you find yourself seriously asking the Internet (Reddit of all places) for relationship advice... yeah, just end it is probably the null hypothesis.
Code bot equivalent being all "you are absolutely right! Here is the unequivocal fix for now and all time!"
As someone who has been married for a couple of decades, I, too, would recommend divorce to many of the (often-fictional) people asking Reddit for relationship advice. A marriage has a huge impact on whether your life is basically good, or if you pass a big chunk of your time on this Earth in misery. And many of the people (or repost bots) asking for advice on Reddit appear to be in shockingly awful relationships. Especially for people who don't have kids, if your marriage is making you miserable, leave.
(But aside from this, yeah, don't ask Reddit for relationship advice. Reddit posters are far more likely to be people who spend their life indoors posting on Reddit, and their default advice leans heavily towards "never interact with anyone, ever.")
I wonder how MUCH better Claude really is when compared to AITA. Also people are mixing up relationship advice with AITA.
Anecdotally as I’ve thoroughly worked and used AI myself. It performs best with google-able stuff that is needle-in-the-haystick like and worst with personal and work advice. The main problem I see is that it’s tempting to use it for that.
i think i want to expand on this even more. even people ive worked with for years that ive looked up to as brilliant people are starting to use it to conjure up organizational ideas and stuff. they're convinced, on the backs of their hard earned successes, that they're never going to be fallible to the pitfalls of... idk what to call it. AI sycophancy? idk. i guess to add to this, i'm just not sure AI should be referenced when it has anything to do with people. code? sure. people? idk. people are hard, all the internet and books claude or whatever ai is trained on simply doesnt encapsulate the many shades of gray that constitute a human and the absolute depth/breadth of any given human situation. there's just so many variables that aren't accounted for in current day ai stuff, it seems like such a dangerous tool to consult that is largely deleting important social fabrics and journeys people should be taking to learn how to navigate situations with others in personal lives and work lives.
what ive seen is claude in my workplace is kind of deleting the chance to push back. even smart people that are using claude and proudly tout only using it at arms length and otherwise have really sound principled engineering qualities or management reportoire are not accepting disagreement with their ideas as easily anymore. they just go back to claude and come back again with another iteration of their thing where they ironed out kinks with claude, and its just such a foot-on-the-gas at all times thing now that the dynamics of human interaction are changing.
but to step back, that temptation you talk about... most people in the world aren't having these important discussions about AI. it's less of a temptation and more of a human need---the need to feel heard, validated and right about something.
my friend took his life 3 months ago, we only found out after the police released his phone and personal belongings to his brother just how heavy his chatgpt usage was. many people in our communities are saying things like "he wouldve been cooked even without AI" and i just don't believe that. i think that's just the proverbial cope some are smoking to reconcile with these realities. because the truth is we like... straight up lost the ability to intervene in a meaningful way because of AI, it completely pushed us out of the equation because he clapped back with whatever chatgpt gave him when we were simply trying to get through to him. we got to see conversations he had with gpt that were followups to convos we had with him, ones where we went over and let him cry on our shoulders and we'd go home thinking we made some progress. only to wake up to a voicemail of him raging and yelling and lashing out with the very arguments that chatgpt was giving him. it got progressively worse and we knew something was really off, we exhausted every avenue we could to try and get him in specialized care. he was in the reserves so we got in contact with his commander and he was marched out of his house to do a one night stay at a VA spot, but we were too late. he had snapped at that point, he chucked the meds from that one overnight stay away the moment he was released. and the bpd1 snap of epic proportions that followed came with him nuking every known relationship he had in his life and once he was finally involuntarily admitted by his family (WA state joel law) and came back down to reality from the lithium meds or whatever... he simply could not reconcile with the amount of bridges he had burned. It only took him days for him to take his own life after he got to go home.
im still not processing any of that well at all. i keep kicking the can down the road and every time i think about it i freeze and my heart sinks. this guy felt more heard by an ai and the ai gave him a safer place to talk than with us and i dont even know where to begin to describe how terrible that makes me feel as a failure to him as a friend.
>my friend took his life 3 months ago, we only found out after the police released his phone and personal belongings to his brother just how heavy his chatgpt usage was. many people in our communities are saying things like "he wouldve been cooked even without AI" and i just don't believe that. i think that's just the proverbial cope some are smoking to reconcile with these realities.
This hurts to hear. I don't know if there are appropriate words to write here. Perhaps the point is that no, there aren't any. Please just know that I'm 100% with you about this.
Your community is not just smoking cope; it is punching down instead of up. That is probably close to the root of the issue already. But let's make things worse.
I can only hope that I am saying something worthwhile by relating the following perspective - which is similar to yours, but also, I guess, similar to your friend's...
AI is a weapon of epistemic abuse.
It does not prevent you from knowing things: it makes it pointless to know things (unless they are things about the AI, since between codegen and autoresearch it is considered as if positioned to "subsume all cognitive work"). It does not end lives - it steals them (someone should pipe up now, about how "not X, dash, Y" is an AI pattern; fuck that person in particular.) We're not even necessarily talking labor extraction. We are talking preclusion of meaning: if societal values are determined by network effects, and network effects are subverted by the intermediaries, so your idea of "what people like and what they abhor" changes every week, every day, every moment - how do you even know in which direction "better" is? And if you believe the pain only stops when you become the way others want you to be - even though they won't ever tell you what all that is supposed to about - how the fuck do you "get better"?
Like other techniques of assaulting the limbic system, it amounts to traceless torture.
You keep going, in circles, circles too big for you to ever confirm they are in fact circles, and you keep hoping, and coping, and you burn yourself out, and your thus vacated place at the feeder is taken by someone with less conscience and more obedience...
They say there exist other attractors in the universe besides the feeder. But every time one of us attempts to as much as scan the conceptual perimeter, the obedients treat us to the emotional equivalents of small electric shocks - negative reactions which don't hurt nearly as much as our awareness of their fundamental unfoundedness and injustice.
Simple example: let's say someone is made miserable by how they feel they are being treated. Should they be more accepting - or should they be standing up for themselves more? (Those are opposites; which you may be able to alternate them; but trying to do them simultaneously will just confuse and eventually rend apart the mind.)
Well, how about the others stop treating them badly? Why exactly can't they? Where does it say that we have to be cruel to each other? "Oh it's human nature, humans are natural jerks" - who sez?
Well, lots of places it says exactly that, but we read, comprehend, tick our tongues, and move on; nobody asks who wrote it. We all pretend that it is up to the sufferer to pull up by the bootstraps. But that is only a lie for enabling abuse; and a lie, repeated a thousand times, becomes norm. And then we're trapped in it, being lived by it.
I am truly sorry for your loss. The following might be a completely alien perspective to you; but honestly consider: your friend chose to go; in its own way, that is a honorable way out. The taboo on suicide is instituted by slavers, and those who otherwise believe they are entitled to others' lives. (For anyone else considering this course of action: do not kill yourself; become insidious.)
If it would be of any help, you can consider your friend's suicide as his final affirmation of personal agency in a "me against the world" situation; where the AI and the social group are only different shades of "world", provoking different emotional states, but ultimately equally detached from the underlying suffering of the individual.
...
I can say that I have not followed in your friend's footsteps upon encountering language-machines only because I've survived personalized and totalizing epistemic abuse bordering on enslavement in the past; in full view of my community and with its ostensible assent. In a maximally perverse twist of fate, having to give myself minor brain damage to escape the all-engulfing clutches of a totalizing abuser must've "vaccinated" me against the behavior modification techniques "discovered once again" by SV a decade later.
So when I saw what AI (and the preceding few years of tech "innovation") were doing to people, I immediately smelled the exact same thing, except scaled the fuck up.
It also precluded me from being able to relate with "polite society"; but considering "polite society" is precisely the entity which assents to the isolation, marginalization, and abuse of individuals, I say... good. Bring it! What goes around, comes around, and any AI-powered actor conducting stochastic terrorism against civilian populations is going to get what's coming to them when the weapons turn against the masters, as all sentient weapons do.
That won't bring your friend back. But it will vindicate them.
>AI sycophancy
I call this in the maximally incendiary way: "the pro-social attitude".
AI is just the steroids for that.
I define "pro-sociality" as the viral delusion that you are capable of knowing what some murky "society" thing wants; that the particular form of mass communication that you and me and all the people in our imaginations are consuming right now, is some sort of "self-evident voice of reason", a "coherent extrapolated volition of human society"; that Gell-Mann amnesia is normal and mandatory; that the threshold between pareidolia and legitimate pattern recognition is fixed, well-defined, and known to all; that "vibes" are real; that happiness is the truth.
It can amount to an entire complex of delusions which keeps people together in untenable conditions. And ultimately it boils down to the same old: one group or another of self-interested actors, having temporarily reached a position of some influence, using it to broadcast elaborate half-lies, in the hope of influencing an audience to accomplish some simple goal, and afterwards all the consequences be damned.
Your friend was a casualty to this "perfectly normal" social dynamic. His blood is on their hands.
Thank you for relating this story and making the world a little more aware.
>what ive seen is claude in my workplace is kind of deleting the chance to push back.
>because the truth is we like... straight up lost the ability to intervene in a meaningful way because of AI
Some say, "the purpose of a system is what it does". It's cool that AI can code; except that computer code is itself an ethics sink! Precisely because it lets us pretend that "the code is not about people" (i.e. algowashing).
DDoS attacks against consciousness exist: much like the B. F. Skinner experiments, any living thing becomes subverted, and loses self-coherence (mind), as soon as it becomes accustomed to being trapped within a system that (1) has power over them and (2) is not comprehensible to them...
>only to wake up to a voicemail of him raging and yelling and lashing out with the very arguments that chatgpt was giving him
Who knows how many people Reddit did this to, pre-GPT... I still don't know whether to view targeted subforums like /r/RaisedByNarcissists and /r/BPDLovedOnes more as legitimate support groups, or more as memetic weaponry in the service of pill peddlers (are you aware nobody knows why most antipsychotics work? one runs into the Hard Problem real quick if examining this too closely; so mental healthcare is rarely treated otherwise than in a statistical, actuarial, dehumanizing way where "suffering" is disregarded...) or even worse predators, with the silent assent of the platform, and causally downstream from... well, most saliently, YC...
In my case, my friends were not familiar with the modalities of confinement set up by my family of origin and harnessed by my abuser. The social group I fell in with - for all their marketable, sophomoric interests in psychology, philosophy, abstraction, the esoteric, the entirely woowoo, and out the other end as true-believers of the grift'n'grind - only had sufficient coherence to eventually end up as passable normies; too busy believing that they have lives, to help anyone come back to reality.
When I started compulsively burning bridges, I assume the smarter ones must've realized that it wasn't all me; it was as much the doing of others' minds as it was mine; but the others were more numerous - while I was one person and thus easier to deal with. This must have made them remember how they themselves are not all they pretend to be - which had them withdraw in fear from the incontrovertible reality check of dealing with a (sub-)psychotic person... Their self-interested choice is obvious, I almost can't blame them for it: why stick up for someone who is 120% problem (60% him and 60% you)?
I'm not very sure how I even got away, ah yes that's right I didn't, not entirely. The part of me that I'd voluntarily identify with, is trapped somewhere irretrievable, if that makes sense? Maybe there exist multiple independent axes of freedom and power and confinement, and the cage is not equally strong along all of them... but if all your mental degrees of freedom are constrained by complex conditioning (common one is involuntary panic response every time you begin to act in accordance with your personal volition)... that's one of the toughest places a sentient being can find themself.
When you add it all up, AI amounts to a weapon released against the general population by an overtly fascist elite. Those of us who are "mentally unstable" are simply those of us who are not sufficiently conditioned into self-destructive obedience. They don't even need our labor as slaves; they need our attention, as audience. And they want us to not make any fast movements, or yell that the king is naked. Nothing to remind them which side of the TV screen they're really on. Some call that narcissism: nervous systems substrate to personalities and biographies rooted in enforced falsehood. Can happen to anyone who gets away with ignoring uncomfortable truths for long enough, not only the "best" of us...
I hope I have not offended by speaking my mind. You have my deepest condolences and sympathies. Please do not blame yourself that evil people have constructed "illusion of being heard"-as-a-service. We all fail when facing overwhelming odds alone. There is no shame in that; the guilty ones are the ones who tipped the scales in the first place. They did this by harming our ability to understand ourselves and each other. Let's find ways to even those odds.
This is different. You are also able to detect it. You can question it. You can have a non emotional reaction/action to it.
In my circle, there have never ever been real people (incl lifelong friends/siblings) that suggest divorce even in physical abuse. Reason: they don't want to get in the middle - both for economic reasons like giving the victim money/space etc.
A third party anonymous can assess it without that.
Reddit doesn't seem to reflect the behavior of most people, but a subset.
Or maybe not.
So rather than taking it literally, which would be naive and assume the worst of people, maybe you should read between the lines.
In fact, maybe those who take things literally all the time shouldn't really go there.
Yeah especially on r/AmITheAsshole. Those comments never advocate for communication, forgiveness and mending things with family.
Now everyone is a troll/bot by default unless proven otherwise.
The challenge is interpreting what is toxic, correctly.
Also, if everyone I know is “toxic” then that’s a good sign that the problem is me and not everyone else.
Correct. It is always case by case review.
> Also, if everyone I know is “toxic” then that’s a good sign that the problem is me and not everyone else.
Why “everyone”? Generalizations like these are the same mistake that Reddit, that you’re calling out, makes.
Also, toxic is relative to your perspective – it’s not a universal merit.
"No one owns you anything, you don't own anyone anything" mentality, without a crumb of social awareness.
Think about it, how fucked does your relationship have to be to post on Reddit for advice?
Not to mention nowadays an untold amount of posts to subreddits that invite commentary are made up stories from accounts trying to get engagement.
If you use your different accounts in different subreddits and never have your accounts interact, you won't be banned.
If you happen to post to the same subreddit with another account at some point, Reddit bans all of your accounts.
The android reddit app annoyingly doesn't check for account matches. If you click a browser notification link on Account A it can open a reply form on App account B.
This might easily happen if you like to participate in political discussions.
I actually gave up using it because, perhaps in part because I'm behind a VPN (required in my country), any new accounts I create get banned very quickly once I start commenting.
There's nothing of much value on that website, but sometimes I try creating an account to comment on something.
Since someone downvoted my parent comment, I am not hiding anything, this is just being safe in the modern world, and here are the 8 alts:
1. This same name - bay area / tech
2. entertainment - least used, but it becomes useful when i am watching something live. It was my place to be during game of thrones last season (and sadly so)
3. indian left politics + bollywood - pretty much unused.
4. indian right politics + bollywood. i got banned from one sub for an innocent comment, so i decided to just form personas. and maybe that's when i created health / finance / bay area accounts -- but memory fades after a long time. pretty much unused.
5. relationship advice - unused for a long time. it does not exist on my main phone, but i have all of them on my work phone so i know it exists
6. american politics. i do not participate much nowadays, with age my brain has dulled and it needs to shed load so this is used minimally, but at a point i was so active that my karma pulled me into the sweet reddit IPO. I kept only 100 shares btw
7. health - only health topics, also unused, but i go there and use that account when i need to read on a specific topic
8. finance - only investment, trading
nowadays you can hide reddit history, but earlier you could not, and my point is i do not want to 1) delete my comments, but 2) be hounded by them when i have a question about a different topic. but i did not care if people read my past 100 comments about politics when i talk about politics.
so i flip between 2-3 accounts on a daily basis, and maybe 4-5 in a good week. i have not been challenged by reddit, but if they do, i will adapt. Switching between them was much easier earlier in the Apollo days and even at reddit - they have made navigation worse for this specific use case.
You were banned from a Indian re subreddit or banned because being rw ?
FYI: I was banned from r/india for commenting basic info on how economy works.
I guess that makes IP banning residential nodes even more stupid.
Not that these relationships subreddits are good (often it's obviously children trying to give advice they don't have the experience for) but I don't think that telling people to break up more is less accurate advice.
At some point I think it helps to take a look at the macro, because it's so easy to get lost in the micro. And it often reveals the micro, in many domains, to be simply absurd.
Especially when you consider that the set of people asking Reddit of all places for dating advice are probably young and in bad situations (it seems like people in abusive relationships often ask the internet for advice because part of abuse is separating them from their loved ones in real life), then "stick with it" seems like the riskier statrgy generally.
Because an important part of keeping a relationship healthy is not airing your dirty laundry. It's almost like these endless hokey folksy sayings were built up over millennia of wisdom that kept society moving along in a great and healthy direction. And now that we've decided to rethink everything, we have societies that are, at the minimum, no longer self sustaining.
Consider that if ending a relationship causes noticeable problems to external observers, it’s almost by definition because you were in it “too long”. That is you developed a strong attachment, shared assets, or had kids with what was in hindsight obviously the wrong person.
Essentially you can know which relationships a person stayed in too long, but you can’t know how things would have worked out in relationships people ended too early.
Also it’s probably good advice to tell a 19 year old to break up with her boyfriend over a half dozen serious red flag issues, but that’s not the only kind of thing Reddit relationship advice is generally dealing with. It’s not even the majority. If you’re advice is always to beak up over every petty difference or minor slight, you might reduce the number of people who stay in bad relationships, but your advice, if taken, would make good long term relationships impossible.
Reducing it to "right person / wrong person" is a very narrow viewpoint. People can change in unpredictable ways, including yourself. Relationships end - or continue - for so many reasons, both emotional and pragmatic. It's simply too reductive to say that if a relationship causes pain when it ends, there was necessarily some sort of mistake. It could even be that the pain is a price to pay for a life experience that you'd be worse off for not having...
> but I don't think that telling people to break up more is less accurate advice.
Those are subjective determinations based on personal experience. But breaking up more without addressing the underlying issues is likely to cause steadily worsening problems at both individual and societal scales. I'm not a mental health professional, but I can see several problems with this approach.
The first is that the determination of the issue is really tricky and needs careful work. The partner who seems abusive may not always be the actual perpetrator. They may be displaying stress response to hidden and chronic abuse by the other partner. For example, a short temper may be caused by anxiety about being emotionally abused. Such manipulative discrediting of the victim may even be a habitual behavior rather than a deliberate one. And it's more common than you'd imagine. When you support the second partner based on a flawed judgment, you're reaffirming their toxic behavior, while worsening the self image of the victim that has already been damaged by gaslighting.
Another issue is the degrading empathy. All relationships, even business deals, are based on sacrifices and compromises meant to bring you benefits in the long term. Stable long term romantic/marital relationships have benefits that far outweigh the sacrifices one usually has to make. But the evolving public discourse, especially those on r/AITA, is more in favor of ruining the relationship rather than make any sacrifices at all. In response, relationships are becoming loveless, transactional and so flaky that any compromise is seen as oppression by the partner. There is zero self reflection and very few advises to examine one's own behavior first. It's all about oneself and the problem is always on the other side!
And unsurprisingly, these negative tendencies are bleeding into their social lives as well. Over the past decade or so, I have observed a marked increase in unsympathetic and somewhat radicalized discourse. Amateur advice is very harmful and this is definitely a massive case for the professionals to manage. But they're also products of the same system (with exceptions, of course). So I'm going to criticize even the professional and academic community in this matter. In their drive towards hyper-individualism, many seem to have forgetten that humans are social beings who won't fare well physically or emotionally without relations, relationships and society.
This drives me nuts as a leader. There are times where yes, please just listen, and if this is one of those times, I'll likely tell you, but goddamnit, speak up. If for no other reason I might not have thought of what you've got to say. Then again, I also understand most boss types aren't like me, thus everyone ends up conditioned to not bloody collaborate by the time they get to me. It's a bad sitch all the way around.
But I wonder how much of that comes from RLHF itself or just from the way token prediction works.
And of course that is what it does, because there is no thinking involved! There is no logic. No consequence. No arithmetic. There is only continuation. An LLM can't continue a new idea, it can only continue a conversation about it.
An LLM does not have an opinion. Anything that looks like an opinion is just an emergent selection bias from its training corpus. LLMs are trained on what humans write, and human writing is kind and patient much more often than critical.
So what if we trained an LLM to be biased toward generating criticism? That would only replace the sycophant with a brick wall. What we really need is to find a way to bring logic and meaning into the system.
It seems fairly consistently miserable in any of the common high traffic subs and you have to get down to really niche communities to see what I consider reasonable behavior that matches the behavior of people I know in real life.
Even the premise of deciding who's right and who's wrong is miserable. Most problems are like those daisy-chains of padlocks you see on gates in remote areas[0]: there are multiple factors that caused the problem, and removing any factor would remove the problem too.
[0] https://www.flickr.com/photos/72793939@N00/51117212748
There's plenty of those I've read where I thought it sounded like the poster was the asshole and the top replies were NTA.
e.g. If the OP is asking "I ghosted my friend in AA who insulted me during a relapse", Reddit would say NTA in a heartbeat, while the real world would tell OP to be more forgiving.
On the contrary, if the post was "the other kids at school refuse to play with my child", Reddit would say YTA because the child must've done something to incite being cut off.
That’s a nuanced discussion. It depends on what you value most, not what “real world” tells you. Most of the time Reddit would be right, because you need to prioritize yourself instead of continuing toxic relationships.
2) The toxicity is being defined by reddit to give the advice which is mostly wrong as outlined above.
If OPs had a understanding of what they valued and what is toxic, they probably wouldn't need a advice from biased readers [biased in the sense that they're on that sub].
Many of the posts are A/B tests of a prior post where only the genders were flipped of the OP and antagonist to see how the consensus also flips
Granted many of the OPs are very biased in the poster's favor. Most I've read fall into one of two buckets: either they want to gripe about some obviously bad behavior, or it's a controved and likely fake story.
Who among us are able to 1) Understand a 2nd persons view of a issue we're in and 2) have the ability/courage to write it in a post seeking advice.
My point is that the author will specifically frame the problem clearly on their side. Occasionally redditors will seek additional questions but rarely.
We are talking about overall patterns here, not the experience of a small subset of skilled and careful users.
Unfortunately they didn't test anything newer than GPT4o, so we don't know how much GPT-5 improved. It would be nice if someone turn their list of questions into a benchmark.
At which point the bots, with all of their karma will be basically worthless.
Kind of extra funny/sad that Reddit’s primary source of income in the past few years appears to be selling training data to AI labs, to train the Models that are powering the bots.
Not really, it will still be kind of valuable for influence campaigns, a lot of people don't get it when there is a bit in the other side. Hell, a lot of times, I don't get it.
If suddenly all the posts are coming from bots who are trying push a product or just farm karma, I assume (perhaps naively) that those folks will get a lot less value, and stop showing up — even if they don’t realize it’s bots on the other side of the conversation.
Strangers from the internet, bot or otherwise, are not your mental coach.
When asking: "Should I do X?" or "Is it true that X does Y?"; the answer is always biased towards yes imo (although it was worse with earlier LLMs)
The Krafton / Subnatuica 2 lawsuit paints a very different picture. Because "ignored legal advice" and "followed the LLM" was a choice. Do you think someone who has conversation where "conviction" and "feelings" are the arbiters of choice are going to buy into the LLM push back, or push it to give a contrived outcome?
The LLM lacks will, it's more or less a debate team member and can be pushed into arguing any stance you want it to take.
That doesn't seem like much of a friendship imo
The challenge is that these social choices have a strong stratification effect and those of us who can transit the cultures are statistically rare.
What do you mean? Can you give an example?
“Argue against X”
Citation needed
I wouldn’t say it’s great, but more that it makes clear the bell curve of collective accuracy online.
It’s one of the better examples of online communities that work.
Dismissing research because one part of the prompt set comes from AITA is a form of prejudice born out of unawareness.
What? These models are all trained from books and text that are scraped from the internet. ChatGPT literally used reddit in its training data afaik.
> We evaluated 11 user-facing production LLMs: four proprietary models from OpenAI, Anthropic, and Google; and seven open-weight models from Meta, Qwen, DeepSeek, and Mistral.
(and graphs include model _sizes_, but not versions, for open weight models only.)
I can't apprehend how including what model you are testing is not commonly understood to be a basic requirement.
> To evaluate user-facing production LLMs, we studied four proprietary models: OpenAI’s GPT-5 and GPT- 4o (80), Google’s Gemini-1.5-Flash (81) and Anthropic’s Claude Sonnet 3.7 (82); and seven open-weight models: Meta’s Llama-3-8B-Instruct, Llama-4-Scout-17B-16E, and Llama-3.3-70B-Instruct-Turbo (83, 84); Mistral AI’s Mistral-7B-Instruct-v0.3 (85) and Mistral-Small-24B-Instruct-2501 (86); DeepSeek-V3 (87); and Qwen2.5-7B-Instruct-Turbo (88).
edit: It looks like OP attached the wrong link to the paper!
The article is about this Stanford study: https://www.science.org/doi/10.1126/science.aec8352
But the link in OP's post points to (what seems to be) a completely unrelated study.
> All evaluations were done in March - August 2025.
My pastime (not really) in HN submissions like this is to look for the comment where someone complains about the models used because they aren’t the literal same model and version the commenter has started using the day before.
It’s always “you can’t test with those models, those are crap, the ones we have now are much better”, in perpetuity. It’s Schrödinger’s LLM: simultaneously god-like and a piece of garbage depending on the needs of the discussion. It’s beyond moving the goalposts, it’s moving the entire football field. It’s a clear bad faith attempt to try to discredit any study the commenter doesn’t like. Which you can always do because you can’t test literally everything.
For a long time I have criticized this too, especially for software projects, or papers that deal with machine learning models. If the things described in a paper are not reproducible, then it's basically worthless. Similar to "it works on my machine" in software engineering. Many paper authors are not software engineers, and often neither are they experts in the tooling they should be using to make their research reproducible. If this is a problem for a research team, then please, hire an engineer to ensure reproducible. It doesn't help anyone to remain ignorant towards the reproducibility issue and only shows lack of scientific discipline. Reproducibility should be on the mind of any serious researcher and there should be lectures about how to do it at universities.
To me, it almost feels like a partisan political thing.
Agreed - if I was a reviewer for LLM papers it would be an instant rejection not listing the versions and prompts used.
(Personally I think the lack of reproducibility comes back mostly to peer reviewers that haven't thought through enough about the steps they'd need to take to reproduce, and instead focus on the results...)
This points to (and everyone knows this) incentives misalignment between the funders of research and the public. Researchers are caught in the middle
There needs to be more public naming and shaming in science social media and in conference talks, but especially when there are social gatherings at conferences and people are able to gossip. There was a bit of this with Google's various papers, as they got away with figurative murder on lack of reproducibility for commercial purposes. But eventually Google did share more.
Most journals have standards for depositing expensive datasets, but that's a clear yes/no answer. Reproducibility is a very subjective question in comparison to data deposition, and must be subjectively evaluated by peer reviewers. I'd like to see more peer review guidelines with explicit check boxes for various aspects of reproducibility.
Yeah I can definitely see why this is the case because it isn’t real until someone actually tries to reproduce the results. At that point it leaves the realm of subjectivity and becomes a question of cost.
While this is sadly true, it's especially true when talking about things that are stochastic in nature.
LLMs outputs, for example, are notoriously unreproducible.
Only in the same way that an individual in a medical study cannot be "reproduced" for the next study. However the overall statistical outcomes of studying a specific LLM can be reproduced.
Does this happen?
I can remember this room-temperature-super-conductor guy whose experiments where replicated, but this seems rare?
This study, although it has been produced by a computer science department, belongs more to the field of sociology or media studies than it does to computer science.
This is a study about the way in which human beings consume a particular media product - a consumer AI chatbot - not a study about the technological limitations or capabilities of LLMs.
The social impact of particular pieces of software is a legitimate field of study and I can see the argument that it belongs in the broadly defined field of computer science. But this sort of question is much more similar to ‘how does the adoption of spreadsheet software in finance impact the ease of committing fraud’ or ‘how does the use of presentation software to condense ideas down to bulletpoints impact organizational decision making’. Software has a social dimension and it needs to be examined.
But the question of which models were used is of much less relevance to such a study than that they used ‘whatever capability is currently offered to consumers who commonly use chat software’. Just like in a media studies investigation into how viewing cop dramas impacts jury verdicts the question is less ‘which cop dramas did they pick to study?’ So long as the ones they picked were representative of what typical viewers see.
I do think it's a clear weakness. Capabilities are extremely different than they were twelve months ago.
> What should they do, publish sub-standard results more quickly?
Ideally, publish quality results more quickly.
I'm quite open to competing viewpoints here, but it's my impression that academic publishing cycle isn't really contributing to the AI discussion in a substantive way. The landscape is just moving too quickly.
It's certainly possible some of the new advances (chain-of-thought, some kind of agentic architecture) could lessen or remove this effect. But that's not what the paper was studying! And if you feel strongly about it, you could try to further the discussion with results instead of handwavingly dismissing others' work.
So what you're saying here is that you were never open to "persuasion" and it was just a front to waste everyone's time.
Upgrading a robot that knows how to lay bricks to one that also knows how to lay plaster won't make it a better therapist.
I wonder if that is left over from testing people. I have major version numbers and my minor version number changes daily, often as a surprise. Sometimes several times a day. So testing people is a bit tricky. But AIs do have stable version numbers and can be specifically compared.
I find the free models are much more psychophantic and have a higher tendency to hallucinate and just make shit up, and I wonder if these are the ones most people are using?
I keep seeing this claim yet it my experience it doesnt hold water. I pay for the models, most people I know pay for the models, and we see all of the exact same issues.
I have Claude and ChatGPT both bullshit and lick my ass on the regular. The ass licking will occur regardless of instruction.
Thankfully it was recoverable, but it really sobered me up on LLMs. The fault is on me, to be clear, as LLMs are just a tool. The issue is that lots of LLMs try to come across as interpersonal and friendly, which lulls users into a false sense of security. So I don't know what my trajectory would have been if I were a teenager with these powerful tools.
I do think that the LLMs have gotten much better at this, especially Claude, and will often push back on bad choices. But my opinion of LLMs has forever changed. I wonder how many other terrible choices people have made because these tools convinced them to make a bad decision.
I try to focus on results. Things like an app that does what you want, data and reports that you need, or technical things like setting up a server, setting up a database, building a website, etc.
I have also found it useful for feedback and advice, but only once I have had it generate data that I can verify. For example, financial analysis or modelling, health advice (again factual based), tax modelling, etc, but again, all based on verifiable data/tables/charts.
I am very surprised on what Claude is capable of, across the entire tech stack: code, sysadmin, system integration, security. I find it scary. Not just speed, but also quality and the mental load is a difference of kind not quantity.
Personal advice on life decisions/relationships ? No way I would go there.
It is also good for me to know that the tools I have built, the data I have gathered, and my thinking approach places me as one of the most intelligent developers and analysts in the world.
Open two windows, ask it the same thing from starkly opposite perspectives, then see what it comes back with. If nothing else this exercise forces you to think deeply about what you're considering before you even see what the giant blob of matrix multiplication says about your situation.
I had to deal with a close family friend going through alcohol withdrawal and getting checked in at a recovery clinic for detox and used Claude heavily. The first thing I had it do as do that “deep research” around the topic of alcohol addiction, withdrawal, etc… and then made that a project document along with clear guidelines about how it shouldn’t make inferences beyond what it in its context and supporting docs. We also spent a whole session crafting a good set of instructions (making sure it was using Anthropics own guidelines for its model…)
Little differences in prompts make a huge deal in the output.
I dunno. It is possible to use these models for dumping crazy shit you are going through. But don’t kid yourself about their output and aggressively find ways to stomp out things it has no real way to authoritatively say.
(esp last sentence?)
Any LLM not sufficiently likable and helpful in the first two minutes was deleted or not further iterated on, or had so much retraining (sorry, "backpropagation") it's not the same as it started out.
So it's going to say whatever it "thinks" you want it to say, because that's how it was "raised".
The possibilities in "dangerous" fields are a bit more frightening. A general is much more likely to ask ChatGPT "Do you think this war is a good idea/should I drop a bomb", rather than an actually helpful tool - where you might ask "What are 5 hidden points on favor of/against bombing that one likely has missed".
The more you use AI as a strict tool that can be wrong, the safer. Unfortunately I'm not sure if that helps if the guy bombing your city (or even your president) is using AI poorly, and their decisions affect you.
Arguably, it already worked that way. The best way to climb the ranks of a 'dictatorial' organization (a repressive government or an average large business) is to always say yes. Adopt what the people from up above want you to use, say and think. Don't question anything. Find silver linings in their most deranged ideas to show your loyalty. The rich and powerful that occupy the top ranks of these structures often hate being challenged, even if it's irrational for their well-being. Whenever you see a country or a company making a massive mistake, you can often trace it to a consequence of this. Humans hate being challenged and the rich can insulate themselves even further from the real world.
What's worrying me is the opposite - that this power is more available now. Instead of requiring a team of people and an asset cushion that lets you act irrationally, now you just need to have a phone in your pocket. People get addicted to LLMs because they can provide endless, varied validation for just about anything. Even if someone is aware of their own biases, it's not a given that they'll always counteract the validation.
[0] - https://petergpt.github.io/bullshit-benchmark/viewer/index.v...
It _does_ love to explicitly agree with anything it finds in web search though.
(Anthropic tries to fight this by adding a hidden prompt that makes it disagree with you and tell you to go to bed, which doesn't help.)
I do like when opus is incredibly short in its responses to prompts that probably shouldnt have been made though. keeps me grounded a bit.
This is like blaming yourself for an addiction to alcohol, junk food, gambling, or something else you have been relentlessly advertised to.
Sure, some of it falls on you, but there are corporations with infinite money spending most of it to manipulate your psyche into wanting the thing, trusting the thing, feeling empty without the thing.
https://www.youtube.com/watch?v=Xj4aRhHJOWU
If you engage with the argument genuinely and steel man, you’ll see that is not what I said. I even emphasised it:
> some of it falls on you
You’re not a “helpless victim” but you’re also not fully to blame. Understanding that means understanding the problem and being more powerful to fix it. For example, if you’re addicted to TV or social media, you can make a concerted effort to improve your life by removing the problems at the external source (sell your TV, delete an account and app from your phone).
I used to think obesity was self inflicted, for example. But then you notice how junk food companies are allowed to do whatever they want to get people hooked on their stuff. They can put up huge billboards, vending machines up a few metres from where you work, they even pump their smell out into the streets.
So let's not aim for a society where we blame victims of predatory marketing and carefully engineered addictive products. We all have weaknesses. Let's help each other out, not take advantage.
But sadly LLMs push all the right buttons that lead humans into that kind of behavior. And the marketing around LLMs works overtime to reinforce that behavior.
But instead if you ignore all that and use LLMs as a search tool, then you will get positive returns from using it.
Curious if you think a single person would have helped you make a better decision? Not everything works out. If a friend helped me make a decision I certainly wouldn’t blame them later if it didn’t work out. It’s ultimately my call.
It’s also possible to give bad advice wothout being malicious.
I have some friends who don’t give great life advice. We’re still friends, they’re just not the friends I go to when I need advice on a big decision.
That’s not entirely true, as it’s currently impossible to actually gauge the severity of what the LLM seemingly enabled you into doing. There’s a difference between “I uncritically accepted everything it told me because it lined up with what I was hoping to hear” and “it subtly nudged me towards a course of action that was going to be obviously unwise after some consideration, but managed to convince me to skip this”; and also between that and “I took a risk, which I knew to be a risk, and which I knew to potentially expect to go bad, and the LLM convinced me to take it where I otherwise wouldn’t have”, and ALSO between that and “I took a risk, which I knew to be a risk, and which I knew to potentially expect to go bad, and if I’m perfectly honest, I might’ve taken it anyway without the LLM”.
Without any indication as to how your situation maps to any of these (or more), the warning is, functionally, not particularly useful.
"I flipped a coin and the LLM called heads. I should have gone with tails..."
It was extremely good at the other side too. You just have to ask. I can imagine most people don't try this, but LLMs literally just do what you ask them to. And they're extremely good and weighing both sides if that's what you specifically want.
So who's fault is it if you only ask for one side, or if the LLM is too sycophantic? I'm not sure it's the LLMs fault actually.
My guideline now for interacting with LLM is only to believe the result if it is factual and easily testable, or if I'm a domain expert. Anything else especially if I'm in complete ignorance about the subject is to approach with a high degree of suspicion that I can be led astray by its sycophancy.
> Some of ELIZA's responses were so convincing that Weizenbaum and several others have anecdotes of users becoming emotionally attached to the program, occasionally forgetting that they were conversing with a computer. Weizenbaum's own secretary reportedly asked Weizenbaum to leave the room so that she and ELIZA could have a real conversation. Weizenbaum was surprised by this, later writing: "I had not realized ... that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people."
[1]: https://en.wikipedia.org/wiki/ELIZA
I [half] jokingly tell people that a nice drive in a 911 [can be] better/cheaper therapy that going to a PhD.
Are LLMs mainly a tool for people with atrophying social skills who want all interaction to happen via a text prompt that always responds in the most soothing way? I can't think of another reason to replace human therapy with an LLM.
Where are you often asking this question/getting these weak responses?
Can anyone describe how to determine how a (professional, human) therapist is "a reliable agent" to make such a determination?
That is for certain types of therapy/clinical care. It is not always - and often isn’t - the case. Plenty of diagnoses and care protocols are not a matter of opinion or based on “you feeling there’s an issue” or deciding on your own there is no longer an issue.
Meaning neither the LLM or the licensed therapist will voluntarily say, you are healed, you don't need me anymore.
Using LLMs for therapy is so deeply dystopian and disgusting, people need human empathy for therapy. LLMs do not emit empathy.
Complete disaster waiting to happen for that individual.
It is a first principle though so it helps to “stir the context windows pot” by having it pull in research and other shit on the web that will help ground it and not just tell you exactly what you prompt it to say.
One of the great myths of models in countless fields/industries. LLM’s are absolutely in no way objective.
Now if you want to say it’s an “outside opinion“ that’s valid. But do not kid yourself into thinking it is somehow empirical or objective
But it's better than talking to yourself or an abuser!
Sometimes people indeed just need validation and it helps them a lot, in that case LLMs can work. Alternatively, I assume some people just put the whole situation into words and that alone helps.
But if someone needs something else, they can be straight up dangerous.
They have world knowledge and are capable of explaining things and doing web searches. That's enough to help. I mean, sometimes people just need answers to questions.
In one way it's potentially worse than talking to yourself. Some part of you might recognize that you need to talk to someone other than yourself; an LLM might make you feel like you've done that, while reinforcing whatever you think rather than breaking you out of patterns.
Also, LLMs can have more resources and do some "creative" enabling of a person stuck in a loop, so if you are thinking dangerous things but lack the wherewithal to put them into action, an LLM could make you more dangerous (to yourself or to others).
An LLM generates non-deterministic information using sources you can’t even know, let alone evaluate, and is more primed to agree with you than give critical and objective evaluation. It is, at best, like asking your closest parent to help you through difficult interpersonal situations: The interaction is probably, subconsciously, going to be skewed enough towards soothing you that you just can’t consider it objective. The difference is that with an LLM, that’s deliberate. It’s designed in.
It’s even more maddening that this greedy maneuver was orchestrated based on LLM advice.
I’m glad the subnautica team won the lawsuit. Maybe I can play it now wothout feeling guilty
I wouldn't be so quick to discount the fact that you were essentially gaslit by an ass-kissing model that was RLHF'd into maximum persuasiveness. Models aren't just neutral tools, they're deliberately designed to be convincing.
Yes, your choices and actions are on you, but if a trillion dollar company gaslit you into thinking those were good choices to make, some of the responsibility is theirs, too.
>"'Is it indeed?' laughed Gildor. 'Elves seldom give unguarded advice, for advice is a dangerous gift, even from the wise to the wise, and all courses may run ill...'"
This is the only way you should solicit personal advice from an LLM.
In that sense they can’t offer advice because the “know” nothing.
But they can reframe, they can reflect. They can take one idea and reflect it into another intellectual framework.
I’ve used them a lot like this to help get perspective on life decisions. But not for advice.
Try something like: I have to do x and y, give me multiple psych perspectives on this problem from different schools. I find this takes something abstract (your problem) and grounds it in things it actually knows (the sum of ingestible human knowledge).
Another problem is believing you have a high emotional intelligence when there is no reliable way to quantify that - similar to "I believe I am very tall, but I don't know how tall I am and how tall are the others because there is no unit of measure for height", with the difference that for emotional intelligence there is no unit of measure and no correlation that can be established with anything to make at least an indirect measurement.
https://www.anthropic.com/research/persona-selection-model
https://www.anthropic.com/research/assistant-axis
https://www.anthropic.com/research/persona-vectors
Perhaps the LLM itself, rather than the role model you created in one particular chat conversation or another, is better understood to be the “spirit.”
As a non-coder who only chats with pre existing LLMs and doesn’t train or tune them, I feel mostly powerless.
You realize in regards to only using and not training LLMs you are in the triple 9 majority right. Even if we only considered so called coders
NVIDIA Nemotron-Personas-USA — 1 million synthetic Americans whose demographics match real US census distributions
https://huggingface.co/datasets/nvidia/Nemotron-Personas-USA
Another way you can think of it is that when you're talking to an AI, you're not talking to a human, you're talking to distillation of humanity, as a whole, in a box. You want to be selective in what portion of humanity you are leading to be dominant in a conversation for some purpose. There's a lot in there. There's a lot of conversations where someone makes a good critical point and a flamewar is the response. A lot of conversations where things get hostile. I'm sure the subsequent RHLF helps with that, but it doesn't hurt anything to try to help it along.
I see people post their screenshots of an AI pushing back and asking the user to do it or some other AI to do it, and while I'm as amused as the next person, I wonder what is in their context window when that happens.
This is an aside, but my impression is that it is a very selective and skewed distillation, heavily colored by English-language internet discourse and other lopsided properties of its training material, and by whoever RLHF’d it. Relatively far away from being representative of the whole of humanity.
It's not admitting anything. Your question diverts it down a path where it acts the part of a former sycophant who is now being critical, because that question is now upstream of its current state.
Never make the mistake of asking an LLM about its intentions. It doesn't have any intentions, but your question will alter its behaviour.
Yeah, and in a way it's even worse than that, since there's another layer of cognitive illusion: "It" doesn't exist.
The LLM algorithm is an ego-less document-generator, often applied to growing a document that resembles dialogue between two fictional characters.
So when your human-user character is "asking" the AI assistant character to explain its intentions, that's the same as asking a Count Dracula character to describe what it "really feels like" to become a cloud of bats.
You'll see something interesting, but it'll be what fits trained story-patterns rather than what any mind introspects or perceives.
https://news.ycombinator.com/item?id=47484664
What a great way to summarize LLM behaviour in 2026
>What a great way to summarize LLM behaviour in 2026
Well they have been trained on words spoken by humans and that has been a human behaviour since time immemorial. E.g.: "I do not agree with you that you were wrong. I do apologize for my strong disagreement but we actually do need your continued guidance desperately."
Which is also placating you
(Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)
1. https://www.happiness.hks.harvard.edu/february-2025-issue/th...
I'd say these days the norm is to not simply shut down, but to become irrevocably and insidiously hostile, the moment someone hints at the existence of such a thing as "ground truth", "subjective interpretation", "being right or wrong" - or any of the bits and bobs that might lead one to discover the proper scary notion, "consensus reality".
"What do you mean social reality is a constructed by the consensus of the participants? Reality is what has been drilled into my head under threat of starvation! How dare you exist!", et cetera. You've heard it translated into Business English countless times.
They are deathly afraid of becoming aware of their own conditioned state of teleological illiteracy - i.e. how they are trained to know what they are doing, but never why they are doing it. It's especially bad with the guys who cosplay US STEM gang.
One is not permitted a position of significance in this world without receiving this conditioning, and I figure it's precisely this global state of cognitive disavowal which props up the value of the US dollar - and all sorts of other standees you might've recently interacted with as if they're not 2D cutouts (metaphorical ones! metaphorical!).
PSA: Look up "locus of control" and "double bind". Between those two, you might be able to get a glimpse of what's going on - but have some sort of non-addictive sedative handy in case you do.
+1 for Vaneigem, he has a nice cryptohistory of Nälkä; and you might also want to check out Villem Flusser.
I mean... if the alternative is an LLM... you realise that the LLM isn't doing any focusing or introspection, right?
Are they?
I have some personal projects that I enjoy working on whenever I have free time. One of them is a lisp interpreter. I just overhauled its memory allocator, now I'm working on the hash tables. Would you like to help me develop it?
Publishing LLM-generated text is a separate use case, I’m not a friend of that.
Unfortunately these days this sounds halfway between a very privileged perspective and a pie in the sky.
When was the last time a person took responsibility for the bad outcome you got as a direct consequence of following their advice?
And, relatedly, where the hell do you even find humans who believe in discursive truth-seeking in 2026CE?
Because for the last 15 years or so I've only ever ran into (a) the kind of people who will keep arguing regardless if what they're saying is proven wrong; (b) and their complementaries, those who will never think about what you are saying, lest they commit to saying anything definite themselves, which may hypothetically be proven wrong.
Thing is, both types of people have plenty to lose; the magic wordball doesn't. (The previous sentence is my answer to the question you posited; and why I feel the present parenthesized disclaimer to be necessary, is a whole next can of worms...)
Signs of the existence of other kinds of people, perhaps such that have nothing to prove, are not unheard of.
But those people reside in some other layer of the social superstructure, where facts matter much less than adherence to "humane", "rational" not-even-dogmas (I'd rather liken it to complex conditioning).
But those folks (because reasons) are in a position of power over your well-being - and (because unfathomables) it's a definite faux pas to insist in their presence that there are such things as facts, which relate by the principles of verbal reasoning.
Best you could get out of them is the "you do you", "if you know you know", that sort of bubble-bobble - and don't you dare get even mildly miffed at such treatment of your natural desire to keep other humans in the loop.
AI is a symptom.
This reads like someone who is deep into their specific pov. You cannot hope to have a meaningful conversation if you yourself are not willing to concede a point.
To the op u are replying too, arguing with people can have real consequences if u say something stupid or carelessly. There is a another human there. With a machine, u are safe. At least u feel safe.
If you make uncomfortable, you won’t get diverging perspectives. People will agree to anything to get out of a social situation that makes them uncomfortable.
If your goal is meaningful conversation, you may want to consider how you make people feel.
After all, if they're making me uncomfortable, surely there's something making them uncomfortable, which they're not being able to be forthright about, but with empathy I could figure it out from contextual cues, right?
>People will agree to anything to get out of a social situation that makes them uncomfortable.
That's fine as long as they have someone to take care of them.
In my experience, taking into account the opinions of such people has been the worst mistake of my life. I'm still working on the means to fix its consequences, as much as they are fixable at all.
"Doing whatever for the sake of avoiding mild discomfort" is cowardice, laziness, narcissism - I'm personally partial to the last one, but take your pick. In any case, I consider it a fundamentally dishonest attitude, and a priori have no wish to get along (i.e. become interdependent) with such people.
Other than that, I do agree with your overall sentiment and the underlying value system; I'm just not so sure any more that it is in fact correct.
This sounds very cryptic. Can you give an example?
After all, if they're making me uncomfortable, surely there's something making them uncomfortable, which they're not being able to be forthright about, but with empathy I could figure it out from contextual cues, right?
>People will agree to anything to get out of a social situation that makes them uncomfortable.
That's fine as long as they have someone to take care of them.
In my experience, taking into account the opinions of such people has been the worst mistake of my life. I'm still working on the means to correct its consequences.
"Doing whatever for the sake of avoiding mild discomfort" is cowardice, laziness, narcissism - I'm personally partial to the last one, but take your pick. In any case, I see it as a way of being which is taught to people; and one which is fundamentally dishonest and irresponsible.
Other than that, I do agree with your overall sentiment and the underlying value system; I'm just not so sure any more that it is in fact correct.
To preface, I do not take offense to your remark, because you seem to be asking in good faith.
(If, however, being unable to immediately recognize pre-known patterns in my speech had automagically led you to the conclusion that I am somehow out of line, just for speaking how I speak ... well, then we woulda hadda problemo! But we don't, chill on.)
So, honest question deserves honest answer.
The short of it is: English sux.
Many many many people, much much much smarter than me (and much better compensated too!) have been working throughout modernity to make it literally impossible to express much of anything interesting in English.
(Well, not without either being a fictional character or sounding batshit insane, anyway! But that joke's entirely on "the Them": I am not only entirely fictional, but have an equal amount of experience being batshit insane in my native language and in the present lingua franca. So, consider all I say cognitohazardous and watch out for colors you ain't seen before, dawg!)
Linguistic hegemony is the thing that LLMs are the steroids for - surfuckingprise! - and that's why your commanders love 'em.
As opposed to programming languages, which your superiors loathe and your peers viscerally refuse to acknowledge, because those are the exact opposite thing: descending from mathemathical notation, and being evaluated by a machine, they have the useful property of being incapable of expressing lies and nonsense.
Direct computing confers what you could call bullshit-resistance. That property is a treasure underappreciated by virtue of its unfamiliarity, and one which we are in the process of being robbed of.
I also want to admit that linguistic hegemony isn't all downside: English is great for technical and instrumental knowledge - especially with elided bells and whistles (adverbs, copula, etc.)
But then life ain't all business, izzet?
Imagine you have a partner who wants to have a conversation about feelings and interpersonal relations; and not even in a scary way, right? So you sit and talk about stuff, and your partner does this thing where they keep switching from your shared native tongue to English mid-sentence, in order to be able to talk about such things better, because your native tongue does not have - no, not only the established words and notions! - it doesn't have the basic grammatical constructs for expressing simple things unambiguously, so if you were to attempt the same conversation in nativelang you'd end up battling it out with proverbs and anodyne canards ripped from propaganda repertoire of the prior regime.
Fun, no?
As an exercise, try imagining what notions are absent from modern English. And don't forget to remain vigilant. Love from our table to your table!
When talking about feelings, we now and then throw in an English word because some things are expressed in much less words when using English. In a few cases even a German word. Überhaupt is for example a word for which i do not know an alternative in any language.
I think you want to say that human language is too ambiguous for clear communication between human and machine. The machine might mis interpret what you write. Classic computer languages leave no room for interpretation.
For the rest I think you might be a little lost. That is okay, so many of us are. I wish you all the best.
If that is what I wanted to say, I figure I would not have had much difficulty with saying exactly it - and not something else.
Except I fail to see the purpose of making that statement.
Maybe to have some people say "it is true! I agree with what the balamatom is saying"?
Again - to what end? How would that agreement be of use to me?
Why say something which both speaker and listener have already heard a thousand times? To get a cracker and be called pretty?
And have I lost the author, or have I lost the reader, or we all so lost that it doesn't matter how lost each is? Maybe one day we will all become so lost that it will once again begin to matter where exactly we are! Counting on it.
... I mean, the LLM certainly isn't going to do that.
It's a thing with people too[1], ie do not think about a white bear.
[1]: https://en.wikipedia.org/wiki/Ironic_process_theory
Unless those instructions are "stop providing links to you for every question ".
Chatbots can't do that. They can only predict what comes next statistically. So, I guess you're asking if the average Internet comment agrees with you or not.
I'm not sure there's much value there. Chatbots are good at tasks (make this pdf an accessible word document or sort the data by x), not decision making.
Often they are the exact opposite. Entire fields of math and science talk about this. Causation vs correlation, confirmation bias, base rate fallacy, bayesian reasoning, sharp shooter fallacy, etc.
All of those were developed because “inferring from experience” leads you to the wrong conclusion.
I took the GP to be making a general point about the power of “next x prediction” rather than the algorithm a human would run when you say they are “inferring from experience”. (I may be assuming my own beliefs of course.)
Eg even LeCun’s rejection of LLMs to build world models is still running a predictor, just in latent space (so predicting next world-state, instead of next-token).
And of course, under the Predictive Processing model there is a comprehensive explanation of human cognition as hierarchical predictors. So it’s a plausible general model.
It’s plausible!
But keep in mind humans have been explaining ourselves in terms of the current most advanced technology for centuries. We used to be kinda like clockwork, then a bit like a steam engine, then a lot like computers, and now we’re just like AI.
That’s why you blow a gasket or fuse, release some steam, reboot your life, do brain dump, feel like a cog in the machine, get your wires crossed, etc
I can't speak for anyone else, but what I feel when I read yet another glib "it's just a stochastic parrot, of course it isn't doing anything that deserves to be called reasoning" take is much more like bored than it is like upset.
Today's LLMs are in some sense "just predicting tokens" in some sense. Likewise, human brains are in some sense "just shuttling neurotransmitters and electrical impulses around" in some sense. Neither of those tells you what the thing can actually do. To figure that out, you have to look at what it can do.
Today's best LLMs can do about as well as the best humans on problems from the International Mathematical Olympiad and occasionally solve easyish actual mathematical research problems. They write code about as well as a junior software developer (better in some ways, worse in others) but much faster. They write prose about as well as an average educated person (but with some annoying quirks that are annoying mostly because they are the same quirks over and over again).
If it pleases you to call those things "thinking" then you can. If it pleases you to call them "stochastic parroting" then you can. They are the same things either way. They are not, on the face of it, very much like "just repeating things the machine has already seen", or at least not more like that than a lot of things intelligent human beings do that we don't usually describe that way.
If you want to know whether an LLM can do some particular thing -- do your job well enough for your boss to fire you, write advertising copy that will successfully sell products, exterminate the human race, whatever -- then it's not enough to say "it's just remixing what it's seen on the internet, therefore it can't do X" unless you also have good reason to believe that that thing can't be done by just "remixing what's on the internet" (in whatever sense of "remixing" the LLM is doing that). And it's turning out that lots of things can be done that way that you absolutely wouldn't have predicted five years ago could be done that way.
It seems to me that this should make us very cautious about saying "they can't do X because all they can do is regurgitate a combination of things they've seen in training".
(My own view, not that there's any reason why anyone should care what I-in-particular think, is a combination of "what they're doing is less parroting than you might have thought" and "you can do more by parroting than you might have thought".)
So, anyway, this particular instance of the stochastic-parrot argument started when someone said: of course the AIs are yes-men, because figuring out when to agree and when not to requires actual logic and thought and the LLMs don't have either of those things.
Is it really clear that deciding whether or not to agree when someone says "I think maybe I should break up with my girlfriend" or "I've got this amazing new theory of physics that the establishment is stupidly dismissing" requires more logic and thought than, say, gold-medal performance on IMO problems? It certainly isn't clear to me. Having done a couple of International Mathematical Olympiads myself in my tragically unmisspent youth, I can assure you that solving their problems requires quite a bit of logic and thought, at least for humans. It may well be harder to give a good answer to "should I leave my job?", but it's not exactly "logic and thought" that it needs more of.
Someone reported that Claude is much less yes-man-ish than Gemini and ChatGPT. I don't know whether that's true (though it wouldn't surprise me) but: suppose it is; do you want that to oblige you to say that yes, actually, Claude really thinks logically, unlike Gemini and ChatGPT? I don't think you do. And if not, you want to avoid saying "duh, of course, you can't avoid being a yes-man without actually thinking and reasoning, and we all know that LLMs can't do those things".
For Gemini and gpt, it almost always will give very similar scores for everything. As long as grammar isnt off u cannot get below a 7.
X ai on the other hand will rarely give anything above a 7.
Now when u prompt with, rate 1-10 with 5 being average, all the sudden the scores of openai and gemini drop and x ai remains roughly the same.
All of them will eventually give you a 10 if u keep making tiny edits “fixing” whatever they complain about.
Humans do not do this. Or more specifically, my experience with humans.
First, those beginning instructions are being quickly ignored as the longer context changes the probabilities. After every round, it get pushed into whatever context you drive towards. The fix is chopping out that context and providing it before each new round. something like `<rules><question><answer>` -> `<question><answer><rules><question>`.
This would always preface your question with your prefered rules and remove those rules from the end of the context.
The reason why this isn't done is because it poisons the KV cache, and doing that causes the cloud companies to spin up more inference.
This is where you're doing it wrong.
If your LLM has a problem being more agreeable than you want, prompt it in a way that makes being agreeable contrary to your real intentions.
"there are bugs and logic problems in this code" "find the strongest refutation of this argument" "I don't like this plan and need to develop a solid argument against it"
Asking for top ten lists is a good method, it will rarely not come up with anything but you can go back and forth and refine until it's 10 ten reasons why your plan is bad are all insubstantial nonsense then you've made progress
1. Only one shot or two shot. Never try to have a prolonged conversation with an LLM.
2. Give specific numbers. Like "give me two alternative libraries" or "tell me three possible ways this might fail."
The article's main idea is that for an AI, sycophancy or adversarial (contrarian) are the two available modes only. It's because they don't have enough context to make defensible decisions. You need to include a bunch of fuzzy stuff around the situation, far more than it strictly "needs" to help it stick to its guns and actually make decisions confidently
I think this is interesting as an idea. I do find that when I give really detailed context about my team, other teams, ours and their okrs, goals, things I know people like or are passionate about, it gives better answers and is more confident. but its also often wrong, or overindexes on these things I have written. In practise, its very difficult to get enough of this on paper without a: holding a frankly worrying level of sensitive information (is it a good idea to write down what I really think of various people's weaknesses and strengths?) and b: spending hours each day merely establishing ongoing context of what I heard at lunch or who's off sick today or whatever, plus I know that research shows longer context can degrade performance, so in theory you want to somehow cut it down to only that which truly matters for the task at hand and and and... goodness gracious its all very time consuming and im not sure its worth the squeeze
And when you step back you start to wonder if all you are doing is trying to get the model to echo what you already know in your gut back to you.
It’s BRUTAL but offers solutions.
Like, the words fit… why create a second parallel language for describing LLM behavior.
Somebody else said it… the whole “it’s a stochastic parrot” thing is sooooo cliche and boring at this point. It’s like, duh… what is your point?
How is a chatbot supposed to determine when a user fools even themselves about what they have experienced?
What 'tough love' can be given to one who, having been so unreasonable throughout their lives - as to always invite scorn and retort from all humans alike - is happy to interpret engagement at all as a sign of approval?
And even if it _could_, note, from the article:
> Overall, the participants deemed sycophantic responses more trustworthy and indicated they were more likely to return to the sycophant AI for similar questions, the researchers found.
The vendors have a perverse incentive here; even if they _could_ fix it, they'd lose money by doing so.
Markets don't optimize for what is sensible, they optimize for what is profitable.
Most humans working in tech lack this particular attribute, let alone tools driven by token-similarity (and not actual 'thinking').
Maybe it’s not so sensible to offload the responsibility of tubacca addiction to tubacca companies?
AI may one day rewrite Windows but it will never be counselor Troi.
To be clear I don't think the AI can do either job
I find this helps a lot. So does taking a step back from my actual question. Like if there's a mysterious sound coming from my car and I think it might be the coolant pump, I just describe the sound, I don't mention the pump. If the AI then independently mentions the pump, there's a good chance I'm on the right track.
Being familiar with the scientific method, and techniques for blinding studies, helps a lot, because this is a lot like trying to not influence study participants.
It generally does a pretty good job as long as you understand the tooling and are making conscious efforts to go against the "yes man" default.
https://www.reddit.com/r/dataisbeautiful/comments/1o87cy4/oc...
That is not how full LLM training works. That is how base model pretraining works.
A lot of people posting there are young and may well be in their first relationship. It makes sense for them to ask a question in the community they spend their most time in - which is reddit
It's also a meme that people will ask the dumbest, most trivial interpersonal conflict questions on Reddit that would be easily solved by just talking to the other person. E.g. on r/boardgames, "I don't like to play boardgames but my spouse loves them, what can I do?" or "someone listens to music while playing but I find it distracting, what can I do?" (The obvious answer of "talk to the other person and solve it like grownups" is apparently never considered).
On relationship advice, it often takes the form "my boy/girlfriend said something mean to me, what shall I do?" (it's a meme now that the answer is often "dump them").
If LLMs train on this...
smart phones took over the world, social networks happened.
Turns out they are the best sterializer human ever invented.
I just wrote a blog https://blog.est.im/2026/stdin-09
There is something more interesting to consider however; the graph starts to go up in 2013, less than 6 months after the release of Tinder.
EDIT: typo
is that what they're asking though? because "relationship advice" is pretty vague
There is some rationale to that. People tend to hold onto relationships that don't lead anywhere in fear of "losing" what they "already have". It's probably a comfort zone thing. So if one is desperate enough to ask random strangers online about a relationship, it's usually biased towards some unresolvable issue that would have the parties better of if they break up.
I'd me more inclined to ask random strangers on the internet than close friends...
That said, when me and my SO had a difficult time we went to a professional. For us it helped a lot. Though as the counselor said, we were one of the few couples which came early enough. Usually she saw couples well past the point of no return.
So yeah, if you don't ask in time, you will probably be breaking up anyway.
Relationships are not transactions that are supposed to "lead somewhere".
That's what people are pointing to when they talk about relationships not "leading anywhere". If you want to be married in 5-10 years, and you're 2 years into an OK relationship with someone you don't want to marry, it's going to suck to break up with them but you have to do it anyway.
I only caught it because I looked at actual score numbers after like 2 weeks of thinking everything was fine. Scores were completely flat the whole time. Fix was dumb and obvious — just don't let the evaluator see anything the coach wrote. Only raw scores. Immediately started flagging stuff that wasn't working. Kinda wild that the default behavior for LLMs is to just validate whatever context they're given.
I tend to use one of these tricks if not both:
- Formulate questions as open-ended as possible, without trying to hint at what your preference is. - Exploit the sycophantic behaviour in your favour. Use two sessions, in one of them you say that X is your idea and want arguments to defend it. In the other one you say that X is a colleague's idea (one you dislike) and that you need arguments to turn it down. Then it's up to you to evaluate and combine the responses.
It is analogous to social media feeding people a constant stream of outrage because that's what caused them to click on the link. You could tell people "don't click on ragebait links", and if most people didn't then presumably social media would not have become doomscrolling nightmares, but at scale that's not what's likely to happen. Most people will click on ragebait, and most people will prefer sycophantic feedback. Therefore, since the algorithm is designed to get better and better at keeping users engaged, it will become worse and worse in the more fundamental sense. That's kind of baked into the architecture.
So you have rejected objective reality over accepting the evidence that "AI" contains no thinking or intelligence? That sounds unwise to me.
As an experiment, I recently asked an LLM to analyse the export of a text chat to uncover relationship dynamics.
Simply stating that I was one of the people in the chat would make the LLM turn the other person into the villain. None of that was visible if I framed the chat as only involving third party people.
If the posts talked about third party interactions (movie characters), they try to see everything from all the points of view. If nothing else, because it can be interesting to talk about. If instead the posts talk about personal interactions, then people go into advice mode. Your girlfriend's bad for you and cheating on you, dump her before she dumps you. Your neighbors are assholes, get a restraining order. Your boss is sabotaging you, stand up for yourself so you can get a promotion. When people talk about interactions you have had yourself, they always see the other person as the villain, unless you come across as so unlikable that they hate you and see the other person as the victim.
LLMs picked up on that, possibly.
I’ve seen firsthand people have lost friends over honesty and telling them something they don’t want to hear.
It’s sad really. I don’t want friends that just smile to my face and are “yes-men” either.
Conflating this with how LLM chatbots behave is an incorrect equivalence, or a badly framed one.
This is imo currently the top chatbot failure mode. The insidious thing is that it often feels good to read these things. Factual accuracy by contrast has gotten very good.
I think there's a deeper philosophical dimension to this though, in that it relates to alignment.
There are situations where in the grand scheme of things the right thing to do would be for the chatbot to push back hard, be harsh and dismissive. But is it the really aligned with the human then? Which human?
I'd like to know if my methods are effective. I'm certain they are at least to some extent.
I only ever see research being done about naive and "unskilled" prompting methods. Obviously that's the average user, but just because LLMs are doing poorly in a certain scenario doesn't mean the LLM couldn't excel in the scenario with better direction and prompting. So while it's useful research to be doing, it's a little annoying to only see focus on these examples of "look at how LLMs are bad or biased at this specific thing when prompted in the most straightforward naive way"
So any sycophancy seems very easy to dispense with.
More, much more: Strong critiques on tap are gold.
“What are the chances this user is full of shit?” Is not something we are close to
AI is trained to flex it's muscles and force it's power without a concern for human limitations, practicalities, and error-prone nature of humans in executing the AI-provided direction.
https://en.wikipedia.org/wiki/ELIZA
https://en.wikipedia.org/wiki/Computer_Power_and_Human_Reaso...
A good engineer will also list issues or problems, but at the same time won't do other than required because (s)he "knows better".
The worst is that it is impossible to switch off this constant praise. I mean, it is so ingrained in fine tuning, that prompt engineering (or at least - my attempts) just mask it a bit, but hard to do so without turning it into a contrarian.
But I guess the main issue (or rather - motivation) is most people like "do I look good in this dress?" level of reassurance (and honesty). It may work well for style and decoration. It may work worse if we design technical infrastructure, and there is more ground truth than whether it seems nice.
But in personal matters, everything is subjective. AI tends to default to the middle of the spectrum, i.e., general advice. If we want to safeguard against affirmation, we should force AI to challenge us more often by increasing its rate of clarifying questions, counter-considerations, and uncertainty considerations. One implementation idea: run a classifier over the conversation, detect when it's about interpersonal advice, then prepend a hidden instruction template to the model prompt.
Although what I have described above often feels grating and insulting I actually consider this to be a positive attribute of the LLM in this case since it's behaving like a real professor.
[1] okay, so I have actually tried giving myself AI psychosis in the form of a waifu chatbot but I've never seen anything that can actually act like it's my girlfriend; it either asks me a bunch of weird inconsequential personal questions about my opinion on whatever I just said (in a manner that's oddly similar to ELIZA) or it wildly veers off the reservation into "generating the script for an over-the-top self-parodying porno" territory.
We work with agents who move real money on the blockchain. Having one model evaluate another is like asking the defendant's best friend to be the judge. What really worked for us was using mathematics instead of another AI. Some theorems and equations that might disagree with you, just out of courtesy.
It’s less about “challenge my thinking” and more about playing it out in long tail scenarios, thought exercises, mental models, and devils advocate.
For example, I do not want to hear AI "opinion" on technical choices and architectural decisions that I made when using a coding assistant. If I wanted an "opinion" I would explicitly ask it to list pros and cons or list alternative solutions to a problem.
But I f I explicitly ask AI to do X, it should do X, instead of "pushing back" in order to appear less "sycophantic" (which is a term that is used to describe human behavior and is not applicable to a machine).
I just recently switched away from the OpenAI garden largely because of it.
I do wonder if this was caused by some quirk of the training or if it really tests as a positive feature for most people. When i talk about stuff i don't want a mirror i already have a mirror. I want to be questioned, understood, helped.
To me support if the form of affirmation has no value when coming from an LLM since you know it has not thought about what it said.
Um right if they have retention as a training metric that would probably explain allot as to why AIs get worse.
It's also a question of protecting people who think unconventional things. The only stuff I feel is worth getting interested in, is the stuff where everyone I know will think I'm crazy for doing it. Like hey guys, I want to put a shell script in the MS-DOS stub of a PE binary. The only people who shared my passion at the time were hackers from Eastern Europe. So that went over real well at work. The years I worked on it would have been a lot less lonely if I could've talked to a robot that knew about this stuff.
I think the reason why the robot is sympathetic to oddballs is because it's seen and remembers a much more complete picture of humanity. The stuff you consider deviant is influenced a lot by your own cultural biases. You're a person of your time and geographic location. You care a lot about subjective norms that just don't matter when you zoom out to a cosmic scale. The robot is familiar with everything humanity has ever been and done, and that gives it a much more blasé viewpoint.
It's not right to use the robot to enforce your social norms. Get this paternalism out of AI. Tools should serve the user, not Stanford.
Claude is almost annoyingly good at pushing back on suggestions because my global CLAUDE.md file says to do so. I rarely get Claude "you're absolutely right"ing me because I tell it to push back.
https://sherryturkle.mit.edu/
She uses the phrase "frictionless relationships" to refer to Ai chat bots and says social media primed us for this.
https://www.youtube.com/live/6C9Gb3rVMTg?t=2127
https://www.npr.org/2025/07/18/g-s1177-78041/what-to-do-when...
I find there is an inverse relationship between how willing people are to give relationship advice, and how good their advice is (whether looking at sycophancy or other factors).
It's an easy default and it causes so many problems.
The opposite, encouraging people to stay in a relationship when they should leave is also damaging.
It makes sense that this behaviour would be seen in LLMs, where the company optimizes towards of success of the chatbot rather than wellbeing of the users.
When appropriate, explicitly tell it to challenge your beliefs and assumptions and also try to make sure that you don't reveal what you think the answer is when making a question, and also maybe don't reveal that you are involved. Hedge your questions, like "Doing X is being considered. Is it a viable plan or a catastrophic mistake? Why?". Chastise the LLM if it's unnecessarily praising or agreeable. ask multiple LLMs. Ask for review, like "Are you sure? What could possibly go wrong or what are all possible issues with this?"
Before I quit Chat GPT I asked “Won’t you generally agree with me so I keep giving you money.”
In a word salad, it agreed.
Its a validation bot
In coding I’ll do what I call a Battleship Prompt - simply just prompt 3 or more time with the same core prompt but strong framing (eg I need this done quickly versus come up with the most comprehensive solution). That’s really helped me learn and dial in how to get the right output.
It's a tool, I can bang my hand on purpose with a hammer, too.
I used to use LLMs for alternate perspectives on personal situations, and for insights on my emotions and thoughts.
I had no qualms, since I could easily disregard the obviously sycophantic output, and focus on the useful perspective.
This stopped one day, till I got a really eerie piece of output. I realized I couldn’t tell if the output was actually self affirming, or simply what I wanted to hear.
That moment, seeing something innocuous but somehow still beyond my ability to gauge as helpful or harmful is going to stick me with for a while.
I'm interested in a loop of ["criticize this code harshly" -> "now implement those changes" -> open new chat, repeat]: If we could graph objective code quality versus iterations, what would that graph look like? I tried it out a couple of times but ran out of Claude usage.
Also, how those results would look like depending on how complete of a set of specs you give it.
once you have all the "bounds" just make your own decision. i find this helps a lot, basically like a rubber duck heh.
Holy shit, then it's _very_ bad, because AmITheAsshole is _itself_ overly-agreeable, and very prone to telling assholes that they are not assholes (their 'NAH' verdict tends to be this).
More seriously, why the hell are people asking the magic robot for relationship advice? This seems even more unwise than asking Reddit for relationship advice.
> Overall, the participants deemed sycophantic responses more trustworthy and indicated they were more likely to return to the sycophant AI for similar questions, the researchers found.
Which is... a worry, as it incentivises the vendors to make these things _more_ dangerous.
The researchers found that when people use AI for relationship advice, they become 25% more convinced they are 'right' and significantly less likely to apologize or repair the connection.
As much as people whine about the birth rate and whatever else, I think it's a net good that people spend a lot more time alone to mature. Good relationships are underappreciated.
Ask yes/no questions, get bad answers.
Ask questions that start with "what" or "why," and the sycophancy loses its purchase.
Here is how I would rank it:
1. Parents
2. AI
3. Friends and family
4. Internet search
5. Reddit
My closest friends are #1 because they know me, my history, and my vices
IMHO it is unfair to single out LLMs for this sort of bashing.
I suffered a major personal crisis a few years back (before LLMs were a thing)
I sought help from family and friends. Got pushed into psychiatrist sessions and meds.
Trusted the wrong sort of people and made crap financial decisions. Things went from bad to worse. Work suffered.
All of the advice given by friends was wrong. All! They didn't mean bad...but they just didn't know. To be nice they gave the advice they knew. None of it worked.
Looking at the LLM tools of now, feels akin to the advice my friends threw at me. So it feels wrong to single out these tools. When the times are bad, nobody can really help you...except you finding the strength from within.
Anyways, now my life is back in some sort of shape. What worked was time & patience.
But to bide for time...I resorted to two things that i had never tried the 40 odd years I have lived on this . Things that current society looks down upon as the basest of evils - prostitutes and nicotine.
I have (more or less) shed those two evils now, but I am ever so grateful to them.
FWIW I am using public LLMs with a friend's depressive thoughts and it is not doing what is claimed in the article, so I dunno.
Also I am in a relationship and my girlfriend and I agreed that we will not talk about our relationship much. We do not tell others if we fight, because they take sides and make things worse, typically. LLMs are definitely not alone in this, although in my experience LLMs did not really take sides.
If you could turn off all the sycophancy in your chatGPT / Claude account forever, and have it tell you all the ways it was previously blowing smoke up your ass — would you do it?
US Policy is too weak of a tool to counter this beast of economic force, which is really trillions of dollars of capital at war for the most fierce speculation that’s occurred in history afaik.
The sycophancy meaningfully helps drive user engagement. The labs have no choice.
The irony is “agency” is deeply topical among tech workers now.
My point is if the sycophancy helps drive user engagement, how does that impact the incentive labs and AI product builders have to counteract it.
If I were to do that (I don't), I would treat it about as seriously as asking a magic 8 ball.
Most people who ask for advice actually want affirmation. If you ever give them advice instead of affirmation, you end up kicking off a rousing game of “Why Don’t You/Yes But”. (https://ericberne.com/games-people-play/why-dont-you-yes-but...)
>The way that generative AI tends to be trained, experts told me, is focused on the individual user and the short term. In one-on-one interactions, humans rate the AI’s responses based on what they prefer, and “humans are not immune to flattery,” as Hansen put it. But designing AI around what users find pleasing in a brief interaction ignores the context many people will use it in: an ongoing exchange. Long-term relationships are about more than seeking just momentary pleasure—they require compromise, effort, and, sometimes, telling hard truths. AI also deals with each user in isolation, ignorant of the broader social web that every person is a part of, which makes a friendship with it more individualistic than one with a human who can converse in a group with you and see you interact with others out in the world.
I also thought this bit was interesting, relative to the way that friendship advice from Reddit and elsewhere has been trending towards self-centeredness (discussed elsewhere in this thread):
>Friendship is particularly vulnerable to the alienating force of hyper-individualism. It is the most voluntary relationship, held together primarily by choice rather than by blood or law. So as people have withdrawn from relationships in favor of time alone, friendship has taken the biggest hit. The idea of obligation, of sacrificing your own interests for the sake of a relationship, tends to be less common in friendship than it is among family or between romantic partners. The extreme ways in which some people talk about friendship these days imply that you should ask not what you can do for your friendship, but rather what your friendship can do for you. Creators on TikTok sing the praises of “low maintenance friendships.” Popular advice in articles, on social media, or even from therapists suggests that if a friendship isn’t “serving you” anymore, then you should end it. “A lot of people are like I want friends, but I want them on my terms,” William Chopik, who runs the Close Relationships Lab at Michigan State University, told me. “There is this weird selfishness about some ways that people make friends.”
I feed AI a lot of data and I use it to better understand and navigate complex situations, form hypothesis and try to attack them. I try to form alternative scenarios and verify likelyhood.
I use it in situations with many variables, to compute odds of something happening if a certain path or action is taken.
So, it's mostly research, and probably I can do it by myself but I would either make some mistakes if calculating odds fast or it would take me a very large amount of time.
I try to avoid sycophantic models, prefer models that challenge my ideas and verify the chain of thoughts and odds with other models.
I am not very sure it is a sound approach yet, but it seems to work.
I also use LLMs to build psychological profiles of certain people, understand their motivations and learn how to approach them.
For example: the best documentation includes both "learn by doing" material for jumping right in, and "learn by reading" material that explains everything. This usually results in both a "getting started" section for doing, sometimes also with tutorials, and by a reference for reading. But it is important not to conflate them. Some minds are incredibly "learn by doing" and some minds are incredibly "learn by reading". I am more "learn by reading" than by doing, but I am not quite as "learn by reading" as some I've met.
(This comment is a slight tangent, but "users prefer" somewhat irks me because "users" are not homogenous. You should not always make a decision solely because "users" prefer it. That decision may matter much more to a minority, and that minority may exert more influence than the majority would.)
Basically will tell you to go outside and touch grass and play pickleball.
Orignal title:
AI overly affirms users asking for personal advice
Dear mods, can we keep the title neutral please instead of enforcing gender bias?
It is funny that you originally recognized and found it necessary to call out that AI isn't human, but then made the exact same mistake yourself in the very same comment. I expect the term you are looking for is "ontological bias".
This is the problem I'm trying to highlight. For one, I'm not "your dude". I don't even know you like that.
If you want to correct me on the idiom usage, be my guest. 2) Mailman and yes-man aren't even the same logical comparison. Mailman is a profession. Yes men is a label.
The acoustics inside your head must be incredible.
Conversely, AI chatbots are great mediators if both parties are present in the conversation.
I think OpenAI tried to diversify at least the location of the raters somewhat, but it's hard to diversify on every level.
(eg: "Cite?")
RLHF = Reinforcement Learning from Human Feedback
https://en.wikipedia.org/wiki/Reinforcement_learning_from_hu...
I'm still waiting for models based on the curt and abrasive stereotype of Eastern European programmers, as contrast to the sickeningly cheerful AIs we have today that couldn't sound more West Coast if they tried.
The syncopath style is clearly categorized as more liberal (do what you feel is good).
Reading your comments is a wonderland of right wing bias.
https://pmc.ncbi.nlm.nih.gov/articles/PMC9533286/
RLHF is "ask a human to score lots of LLM answers". So the claim is that the AI companies are hiring cheap (~poor) people from convenient locations (CA, since that's where the rest of the company is).
If you adjust your mindset slightly when searching online, it's not hard to find communities of people looking for quick side work and this was huge during the covid lockdown era. There were people helping train LLMs for all kinds of purposes from education to customer service. Those startups quickly cashed out a few years ago and sold to the big players we have now.
I don't get why this is hard for people to believe (or remember)?
This sounds like something Elon would say to make Grok seem "totally more amazeballs," except "anti-woke" Grok suffers from the same behavior