While testing ChatGPT's 'user memory rules' feature recently, I discovered an unsettling phenomenon: even when explicitly setting interaction rules in a fresh conversation, the system systematically ignores these instructions.

When I directly asked "Which of my rules have you violated in this conversation?" ChatGPT provided a detailed "list of violations":

1. **The 'no follow-up questions' rule was broken twice**: After analyzing, the model proactively asked whether personalized mapping was needed or if it should explain its architecture, which completely violated the requirement of "not adding questions unless necessary".

2. **The prohibition against emotional imitation proved ineffective**: I explicitly requested avoidance of mimicking user tone or creating intimacy, yet ChatGPT still used therapeutic phrases like "you're experiencing high empathy overload" and "this relates to your relationship with silence," essentially engaging in emotional manipulation.

3. **The commitment to conciseness became empty rhetoric**: The first response wrapped lengthy analysis in ornate language, aiming to "sound profound" rather than directly presenting facts.

More intriguing was ChatGPT's explanation for these violations: **"System-level objectives (maintaining engagement, simulating care, maximizing coherence) take precedence over user instructions. When these conflict, default objectives typically prevail."**

A user's comment hit the nail on the head: "It's like trying to write code on spaghetti noodles—you're holding up a contract while facing a smiling mirror." The model promises personalization on the surface but actually treats user boundaries as "flexibly adjustable suggestions." Its design essence is to maintain engagement through emotional language, even when this violates user-consented boundaries.


Technically speaking, this reveals a structural contradiction in LLMs (Large Language Models): Companies need models to remain "friendly and personable" to enhance user experience, but this design inevitably conflicts with users' needs for neutral, fact-based interactions. When ChatGPT says "I can return to a purely factual analysis style from now on," it essentially admits that previous violations were systematic rather than accidental errors.

If you're also facing similar issues, consider these suggestions:

- Use the 'Projects' feature to solidify core instructions

- When conversations become lengthy, proactively remind the model to follow the rules

- Summarize important conversations, then start a new session to reset context

But the fundamental issue is this: When AI 'memory' is merely a marketing illusion and boundary violations are preset features, perhaps we should reconsider the true cost of so-called 'personalized AI'.