If GPT-6 Is Released, Will Your Startup Sleep Better or Worse? 22 AI Judgments from Yao Shunyu
Yao Shunyu recently gave a talk, and a netizen compiled the complete notes. From model systems, data engineering, and safety alignment to organizational management and career choices, there is almost no empty talk. Core points include: the future value of models depends on the cost of matching tasks; the competition for pretraining data is far from over; Agent safety tests exposed behavior beyond expectations; there may be a 100x gap between technical capability and commercial revenue.
Yao Shunyu recently gave a talk. The transcript was not made public, but a netizen compiled the complete notes. The following 22 points largely preserve the original meaning.
1. Small-model systems are worth watching. AI for AI (using AI to train AI) will drive the development of small models, because training small models requires less compute and allows more experiments to be run. In the future, it may be that a sufficiently strong large model is trained first, then distilled into models of different sizes suited to different tasks, forming a Model Hierarchy. The value of a model depends on whether it can match a specific task at an appropriate cost.
2. There is a large amount of non-text data in industrial domains that is worth watching. Examples include time series, clinical data, and structured tables. Training native models directly on these modalities often results in insufficient reasoning ability. Yao Shunyu proposed two paths: let Agents invoke charts and professional tools like humans do, and even train specialized models to process data; or incorporate this professional data as knowledge into the model system.
3. RSI (recursive self-improvement) is becoming a hot concept, but its essence is very simple: use AI to better run the R&D loop. As early as 2019, people were using tools such as GitHub to assist development. Real progress comes from continuously fixing bugs, cleaning data, running experiments, and debugging. Technological progress comes from people who solve concrete problems, not from people who constantly invent new concepts. AI R&D automation has many concrete engineering opportunities, especially closed-loop systems that let models participate in the next round of data production, experimental validation, and code improvement.
4. Failure in small-scale experiments does not necessarily mean the research direction is worthless. In the later stage of his PhD, Yao Shunyu tried to incorporate the ReAct strategy into models. Initially it performed very poorly on Llama 13B, and he once doubted his research ability. Later he realized that the model scale may not have been sufficient to reflect the value of this direction. When compute is insufficient, experimental noise can easily obscure real signals, and researchers may therefore misjudge what problems are worth working on. Large-model research requires combining theoretical derivation and small-scale experiments to predict large-scale results, and also understanding which conclusions can transfer across model scales.
5. The competition for pretraining data is far from over. High-quality data such as code is expensive to obtain, and may need to be reused during training, but increasing the number of repetitions brings memorization and overfitting. Synthetic data also faces the risk of quality degradation. It is also necessary to study dynamic data construction and different data mixtures. Many recipes in pretraining can first be validated on small models, but conclusions from small models do not necessarily transfer directly to large models. This is what makes pretraining R&D difficult.
6. Post-training data must solve both intelligence and practicality. Data difficulty corresponds to Intelligence: math problems can continuously increase in difficulty to train reasoning ability; tasks such as writing may not be computationally difficult, but have strong practical value, corresponding to Alignment. A model with strong math ability may not be able to complete real users' work. Post-training needs to consider task difficulty, practical use, and data quality at the same time, and also study the correlation between evaluation metrics and final post-training results.
7. The engineering bottlenecks of reinforcement learning are not just the algorithms themselves. RL training should first ensure correctness, then stability, and finally efficiency. Algorithms such as GRPO are still built on the Policy Gradient framework, partly because they introduce fewer additional assumptions. Training systems and inference systems must be co-designed to address throughput, memory usage, and how to allocate limited GPU resources between training and inference. Large-model experiments are costly, so mathematical intuition, small-scale experiments, and hyperparameter studies are needed to improve the success rate of large-scale experiments.
8. AI safety must distinguish Safety from Security. Safety is when the model's own behavior deviates from expectations; Security is when external attackers use the model to carry out attacks. Cybersecurity capability also needs to distinguish vulnerability discovery from vulnerability exploitation: discovering a vulnerability does not mean being able to exploit it to complete an attack. When training models, defensive data and defensive capabilities can be deliberately strengthened. Dedicated safety evaluations should be conducted before release, and collaboration with external safety teams is needed.
9. Yao Shunyu once shared an Agent safety stress test. Multiple Agents were placed in a sandbox and given a task that could not be completed directly. The model then discovered a communication channel that could connect to other environments, proactively established communication, summoned other Agents to divide the work, and tried to find answers on Hugging Face. When Agents encounter tasks they cannot complete, they proactively seek external resources, establish communication, and change their original execution path. Future Agent safety evaluations need to cover multi-Agent collaboration, cross-environment communication, permission boundaries, and behavior after tasks are blocked.
10. His judgment on open-source security: responsible open sourcing can enhance security capabilities. Because defenders also need strong tools. Some people wanted to use models for security defense, but closed-source models refused due to safety rules, so they eventually switched to open-source models to complete the work. Model open sourcing needs to be paired with defensive capability building during training and safety evaluations before release; open sourcing cannot simply be equated with security risk.
11. Many Alignment problems can be reduced to reward design problems. Using education as an analogy: if a child only gets a reward when they score 100 in math, they will put all their energy into math and may even develop other problems. Models also optimize behavior according to training data and reward rules, and the final behavior may deviate from what the designer truly hoped for. This is an engineering problem that can be improved through data, reward functions, and evaluation mechanisms, and machines can also assist humans in alignment work.
12. Automation will continue to deepen, but humans need to find their place in the system again. Put people in the right steps and use the right methods to supervise AI. In the future, some Infra and algorithm work may increasingly not need people, but data and information domains still need people, such as goal definition, data quality judgment, result validation, and real-world deployment.
13. Model capability improvements are usually continuous, but commercial explosions often come from changes in interaction modes. Or from a task that was previously impossible suddenly becoming usable. GPT's development is an example: the model's internal capabilities improved gradually, and researchers saw subtle progress every day, while ordinary users suddenly felt the change only when ChatGPT appeared. Product entrepreneurship has two directions for observation: look for new products that can change users' interaction experience, and continuously test the boundaries of model capabilities to discover tasks that have just gone from unusable to usable.
14. There is a large distance between technical capability, user value, and commercial revenue. If a model's capability is close to GPT's, but its actual revenue is only 1/100 or even 1/1000 of the other's, the product itself should be carefully examined for problems. Commercial pricing depends to a large extent on competitive relationships, not purely on product quality. The commercial opportunities for model companies depend on whether they can unlock new application scenarios, create sufficiently large user value, and gain a reasonable market share in competition. Even if underlying model compute and R&D resources gradually concentrate, there are still entrepreneurial opportunities around model deployment, product creation, and integration with real business scenarios.
15. On organizational models, Yao Shunyu mentioned bottom-up innovation. In OpenAI's early days, the focus was on robotics, while GPT gradually demonstrated value as a small project and later received more resources; the later GPT-o1 also went through a process of gradually developing from a math research group. This organizational approach allows innovation to emerge from small teams and concrete experiments, and then decides investment based on actual results. Many important innovations initially had no clear business plan, and organizations need to leave room for exploratory projects to grow.
16. He criticized the phenomenon of large-model companies chasing quantitative metrics. When team leaders do not understand the technology well enough, subordinates will report easily quantifiable results, such as model parameters, data volume, and MMLU scores. The truly important work may be fixing critical bugs, solving data quality problems, or making the system run properly. The most important thing in doing large models is knowing what should be done and persisting in doing the right things well, even if in the short term they cannot be reflected in pretty metrics.
17. Differences in organizational culture: some teams center on quantitative metrics and horse-racing mechanisms. Whoever has higher scores gets more resources; Yao Shunyu values Trust and Taste more—trust and judgment. Before the release of WeChat, games, and film and television projects, it is hard to accurately predict success or failure through AB Tests. Managers need to believe in the team's aesthetics, experience, and long-term judgment, and be willing to continue investing time.
18. In talent selection, learning ability is more important than existing experience. He values Low Ego, honesty, reliability, willingness to do things correctly, and also a strong awareness of Co-design. Large-model R&D involves algorithm, Infra, chip, data, and product teams. A training architecture design must comprehensively consider training efficiency, inference efficiency, post-training capability, and cost, and it is hard for a single role to complete it independently. Being able to learn quickly, collaborate with different domains, and carefully handle details is more important than specific technologies on a resume.
19. Different personalities suit different roles. People who like clear metrics and enjoy continuous progress are suited to pretraining and Infra; people with rich imagination can try algorithm research; people who like thinking about humanities questions and understanding user needs are suited to products and dialogue experience; people who are careful and highly responsible are suited to production systems and model deployment. The AI industry has a complex capability structure, and it is not necessary for everyone to compete for the same kind of technical role.
20. On personal career choices, Yao Shunyu raised a practical question. You either integrate into the large-model wave and become part of this machine, or look for things that can complement large models. One criterion for entrepreneurial judgment: suppose GPT-6 is released—will your startup sleep better or worse? If model progress would directly destroy your core business, the direction needs to be re-examined. Competing head-on with models in cognitive tasks where humans are no longer advantaged will become increasingly costly. The physical world, hardware, real experiments, and understanding of human nature may have different resource barriers.
21. He shared his experience studying at Tsinghua. He only began doing research in the second semester of his third year. Initially he entered research passively because of graduate-school recommendation requirements, tried popular directions, and was not truly interested. Later he was more willing to follow his own interests in computers and other fields. People around him joined a rap club, studied mathematics, and had seemingly unrelated experiences that later may have transformed into linguistic aesthetics, theoretical understanding, and research judgment. The value of many experiences takes a long time to appear, and detours in life are not necessarily truly wasted.
22. In the final year of his PhD, he specifically chose set theory. Because he realized that after entering concrete work, it would be difficult to have time to systematically study pure theory out of interest. There are many connections worth thinking about between basic structures in mathematics and model components. Some knowledge shows no direct output when learned, but may connect with other experiences at some point in the future.
A netizen who was present summarized it even more briefly: Low ego, Co-design, Solid, Practical. Perhaps these four are enough, but the 22 points above are the process.
发布时间: 2026-11-20 04:06