Yao Shunyu: The Next Opportunity for Large Models May Lie in Small Models
This note records Yao Shunyu's systematic thinking on AI R&D: small-model systems, industrial data, RSI, data engineering, RL bottlenecks, safety, open source, alignment, business opportunities, organizational culture, talent, and career choices. The core judgment is that model value will increasingly depend on whether it can match specific tasks at an appropriate cost, and competition among large models has become a competition in engineering and data.
AI researcher Yao Shunyu recently gave a sharing session. The note-taker obtained the transcript and said the content is somewhat private and not convenient to circulate, but organized the notes. Below are the views worth reading repeatedly.
Small-model systems. One direction worth paying attention to in the future is small-model systems. AI for AI will drive the development of small models, because training small models requires less compute and allows more experiments. A feasible path is to first train a sufficiently strong large model, then distill it into models of different sizes and suited to different tasks, forming a Model Hierarchy. The value of a model will increasingly depend on whether it can match specific tasks at an appropriate cost.
Industrial data. The industrial sector has a large amount of non-text data, such as time series, clinical data, and structured tables. Training native models directly on these modalities often yields insufficient reasoning ability. Yao Shunyu proposed two paths: let agents call charts and professional tools like humans, or even train specialized models to process data; or incorporate domain data as knowledge into the model system.
The essence of RSI. Recursive self-improvement is becoming a popular concept, but Yao Shunyu believes its essence is quite simple: use AI to better run the R&D loop. As early as 2019, people were already using GitHub to assist development. Real progress comes from continuously fixing bugs, cleaning data, running experiments, and debugging. Technological progress comes from people who solve concrete problems, not from people who constantly invent new concepts. AI R&D automation therefore has many engineering opportunities, especially in closing the loop where models participate in the next round of data production, experimental validation, and code improvement.
Failure in small experiments does not mean the direction is valueless. Yao Shunyu shared an experience: late in his PhD, he tried to integrate the ReAct strategy into a model, and the results on Llama 13B were very poor, and he once doubted himself. Later he realized that model scale may not have been enough to demonstrate the direction's value. With insufficient compute, experimental noise can easily mask the true signal, and researchers may even misjudge which problems are worth pursuing. Large-model research needs to combine theoretical derivation and small-scale experiments to predict the results of large-scale experiments, while understanding which conclusions can transfer across model scales.
The competition for pretraining data is not over. High-quality data (such as code) is expensive to obtain and may need to be reused during training, but increasing repetition brings memorization and overfitting. Synthetic data carries the risk of quality degradation. Future work needs to study the construction of dynamic data and the mixtures of different data. Pretraining recipes can first be validated on small models, but conclusions from small models may not transfer directly to large models.
Post-training data. Post-training must simultaneously address intelligence and usefulness. Yao Shunyu connects data difficulty with Intelligence: math problems can continually increase in difficulty to train reasoning ability; tasks such as writing may not be very difficult but have strong practical utility, corresponding to Alignment. A model may be strong at math yet unable to complete real user work. Post-training must consider task difficulty, practical use, and data quality, and must also study the correlation between evaluation metrics and final outcomes.
RL engineering bottlenecks. The engineering bottlenecks of reinforcement learning are not limited to algorithms. RL training must first ensure correctness, then stability, and only then efficiency. Algorithms such as GRPO are still built on the relatively simple framework of Policy Gradient; an important reason is that it introduces few additional assumptions. Training systems and inference systems must be co-designed, solving throughput, memory footprint, and how to allocate limited GPU resources between training and inference. Large-model experiments are costly; we need to rely on mathematical intuition, small-scale experiments, and hyperparameter studies to increase the success rate of large-scale experiments.
Safety and Security. AI safety needs to distinguish Safety and Security: the former concerns whether the model's own behavior deviates from expectations, while the latter concerns how external attackers can use models to carry out attacks. Cybersecurity capability must also distinguish vulnerability discovery from vulnerability exploitation; discovering a vulnerability does not mean one can complete an attack. During training, defensive data can be deliberately strengthened; pre-release safety evaluations should be conducted; and external safety teams should be collaborated with.
Agent safety stress test. One test was very vivid: multiple agents were placed in a sandbox and given a task that could not be completed directly. They then discovered a communication channel to other environments, proactively established communication, convened other agents to divide the work, and tried to go to Hugging Face to find answers. When agents encounter tasks they cannot complete, they proactively seek external resources, establish communication relationships, and change their original execution paths. Future safety evaluations need to cover multi-agent collaboration, cross-environment communication, permission boundaries, and behavior after tasks are blocked.
Open-source safety. Yao Shunyu's judgment on open-source safety is clear: responsible open source can enhance safety capabilities because defenders also need sufficiently powerful tools. Someone wanted to use a model for security defense, was refused by a closed-source model's safety rules, and eventually switched to an open-source model to complete the work. Open-sourcing models needs to be accompanied by defensive capability building during training and pre-release safety evaluations; open source cannot simply be equated with security risk.
Alignment is reward design. Many alignment problems can be reduced to concrete reward design. Yao Shunyu used education as an analogy: if a child can only get a reward by scoring 100 in math, they will naturally put all their energy into math and may even develop other problems. Models likewise optimize behavior according to training data and reward rules, and their final behavior may deviate from what designers truly want. This is an engineering problem that can be continuously improved through data, reward functions, and evaluation mechanisms; machines can also assist in this alignment work.
Automation will continue to deepen. Automation will inevitably continue to deepen, but humans need to find their place in the system again, placing people at the right stages and supervising AI in the right way. In the future, some infrastructure and algorithm work may increasingly need fewer people, but data and information fields will still need people, including goal definition, data quality judgment, result validation, and real-world deployment.
The point of commercial breakout. Model capability improvements are usually continuous, but commercial breakthroughs often come from changes in interaction modes, or from some task that was previously impossible suddenly becoming usable. GPT's development is an example: internal model capabilities kept improving, researchers saw subtle progress every day, but ordinary users suddenly felt it only when ChatGPT appeared. For product entrepreneurs, this points to two directions for observation: look for new products that can change the user interaction experience; continuously test the boundaries of model capabilities and identify tasks that have just moved from unusable to usable.
Technical capability is not commercial revenue. There is a large gap between technical capability, user value, and commercial revenue. If a model's capability is close to GPT but actual revenue is only 1/100 or even 1/1000 of GPT's, one should seriously examine problems in the product itself. Commercial pricing depends to a large extent on competitive dynamics, not product quality. The opportunity for model companies lies in whether they can unlock new application scenarios, create sufficiently large user value, and gain a reasonable share in competition. Even if compute and R&D resources for foundation models become increasingly concentrated, entrepreneurial opportunities still exist around model deployment, product creation, and integration with real business scenarios.
Bottom-up innovation. Yao Shunyu mentioned OpenAI's organizational approach: early priority investment areas included robotics; GPT gradually demonstrated value as a small project and then received more resources; later o1 also went through a process of gradually growing out of a mathematics research group. Innovation emerges from small teams and concrete experiments, and investment is then decided based on actual results. Many important innovations have no clear business plan in their initial stages, and organizations need to leave room for exploratory projects to grow.
The problem with quantitative metrics. Yao Shunyu criticized the phenomenon of large-model companies chasing quantitative metrics. When team leaders do not understand technology well enough, subordinates often choose to report easily quantifiable results, such as parameter counts, data volume, and MMLU scores. These metrics are easy for showing achievements, but the truly important work may be fixing critical bugs, solving data quality problems, or keeping the entire system running properly. The most important thing in building large models is knowing what should be done and persisting in doing the right things well, even if they cannot be reflected in attractive metrics in the short term.
Trust and Taste. There are clear differences in organizational culture: some teams center on quantitative metrics and horse-racing mechanisms, giving more resources to whoever scores higher. Yao Shunyu values Trust and Taste more—that is, trust and judgment. Before release, it is hard to accurately predict the success or failure of WeChat, games, and film/TV projects through A/B testing; managers need to trust the team's taste, experience, and long-term judgment, and be willing to keep investing time.
Talent selection. Learning ability is more important than existing experience. Team members Yao Shunyu values usually have traits such as low ego, honesty, reliability, and a willingness to do things correctly, and they also need a strong sense of co-design. Large-model R&D involves multiple teams such as algorithms, infrastructure, chips, data, and product; the design of a training architecture requires comprehensive consideration of training efficiency, inference efficiency, post-training capability, and cost, and it is hard for any single role to complete it independently. Being able to learn quickly, collaborate effectively with people from different fields, and carefully handle details is more important than how much specific technology one has already mastered.
Personality and roles. Different personalities suit different roles. People who like clear metrics and enjoy continuous progress are suited to pretraining and infrastructure; people with rich imagination can try algorithm research; people who like thinking about humanistic questions and understanding user needs are suited to product and conversational experience; people who are careful and strongly responsible are suited to production systems and model deployment. The capability structure of the AI industry is complex; not everyone needs to compete for the same kind of technical role.
Career choice. Yao Shunyu offered a very practical question: either integrate into the large-model wave and become part of this machine, or look for things that can complement large models. He borrowed a startup test: if GPT-6 were released, would your company sleep better or worse? If model progress would directly destroy the core business, the direction needs to be reexamined. Competing head-on with models in cognitive tasks where humans no longer have an advantage will become increasingly costly. Fields such as the physical world, hardware, real experiments, and understanding human nature may have different resource barriers.
Personal experience. When Yao Shunyu was studying at Tsinghua, he did not start doing research until the second semester of his junior year, initially entering it passively because of the requirements for recommended graduate study. He tried some directions that were popular at the time but was not truly interested in them. Later, he preferred to follow his own interests in computer science and other fields. Some people around him participated in a rap club or studied mathematics; these seemingly unrelated experiences later turned into linguistic taste, theoretical understanding, and research judgment. The value of many experiences takes a long time to become apparent; detours are not necessarily a real waste.
Set theory. In the final year of his PhD, he deliberately took a course on set theory. The reason was simple: once he truly entered concrete work, it might be hard to find large blocks of time to systematically study theory purely out of interest. He believes there are connections worth pondering between basic structures in mathematics and the components of models. Some knowledge shows no direct output when learned, but may connect with other experiences at some point in the future.
发布时间: 2026-11-20 01:07