Applying for post-training roles, but everyone who contacted me was from a data company
Jinyan Su recorded the data business she saw while looking for post-training roles. Synthetic data is cheap to produce, quality is hard to define, and connections are the real moat. Model labs' budgets, compute anxiety, and gray areas sustain a business with no moat.
Jinyan Su published a long post on X recording the data business she saw this year while looking for post-training roles. She was applying in the post-training direction, and almost all the startups that contacted her were doing data. Even now, she still receives DMs from headhunters working on RL environments, data startup founders, and people who want to switch into this direction. There is more and more noise on the internet, time and attention are becoming more expensive, and yet the cost of sending someone a DM is very low. For people one does not know well, refusing is often something one is forced to do.
Post-training in academia can be small-scale: tweak algorithms for one's own amusement, without worrying about how general the resulting algorithm is or how large a model it can scale to. Industry is different. Existing LLM training recipes, especially the post-training part, are almost mature, and algorithmic innovation is becoming rarer. Open-source RL training frameworks and internal model-training frameworks at companies are mature and stable, and model improvements almost all come from data.
People doing pretrain clean data, study architecture, scaling, and data mixing ratios; people doing post-train mainly deal with various RL environments; in between there is mid-train, training on high-quality data to strengthen the model's domain-specific capabilities. Most people doing post-training in industry are also handling data. If you want the model to do various tasks, you have to train it on relevant data. The more task types there are, the more diverse the data needs to be.
It is not known to outsiders what data frontier labs collect separately. Some estimate that RL environments are used for RL training, and high-quality trajectories produced by models are used for off-policy cold start, that is, SFT, followed by on-policy distillation or RL. The types of data provided by data vendors are very different. Many very small startups operating under the radar are almost all synthetic data, with people taking a brief look afterward. Startups with recognizable names in the field, such as Scale AI and Mercor, still find real experts to label data.
Creating synthetic data costs almost nothing, depending on the task. Tasks that require GPU cost more; browser and computer use only require token costs. Most small data workshops directly use personal accounts, so the cost is almost negligible. Costs are low and there are no technical barriers, so everyone wants to try.
At a data event in Shenzhen in October, a recent PhD graduate pitched passionately. Jinyan Su described his expression as overly vivid: all the features on his face, especially his eyebrows, were dancing. She asked him: with so many data companies now, what is your advantage? He said: the company is just me, there are no extra expenses, and I'll do it like this for half a year and see what happens.
She has asked many people this question. Most answers are: we have better technology, we've accumulated a lot of know-how, and frontier labs that used our data all said it was better than other data vendors. It seems every data startup has its own know-how, its own unique advantage, and is recognized by clients. It's like how every paper claims to have novelty and be state of the art. When first entering the field, one can still look at the person saying this with a face full of admiration. After asking many companies, she still hasn't heard a refreshing viewpoint. Some startups also say they maintain a strong network of vertical-domain experts. What counts as an expert is unknown.
These answers are talking points put on the table to fool investors. The only thing that can become a data startup's unique advantage is connections. Knowing people at frontier labs, or having worked at a frontier lab oneself, knowing what they want, and being able to sell data through connections. In China, this probably also includes dining culture, drinking culture, and gift-giving culture.
Quality matters, but it is very hard to make it a moat. There are enough startups that can bring quality to a certain baseline, and from a quality perspective it is hard for everyone to differentiate. The word quality itself is also hard to define. When model labs buy data, they almost always inspect quality with their own eyes, or have data vendors train it on a small model. There have also been cases where the same data worked well after training on one company's model but poorly on another's. Often, the details that were strictly controlled and took a lot of time may have no impact at all on the final result.
There are more ways to sell data than one would imagine. The first is crowdsourcing task contributions. In academia, contributing tasks and building benchmarks through crowdsourcing is reasonable because resources are insufficient. Some PhDs from academia have carried crowdsourcing forward in startups. Academic participants want a paper authorship position and don't care much about monetary rewards. The work done by current PhD students has less and less influence, and everyone wants a chance to participate in projects that look more influential. Some Chinese students are more inclined to participate because publication helps with later green card applications.
For example, a certain startup imitates terminal bench to crowdsource tasks and recruits participants on Xiaohongshu. Recently there have been posts on Xiaohongshu criticizing such startups. Commenters' attitudes are mixed; the main criticisms are: compared with terminal bench, the tasks and process are unclear; some people spent time but in the end were not included as paper authors; promised authorship was actually a promise to submit the paper to ICLR and Nature, which is essentially a blank check because the paper may not be accepted; and the benchmark is currently not open source. Academic crowdsourced research at least ends up open source, and everyone volunteers for the love of it. Once it is used for a founder's personal business, contributors should have the right to be informed and receive a reasonable explanation and compensation. This passage is Jinyan Su's paraphrase of the content of Xiaohongshu posts, not her personal opinion.
The second is to first make a public benchmark and then sell data. If there is no demand, create demand; if people don't feel anxious, create anxiety. When making a benchmark, you want the model's performance to be low; otherwise who would buy data? From an academic perspective, if a model performs too well on a benchmark, it means the research question is not important, so you also need to make performance low. Both academia and startups have enough motivation to make model performance worse. A bunch of unreasonable benchmarks have appeared: verifiers are too strict, and a model may solve the task correctly in another way but receive a low final score; or the task itself is problematic, with many ambiguous things not told to the model, and multiple possibilities are possible while the model is expected to follow one particular one.
The third is to build agent applications. Some companies, when they were about to go bankrupt, discovered that someone was willing to buy data at a high price, so they transformed on the spot into data vendors. Some operate well, do their main business while also generating some synthetic data from user data to sell. The fourth is free token sites. They directly give away tokens to use, but the data is taken. When something appears free, you should think clearly about what you are paying without realizing it. The fifth is relay/proxy sites. They collect money for tokens from users on one side and collect money for data from downstream on the other. Users pay to buy tokens, and it turns around and packages the call data to sell, making money on both ends. She heard that a certain model company went to buy data and found it had bought its own data, because a relay station they were using had sold the data.
There are also middlemen. People with connections can contact buyers directly. People without connections rely on a do-or-die, fearless, hard-working entrepreneurial spirit to reach out themselves, ask for introductions, and gradually contact buyers. There are also people who don't understand the industry, have no connections, and have no sales ability, but have a batch of data on hand; they heard from hearsay that data can be sold for money and is very profitable, but they don't know how to sell it. Finding a middleman willing to buy at a low price is relatively easy. Give the data to the middleman, and the middleman sells it to model labs, earning a price difference.
Why does this business seem to have no moat, yet still flourish and everyone still makes money? Jinyan Su's initial judgment was expert data: data that requires human labeling will definitely be outsourced. But among it there is also a lot of synthetic data. Many small data vendors have no relevant domain expertise and also make synthetic data. Models are getting stronger and stronger, and less and less data can be called expert data; data manually labeled by human experts may not even be as good as what the model generates itself. Apart from data vendors that truly possess top experts, do other data vendors still have value? The Mistral Large 4 technical report says that the environments, tasks, reference answers, and verifiers used in post-training were mainly built and produced by agents themselves, and it did not mention externally purchased data or human annotation.
There is no threshold for generating synthetic data. Previously it might have required a sufficiently smart researcher; now with LLMs, even someone who doesn't understand research and has no industry experience can learn how to do it with the help of an LLM. Model companies with insufficient funds do it themselves. Model improvement depends on compute and data. Model companies with sufficient funds allocate enough money to both compute and data. Since they have allocated it, why not use it all? Compared with not buying data, buying data outside will generally not make the results worse. Another consideration is that even if they want to buy more compute, it may not be so easy to buy. Data is easier to buy, so they might as well buy more data. There are also legal issues. As a gray industry, data contains many illegal aspects. Model companies are unwilling to handle it themselves, while data vendors can operate on the edge of the law as cutouts.
Under huge profits, some researchers who look glamorous on the outside are also willing to do unscrupulous things. For example, after building agent applications became unsustainable, they started selling data; two cofounders parted ways. Before the split, founder A copied a copy of the data for himself. A few months later, he logged onto the server maintained by founder B again, wanting to steal more data, and was discovered by founder B. In the end, data ownership was defined through legal means. There are also founders who hint that employees of companies or data vendors steal data from the inside. As an observer trying hard to stay out of it, Jinyan Su has also seen various outrageous behaviors by people around her after facing temptation and interests. She has seen outrageous behavior from more senior people in academia, such as professors, but rarely outrageous behavior among peers. These things, including the 'Tian Keyu poisoning' incident that she used as an example in the article, made her realize that she is already senior enough, because her peers are already old enough to do all kinds of outrageous things.
Experts who label data, people who saw the trend and started data businesses—it is like when a building is about to collapse: everyone does not want the building to collapse, but everyone is accelerating its collapse. If you don't sell data, someone else will. If you don't help label data, some other expert will. If you don't earn the money, someone else will. Why not earn it yourself? It's just like everyone is building stronger models and no one can stop. If one model company stops and the others don't, it falls behind. As an expert labeling data, as a data vendor chasing benchmarks to sell data, and choosing whether to obtain data by stealing—all of these are optional. Then when a certain company forces employees to record their screens and sends employees off to label data to improve models, how much of that is their own choice?
In the replies to the tweet, Kevin Wang said that many data companies are not selling data, but selling the anxiety of model companies about lacking data. Wei Jia said that applying for post-training roles but having all the contacts come from data companies says more than the title. Jie said they had worked at a data vendor, and there were also various outsourcing distributions and reselling to each other. Zijian Hu said there is another type of data company that acquires data from failed startups and resells it. Jinyan Su mentioned that Google and Mercor previously competed to acquire Spirit Airlines. WQ said, data moats are just vibes and ndas. The prosperity of the data business looks like a technology business, but is actually a combination of connections, budgets, anxiety, and gray areas. No moat does not mean no business.
Original link: https://x.com/i/article/2121808409062416385
English version link: https://jinyansu.substack.com
发布时间: 2026-11-15 18:28