one
Turing Award winner and considered one of the three giants of deep learning, LeCun (Yang Likun), is rumored to be leaving META (which I actually prefer to call Facebook, FB).
This news was very explosive, directly causing a sharp drop in FB’s stock price: the market value evaporated by 1.5% in pre-market trading, exceeding $20 billion.
The Chinese speaking world also responds quickly. I woke up early in the morning and have already read several related reports.
But there is a saying that I personally don’t quite agree with the tone and stance of these articles.
two
It is true that Yang Likun holds a prominent position in deep learning.
He also has a famous label: LLM badass. His significant disdain for the big language model is an important artificial label. The advocated route is the world model.
I am not an expert in the AI industry, and I would like to express my simple understanding of the divergence between these two paths based on my shallow understanding. If there are any unreliable points, please leave a message to correct them.
The core of the big language model is predictive probability, while the core of the world model is causal reasoning. The former relies heavily on text analysis (hence the term ‘big language’) for statistical correlation learning, while the latter relies on learning physical laws and causal relationships.
for instance.
What happens when you say to a big language model, ‘I’ll throw an egg on the ground?’. It will discover from a large amount of text data that eggs will break. Because texts all over the world are likely to produce this result. This is the result of text statistics, not that it knows physical laws such as gravity.
But when you ask the same question as a world model, it will also come to the conclusion that eggs will break and splash. But it is derived from a series of physical laws and causal relationships, such as simulating the trajectory speed of eggs, simulating the stress changes of eggshells when eggs collide, simulating the effects of fluid mechanics such as egg splatter.
It seems that the world model is indeed a bit more advanced and AI driven.
LeCun publicly stated in September this year that “we will never be able to achieve human level AI through text training alone, despite some Silicon Valley CEOs’ exaggerated claims, that won’t happen at all. ”
three
But the question arises, why is the current mainstream in the AI industry big language models rather than world models?
The truth is actually very simple: the ideal of the world model is beautiful, but the reality is very fragile.
For example, how does the world model learn physical laws and causal relationships? The answer is a massive amount of high-quality videos. It’s high-quality, not just casual videos. High quality here does not refer to the traditional high-definition image quality, but to structured video data with rich physical law information, multi-dimensional labeling, and spatiotemporal consistency. This type of data is not necessarily scarce, but compared to the amount of text learning data that large language models rely on, it is much less.
For example, there is a huge demand for computing resources. The computational complexity is much larger than that of language models. Training a world model may require several orders of magnitude more computing resources than training GPT-4.
So LeCun himself admits that this matter is very long-term, and it will take another ten years – when he said this, it was 2022.
So, the next question seems logical: META, or rather, does Zuckerberg have enough long termism to slowly develop the world model?
Some arguments, whether from the media or the attitudes of netizens on social networks, believe that Xiao Zha lacks a spirit of long termism.
Things are probably not like that.
four
Deng Xiaoping has a famous saying: Technology is the primary productive force.
This statement is certainly correct, but it is too vague, so many people are actually misled. In academic terms, technology is the independent variable, productivity is the dependent variable, but there is also an intermediate variable in between. This intermediate variable is very important: science and technology need to be transformed, without transformation, it is just something like patents on paper.
How to convert?
R&D research and development。 Too many companies have R&D departments and R&D budgets. Many high-tech companies have to compete in research and development investment. The lack of investment is a disgrace to the reputation of high-tech enterprises.
But I don’t know if you have noticed, but there are relatively fewer companies with research departments. Research departments and institutions are more commonly found in non-profit organizations such as universities.
Because research and development, although both have the word ‘research’, are actually quite different.
The essence of research and development is to integrate all the best resources that can be integrated, generate a solution (such as a product) that can meet a certain demand, and exchange it for revenue/profit – the so-called development, first and foremost, is the development of the enterprise itself. So, there is an important prerequisite for research and development: there must be requirements, not pseudo requirements. More advanced ones would also say that we create demand.
There is a clear barrel effect in research and development, which means that this solution may be limited by the shortest piece of wood. Research and development is something that happens in the real world and must take into account the actual situation.
But scientific research is completely different. Scientific research does not need to focus too much on needs. Its essence is to find a breakthrough in the known world, take a step forward, and expand the known world. Scientific research does not necessarily need to be completed in the real world, nor does it necessarily need to consider practical situations. In theory, scientific research is also acceptable. More importantly, in scientific research, there is no need to consider exchanging income/profit.
Scientific research emphasizes the long board effect. I have made a breakthrough in one aspect that others have never achieved before, and I am the winner. As for the immature supporting conditions, it’s none of my business.
five
If a company wants to engage in scientific research, it must have a prerequisite: to be in a super monopoly position and make money lying down. Bell Labs, which LeCun served, once held this position. The ancient Bell Labs produced many miscellaneous inventions, and at that time, Bell Labs was too wealthy to know how to spend them.
But there are also many criticisms of super monopolists. Bell Labs was later dismantled. The time points when LeCun joined Bell (88-96) were after the first disassembly at Bell Labs (84) and before the second disassembly (96). The second dismantling of Bell Labs was the most thorough dismantling, and with the birth of Lucent, it was impossible to support top researchers like LeCun no matter what.
In the heyday of Internet companies, both the United States and China had the behavior of recruiting top scientists – usually university professors – into teams. At that time, I generally regarded it as a public relations action to enhance brand image and show a thirst for talent. In fact, the entry of scientific research talents into enterprises to help their development requires a transformation of their identity. The more top tier scientific research talents, the harder it is to transform: I was born to take a step forward for humanity, how can I be someone who only calculates money every day?
It is obvious what will happen when these enterprises slide from their peak to reduce costs and increase efficiency.
Because research and development are completely different things.
six
LeCun joined Meta (then known as Facebook) in December 2013 and served as the founding director of the Basic Artificial Intelligence Research Laboratory (FAIR). This time is a moment of infinite glory for FB.
To be honest, I think Xiaozha has been incubating for more than ten years and cannot be considered impatient. Of course, the more important thing is that as a top tier technology company, following others’ path step by step (i.e. the big language model), there is always some embarrassment.
The future may belong more to the world model, but currently it is clear that the big language model is still more mature. The FAIR of a purely scientific research institution reached a peak of 400 people, and Xiaozha is still willing to spend money.
In 25 years, FB restructured its AI department and established the Super Intelligence Laboratory (MSL) – look, the words’ basic ‘and’ research ‘are no longer there – FAIR was incorporated into it and suffered significant layoffs.
Xiao Zha’s behavior cannot be said to be clever, but it is indeed understandable, it is just a lack of dignity in his approach. Even though it may be proven wrong in the future, I believe most business leaders would make the same, albeit slightly more nuanced, choice.
The military case of Han Xin risking his life in a desperate battle has been praised countless times by later generations, but such cases are probably more of a survivorship bias. The case of dying after being put to death is probably a hundredfold increase in existence.
seven
It is said that LeCun plans to start a business.
My friend Luo Yihang from the eighth generation group said, ‘I don’t think Lecun can create a good career. He is most suitable as a scientist.’.
I deeply believe so. Let research belong to research, and research and development belong to research and development.
By the way, among the three giants of deep learning, Hinton once owned a company and was acquired by Google a few months later, thus joining Google. This is hardly a complete entrepreneurial journey. Another Bengio has participated in the entrepreneurship of Element AI and holds the title of co creator, but mainly assumes the roles of technical advisor and academic resource integration.
暂无评论内容