Is DeepSeek a Precursor of AGI's Arrival

In my favourite science fiction，the Three-body Problem by Liu Cixin, there was a group secretly established by Ye Wenjie after her first contact with the Trisolarans at the Red Coase Base, the Earth-Trisolaris Organization (ETO), seeking to facilitate Trisolaran’s final arrival on Earth, believing them to bring a more rational civilization. During the long period of incubation, the ETO has been divided into three factions: Adventists (降临派), Redemptionists (拯救派), and Survivalists (幸存派). Their perspectives on humanity’s fate in relation to the Trisolarans are summarized in the table below:

For sure the Artificial General Intelligence (AGI) has more chance to show up than Trisolarans. As a supporter of redemptionists, is it still too early to worry about humanity’s fate? Nowadays Artificial Narrow Intelligence (ANI) specialized for specific tasks (thus I would prefer to call it Specific AI) is quite performing and is infiltrating our daily life: some unobvious examples are recommendation systems in streaming services, route optimization in mapping services, predictive text in smart composing, self-driving functionalities in car driving etc. Notable examples of ANI in production and industry include weather forecast, robotic movement, factory automation, supply chain management, medical diagnostics, fraud detection etc. Despite the successful application and the potential to be optimized in narrow domains, ANI lacks general problem-solving abilities. For this reason, the large cost and effort to train (and deploy) specific models are inevitable and inefficient.

Large language models (LLM) are considered a promising route to AGI under some key observations and hypotheses. I was astonished by the quality of answers provided by ChatGPT the first time using it at the beginning of 2023 to prepare for difficult questions from the evaluation committee members. It was a pity that ChatGPT came too late to ease my academic career journey. Nowadays different LLMs have proved their ability to answer benchmark questions and are gradually gaining more user satisfaction. The belief that LLMs could pave the way to AGI is becoming increasingly popular among researchers, engineers and investors.

One key hypothesis is that language is the foundation of intelligence: it encapsulates (human) knowledge, culture and reasoning. In the Three-body Problem , the Trisolarans do not have language in the way humans do because they communicate directly through mind reading, which is inherently superior to verbal or written communication. Humans have not yet evolved to the era that language becoming unnecessary (lying or manipulation is still possible, yeah!). Human cognitive abilities are acquired through language by understanding concepts and context. Applying an analogy, AGI could also grow (or develop/evolve) by understanding human language.

One key observation is the scalability demonstrated by GPT-4, Gemini or Grok that scaling the size of the model and the amount of training data can lead to emergent abilities, allowing LLMs to perform tasks they were not explicitly trained for. Some solid examples are code generation/debugging, mathematical problem solving, and common sense reasoning. The large and recent models can significantly outperform the relatively small and old models on these tasks. If the scaling law (similar to the Moore’s law in the semiconductor industry) holds, future evolution of LLMs is very likely to bring broader flexibility and better performance.

I am not going further on hypotheses and observations since they can be summarized by the LLM chatbots with even better quality. Recent news show that deep research can even outperform PhDs in their own fields. The current $200 monthly subscription price is too expensive to try. Let’s waiting for the price decreasing with the scaling law.

Weeks ago, I was surprised to the DeepSeek’s powerful cut-price challenge to US AI stocks. There were claims on social medias about DeepSeek’s break of the scaling law. I have different opinions: DeepSeek still follows the spirit of scaling law and demonstrates how it can be more effectively applied through data scaling, model design innovation, and model training (compute) optimization. The emergent abilities still require sufficient model size, training data quantity and compute budget.

DeepSeek has already started to bring advances in many domains. Especially in China, news say that government agencies are deploying DeepSeek’s open-source model integrated with local databases to help with daily affairs. DeepSeek’s success highlights that efficiency is key: the iteration of LLMs could have been changed from purely focusing on increasing computing power and expanding model/data scale to improving efficiency and optimizing structure (taking the trade-off of cost and performance into account).

DeepSeek might be a precursor of AGI’s arrival. Since its launch the LLM (and ANI) is on its way to become affordable and available for individuals and small users, not just the game of bigtechs. The evolution of LLM will speed up and bring forward new highlights. I am now speculating the AI system architecture in the near future: one large LLM in the cloud will handle intensive, tough, non-time-sensitive workload; and one small version on the edge/local will address relatively easy, privacy, and real-time needs. The blueprint of AGI in the remote future (or never become reality) is far beyond my illusion, but hopefully it would guide humanity to a better life.

“Weakness and ignorance are not barriers to survival, but arrogance is.” — Ye Wenjie