Jensen Huang Revisits AI Risks: Many Predictions Fabricated, China to Become Key Open-Source Force

By Helen Li

Edited by Xu Qingyang

On September 14, U.S. local time, at the All-In Summit 2026, an in-person event hosted by the renowned tech podcast All-In Podcast, NVIDIA CEO Jensen Huang discussed with the four hosts topics including AI safety, AI doomsday theories, recursive self-improvement, open-source models, and AGI.

The four hosts are also co-founders of the All-In Podcast: angel investor Jason Calacanis, former Facebook executive Chamath Palihapitiya, prominent "PayPal Mafia" member David Sacks, and CEO of agricultural biotech company Ohalo Genetics and investment firm The Production Board, David Friedberg.

The interview explored a core question: what safety boundaries are needed for AI development. Anthropic CEO Dario Amodei recently discussed AI safety, and some leading labs have called for slowing down. Huang believes safety and innovation are not contradictory. He stated that safety incidents that have occurred should be analyzed as engineering problems and controlled through technology, processes, and testing.

Huang addressed the frequent "doomsday predictions" about AI in recent years. He cited examples that have been proven inaccurate, such as "radiologists will be replaced by AI," "90% of code will be generated by AI in the short term," and "50% of entry-level jobs will disappear soon." He believes such predictions lack sufficient scientific basis and should not be used to create social panic.

When discussing recursive self-improvement (RSI), Huang said RSI is not a mysterious technology that emerged suddenly, but a combination of existing technologies such as context, reinforcement learning, synthetic data, and LoRA.

As for the debate between open-source and closed-source models, he clearly stated that both are indispensable. He also believes that China, with its large number of engineers and talent in science and mathematics, may become an important contributor to the global open-source ecosystem in the future.

The conversation then turned to "AGI and superintelligence." When asked whether the standard for AGI is reaching human-level intelligence and whether that stage has arrived, Huang briefly replied, "We have reached it."

The following is an edited transcript of Huang's latest interview, lightly edited for clarity:

01 AI Safety Cannot Be Opposed to Innovation

Q: Recently there has been increasing discussion about AI safety. Dario Amodei published an article on AI safety, and former Anthropic researcher Jacob Cockson also left due to related issues. What do you think about these developments?

Huang: Safety issues must be taken seriously, but safety and staying ahead are not in conflict. We can innovate quickly, execute rapidly, and keep the U.S. ahead while ensuring safety. Viewing these two as mutually exclusive is a mistake.

Cockson showed courage by speaking publicly about these issues. As a whistleblower, he has the right to do so, but I think his subsequent predictions about the future lack sufficient scientific basis, because they are not based on real scientific research.

Many leading labs are transitioning from research to engineering, and the two are fundamentally different. If control or management problems arise during the transition, that is another matter. As for what Cockson actually saw, I don't know.

Q: In recent years, the AI industry has seen many very pessimistic predictions, such as "AI could destroy humanity and cause mass unemployment." What do you think about these statements?

Huang: I think we should not casually attach labels like "probability of human extinction" or "destruction of civilization" to these things, because many of them are actually fabricated. When you label someone as a researcher or scientist and then make a sensational prediction, it can easily cause public panic, and I think that is irresponsible.

We have seen many such predictions in the past. Someone predicted that radiologists would be completely replaced by AI within five years, and radiologists would no longer be needed in the future. The reality is that the world needs more radiologists; AI is indeed used in medical image analysis, but that is completely different from "radiologists disappearing."

Others predicted that "within the next 6 to 12 months, 90% of code will be generated by AI, and 50% of entry-level jobs will disappear within 6 to 9 months." In fact, none of these predictions came true. Previously, some said "GPT-2 and Llama 3 are too dangerous to release," and even predicted that "half of white-collar jobs will disappear the next year," or that the so-called "employment doomsday" is coming.

I think these predictions should be recorded. Years later, when we look back, we will know which predictions were correct and which were just creating panic.

Q: So you think AI safety issues should be handled more as engineering problems?

Huang: If a safety incident occurs, we should first conduct a root cause analysis from an engineering perspective, understand what happened, where the problem lies, and what measures can be taken to prevent similar incidents from happening again, and then turn these measures into systems, processes, and technical controls.

I believe these companies are already doing these things, including better sandboxes, stricter runtime environments, improved monitoring, and continuous monitoring systems. I would even bet that these problems can be controlled and prevented.

If an engineering company really doesn't know what happened and doesn't know how to control it, engineers can be sent to help them. But I don't think it will come to that. These labs have very talented people, and I believe they have already analyzed the relevant issues and taken appropriate measures.

Q: If AI safety issues require regulation, where should regulation start?

Huang: Regulation should target real problems. The issues that really need attention mostly come from leading AI labs, because they have the most computing resources and are working on the most cutting-edge problems. A high school student or an ordinary startup is unlikely to have enough computing power to create these problems.

These labs are transitioning from research to engineering, while also building companies, cultures, technologies, and products, which is not an easy process. Therefore, attention should be paid to how they build more complete engineering systems to ensure that technology development and testing processes are safe enough.

Q: Is there a need for third-party evaluation agencies?

Huang: I think there can be multiple evaluation and audit agencies. They don't need to know everything, but they should know what questions to ask. This is somewhat similar to financial auditing. If there is only one evaluation agency, it may be influenced; if there are multiple evaluation agencies, they can check and balance each other.

02 "Recursive Self-Improvement" Is Not That Mysterious

Q: There is increasing discussion about recursive self-improvement, that is, AI generating data, self-training, and continuously accelerating. What do you think?

Huang: I think "recursive self-improvement" is a new label that has become popular recently, but behind it is actually a combination of many existing technologies, including context, skills, reflection, reinforcement learning, synthetic data generation, and LoRA.

These methods can indeed allow AI to accumulate experience during task execution and then continuously improve. For example, through synthetic data and reinforcement learning, model capabilities can be improved without retraining the entire base model, and the accumulated experience can be further used to train the base model. I believe almost every company is using some of these technologies.

The term "recursive self-improvement" may sound like the system will become completely out of control, but in reality, products still need to undergo evaluation, retesting, and regression testing before release, which are the most basic engineering controls.

These leading labs are transitioning from research institutions to engineering institutions, and I believe they will have better methods, knowledge, practices, tools, and technologies in the future to control, verify, and evaluate these systems. Recursive self-improvement can happen internally, but verified products can still be released safely.

Q: What do you think about the relationship between open-source models and closed-source models?

Huang: The world needs both open-source models and closed-source models. I myself also use closed-source frontier models. They are a bit like bottled water; water itself is free, but different forms of products can meet different needs.

Open-source models are very important for sovereignty, privacy, and enterprise proprietary technology. In the past six months, about $400 billion in venture capital has flowed into AI-native companies, of which about 80% use open-source models. Without open-source models, many startups would not be able to achieve their goals, because what they want to do is different from what leading labs are doing.

The real advantage of the U.S. lies in having multiple forms of innovation. To win the AI race, we must allow open-source models to develop.

Winning the AI race is not just about a few American companies winning, but about American companies, industries, researchers, educators, students, and entrepreneurs all being able to seize opportunities from this technology. Some will use closed-source models, many will use open-source models, and both are needed.

Q: Do you think China will become an important contributor to the global open-source ecosystem?

Huang: I believe China may contribute a significant portion to the global open-source ecosystem. China has a large number of engineers, and many students studying science and mathematics. Top universities like Tsinghua produce many outstanding talents every year.

For example, Chinese engineers have contributed to Linux, Kubernetes (open-source container orchestration system), and many software projects. One characteristic of open source is that once you download it, it belongs to you; you can continue to modify, optimize, and make it your own.

The AI race ultimately comes down to who can best utilize this technology. Many important inventions of the past industrial revolution came from Europe, but the U.S. later made good use of those technologies. I hope the next industrial revolution can develop in a similar way.

I think the discussion in China is more pragmatic, focusing more on economic development and social progress, rather than constantly discussing doomsday or civilization destruction. If these doomsday predictions were true, we should certainly discuss and respond to them, but there is no need to create panic. Our job is to make it happen.

Q: If AI eventually automates a large amount of programming work, what will engineers still need to do?

Huang: Engineering work will still exist. My generation of engineers did not spend a lot of time writing code when software was not yet widespread. Now software engineers spend a lot of time writing code. In the future, if a large part of that work is automated, humans will still be engaged in engineering work. One habit of mine is that my favorite keyboard key is Backspace, because the best software often means less code.

03 Why NVIDIA Is Deeply Involved in the AI Industry Chain

Q: The U.S. government initially emphasized jobs, reindustrialization, energy, and supply chains. These things are indeed happening now. NVIDIA has become an important capital provider for the entire AI industry. How do you decide the direction of resource allocation?

Huang: AI is a new industrial revolution. This industry includes many components, such as models, chips, applications, data centers, buildings, electricity, and power generation facilities. I have been observing the entire ecosystem to identify bottlenecks.

If a very good company is limited by a certain link, I will pay attention to that problem. It could be the supply chain, or land, electricity, or factories. NVIDIA's scale is already large enough, so we have to think about the supply chain many years in advance. Companies like Corning, Lumentum, TSMC, and memory suppliers are all companies we need to pay long-term attention to.

Q: Some people believe that most of the profits in the AI industry will eventually shift to the application layer. Why does NVIDIA want to enter higher-level areas such as models and Hugging Face?

Huang: NVIDIA can run almost all models in the world. About a year and a half ago, people mainly discussed OpenAI's models, but now there are Meta's Muse, Grok, Gemini, Anthropic, and many other models. Many frontier AI labs are built on NVIDIA's platform.

Our strategy is to help everyone succeed, not to take over other people's businesses. We will stay as low in the technology stack as possible, and only move up when necessary. CUDA supports a large number of frameworks, and Megatron and Megatron Core support large-scale training. We invent necessary technologies and then let the entire ecosystem develop, allowing more companies to participate.

Q: Why are regional cloud service providers becoming increasingly important now?

Huang: Regional cloud service providers are usually more flexible than large hyperscale cloud providers because they can find land, power, and factories locally, and they better understand local needs.

In the future, a distributed enterprise network will form, and more and more countries will regard AI as a strategic industry. We have already seen some regions begin to increase AI computing infrastructure.

Q: NVIDIA is now developing its own open-source models, including models in autonomous driving and biomedicine. Why do these things?

Huang: Because customers need them. We have the ability to do the best, so we will do it. For example, Alpamayo, which is our autonomous driving model. It is an autonomous driving system with reasoning capabilities. In the past, autonomous driving required billions of hours of road data, but reasoning capabilities can allow the system to think about similar situations, thereby reducing reliance on large amounts of actual road data.

The future of autonomous driving includes not only passenger cars, but also agricultural equipment, trucks, vans, and many other mobile devices. Many companies do not have a complete technology stack, so NVIDIA can provide a complete technology stack, and customers can make last-mile adjustments according to their needs.

The same is true in the biomedical field. We developed ESM2, ESMFold, OpenFold, AlphaFold 2, and some equivariant models and Protein Complexa (a generative AI framework for all-atom protein complexes), because pharmaceutical companies really need these technologies. My starting point is always: someone needs this technology, so we do it.

04 Musk's Terafab

Q: Musk announced the Terafab chip factory plan, which is very large in scale. What do you think?

Huang: NVIDIA is very familiar with process technology, and we have been pushing the limits of the process, so of course we can discuss this issue. Once Musk decides to do something, it is difficult to stop him from continuing, and that is also his superpower.

Q: So can NVIDIA produce chips in his factory in the future?

Huang: We can discuss it.

Q: If the definition of AGI is reaching human-level intelligence, do you think we have entered this stage? What about superintelligence?

Huang: We have reached it, and we can even say that in certain specific fields, we have entered the stage of superintelligence.

Q: Why?

Huang: Because in some very specific fields, AI has already surpassed humans. Autonomous driving is an example. If a car can drive better than a human, then in the driving domain, it is superintelligence.

Currently, the accident rate of autonomous driving systems can reach one-tenth of that of humans. The same is true in the biomedical field. Tasks such as synthetic proteins and virtual screening have reached the level of superintelligence. In these specific fields, AI is already able to perform tasks that exceed human capabilities.

Q: What is it like to be at the forefront of such technology?

Huang: I like this feeling. The future is bright. Maybe many people won't need to work in the future, but I don't want to miss this era.

I hope all of us can move toward that future together, and humanity will ultimately succeed together. We need to encourage everyone and reduce unnecessary dramatic arguments. Most importantly, we need to ensure that the entire U.S. can participate.