Skip to Content
皮衣与硅片
第四部:引爆AI奇点(2012—2022)

第十二章:多伦多卧室传来的惊雷(2012—2015)

Chapter Twelve: The Thunderclap from a Toronto Bedroom (2012–2015)

在人类科技史上,真正改写文明航向的奇点,往往并不诞生在耗资百亿的国家实验室里,而是诞生在某个连空调都吹不凉的简陋房间。

In the history of human technology, the singularities that truly bend the course of civilization are seldom born in national laboratories that burn through billions; more often they are born in some shabby room so crude that even the air conditioning cannot keep it cool.

2012年夏天,加拿大多伦多郊区一栋普通民宅的二楼卧室里,两块插在台式机主板上的英伟达GeForce GTX 580游戏显卡,正发出如同直升机螺旋桨般的刺耳轰鸣。由于3GB显存和数百瓦功耗散发出惊人的热量,二十六岁的乌克兰裔博士生亚历克斯·克里泽夫斯基(Alex Krizhevsky)不得不打开窗户透气,任由滚烫的气流在床铺与书桌之间打转。

In the summer of 2012, in a second-floor bedroom of an ordinary house in the Toronto suburbs, two NVIDIA GeForce GTX 580 gaming graphics cards slotted into a desktop motherboard were shrieking like the rotors of a helicopter. With 3GB of video memory and several hundred watts of power pouring out astonishing heat, the twenty-six-year-old Ukrainian-born doctoral student Alex Krizhevsky had no choice but to open the window for air, letting the scalding current swirl between his bed and his desk.

两块 GeForce GTX 580:多伦多卧室里轰鸣六天六夜的深度学习熔炉(资料图)
两块 GeForce GTX 580:多伦多卧室里轰鸣六天六夜的深度学习熔炉(资料图)

指导他做这个疯狂实验的,是六十四岁的多伦多大学计算机科学教授杰弗里·辛顿(Geoffrey Hinton),以及另一位眼神如苦行僧般狂热的年轻博士生——日后成为OpenAI联合创始人兼首席科学家的伊利亚·苏茨克维(Ilya Sutskever)。

The man directing this mad experiment was the sixty-four-year-old University of Toronto professor of computer science Geoffrey Hinton, together with another young doctoral student whose eyes burned with an ascetic's fanaticism — the future OpenAI co-founder and chief scientist Ilya Sutskever.

在此之前的整整三十年里,“人工神经网络(Artificial Neural Networks)”在主流计算机科学界是一个近乎异端的笑话。学院派权威们信奉的是由人类专家编写精密规则与特征提取公式的传统机器学习;而辛顿却像中世纪固执的炼金术士一样,坚信只要模仿人类大脑皮层神经元的连接方式,用足够多的数据去“喂养”多层神经网络,机器就能自己学会看懂这个世界。

For the full thirty years before that, "Artificial Neural Networks" had been a joke bordering on heresy in mainstream computer science. The academic establishment believed in traditional machine learning, in which human experts hand-crafted intricate rules and feature-extraction formulas; Hinton, however, like some obstinate medieval alchemist, held the conviction that if one only imitated the way neurons connect in the human cerebral cortex and fed enough data to a multilayered neural network, the machine would teach itself to see and understand the world.

然而,神经网络有一个致命的死穴:算力黑洞。

Yet neural networks carried one fatal weak point: a black hole of computing power.

要让一个包含六千万个参数、六十五万个神经元的深度卷积网络,去学习斯坦福大学华裔学者李飞飞团队构建的ImageNet数据库里那一百二十万张高清图片(涵盖一千个物种类别,从阿富汗猎犬到消防车),所涉及的矩阵乘法运算量高达天文数字。如果用当时最顶级的英特尔多核CPU集群来跑,训练一次至少需要好几个月——还没等跑出结果,研究经费和博士生的青春就先耗光了。

To train a deep convolutional network of sixty million parameters and six hundred and fifty thousand neurons on the 1.2 million high-resolution images in ImageNet, the database built by the team of the Stanford scholar of Chinese descent Fei-Fei Li (a thousand object categories, from Afghan hounds to fire engines), demanded a quantity of matrix multiplication on an astronomical scale. Run on the most powerful multicore Intel CPU clusters of the day, a single training pass would take many months at the least — long before results arrived, the research funding and the doctoral student's youth would both have burned away.

绝境之中,伊利亚说服囊中羞涩的课题组去电脑城买回了两块打游戏用的GTX 580显卡(辛顿后来幽默地回忆,他曾向英伟达申请免费赠送一块几千美元的Tesla科研卡遭到了基层销售的婉拒,逼得他们只能自己掏钱去买消费级游戏卡)。亚历克斯利用黄仁勋顶着华尔街唾骂塞进每一张显卡里的CUDA底层接口,亲手写出了跨越两块GPU并行通信的卷积运算代码。

In this desperate strait, Ilya talked their cash-starved lab into walking into a computer store and buying two GTX 580 gaming cards (Hinton would later recall with humor that his request to NVIDIA for a complimentary Tesla research card worth several thousand dollars had been politely turned away by a frontline salesperson, leaving the group no choice but to pay for consumer gaming cards out of its own pocket). Drawing on the CUDA low-level interface that Jensen Huang, braving the curses of Wall Street, had wedged into every graphics card, Alex wrote with his own hands the convolution code for parallel computation across two GPUs.

连续轰鸣运转了整整六天六夜之后,这个后来被载入史册的模型——AlexNet——走上了2012年9月底意大利佛罗伦萨的ImageNet国际视觉识别挑战赛擂台。

After six days and six nights of uninterrupted thunderous operation, this model that would later enter the history books — AlexNet — stepped onto the arena of the ImageNet international visual recognition challenge in Florence, Italy, at the end of September 2012.

结果揭晓的那一秒,统治计算机视觉界数十年的传统派学者们感觉天塌了。

The instant the results were announced, the traditionalist scholars who had ruled computer vision for decades felt the sky fall in.

此前,全世界最顶尖的传统算法团队穷尽毕生数学技巧,每年只能把图像识别的前五错误率(Top-5 Error Rate)艰难地往下挤百分之一左右,徘徊在26%上下;而辛顿师徒用两块游戏显卡跑出来的AlexNet,直接将错误率暴砍了十几个百分点,一举砸穿到了惊世骇俗的15.3%!

Before that, the world's finest traditional algorithm teams, exhausting a lifetime's mathematical craft, could only grind the Top-5 Error Rate of image recognition down by about one percentage point a year, hovering around 26%; the AlexNet that Hinton and his students ran on two gaming cards hacked the error rate down by more than ten percentage points at a stroke, smashing through to a staggering 15.3%!

更令内行脊背发凉的是,AlexNet代码里没有任何一条关于“猫的耳朵是尖的”或“汽车有四个轮子”的人工规则——机器纯粹是靠着CUDA并行算力的暴力喂养,在硅片矩阵里自己“顿悟”出了万物的视觉法则!

What sent a colder chill down the spines of the experts was that AlexNet's code contained not a single hand-written rule about "cats having pointed ears" or "cars having four wheels" — the machine, fed purely on the brute force of CUDA's parallel computing power, had "awakened" all by itself, inside a matrix of silicon, to the visual laws of all things!

同一时刻,大洋彼岸的加州斯坦福大学与谷歌X实验室里,另一位华人人工智能先驱吴恩达(Andrew Ng)也刚刚得出一个震撼的结论:过去谷歌用一千台服务器、一万六千颗英特尔CPU核心耗费巨资才在YouTube视频里识别出“猫脸”的神经网络,他在斯坦福与英伟达首席科学家比尔·达利(Bill Dally,2009年接替大卫·柯克加盟英伟达的斯坦福计算机系主任)合作测试后发现,仅仅用三台服务器、十二块英伟达GPU,就能跑出完全一样的效果,而硬件成本和耗电量暴降了百分之九十以上!

At the very same moment, across the ocean at Stanford in California and in the Google X laboratory, another Chinese-born pioneer of artificial intelligence, Andrew Ng, had just reached a conclusion no less thunderous: the neural network for which Google had once spent a fortune on one thousand servers and sixteen thousand Intel CPU cores to recognize "cat faces" in YouTube videos could, as he found in tests run at Stanford together with NVIDIA's chief scientist Bill Dally (the Stanford computer-science department chair who had joined NVIDIA in 2009, succeeding David Kirk), produce exactly the same result on three servers and twelve NVIDIA GPUs — with hardware cost and power consumption slashed by more than ninety percent!

这两声惊雷,在2012年深秋同时传到了圣克拉拉黄仁勋的办公桌上。

Those two thunderclaps reached Jensen Huang's desk in Santa Clara in the same late autumn of 2012.

那一年,黄仁勋四十九岁。在商业史上,无数伟大的技术发明者都曾与财富擦肩而过——施乐帕洛奥图研究中心发明了图形界面和鼠标,却白白便宜了乔布斯;柯达发明了第一台数码相机,却为了保护胶卷利润亲手把它锁进了抽屉。当时硅谷几乎所有半导体巨头(包括英特尔、AMD和高通)看到AlexNet的新闻时,都只把它当成学术界一个有趣的小插曲:不就是几百个大学教授买几百张显卡搞图像识别吗?这能卖出几个钱?

Jensen Huang was forty-nine that year. Business history is strewn with great inventors who brushed past fortune — Xerox PARC invented the graphical interface and the mouse, only to hand them as a free gift to Steve Jobs; Kodak invented the first digital camera, then locked it in a drawer with its own hands to protect the profits of film. When the news of AlexNet broke, nearly every semiconductor giant in Silicon Valley (Intel, AMD, and Qualcomm included) saw it as no more than an amusing academic footnote: a few hundred professors buying a few hundred graphics cards to do image recognition — how much money could that possibly sell for?

唯独黄仁勋,在听完比尔·达利与首席技术团队长达数小时的深度推演后,瞳孔剧烈收缩。

Jensen Huang alone, after sitting through hours of deep deliberation with Bill Dally and the chief technology team, felt his pupils contract sharply.

凭借二十年游走于晶体管物理极限与软件架构之间的顶级直觉,黄仁勋在短短几天之内,看穿了一个超越时代的终极本质:人类编写软件的方式,要变天了。

With the first-rate intuition he had honed for twenty years along the boundary between the physical limits of transistors and the architecture of software, Jensen Huang saw through, within mere days, an essential truth that outran its era: the way humanity writes software was about to change forever.

在过去的半个世纪里,“软件1.0”是由人类程序员坐在键盘前,一行一行敲出逻辑指令(If-Then-Else),再交给CPU去按顺序执行;而深度学习开启的“软件2.0”,不再由人类写代码,而是由人类收集海量数据,用GPU组成的超级算力熔炉去“炼”出一个神经网络权重模型——换句话说,深度学习不是某一种软件应用,它是未来所有软件的母体!

For the previous half-century, "Software 1.0" had been written by human programmers seated at keyboards, tapping out logical instructions (If-Then-Else) line by line for a CPU to execute in sequence; "Software 2.0," the era deep learning opened, would no longer be coded by humans — humans would gather oceans of data and smelt, in a supercomputing furnace built of GPUs, a model of neural network weights. In other words, deep learning is not one software application among others; it is the mother of all software to come!

更让他浑身战栗的是,神经网络的底层数学逻辑,完完全全就是海量神经元权重的矩阵乘加运算(Matrix Multiplication)——这简直就是老天爷专门为GPU那数千个并行CUDA流处理器量身定做的天命战场!

What made him tremble from head to toe was that the underlying mathematics of neural networks was, precisely and completely, the matrix multiply-accumulate operation (Matrix Multiplication) at the scale of billions of neuron weights — as if heaven itself had tailored this battlefield of destiny for the thousands of parallel CUDA stream processors on a GPU!

没有任何犹豫,也没有搞什么“先成立一个十人创新小组试水两年”的温和改良。2012年底的一个周五,黄仁勋在圣克拉拉总部发动了一场近乎独裁式的全公司战略大转向(Pivot)。

There was no hesitation, and none of the gentle gradualism of "forming a ten-person innovation group to test the waters for two years." On a Friday at the end of 2012, Jensen Huang launched at the Santa Clara headquarters an almost autocratic company-wide strategic pivot.

在周末召开的核心高管闭门会议上,黄仁勋在白板上写下了几个大字:深度学习不再是一个科研项目,从星期一开始,英伟达就是一家人工智能公司。

At the closed-door meeting of core executives convened over the weekend, Jensen Huang wrote several large words on the whiteboard: Deep learning is no longer a research project. From Monday morning, NVIDIA is an artificial intelligence company.

高管们全懵了。当时英伟达百分之八十以上的营收依然来自游戏玩家买显卡打《英雄联盟》和《战地》,深度学习带来的收入在财务报表上连小数点后三位都凑不够,是个彻头彻尾的“零亿美元市场”。要把全公司最精锐的几千名软硬件工程师从赚钱的图形、移动业务抽调出来,去赌一个连商业模式都没影子的学术概念?万一人工智能像七八十年代那样再次迎来“AI寒冬”泡沫破裂,英伟达岂不是要万劫不复?

The executives were struck dumb. More than eighty percent of NVIDIA's revenue still came from gamers buying cards to play League of Legends and Battlefield; deep learning's contribution could not even fill three places after the decimal point on the financial statements — a market of, quite literally, "zero hundred millions." Divert the company's several thousand finest software and hardware engineers away from the profitable graphics and mobile businesses to wager on an academic concept with no visible business model? And if artificial intelligence were to suffer another burst bubble, another "AI winter" like the seventies and eighties, would NVIDIA not be plunged beyond all recovery?

面对质疑,四十九岁的黄仁勋拿出了比2006年强推CUDA时更加狂暴的铁腕。

Confronted with the doubts, the forty-nine-year-old Jensen Huang brought out an iron hand even fiercer than the one he had wielded in forcing CUDA through in 2006.

“如果深度学习只是帮人认出照片里的猫,那它一文不值;但如果它能看懂图像,它就能听懂语音、读懂人类语言、看懂医学X光片、学会自动驾驶汽车!”黄仁勋环视全场,下达了不留任何退路的军令,“把公司所有的研发优先级全部重排,把每一分能抠出来的预算全部压上去——我们要在全世界反应过来之前,把AI的底层地基彻底焊死在英伟达的架构上!”

"If deep learning does nothing more than help people recognize cats in photographs, it is worth nothing; but if it can understand images, it can understand speech, read human language, read medical X-rays, and learn to drive a car!" Jensen Huang swept his gaze around the room and issued an order that left no line of retreat: "Reorder every research priority in this company. Pile on every dollar of budget that can be squeezed out — before the world knows what has happened, we will weld the foundations of AI irrevocably onto NVIDIA's architecture!"

在接下来的三年(2013—2015)里,当英特尔还在傲慢地守着数据中心99%的至强(Xeon)CPU王座打盹、AMD还在忙着给索尼PS4和微软Xbox One定制微薄利润的游戏主机芯片回血时,黄仁勋在无人竞争的旷野上展开了一场疯狂的“圈地筑墙运动”。

Through the three years that followed (2013–2015), while Intel dozed arrogantly on its throne guarding 99% of the data center's Xeon CPU market and AMD busied itself nursing its wounds with thin-margin custom console chips for Sony's PS4 and Microsoft's Xbox One, Jensen Huang waged a frenzied campaign of land-grabbing and wall-building across an open field where no rival stood.

他的杀招不是只造更快的芯片,而是直接杀进AI研究者的每一行代码里。

His killing move was not merely to build faster chips, but to strike directly into every line of code an AI researcher would ever write.

当时像辛顿、杨立昆(Yann LeCun,纽约大学教授兼Facebook AI掌门人)、吴恩达这样的顶尖学者虽然用上了CUDA,但用底层C语言手写神经网络的卷积、池化与反向传播代码依然痛苦不堪。黄仁勋立刻命令伊恩·巴克的软件军团与全球所有顶尖AI实验室贴身结对子:“去问多伦多、问斯坦福、问伯克利、问百度和谷歌的大脑团队,他们最头疼什么算子,我们就直接把它写成现成的免费底层库!”

Scholars of the first rank — Hinton, Yann LeCun (the New York University professor who headed Facebook AI), Andrew Ng — had by then taken up CUDA, yet hand-writing the convolution, pooling, and backpropagation code of neural networks in low-level C remained pure torment. Jensen Huang at once ordered Ian Buck's software legions to pair themselves in intimate partnership with every top AI laboratory on Earth: "Ask Toronto, ask Stanford, ask Berkeley, ask the brain teams at Baidu and Google which operators give them the worst headaches — and we will write those operators, ready-made and free, into our low-level libraries!"

2014年,英伟达正式推出了这道日后令所有竞争对手绝望窒息的钢铁长城——cuDNN(CUDA深度神经网络加速库)。

In 2014, NVIDIA formally unveiled the steel rampart that would one day suffocate every competitor with despair — cuDNN, the CUDA Deep Neural Network library.

紧接着,奇迹般的飞轮效应转起来了:谷歌开源TensorFlow、脸书开源PyTorch、伯克利推出Caffe……全世界所有主流的AI深度学习框架,为了跑出最快的速度,在底层全部默认调用了英伟达的cuDNN与CUDA!这意味着,全球数百万名涌入AI浪潮的年轻算法工程师,哪怕一辈子没见过显卡长什么样,他们敲下的每一行Python代码,都在不知不觉中加深着整个科技界对英伟达硅片的生理成瘾。

At once the flywheel of a miracle began to turn: Google open-sourced TensorFlow, Facebook open-sourced PyTorch, Berkeley produced Caffe... and every mainstream AI deep-learning framework in the world, in its pursuit of the fastest speed, defaulted at the lowest level to NVIDIA's cuDNN and CUDA! It meant that the millions of young algorithm engineers flooding into the AI wave, even those who would never in their lives see what a graphics card looked like, were deepening, one line of Python at a time and without ever knowing it, the whole technological world's physiological addiction to NVIDIA silicon.

硬件层面,黄仁勋更是彻底放开了手脚。在开普勒(Kepler)、麦克斯韦(Maxwell)架构一路狂飙的同时,他在内部绝密立项了代号帕斯卡(Pascal,2016年问世)与伏打(Volta,2017年问世)的新一代架构——他做了一个惊世骇俗的决定:在未来的GPU硅片里,不仅要保留图形核心和通用CUDA核心,还要专门辟出一大片宝贵的晶圆面积,塞进只做一件事的专用特种部队——Tensor Cores(张量核心),并引入半精度浮点运算(FP16),专门用来以十倍、百倍的速度暴力碾碎深度学习矩阵!

On the hardware front, Jensen Huang unbound his hands entirely. Even as the Kepler and Maxwell architectures thundered forward, he opened top-secret internal programs for the next generation of architectures codenamed Pascal (arriving 2016) and Volta (arriving 2017) — and he made a decision that stunned all who learned of it: future GPU silicon would keep not only the graphics cores and general-purpose CUDA cores, but would carve out a broad stretch of precious wafer area for a special-purpose strike force with a single mission — Tensor Cores — while introducing half-precision floating-point arithmetic (FP16), built for one purpose alone: to crush deep-learning matrices by brute force at ten and a hundred times the speed!

到了2015年的GTC大会上,站在圣何塞舞台中央的黄仁勋,已经脱胎换骨。

By the GTC conference of 2015, the Jensen Huang standing at the center of the stage in San Jose had been remade entirely.

台下不再是2009年那群为了蹭免费盒饭而来的穷博士生。谷歌、微软、亚马逊、脸书、阿里巴巴与百度的首席架构师们把过道挤得水泄不通。身穿黑色皮衣、两鬓开始染上银霜的黄仁勋,背后大屏幕上闪烁着密密麻麻的神经网络拓扑图。他在那个春天向全世界抛出了一个当时听起来像科幻小说、十年后却成真得令人毛骨悚然的预言:

The audience was no longer the penniless doctoral students of 2009, come to scrounge a free boxed lunch. The chief architects of Google, Microsoft, Amazon, Facebook, Alibaba, and Baidu had packed the aisles until not a gap remained. In his black leather jacket, silver beginning to frost his temples, Jensen Huang stood before a vast screen flickering with densely woven diagrams of neural network topology. That spring he flung a prophecy at the world that sounded like science fiction at the time and came true, ten years on, with a chill that crawls the skin:

GTC 舞台上的黄仁勋:台下挤满了谷歌、微软、亚马逊的首席架构师(公开资料图)
GTC 舞台上的黄仁勋:台下挤满了谷歌、微软、亚马逊的首席架构师(公开资料图)

“未来的每一家公司都将成为AI公司,未来的每一座数据中心都将变成一座生产智能的工厂。”

"Every company of the future will be an AI company, and every data center of the future will become a factory for the manufacture of intelligence."

就在这一年(2015年)年底的旧金山,为了对抗谷歌吞并DeepMind后可能形成的AI垄断霸权,硅谷投资奇才萨姆·奥特曼(Sam Altman)、特斯拉掌门人埃隆·马斯克,连同刚刚从谷歌跳槽出来的AlexNet核心缔造者伊利亚·苏茨克维等十一位极客,在一家酒店的晚宴上宣布共同出资十亿美元(实际早期到位仅一亿多美元),创立一家毫不起眼的非营利性通用人工智能研究机构——OpenAI。

At the end of that same year (2015) in San Francisco, to stand against the AI monopoly that might congeal after Google swallowed DeepMind, the Silicon Valley investment prodigy Sam Altman, the master of Tesla Elon Musk, and Ilya Sutskever, core creator of AlexNet and a fresh defector from Google, joined by other hackers to make eleven geeks in all, announced over a hotel dinner a joint fund of one billion dollars (in fact only a hundred million and a little more arrived in the early days) to found an unremarkable nonprofit research institute for artificial general intelligence — OpenAI.

消息公布后,大多数人只当这是硅谷亿万富豪们又一次理想主义的玩票。但嗅觉如鲨鱼般敏锐的黄仁勋,却在第一时间盯上了旧金山先锋大厦(Pioneer Building)里那间只有几十个人的简陋实验室。

When the news broke, most people filed it away as another idealist flourish from Silicon Valley's billionaires. But Jensen Huang, whose nose was sharp as a shark's, fixed his sight at once on that bare laboratory of a few dozen people in San Francisco's Pioneer Building.

他决定亲自给这群疯子锻造一柄前所未有的重剑——并在2016年那个炎热的夏日午后,亲手抬着它敲开OpenAI的大门,亲手点燃那根在七年后彻底引爆人类世界的导火索。

He resolved to forge, with his own hands, a sword unlike any before it for this band of madmen — and on one burning summer afternoon in 2016, to carry it up the stairs himself, knock on OpenAI's door with his own hands, and light with his own hands the fuse that would, seven years later, detonate the whole human world.

第十二章:多伦多卧室传来的惊雷(2012—2015) · 第 1/35 段