On Monday, IBM announced a breakthrough in optics technology that promises to dramatically improve the way data centers train and run generative artificial intelligence (AI) models. Its new co-packaged optics (Co-packaged optics) technology basically enables the power of optics to be brought inside the chip, allowing connections in data centers to reach the speed of light.
Speaking to reporters at a media briefing, Mukesh Khare, general manager of IBM Semiconductor, said that the telecom industry has made significant strides in making faster chips, but the speed of communication between those chips hasn't kept pace with that. There is a gap of several orders of magnitude between the rate of growth in computing power and the rate of growth in the speed of communication between chips.
In fact, at a more basic level, chips still communicate through electricity, they use copper wires," he noted. And we all know that the best communication technology is fiber optics, which is why it's being used everywhere in long-distance communications."
Although co-encapsulated optics has been around for some time, IBM has revolutionized co-encapsulated optics by developing a new polymer optical waveguide (PWG) technology that allows chipmakers to place silicon photonic chips in the edge region - a valuable space known as "prime real estate" - and then use it for the first time. The PWG technology enables chipmakers to place six times more optical fibers in the edge area of a silicon photonic- a valuable space known as "prime real estate" - than ever before. Each fiber is roughly three times as wide as a human hair, can extend from a few centimeters to hundreds of meters, and can transmit data at terabits per second.
What does it all mean?
The company says the technology will allow for 80 times faster communication bandwidth between chips than is possible today using electronics, and reduce energy consumption by more than five times.
It could also make training large language models (LLMs) up to five times faster, reducing the time it takes to train a standard LLM from three months to three weeks, with performance gains increasing as larger models and more GPUs are used.
In addition to allowing GPUs and gas pedals to communicate with each other faster, this technology has the potential to redefine the way the computing industry transmits high-bandwidth data through boards and servers.
Khare said, "We are very excited to introduce the power of light to radically accelerate the world of AI and many other applications."
When asked when the technology would be commercialized, Khare said IBM's research department is "ready to put it to use."
