One FPGA card is assigned to every new Azure server and is connected to its network interface card (NIC), PCIe bus and the top-of-rack network switch. That means it’s possible for each FPGA to talk with others, and Microsoft can leverage many FPGAs across its data centres for big jobs, with low latency. That’s especially important for massive machine-learning applications.
“If we want to allocate 1,000 FPGAs to a single [deep neural net] we can,” Burger said. “We get that kind of scale.”
That scale can provide massive amounts of computing power. If Microsoft used Azure’s entire FPGA deployment to translate the English-language Wikipedia, it would take only a tenth of a second, Burger said on stage at Ignite.
Custom silicon
Microsoft is not the only company turning to custom silicon for this sort of work. Google unveiled a Tensor Processing Unit earlier this year that’s supposed to accelerate some machine learning tasks in its cloud. The TPU is an Application-Specific Integrated Circuit or ASIC — a purpose-built chip.
Google used ASICs instead of FPGAs because of speed and efficiency. So, why did Microsoft choose FPGAs?
The industry moves far too quickly for him to be confident a particular ASIC will do what needs to be done over time, Burger said. While using only the reprogrammable hardware in an FPGA wouldn’t be great for performance, the hardened SRAM and DSP chips on the FPGA board can speed up certain applications, shrinking the performance gap.
“I’m not comfortable locking the control path down for three years and saying ‘I know what to do now,'” Burger said.
Right now, Accelerated Networking is only available for DS15v2 instances in Azure’s West Central US and Western Europe regions. It is only compatible with Windows Server 2012 R2 and Windows Server 2016 Technical Preview 5, though Microsoft plans to make it work with Linux instances soon.
Future plans
In the future, the Accelerated Networking service will expand to Azure’s other virtual machine types and operating systems. It will go from being an opt-in enhancement to being a free, opt-out benefit that will increase networking speeds available by default.
Looking to the future, Microsoft has said that the FPGAs will be put to use in machine-learning applications. Burger said the company has set up code for its Cognitive Services to run in an FPGA-accelerated mode, so those may be next.
“This is going to be a journey for how we expose this capability to customers,” Russinovich said. “I think the first thing that we’re talking about doing is [deep learning] where we train the models and then let customers run them on CPUs or GPUs in our data centre. In the future, they’ll be able to run the scoring on FPGAs, and potentially also train models themselves if they want to on the FPGAs. But we’re a ways off.”
For Burger, one of the biggest questions will be what the right mix of FPGAs and CPUs is inside an Azure datacentre. Even though Microsoft has hundreds of thousands of FPGAs already deployed, they aren’t enough to meet the company’s needs as more teams start using them.
“The CPUs are important and will continue to be important for all the software and all these products and services we have,” he said. “But I think for applications, the big breakthrough at scale is going to come from non-CPU technologies.”
IDG News Service
Ltd
