When it comes to high-speed data processing, RAM has always been the go-to memory for computers because it is tens of thousands of times faster than disk drives and many times faster than NAND flash.
Researchers at MIT, however, have built a server network that proves for big data applications, Flash is just as fast as RAM and vastly cheaper.
In the age of Big Data, where massive data sets are used to uncover the purchasing trends of millions of people or predict financial market trends based on millions of data points, a single computer’s RAM won’t do.
For example, the data needed to process a single human genome requires between 40 and 100 typical computers.
NAND flash is about a tenth as expensive as RAM and it also consumes about a tenth as much power. So at the International Symposium on Computer Architecture last month, MIT researchers revealed a new system that proved Flash memory is just as efficient as conventional RAM, but it also cuts power and hardware costs.
In fact, even without their new network configuration, the researchers showed that if servers working on a distributed computation use disk drives to retrieve data just 5% of the time, performance is the same as if it were using flash.
For example, 40 servers with 10Tb of RAM could not handle a 10.5Tb computation any faster than 20 servers with 20Tb worth of Flash memory. And, the flash would cost less and consume a fraction of the power.
The researchers were able to make a network of 20 Flash-based servers competitive with a network of RAM-based servers by moving some of the computational power off the servers and onto the Flash drives’ controller chips.
The researchers used flash drives to pre-process some of the data before passing it back to the servers, increasing the efficiency of the distributed computation.
Each server was connected to a field-programmable gate array (FPGA), a kind of chip that can be reprogrammed to mimic different types of electrical circuits. Each FPGA, in turn, was connected to two 500GB flash chips and to the two FPGAs nearest it in the server rack.
Networked together, the FPGAs became a fast network that allowed any server to retrieve data from any Flash drive. The FPGAs also controlled the Flash drives.
“This is not a replacement for DRAM [dynamic RAM] or anything like that,” Arvind Mithal, the Johnson Professor of Computer Science and Engineering at MIT, told MIT News.
Arvind, as Mithal typically goes by, performed the work with a group of graduate students and researchers at Quanta Computer. The research showed there are likely many applications that can replace RAM and take advantage of a Flash-based computer architecture’s lower cost.
“Everybody’s experimenting with different aspects of Flash. We’re just trying to establish another point in the design space,” Arvind said.
Lucas Mearian, IDG News Service
Ltd
