With the proliferation of compute-intensive machine learning applications, such as chatbots that perform real-time language translation, device manufacturers are investing in specialized hardware to quickly move and process the large amounts of data these systems demand. They often include components.
Choosing the best design for these components, known as deep neural network accelerators, is difficult because there is a huge range of design options. This difficult problem becomes even more difficult when designers try to add cryptographic operations to protect data from attackers.
Now, MIT researchers have developed a search engine that can efficiently identify the best designs for deep neural network accelerators that improve performance while maintaining data security.
Their search tool known as secure loopis designed to take into account how adding data encryption and authentication measures impacts accelerator chip performance and energy usage. Engineers can use this tool to obtain optimal designs of accelerators for neural networks and machine learning tasks.
Compared to traditional scheduling techniques that do not consider security, SecureLoop can improve the performance of accelerator designs while protecting data.
SecureLoop can be used to improve the speed and performance of demanding AI applications such as self-driving cars and medical image classification, while keeping sensitive user data safe from certain types of attacks.
“If you’re interested in performing computations that maintain the security of your data, the rules you used to find the optimal design are now broken. So all that optimization has to do with this new, more complex You need to customize it to your constraint set. [lead author] Kyungmi has done just that with this paper,” said Joel Emer, an MIT professor specializing in the practice of computer science and electrical engineering and co-author of the paper on SecureLoop.
Emmer is joined on the paper by first author Kyungmi Lee, a graduate student in electrical engineering and computer science. Mengjia Yan, Homer A. Burnel Career Development Assistant Professor of Electrical Engineering and Computer Science and member of the Computer Science and Artificial Intelligence Laboratory (CSAIL). and lead author Anantha Chandrakasan, dean of the Massachusetts Institute of Technology’s School of Engineering and the Vannevar Bush Professor of Electrical Engineering and Computer Science. This research will be presented at the IEEE/ACM International Symposium on Microarchitecture.
“The community reluctantly accepted that adding cryptographic operations to the accelerator would incur overhead. They thought it would only make a small difference in the design trade-off area. But , this is a misconception. In fact, cryptographic operations can greatly distort the design space of energy-efficient accelerators. Kyungmi has done a great job in identifying this problem,” Yan added. I did.
reliable acceleration
Deep neural networks consist of many layers of interconnected nodes that process data. Typically, the output of one layer becomes the input of the next layer. Data is grouped into units called tiles for processing and transfer between off-chip memory and the accelerator. Each layer of a neural network can have its own data tiling configuration.
A deep neural network accelerator is a processor with a set of computational units that parallelizes operations such as multiplication at each layer of the network. Accelerator schedules describe how data is moved and processed.
Space on the accelerator chip is at a premium, so most data is stored in off-chip memory and fetched by the accelerator as needed. However, because the data is stored off-chip, it is vulnerable to attackers who can steal the information or change some values, potentially causing the neural network to malfunction.
“As a chip manufacturer, we cannot guarantee the security of external devices or the entire operating system,” Lee explains.
Manufacturers can protect their data by adding authenticated encryption to their accelerators. Encryption uses a private key to scramble data. Authentication then splits the data into uniform chunks and a cryptographic hash is assigned to each chunk of data and stored with the data chunk in off-chip memory.
When the accelerator retrieves an encrypted chunk of data, known as an authentication block, it uses a private key to restore the original data, verify it, and then process it.
However, the tile sizes of the authentication block and data do not match, which can result in multiple tiles in a block or a tile split into two blocks. Since the accelerator cannot arbitrarily retrieve parts of the authentication block, it ends up retrieving extra data, which consumes additional energy and can slow down computation.
Additionally, the accelerator still needs to perform cryptographic operations on each authentication block, further increasing computational cost.
efficient search engine
MIT researchers use SecureLoop to discover how they can identify the fastest and most energy-efficient accelerator schedules, allowing devices to use off-chip memory to retrieve additional blocks of data for encryption and authentication. We looked for ways to minimize the number of times you need to access .
They started by extending an existing search engine called Timeloop, which Emer and his collaborators had previously developed. First, we added a model that can account for the additional computations required for encryption and authentication.
We then reformulated the search problem into a simple mathematical equation. This allows SecureLoop to find the ideal true block size in a much more efficient way than searching through all possible options.
“Depending on how you allocate this block, you can increase or decrease the amount of unwanted traffic. If you allocate cryptographic blocks well, you only need to retrieve a small amount of additional data,” Lee says.
Finally, we included heuristics to ensure that SecureLoop identifies the schedule that maximizes the performance of the entire deep neural network, not just a single layer.
Finally, the search engine outputs an accelerator schedule that includes a data tiling strategy and authentication block size that provides the highest possible speed and energy efficiency for a given neural network.
“The design space for these accelerators is vast. What Kyungmi has done is a very practical approach to make that search tractable so that you can find a suitable solution without having to search the space exhaustively. It was about figuring out a way to do it,” Emmer says.
When tested in a simulator, SecureLoop identified schedules that were up to 33.2 percent faster and had a 50.2 percent better energy-delay product (a metric related to energy efficiency) than other security-aware methods.
The researchers also used SecureLoop to investigate how the accelerator design space changes when security is considered. They learned that allocating a little more area on the chip to the crypto engine, sacrificing space for on-chip memory, could lead to improved performance, Lee says.
In the future, the researchers hope to use SecureLoop to find accelerator designs that are resistant to side-channel attacks that occur when an attacker gains access to the physical hardware. For example, even if the data is encrypted, an attacker can monitor the device’s power consumption patterns and obtain sensitive information. They also extend her SecureLoop to be applicable to other types of computations.
Funding for this research was provided in part by Samsung Electronics and the Korea Advanced Research Foundation.