"SoftNeuro™" is one of the world's fastest (*1) deep learning inference engines. It operates in multiple environments, utilizing learning results that have been obtained through a variety of deep learning frameworks. Moreover, SoftNeuro is not limited to image recognition. It is a general-purpose inference engine that can be used for text analysis, voice recognition, image recognition, etc.
Morpho not only licenses SoftNeuro as a standalone inference engine, but also brings about a large increase in the processing speed of their existing products by embedding "SoftNeuro" into them. An example is "Morpho Deep Recognizer™", which is Morpho’s image recognition engine.
1.One of the fastest in the world (According to Morpho’s research)
“SoftNeuro" is faster than or as equally fast as the mainstream inference engines - when run on CPUs - while providing a set of key features as described later. This high-speed performance has been achieved through a variety of optimizations (neural network, memory usage and others) for each platform. Please refer to *1 for details.
2.Supports multiple frameworks
“SoftNeuro” achieves fast processing by utilizing the learning results of these major frameworks (Fig.1). It is possible to achieve fast inference processing and multi-platform compatibility without discarding the learning assets that users have built up so far. Further, the compatibility of “SoftNeuro” with frameworks and layers will be expanded sequentially.
3. Multi-platform compatibility
“SoftNeuro” is scheduled to be applied to a variety of platforms, and appropriate optimization (CPU speed-up instructions, use of GPU, DSP and other specialized hardware) for each platform will be performed.
Multi-platform compatibility facilitates deploying the learning results on a wide range of platforms. It also provides flexibility to quickly respond to changes in platforms and hardware trends.
4.Compatible with secure file formats
“SoftNeuro” is capable of encrypting the trained networks, minimizing the risk of leaking the machine learning know-how and the learned model parameters.
*1：One of the Fastest in the world (According to Morpho’s research)
We compared the inference speed of “SoftNeuro” with that of widely-used inference engines that we could obtain, on ARM and intel CPU architectures (at the current state, “SoftNeuro” does not utilize GPU).
Figures 2-1 and 2-2 present the results of the evaluation for these two architectures.
●Figure 1. Integrating “SoftNeuro” into Machine Learning Systems
●Figure 2-1. Comparison of Inference Speeds (CPU: ARM)
Measured on: Qualcomm Snapdragon 835 (Image size used for inference : 224×224 pixels)
●Figure 2-2. Comparison of Inference Speeds (CPU: Intel)
Measured on: Intel® Core™ i7-6700K CPU @ 4.00 GHz (Image size used for inference : 224×224 pixels)