GSI Technology has been granted a patent for a system that trains a neural network-based floating-point-to-binary feature vector encoder. The system aims to preserve the locality relationships between samples in an input space and an output space. It includes a neural network with floating-point inputs and outputs, a probability distribution loss function generator, and various other components for generating proxy sets and reference sets. The patent claim describes a method for training the neural network and generating a loss function based on input and output probability distributions. GlobalData’s report on GSI Technology gives a 360-degree view of the company including its patenting strategy. Buy the report here.

According to GlobalData’s company profile on GSI Technology, convolutional neural networks (CNNs) was a key innovation area identified from patents. GSI Technology's grant share as of September 2023 was 58%. Grant share is based on the ratio of number of grants to total number of patents.

Training a neural-network-based floating-point-to-binary feature vector encoder

Source: United States Patent and Trademark Office (USPTO). Credit: GSI Technology Inc

A recently granted patent (Publication Number: US11763136B2) describes a method for training a neural network to preserve locality relationships between samples in an input space and an output space. The method involves using a neural network with floating-point inputs and floating-point pseudo-bipolar outputs. A loss function is generated to compare the input probability distribution, constructed from floating-point cosine similarities of the input space, with the output probability distribution, constructed from floating-point pseudo-bipolar pseudo-Hamming similarities of the output space.

The method further includes calculating the output probability distribution between a floating-point pseudo-bipolar encoded sample vector and a set of pseudo-bipolar encoded reference vectors. Additionally, a random sampling of vectors from a training vector set is taken to generate a representative proxy vector set. A sample vector is selected from the training vector set, and a set of k nearest neighbor vectors from the proxy vector set, closest to the sample vector, is found to generate a reference vector set to be encoded by the encoder. This process is repeated for each training iteration, and multiple sample vectors are selected per training iteration to generate a plurality of sample vectors and reference vector sets.

The loss function is calculated using a Kullback-Leibler divergence from the input probability distribution and the output probability distribution. The neural network used for training includes an output layer that generates the floating-point pseudo-bipolar encoded sample and reference vectors using a beta-scaled tan h layer. The pseudo-Hamming similarities in the output space are calculated using an inner product, and both the cosine similarities and pseudo-Hamming similarities are normalized to be within the same range of values. The normalization process involves using a binary code length for the pseudo-Hamming similarities and converting both the cosine similarities and pseudo-Hamming similarities to probabilities.

Once the neural network is trained, an inference neural network is produced from the trained neural network. This involves removing the pseudo-bipolar output layers and adding at least one binary output layer to generate the inference neural network. The true binary vectors outputted by the inference neural network can be used in approximate nearest neighbor searches.

Overall, this patented method provides a technique for training a neural network to preserve locality relationships between samples in an input space and an output space, with potential applications in various fields such as image recognition, natural language processing, and recommendation systems.

To know more about GlobalData’s detailed insights on GSI Technology, buy the report here.

Premium Insights

From

The gold standard of business intelligence.

Blending expert knowledge with cutting-edge technology, GlobalData’s unrivalled proprietary data will enable you to decode what’s happening in your market. You can make better informed decisions and gain a future-proof advantage over your competitors.

GlobalData

GlobalData, the leading provider of industry intelligence, provided the underlying data, research, and analysis used to produce this article.

GlobalData Patent Analytics tracks bibliographic data, legal events data, point in time patent ownerships, and backward and forward citations from global patenting offices. Textual analysis and official patent classifications are used to group patents into key thematic areas and link them to specific companies across the world’s largest industries.