Text Classification in Memristor-based Spiking Neural Networks
Memristors, emerging non-volatile memory devices, have shown promising potential in neuromorphic hardware designs, especially in spiking neural network (SNN) hardware implementation. Memristor-based SNNs have been successfully applied in a wide range of applications, including image classification and pattern recognition. However, implementing memristor-based SNNs in text classification is still under exploration. One of the main reasons is that training memristor-based SNNs for text classification is costly due to the lack of efficient learning rules and memristor non-idealities. To address these issues and accelerate the research of exploring memristor-based spiking neural networks in text classification applications, we develop a simulation framework with a virtual memristor array using an empirical memristor model. We use this framework to demonstrate a sentiment analysis task in the IMDB movie reviews dataset. We take two approaches to obtain trained spiking neural networks with memristor models: 1) by converting a pre-trained artificial neural network (ANN) to a memristor-based SNN, or 2) by training a memristor-based SNN directly. These two approaches can be applied in two scenarios: offline classification and online training. We achieve the classification accuracy of 85.88% by converting a pre-trained ANN to a memristor-based SNN and 84.86% by training the memristor-based SNN directly, given that the baseline training accuracy of the equivalent ANN is 86.02%. We conclude that it is possible to achieve similar classification accuracy in simulation from ANNs to SNNs and from non-memristive synapses to data-driven memristive synapses. We also investigate how global parameters such as spike train length, the read noise, and the weight updating stop conditions affect the neural networks in both approaches. This investigation further indicates that the simulation using statistic memristor models in the two approaches presented by this paper can assist the exploration of memristor-based SNNs in natural language processing tasks.
A tool for emulating neuromorphic architectures with memristive models and devices
Memristors have shown promising features for enhancing neuromorphic computing concepts and AI hardware
accelerators. In this paper, we present a user-friendly software infrastructure that allows emulating a wide range of
neuromorphic architectures with memristor models. This tool empowers studies that exploit memristors for online learning and online classification tasks, predicting memristor resistive state changes during the training process. The versatility of the tool is showcased through the capability for users to customise parameters in the employed memristor and neuronal models as well as the employed learning rules. This further allows users to validate concepts and their sensitivity across a wide range of parameters. We demonstrate the use of the tool via an MNIST classification task. Finally, we show how this tool can also be used to emulate the concepts under study in-silico with practical memristive devices via appropriate interfacing with commercially available characterisation tools.
Hardware-efficient compression of neural multi-unit activity using machine learning selected static Huffman encoders
Recent advances in intracortical brain machine interfaces (iBMIs) have demonstrated the feasibility of using our thoughts; by sensing and decoding neural activity, for communication and cursor control tasks. It is essential that any invasive device is completely wireless so as to remove percutaneous connections and the associated infection risks. However, wireless communication consumes significant power and there are strict heating limits in cortical tissue. Most iBMIs use Multi Unit Activity (MUA) processing, however the required bandwidth can be excessive for large channel counts in mm or sub-mm scale implants. As such, some form of data compression for MUA iBMIs is desirable.
A CMOS-based Characterisation Platform for Emerging RRAM Technologies
Mass characterisation of emerging memory devices is an essential step in modelling their behaviour for integration within a standard design flow for existing integrated circuit designers. This work develops a novel characterisation platform for emerging resistive devices with a capacity of up to 1 million devices on-chip. Split into four independent sub-arrays, it contains on-chip column-parallel DACs for fast voltage programming of the DUT. On-chip readout circuits with ADCs are also available for fast read operations covering 5-decades of input current (20nA to 2mA). This allows a device's resistance range to be between 1k and 10M with a minimum voltage range of 1.5V on the device.
How to Shrink My FPGAs—Optimizing Tile Interfaces and the Configuration Logic in FABulous FPGA Fabrics
Commercial FPGAs from major vendors are extensively optimized, and fabrics use many hand-crafted custom cells, including switch matrix multiplexers and configuration memory cells. The physical design optimizations commonly improve area, latency (= speed), and power consumption together. This paper is dedicated to improving the physical implementation of FPGA tiles and the configuration storage in SRAM FPGAs. This paper proposes to remap configuration bits and interface wires to implement tightly packed tiles. Using the FABulous FPGA framework, we show that our optimizations are virtually for free but can save over 20% in area and improve latency at the same time. We will evaluate our approach in different scenarios by changing the available metal layers or the requested channel capacity.
The Future of FPGA Acceleration in Datacenters and the Cloud
In this article, we survey existing academic and commercial efforts to provide Field-Programmable Gate Array (FPGA) acceleration in datacenters and the cloud. The goal is a critical review of existing systems and a discussion of their evolution from single workstations with PCI-attached FPGAs in the early days of reconfigurable computing to the integration of FPGA farms in large-scale computing infrastructures. From the lessons learned, we discuss the future of FPGAs in datacenters and the cloud and assess the challenges likely to be encountered along the way. The article explores current architectures and discusses scalability and abstractions supported by operating systems, middleware, and virtualization. Hardware and software security becomes critical when infrastructure is shared among tenants with disparate backgrounds.
An Adiabatic Capacitive Artificial Neuron with RRAM-based Threshold Detection for Energy-Efficient Neuromorphic Computing (2022)
In the quest for low power, bio-inspired computation both memristive and memcapacitive-based Artificial Neural Networks (ANN) have been the subjects of increasing focus for hardware implementation of neuromorphic computing. One step further, regenerative capacitive neural networks, which call for the use of adiabatic computing, offer a tantalising route towards even lower energy consumption, especially when combined with `memimpedace' elements. Here, we present an artificial neuron featuring adiabatic synapse capacitors to produce membrane potentials for the somas of neurons; the latter implemented via dynamic latched comparators augmented with Resistive Random-Access Memory (RRAM) devices. Our initial 4-bit adiabatic capacitive neuron proof-of-concept example shows 90% synaptic energy saving. At 4 synapses/soma we already witness an overall 35% energy reduction. Furthermore, the impact of process and temperature on the 4-bit adiabatic synapse shows a maximum energy variation of 30% at 100 degree Celsius across the corners without any functionality loss. Finally, the efficacy of our adiabatic approach to ANN is tested for 512 & 1024 synapse/neuron for worst and best case synapse loading conditions and variable equalising capacitance's quantifying the expected trade-off between equalisation capacitance and range of optimal power-clock frequencies vs. loading (i.e. the percentage of active synapses).
Selectively biased tri-terminal vertically-integrated memristor configuration (2022)
Memristors, when utilized as electronic components in circuits, can offer opportunities for the implementation of novel reconfigurable electronics. While they have been used in large arrays, studies in ensembles of devices are comparatively limited. Here we propose a vertically stacked memristor configuration with a shared middle electrode. We study the compound resistive states presented by the combined in-series devices and we alter them either by controlling each device separately, or by altering the full configuration, which depends on selective usage of the middle floating electrode. The shared middle electrode enables a rare look into the combined system, which is not normally available in vertically stacked devices. In the course of this study it was found that separate switching of individual devices carries over its effects to the complete device (albeit non-linearly), enabling increased resistive state range, which leads to a larger number of distinguishable states (above SNR variance limits) and hence enhanced device memory. Additionally, by applying a switching stimulus to the external electrodes it is possible to switch both devices simultaneously, making the entire configuration a voltage divider with individual memristive components. Through usage of this type of configuration and by taking advantage of the voltage division, it is possible to surge-protect fragile devices, while it was also found that simultaneous reset of stacked devices is possible, significantly reducing the required reset time in larger arrays.