Manal Talib Ali; Bassam H. Abed
Abstract
Hyperbolic tangents and Sigmoid are commonly used for Artificial Neural Networks as activation functions. The complex equation of the activation function is one of the most difficult ...
Read More ...
Hyperbolic tangents and Sigmoid are commonly used for Artificial Neural Networks as activation functions. The complex equation of the activation function is one of the most difficult to be implemented in hardware because containing division and exponential, which gives non-linear behavior. The challenge is building a tan-sigmoid function in hardware with efficient performance. Therefore, this work will focus on implementing the activation function in FPGA. To overcome this challenge, a different approach was proposed in this paper, efficient hardware-implemented for tan-sigmoid in terms of the number of slices occupied and the resources utilization are designed. In this work, three approaches are proposed: tan-sigmoid using the log approximation method, tan-sigmoid using segmentation method, and tan-sigmoid using the polynomial method. These approaches are efficiently implemented in the Xilinx Spartan-3A xc3s700a-4fg484 platform. Hardware synthesis and FPGA implementations illustrate that the proposed tan-sigmoid only takes up to 1% of logic resources in the first and second proposed approaches. While, 4% showed in the third proposed approach, with the best efficiency and significantly confirmed the lowest implementation costs than the traditional approach.
