This thesis work aims to overview the literature in the field of Fuzzy Flip-Flop Neural Networks (FNN), which is a combination of the fuzzy model and the neural networks. Activation functions play an important role in neural networks, making the neural networks approximate any nonlinear function arbitrarily. In FNN, fuzzy J-K and D flip-flops are connected to neural networks as the activation function. Although the role of the activation function in the networks is the same, their effects are different according to the specific equations. In some cases, the fuzzy J-K and D flip-flops have a better behavior compared with the traditional activation functions. This thesis aims to build a framework to cover certain activation functions, analyzing the behavior of neural networks with various activation functions and try to find the explanation and foundation in theory.
