All technological notes.
Long Short-Term Memory (LSTM):
Applications:
Disadvantages


Forget gate: decide forget
Longe-Term Menory to be remembered.sigmoid function decides it.
previous state (ht-1) 前状态content input (Xt) 内容输入0(omit this) and 1(keep this) for each number in the cell state Ct-1Input gate:
New Long-Term MemorySigmoid function:
Potential Long-Term memory to add to the Long-Term Memorytanh function
Short-Term Memory and the InputPotential Long-Term MemoryOutput gate
Sigmoid function decides which values to let through 0 or 1.
Potential memorytanh function gives weightage to the values which are passed, deciding their level of importance ranging from -1 to 1 and multiplied with an output of sigmoid.
Potential Short-Term memoryvs RNN
Hidden Layer Structure
Gating Mechanisms
sigmoid gates and 2 Tanh layer.
Gates were added to restrict the information that goes through cells.
0-1
0 is a reference to “reject all’1 means “include all.”Activation Functions:
sigmoid activation functions (0,1)tangent activation functions (-1,1)