The SiLU activation function was introduced in "Gaussian Error Linear Units
(GELUs)" Hendrycks et al. 2016 and
"Sigmoid-Weighted Linear Units for Neural Network Function Approximation in
Reinforcement Learning"
Elfwing et al. 2017 and was independently
discovered (and called swish) in "Searching for Activation Functions"
Ramachandran et al. 2017
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2021-08-16 UTC."],[],[]]