Attention to Rows and Columns: Altering Transformers’ Self-Attention Mechanism for Greater Efficiency
https://read.deeplearning.ai/the-batch/pale-transformer/?utm_campaign=The%20Batch&utm_content=220752275&utm_medium=social&utm_source=linkedin&hss_channel=lcp-18246783
Viz
https://twitter.com/bradyajohnston/status/1438840382385573890
https://twitter.com/DarkStarSystems/status/1373982932835127300