Dear all, Joris Plettscher will deliver the next Oberseminar (19th December,10:00, via Zoom): Title: „Influence of Pruning on Polysemanticity in Artificial Neural Networks“ Abstract: „Features are measurable properties of the data that are relevant to solving a certain task. When multiple unrelated features significantly impact the activation of an individual neuron in a neural network, it is called polysemanticity. This thesis aims to investigate the influence of pruning, meaning the removal of parameters from a model, on the degree of polysemanticity in neural networks. To solve this question, a method to generate toy data with known ground truth features is presented and used to create a controlled environment in which polysemanticity can be measured. Starting with a large model that sufficiently represents the ground truth features, different pruning ratios are applied and the pruned model retrained. The polysemanticity in the retrained model is then measured. Assuming that the number of represented features is preserved through pruning, the degree of polysemanticity could be expected to increase since there are fewer neurons to represent the same amount of features. In addition to that, the Dropout technique will be examined in the context of pruning, as it encourages neurons to adapt to the potential absence of other neurons and promotes redundancy in feature representations. The experiments of this thesis could show that the training process includes an adjustment of polysemanticity, often reducing it until it converges to a certain level. The impact of pruning on polysemanticity was not definitive, though a tendency for polysemanticity to decrease after pruning was observed in some cases." **We cordially ask the attendees to turn on their cameras and turn off their microphones during the presentation. Zoom Room: https://eur05.safelinks.protection.outlook.com/?url=https%3A%2F%2Ffernuni-hagen.zoom.us%2Fj%2F62388524887%3Fpwd%3DWmJRRUpDZWZ4WlJoeFlGQTBESWplQT09&data=05%7C02%7Csteffi.bluemel%40fernuni-hagen.de%7C42fe776c1f324ba6b7aa08dc223844de%7Cc2bba033db424b299adb6ac379499ef7%7C1%7C0%7C638422872265614676%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=X%2F4UFs7fFH%2BIDLw7SsgOOqv3FUyNoCPE7k9B7YFDaGE%3D&reserved=0<https://fernuni-hagen.zoom.us/j/62388524887?pwd=WmJRRUpDZWZ4WlJoeFlGQTBESWplQT09> --- Current Scheduled Talks ---- * 19th December, Joris Plettscher, BSc defense * 09th January, Lucas Filipiak, BSc proposal * 16th January, Roman Ziske, MSc proposal * 30th January, Ute Schmid, invited speaker If you do not wish to receive messages from this mailing list anymore, please let me know by simply responding „unsubscribe“ to this mail (steffi.bluemel@fernuni-hagen.de<mailto:steffi.bluemel@fernuni-hagen.de>). Best regards, Steffi Blümel - Sekretariat - ---------------------------------------------------- FernUniversität in Hagen Fakultät Mathematik und Informatik / Lehrgebiet Künstliche Intelligenz Universitätsstraße 1 58097 Hagen Fon: + 49 23 31 - 9 87 40 06 E-Mail: steffi.bluemel@fernuni-hagen.de<mailto:steffi.bluemel@fernuni-hagen.de>