Till startsida
Sitemap
To content Read more about how we use cookies on gu.se

Marco Baroni "Tabula nearly rasa: Probing the linguistic knowledge of character-level neural language models trained on unsegmented text"

Research profile seminar

Marco Baroni "Tabula nearly rasa: Probing the linguistic knowledge of character-level neural language models trained on unsegmented text"

Tabula nearly rasa: Probing the linguistic knowledge of character-level neural language models trained on unsegmented text

Work in collaboration with Michael Hahn

As recurrent neural networks (RNNs) have recently reached striking performance levels in a variety of natural language processing tasks, there has been a revival of interest in whether these generic sequence processing devices are effectively capturing linguistic knowledge. Nearly all studies of this sort, however, initialize the RNNs with a vocabulary of known words, and feed them tokenized input during training. We are instead running an extensive, multi-lingual (English/German/Italian) study of the linguistic knowledge induced by RNNs trained at the character level on input data with whitespace removed. Our networks, thus, face a tougher and more cognitively realistic task, having to discover all the levels of the linguistic hierarchy from scratch. Our current results show that these "near tabula rasa" RNNs are implicitly encoding a surprising amount of phonological, lexical, morphological, syntactic and semantic information, opening the doors to intriguing speculations about the degree of prior knowledge that is necessary for successful language learning.

Lecturer: Dr Marco Baroni

Date: 10/22/2018

Time: 1:15 PM - 3:00 PM

Categories: Linguistics

Location: Department of Philosophy, Linguistics and Theory of Science
T116

Contact person: stergios chatzikyriakidis

Page Manager: |Last update: 9/15/2015
Share:

The University of Gothenburg uses cookies to provide you with the best possible user experience. By continuing on this website, you approve of our use of cookies.  What are cookies?