At MIT's 2011 Brains, Minds, and Machines symposium celebrating the institute's 150th birthday, Technology Review reports that Prof. Noam Chomsky criticized machine learning researchers who rely on purely statistical methods. Chomsky argues these researchers produce behavior mimicking the world without attempting to understand its meaning. He interprets scientific success as approximating unanalyzed data, a notion he believes is novel in the history of science.
The remarks were a response to Steven Pinker's inquiry about probabilistic models trained with statistical methods. Chomsky posits that statistical language models have had engineering success but this is irrelevant to real science. He compares accurate modeling of linguistic facts to butterfly collecting, asserting that underlying principles matter far more.
He contends statistical models are incomprehensible and provide no insight, arguing people do not decide sentence words by consulting probability tables. Instead, he claims humans map from an internal semantic form to a syntactic tree-structure, linearized into words, without probability or statistics. Furthermore, he asserts statistical models have proven incapable of learning language, suggesting language must be innate and questioning why researchers pursue what he views as the wrong enterprise.
The debate continues, with responses agreeing engineering success is not the sole goal of science, but noting it serves as evidence of scientifically successful models. The science of understanding language is viewed as no different than other sciences, where fact accumulation is the dominant mode. Responses acknowledge the difficulty of interpreting models with billions of parameters but suggest insight comes from examining model properties.
While agreeing a simple Markov model cannot cover all language, the counter-argument proposes a probabilistic model covering words, syntax, semantics, context, and discourse is necessary, rather than dismissing all probabilistic models due to shortcomings of a 50-year-old model.
Source: Hacker News · Summarized by HeadlinesBriefing