Connectionist Knowledge Representation

 Scientific Discovery Through Large Language Models or Knowledge Systems?

This important paper could be a foundation for a bridge between connectionist and explicit knowledge representation.

In the last decades, the focal point of AI moved progressively farther away from the knowledge-driven, descriptive systems. Instead, the center stage occupy huge deep learning connectionist models. As a result, R&D for scientific discovery is facing new opportunities for #GPT-style implementations and, at the same time, less scientific interest in logic-driven solutions. 

Many of the fundamental design assumptions behind computational scientific discovery remain valid. For example, the representation of scientific laws and theories is still a foundational problem. We can explore deep connectionist models but the interpretation of simple models that semantic concerns of the relationships between activation patterns and the real world remain a very new research area. On the other hand, decades of research focus on model-theoretic semantics of predicate calculus and logic, so we know the complexity of the issues quite well. 

One of the most powerful methods in "classical" systems is analogical problem solving and the simulation infrastructure for applying base solutions to new target problems. It is unclear if black box system approaches will be practical because spreading activations between explicit concept representations differ from connectionist methods. However, many convolution-based deep-learning applications provide a certain level of insight into the training process. The new research focus on deep connectionist methods for extracting logical consequences from current beliefs and facts, and adding them to the set of beliefs will surely be an important research area.

Cloud computing has enabled massive models despite the computational cost. So perhaps, at this point, concerns about the very high price of theorem-proving systems are no longer valid. We probably need more emphasis on theorem-proving, deduction, induction, and extralogical models. Perhaps concept formation from connectionist models would be a great start.

#deeplearning #logic #deduction #abduction #deduction #computationalscience #problemsolving #rulebasedsystems #predicatecalculus #theoremproving #neuralnetworks #AI #machinelearning 

https://arxiv.org/pdf/2110.13665.pdf