Balancing data and human knowledge could fuel scientist-like AI

imeng.vip:03月-10日  Researchers report they have developed a framework for assessing the relative value of rules and data in “informed machine learning models” that incorporate both.

When you teach someone to play chess, you tell them the rules of the moves made by the bishops, pawns, kings, and queens of the black and white board.

Or you could just leave them hanging and expect them to learn it independently. That would take longer and be a bit more frustrating for the individual.

In a similar manner, we can also improve artificial intelligence (AI) by incorporating fundamental rules, like the laws of physics, into their training. 

By doing this, AI could become more efficient and better at understanding the real world. However, determining which rules are most valuable for AI can be difficult.

How both data and rules affect predictions
Deep learning has changed how scientists research by finding important connections in lots of data. However, there are still problems with using only data. 

Generative AI models like ChatGPT and Sora learn solely from data, lacking an understanding of physical laws and struggling with new situations. 

Informed machine learning, on the other hand, offers an alternative, incorporating rules to guide training, but the importance of rules vs data remains unclear. 

So, the researchers developed a framework to assess rule contribution to model accuracy, optimizing by adjusting rule influence and filtering out redundant ones. 

This approach, demonstrated in engineering, physics, and chemistry, improves model performance and can optimize experimental conditions.


“Embedding human knowledge into AI models has the potential to improve their efficiency and ability to make inferences, but the question is how to balance the influence of data and knowledge,” said Hao Xu of Peking University and the study’s first author. 

“Our framework can be employed to evaluate different knowledge and rules to enhance the predictive capability of deep learning models,” added Xu.

Making models work in a dynamic scenario
Data-driven models are good at tasks they were trained on but not so good when things change. On the other hand, rule-driven models do better when things change. 

“We are trying to teach AI models the laws of physics so that they can be more reflective of the real world, which would make them more useful in science and engineering,” said Yuntian Chen of the Eastern Institute of Technology, Ningbo, and senior author of the study.


Combining data and rules gives us moderate predictions in both scenarios, showing a connection between data and rules.

In their study, the researchers focused on three main questions: How do we know if the rules we add are useful? What’s the relationship between data and rules? How can we make rules work better in machine learning?

“We found that the rules have different kinds of relationships, and we use these relationships to make model training faster and get higher accuracy,” said Chen.

“We want to make it a closed loop by making the model into a real AI scientist,” said Chen. “We are working to develop a model that can directly extract knowledge from the data and then use this knowledge to create rules and improve itself.”

The study was published in the journal Nexus.

Study abstract:

Knowledge constitutes the accumulated understanding and experience that humans use to gain insight into the world. In deep learning, prior knowledge is essential for mitigating shortcomings of data-driven models, such as data dependence, generalization ability, and compliance with constraints. Here, we present a framework to enable efficient evaluation of the worth of knowledge by the derived rule importance. Through quantitative experiments, we assess the influence of data volume and estimation range on the worth of knowledge. Our findings elucidate the complex relationship between data and knowledge, including dependence, synergistic, and substitution effects. Our model-agnostic framework can be applied to a variety of common network architectures, providing a comprehensive understanding of the role of prior knowledge in deep learning models. It also offers practical utility for knowledge identification and model construction within interdisciplinary research by improving the performance of informed machine learning and distinguishing improper prior knowledge.