Project title:
Testing AI as a Rational Theologian
Years:
2023-2024
Granting institution:
Ian Ramsey Centre, University of Oxford
Programme:
New Horizons for Science and Religion in Central and Eastern Europe (CEE)
Project Description:
LLM-powered chatbots can effectively formulate arguments, understand them and reply to them. However, can AI give reasonable answers to the questions about theology and religion?
This project is aimed to outline the possibilities, limitations and challenges with respect to the use of AI as a rational theologian. The main task of the project will be to prepare tests (including scenarios for structured interviews) and benchmarks for AI to measure its theological competence and identify when it becomes a “mature” theologian, as well as a scenario for an advanced interview to be conducted with a mature AI-theologian concerning big questions of philosophy and theology.
The project will help to assess when AI is ready for rational and deep discussion about theological problems and argumentation. It will be a moment in which we will be able to get from AI instructive feedback concerning metaphysical theories, arguments for the existence of God or the status of sacred texts. It will be a valuable material in the debate on the rationality of faith. However, first, we need to be able to measure the competence in theology of bots which are based on particular models, to know how accurate response we can expect from them.
The relevant part of the project will include the following activities:
- Creating the conception of AI-bots survey (including testing bot’s: theological knowledge, resources it uses, sincerity, hermeneutic skills etc.);
- Creating tools: a) scenarios of structured interviews with AI-bots; b) assessment rules and benchmarks for AI theological development;
- Interviews with selected bots (including an advanced interview if any available bot is ready for that).
These activities, first, will result in an answer to the research question 1: how to measure AI competence in theology? Second, we will try to get an answer to question 2: are the existing models sufficiently trained to be rational theologians and to solve theological problems?
If so, we will be able to answer question 3: what kind of valuable feedback about rationality of religion can we get from AI? Finally, we will consider question 4: whether the existing models need additional (theological) training?
Thus, at the end of the project, we expect to know: when AI is a “mature” theologian, whether any existing AI-bot can be helpful in the debate about the rationality of faith, and how it should be trained to be more effective to reach such goals.
Principal Investigator:
Dr Marcin Trepczyński – assistant professor at the Faculty of Philosophy of the University of Warsaw, specialised in medieval philosophy, including logic and methodology of science, as well as the relationships between logic and religion. He was an expert of the Education Research Institute, specialised in learning outcomes assessment. He is a member the management board of the Logic and Religion Association. Publications: ORCID | Academia.edu
Project publication:
- Trepczyński M. 2023. Religion, Theology, and Philosophical Skills of LLM–Powered Chatbots, Disputatio philosophica: International Journal on Philosophy and Religion 1(25), 277-296, DOI: https://doi.org/10.32701/dp.25.1.2.