There is a lot of legitimate concern over the net effects of students using AI to do their school work. If you let AI do your writing, then you will not learn how to write, for example. Or worse – if you let AI think for you, then you will not learn how to think. I have spoken with many teachers at many levels about this and they range from those who take diligent steps to minimize AI “cheating”, to the other end where some throw up their arms and essentially say, “if students us AI to cheat, they are just cheating themselves out of an education.” For me the bigger question is – what happens to society if we raise a generation intellectually crippled by their dependence on AI?

Certainly, the educational system at every level needs to deal with the reality of AI, and needs to protect the legitimacy of their methods for evaluating students. But also, schools need to face the reality of a world with AI and prepare their students for that world, which may require leveraging AI itself as a teaching tool. In fact, using AI as part of the teaching itself may secondarily solve (or at least mitigate) the AI-cheating problem. For example, if you develop a workflow that includes human and AI elements, then use of AI is already baked in and accounted for. Ideally, the outcome will be optimal when the human adds value to the process and uses AI effectively. The outcome should be better than AI alone, or human alone, could generate. And if you have to show your work throughout the workflow, it becomes hard to fake your contribution to the process.

I don’t think researchers were even thinking of the AI-cheating problem when they developed this AI assisted workflow to teach undergraduates how to develop proper research questions. They were trying to more effectively teach students how to develop research questions. But their model may provide a useful template in many other educational contexts, and also help the AI-cheating problem.

Here is the problem they were trying to address – undergraduate students often have a difficult time developing useful research questions. It’s a lot harder than you might think – “Students can usually name a topic they care about, but often lack strategies for turning that interest into a question that is grounded in evidence, scoped to what is feasible, and aligned with available methodologies.” This has long been identified as a bottleneck in undergraduate research education. Developing a solid research question is often the most critical step in any research, and is often the most challenging thing to do. 

What these researchers did was operationalize the process of developing a research question into a specific workflow, using human only and AI-assisted steps along the way. The goal was primarily to help teach students what those steps are, improving their critical thinking skills with regard to research questions. It also replaces a model that is more intuition-based – dependent on a moment of inspiration – with a technique that anyone can apply. Here’s the workflow:

First, select a broad topic of interest (human only). Second (AI-assisted) make a list of subtopics to help narrow the potential research topic as much as possible. Third (AI-assisted) build a list of keywords and search terms that relate to the narrowed topic. Fourth (AI-assisted) conduct a literature search using those terms and synthesize that literature into what is currently known. Fifth (AI-assisted), identify a gap in the current knowledge of that narrow topic. Sixth (AI-assisted) turn that gap into a testable question. Seventh (human only) verify all claims up to this point against the literature. And finally eighth (human only) seek peer-feedback from experts in the field.

It is also critical to understand how the AI-assistance works. The AI is called a Socratic challenger. What it does is ask questions of the user to challenge any of their ideas. It does not just generate their work for them – it challenges them to justify and question their work, to iterate it and make it better. AI is acting as a thought collaborator, challenging the user to question and revise their ideas.

The study followed students using the software over the course of a semester where they developed a specific research question. They wanted to know what students thought of each step of the process, to see if some worked better than others. All of the steps, on average, were found to have “helped a lot”. They also found internal consistency in the results. Qualitatively students found that the AI was an effective “thought partner” helping them improve and refine their ideas throughout the process.

The authors partly conclude that the key to the success of this method was that it was structured – it involved a workflow of 8 specific steps with AI having a defined role in each step. Prior literature shows that using AI research development tools in an unstructured way can lead to “weakly justified” conclusions and “artificially hallucinated” outputs. This is because LLMs are great at mimicking the structure of human output, but this can be very superficial and not anchored to reality.

Interestingly, nowhere in the paper do they discuss the “sycophancy” problem with many LLMs. When I have used LLMs as a “thought partner” for some creative projects (mostly just to experience first-hand how they work) I have found that to be the biggest problem. All of my ideas are simply wonderful. I have to actively work against it. But in this study it is implied that the Socratic challenger method inherently solves the sycophancy problem because the entire point is to constantly challenge the users ideas.

I found this study to be very provocative on many levels. First – the process of developing research questions is very interesting to me. I have encountered it as an academic, and I am constantly aware of it as a science communicator. It is an underappreciated (by the general public) but critical aspect of scientific research. And second, I think the best way forward through the AI issue with education is mostly (but not entirely) to leverage AI to improve education. Make it part of the process. But you can’t simply just throw it into the mix without thinking carefully about how it will be used, evaluating its use, and then making adjustments.

In the end, the advent of LLMs and modern AIs may force us to think more deeply about thinking (always a good thing, in my opinion). AIs don’t really think, but they can superficially look as if they are, creating a simulacrum that can be impressive but is ultimately shallow and soulless (the AI slop phenomenon). What is it that humans are doing that AIs are not doing? How can we operationalize that? How can we teach students to be better critical thinkers, to be more creative, and to produce output that is clearly above the level of AI-slop? That would be the best outcome of the disruption of AI, and one worth fighting for.