We prepared OSYM Style Exam Questions for Artificial Intelligence ChatGPT!

Recently, ÖSYM President, “Artificial intelligence will create the exam questions.” shaped He made a statement. Minister, human factor Although he has stated that he will not be put on the backburner, it seems that there will be many question marks in mind.

We present to you the most popular artificial intelligence tool today. We had ChatGPT 4 prepare OSYM style questions and asked for their answers. Then, we asked one of our teachers about the results. Let’s see what kind of results were obtained and discuss together:

First of all, you can download the visual of the 2021-AYT/Science question from the ÖSYM website. By giving it to ChatGPT We asked him to pose a similar question:

Note: Since ÖSYM does not allow sharing the question it has prepared, we have to censor it.

Then ChatGPT prepared a question for us as follows:

In fact, it looks very well prepared, doesn’t it? You may not have seen the question we asked as an example because we censored it, but He prepared an almost similar question. He even paid attention to his painting and created an original question. So is it really as good as it seems?

We asked him to solve this question in detail and then give the answer:

To summarize: We can say “Spider-Man pointing at each other” because he found the answer “20”. As well as solved not once but twice and again reached the same conclusion. After finding the answer in the first one, he realized that this answer was not in the options and contradicted himself. “There is no this option in the question, so there must be an error in the solution.” he said and started to solve it again.

After exhausting his brain for a long time, he found the answer “20” again and “The question is incorrect or there is a typo.” he said and talked to himself. We all think, “Well, you prepared the question, how could the question be wrong?” we said and laughed a lot. There’s another funny side to this…

We reached out to Chemistry Teacher Muzaffer Baz to confirm the “20” answer he found, and we were even more surprised when he gave us the following answer:

“Now, there is a logical error at the beginning of the question. The beginning part of the question is not clear. Is there a solution at first or not? Or does it become saturated when you add 40 g of salt to the water? None of these are stated. At first, I thought the question was like there was salt in the solution. I solved it and the result was 1300/85=15.29 and there was no integer.”

The reason why artificial intelligence found the answer wrong is probably as our teacher said. It’s because the problem is wrong. Because if you go up and read a little bit, he tried to solve it based on assumption and got confused. In addition, our teacher also gave the following answers despite the questions being prepared with artificial intelligence:

“Questions need to be created by humans, not artificial intelligence, because the person who will analyze the question is a human. For this reason, students need to be asked questions in a more understandable language. Questions that can confuse the student and cannot properly explain what they want are not of high quality. In addition, questions made with this system are not of high quality.” The difficulty level may not be determined in an exam. Taking everything into account, the current situationartificial intelligence with iIt does not seem possible for him to prepare questions in any way. Of course, if a question pool is created by humans in the future and an artificial intelligence trained with this question pool creates questions, a more comprehensive, beautiful and non-misleading exam will emerge.

We thank him for his answers.

Maybe some of you will say, “You asked the question visually, try it in text as well.” We gave him another chance and had him prepare a math question:

We were initially hopeful because it was a math question, but the result was not as we expected. question Let us write in text and change the prompt. However, this time we faced a much worse result. In fact, this time the result was so bad that it even caused an error.

“Make sure that the answer to the question you pose is in the options.” As we said, he started solving the question as soon as we posed it and He couldn’t get out of it.

As a result, artificial intelligence does not seem to be a reliable source right now.

Yes, artificial intelligence is very popular today and will definitely reach incredible levels as time goes by, but as it is, it will not be possible to answer the questions prepared by our teachers. does not have the competence to prepare.

Artificial intelligence is now more 2-3 years old And like a baby who is just learning everything. You give it something, you educate it and teach it. In return, he gives you similar answers as you taught him.

We used ChatGPT 4 here and without any training We asked him to prepare similar questions. Maybe if we had asked OSYM to ask more than one question and learn the question styles and writing styles, the results would have been different, we do not know.

The version we are currently using can pull data from the internet, but it cannot act on its own. Even Even if it pulls data from the internet, it sometimes makes up the information may give incorrect information. Once upon a time, “Who Wants to Be a Millionaire?” He also gave wrong answers to some questions of the competition.

We do not know exactly how ÖSYM will handle this issue.

Presumably they will not do anything like we did. If what we don’t know is in the background have a proper data set, are trained and designed If there is a system, they can achieve much more efficient results compared to these simple examples. Therefore, it may not be logical to immediately come to the conclusion we stated in the title.

There may come a day when the obstacles to artificial intelligence will be removed. a decent learning algorithm If it can be developed and can quickly learn and comprehend every data on the internet or the world’s agenda, then perhaps they will be writing these articles, not us.

What are your opinions on this subject?

Our other artificial intelligence content that may interest you: