Seems like proctored, handwritten essay exams are the best way to deal with it all.
Yeah but then someone has to actually read it to grade it
Then use locked down computers that doesn’t do anything, but display the exam and student can select multiple choice and type where prompted?
Its all solutions that existed before AI. Nothing has really changed over the past decade when it comes to not allowing or restricting electronics during exams other than schools forgetting they can do that.
Students looking up answers over the internet or getting text messages were just as much of a problem as AI if exams allowed students to bring those devices and use it freely. Even graphing calculator usage was restricted by smarter teachers aware of its note storing and program capabilities.
So the problem here is degree inflation. 20 years ago when I was getting my first master’s degree it was a 5 years of in person classes. In the meantime degrees became required for lots of positions. Teachers need to get additional degrees to get promotions, you need a degree to apply for a job in the public sector and so on. So what used to be a proof of your qualifications became just a certificate you need to get. Now I’m getting my second master’s degree and it’s a year of online classes, without any in person evaluation. We will either start distinguishing between bullshit degrees from online universities and real ones or all of them will lose any value.
I work in higher education at legitimate public colleges, and increasingly, many degrees feel like little more than credentials sold for tuition. Instructors may never meet their students, courses often consist of recycled online content and multiple-choice quizzes, and students commonly rely on ChatGPT.
The quality of instruction can be equally troubling. I have seen someone earn a cybersecurity certificate and begin teaching the subject the following semester, while another instructor requires programming assignments to be printed on paper. After working at two reputable colleges within a large public system, and seeing the poor quality of my wife’s recent master’s program, it is difficult not to view much of higher education as a sham.
It’s been a sham for a long time now. I would argue it’s basically always been a sham, little more than a way for the upper class to mingle and gain connections.
It had a short run of time where it had a general use case for your avg person, but that in the grand scheme of things was the expection not the rule. It has just gone back to being a sham entirely.
I saw rants very much like yours 30 years ago - before you first masters…
A masters has always to my knowledge been one additional year after you have a 4 year degree. So that they only require 1 year for the second degree is expected - you already have a 4 year degree, a masters is one additional year of advance study on top of it.
Yes, degree inflation is nothing new. I remember my uncle telling me how difficult his high school graduation exams were and how easy they got later.
We already had private school giving diplomas to politicians without attending any classes (I know about such instances in Spain and Poland). With AI getting some type of a degree will be just a formality for most people. No idea if it’s an issue or not. If employers want to use degree as a proof for anything they will have to verify which ones mean something. If they don’t care then I guess no one will.
Depends. 1 for many vocational degrees, 2 for most academic or engineering. Also usually only the 1s accommodate completion by working professionals. 2s tend to assume full-time study.
The gold standard would be to make each student do an oral defense of their work. Hand-written exams are good too. Both of these approaches are time-consuming to mark, though.
Assessment design can require students to perform analysis of rare and obscure source material, so that there are no published materials for AI to draw on (which can make it struggle). Similarly, asking for a good bibliography that lists the sources used in the text and formats it correctly would show up the hallucinatory tendencies of AI, I think — though I haven’t tried the latest models.
Agreed 100%.
Every exam should have some oral component. Doesn’t need to be the whole of one’s grade, but 10% sounds about right: significant enough to be impactful, but not so significant that people with severe anxiety can’t otherwise pass.
My biggest fear was this. I hated getting up information of a class
Same, but it motivated me to do the work, and it’s a skill that crosses over into work life significantly.
Doesn’t scale.
It works great. That’s basically how it’s done in aviation, flight instruction is inherently 1 on 1, so I’d assign my students to do some reading and then it was pretty obvious if they’d done that the next flight based on who was asking questions. The ones that did the reading will come in asking me to clarify something, if they say “Yeah I read it” and I have to do the clarifying they didn’t read it. Oral and practical tests are the standard, show me you can fly the plane.
To make that work, most professional pilots have to be flight instructors at some point. It’s almost certainly the first flying job you’ll get after earning a commercial certificate, and thus spend at least some time doing that job. You’re not going to get that in most college classes, so where are you going to get the labor necessarily to give an oral exam to every student in every class?
Yes — I think if I were to teach a large undergraduate class tomorrow I would go with a combination of handwritten exams and, for homework assignments, critical analyses of the most obscure examples of whatever is being studied possible so that AI has no idea what you’re even on about (in my field, ask it to analyse music; it’s hopeless). Multiple choice is fine for stuff that can be rote learned.
I can think of a couple of rules that could prevent a lot of the more blatant cheating:
-
All computer generated coursework must have some form of change tracking enabled, thus the assessor can see the way the essay was written.
-
All computer generated coursework must come with an accompanying oral assessment where the student has to discuss their work and research methodology.
The former would make it difficult to copy directly from ChatGPT, as the lack of any editing would leave an obvious fingerprint
… And passing the latter would require a level of understanding that would be difficult to replicate if you hadn’t done the research yourself.
Passing the latter is basically the same step as a tradesperson needs to complete to show they know what they are doing. It is completely reasonable and would also catch plagiarism and paying some other person to do the work in addition to ai.
-
remember these?
Yeah, it’s a made up problem. In my experience, generated by two factors: universities wanting to increase the number of tests for each subject, and wanting students to be experienced with using computers. Thus many tests become at home, with the computer, where cheating is soooooo easy.
Yeah, cheating with electronics is nothing new. Laptop and phones and internet has existed long before AI, and people could get outside help through messages to have people feed them answers.
Yet, schools dealt with it by either doing in class paper and pencil exams, or digital in class exams on locked down computers with no internet access and no desktop access with software like examsoft.
I’ve had even math teachers ban graphing calculators or provide use of in class graphing calculators well aware of notes that could be typed into them and programs made to solve answers before hand.
So all these current issues seem to be the result of either schools moving to take home, or letting people use personal electronics with full internet and file access during exams. Its like teachers lost the ability to say no to what is and isn’t allowed during exams.
In class graphing calculators would be good.
Our teachers instead manually wiped our ti-83+ before the exam.
Fortunately for us, we had a program that emulated the system menus, so the teacher could think they were resetting it without actually doing it.
Some people used it to cheat but most of us just didn’t want to lose all our games.
Yeah, I think archiving the programs would lead to surviving a reset.
They didn’t. There was a type of reset that restores the calculator to factory settings iirc
Looks like that type was ‘All Memory’
https://education.ti.com/en/customer-support/knowledge-base/ti-83-84-plus-family/product-usage/34871
As someone who is terrible at project work but great at tests, maybe now is finally my time to get a degree. Because tests seem the only thing you can’t cheat on now.
Reap what you’ve sown, assholes.
What are you on about? This is about how universities have a hard time managing AI use and misuse by students. Have they sown those AI tools?
The essence of an academic paper is that a student shows that they have learned all the goals that were set for that course or courses. If a student lets tgere paper be generated by AI, they can not prove that they have learned those goals. That is why universitues have (or should have) strict AI guidelines, so that students know which kinds of use are allowed and which are not.
This is a hard nut to crack. Of course you will see overfitting and underfitting until someone somewhere fimds a universal approach. I personally think AI-detection tools are shit, but you cannot fault universities for trying it. My university handles AI fraud manually, with a committe. That is very labor intensive. We only have a couple hundred students, but if you have thousands, it quickly becomes impossible.
A lot of unis give students free AI access. Companies like Microsoft bundle Copilot into their education packages, so that students will graduate without learning to write essays on their own. Then they’ll be working, paying professionals who can’t go without AI because they didn’t actually earn their degrees. It’s in the business model.
I thought that they were trying to get at the hypocrisy of using ai tools to detect ai writing, which has a dismal success rate and ends up with a crazy amount of false positives.
I see that the growth of computer/digital testing directly led to student abuse of ai. Once students were no longer called on to write a blue book or even just generate responses to short answer questions, we failed our students. I understand that term papers and larger writing assignments have their place, but there really is nothing like being able to sit down and show your knowledge in an on the spot essay (at least for a lot of humanities courses where I see ai being abused).
A systemic issue in this space comes from universities and licensing entities giving testing companies carte blanche to race to the bottom in terms of effective testing tools. For licensing, apprenticeship/mentorship and observation are always better indicators of on the job performance, but that doesn’t have the ease and profit margins of big bullshit testing, so companies would never go for it unless states mandated such pathways. With regard to a lot of current digital university testing, a lot of that was Frankensteined from existing licensing exam tools/strategies and similarly does not effectively assess student/candidate aptitude.
Even if this was not the intent of the commentor, my argument is that carelessness and disingenuous/gatekeeping testing practices from universities and licensing institutions certainly bred a significant level of disengagement in students which leads to them phoning it in with ai slop. If we wanted rigorous work from our students, we should have kept the same standards for ourselves, collectively.
You pretty much got it. I’m a little surprised more didn’t tbh.
Can you elaborate?
Universities started pushing AI as a learning tool. They pay for subscription AI services for enrolled students. So I’m not sure this isn’t them doing this to themselves.
These places that went all in on hyped up “AI” and dystopian surveillance can take their lumps. They’re “realizing” what they’ve been told from day one.
Happy cake day!
