Test-Prep Advice Post

When charged with a crime you didn't commit, would you hire AI representation, instead of an actual lawyer?  

When filing a complicated tax return, would you allow AI to fill out the forms, instead of a certified public accountant?  

Sure – I might choose to trust a Waymo automated car driver more than a random Uber one, and doctors are known for using technology and computers to aid with surgery.  However, one thing you will notice is that when the task is complex and/or highly consequential, a real human expert is almost always there, for safety and accuracy reasons.

We already know, for example, that chatbots can dispense highly dangerous, unproven advice in response to both physical and mental health concerns — and that despite human aviation's less-than-perfect historical safety record, and the rising usage of autopilot, few would trust AI to fly passenger planes all by themselves.

As an experiment, I recently took a hard LSAT question and a hard SAT question, and entered each verbatim into 3 of the most popular AI engines (ChatGPT, Claude, and Gemini), asking only for the correct answer.  

Guess what?  A highly confident 0 for 6: they chose the wrong answer every single time.  Then, once I corrected these LLM "agents" by providing the actual correct answers, the bots each capitulated, immediately admitting the mistakes in their prior approaches.

My conclusion? AI isn’t qualified to replace professional test-prep tutors like me anytime soon.  And good luck having an actual spoken conversation with these "chat" bots, instead of just relying on their usual text explanations: the voice chat features are even worse.

Another consideration: free written explanations to official SAT, ACT, GRE, GMAT, and LSAT questions have already existed for decades, written by real people like me and hosted on websites such as the PowerScore LSAT discussion forums.  However, students often discover that many of these questions, especially the hardest ones, will still require a discussion with a human expert to fully unlock.

In other words, there is a difference between having access to written (or even video) explanations, and being able to truly comprehend them.  This dichotomy applies to even the best and most detailed explanations, written by true experts such as Bunuel on GMAT Club.  Educational studies have consistently shown that active engagement with the material (notes, conversation, blind review, etc.) is nearly always superior to simply reading a text explanation, or watching a video.   So why are we expecting AI-created explanations, which often "hallucinate" nonexistent facts, are programmed to flatter users and minimize their mistakes, and are mostly derived from already-existing human explanations, to be any better at helping us learn?

In reality, AI question solutions exhibit the same flaws as their human-created counterparts: they are are too complicated, reverse-engineered from the correct answer, over-simplified, incomplete, contradictory, hard to understand and/or categorize, redundant, difficult to execute under the time pressure of the actual exam, et cetera.  And AI videos, though easy to watch, often make for even worse learning tools due to the proven drawbacks of passive learning.  These downsides also apply to AI-written text explanations — namely, poor retention due to low engagement.

Until AI can answer every standardized-test question correctly without first checking the official answer key, and until it can also hold a real, natural, customized, engaging conversation about the nuances of the test and best preparation methods, you’re certainly better off choosing an experienced private tutor for the task. 

After all, though mastery of the exam is a prerequisite, mastery alone is not a guarantee of a skilled instructor who can simplify complex tasks with straightforward and understandable language.  In the same way that elite athletes don't always make the best coaches, students who ace the exam themselves are often surprisingly lousy at teaching it.  This applies to the computers, too. 

The best test-prep instructors are real people with decades of teaching experience, who can not only explain the exam questions in an expert, understandable manner, but are skilled at precisely monitoring the complex emotional ranges and varying attention spans of their fellow human students: encouraging them when they are feeling down, bringing them back to earth with blunt realities when their confidence gets too high, lightening the mood by cracking a joke or changing the subject, asking the right questions at the right times, and motivating students to work harder when necessary.  AI is well-known as a relentless cheerleader for the user, but human tutors understand that students also benefit from constructive criticism, and sometimes even need the occasional distraction or tangential conversation to learn best.

Finally, tests such as the SAT, ACT, LSAT, GRE, GMAT, ISEE and SSAT are assigned outsize importance during the admissions process, and should be prioritized accordingly.

Am I biased?  Of course: test-prep tutoring is how I make a living.  But after 25 years and counting as a pro tutor, I also know that private tutoring gets results — and that in the vast majority of cases, artificial intelligence does not.  You can’t AI your way into a great college or grad school in 2026.  Having access to an (often incorrect) written explanation is not the same as actually understanding and fixing your mistakes by discussing them with an experienced teacher — a real person who can not only help simplify the questions, but who can also challenge you, critique your assumptions, and relate to your struggles on a personal level.  In fact, it's not even close.

Of course, we all use AI for grunt, repetitive tasks sometimes, but standardized test scores are simply too important to trust to a generic computer algorithm: some law-school admissions experts estimate, for example, that for every 1-point improvement in one's LSAT score, students are eligible for an additional $10,000 in scholarships and grants.  Though deciding to use AI for your prep is certainly the cheaper option, that choice may well end up costing you in the long run, both financially and academically.

Enjoying genuine, Socratic discussions with master educators is a key aspect of optimal learning, one that the machines will never be able to fully replicate, because they will never fully understand what it's like to have a human mind and body like ours.

Soon the AI bubble will pop, and we will go back to giving experts in education their proper due.  Until then, I’ll still be chugging along, adding to my 25,000 hours of experience ... and maybe writing a blog post to complain every once in a while.