I Taught a Course Heavily Written by AI. Here's What Went Wrong.

In 2013, I left my job as a software architect and went all-in on coding education, launching one of the first .NET and Java bootcamps in the country (US). I have over a decade of experience taking people from zero to entry-level professional. So, when I was engaged to teach a full-stack bootcamp experience earlier this year, I was excited to get back to my roots. 17 learners, intensive training, big-name client, high stakes for the participants.

What I didn't expect was that most of the curriculum had been written and assembled by AI, without a qualified subject matter expert (SME).

This could have been a complete disaster, but fortunately, I was able to play some “hero-ball,” and my students loved the course and achieved their learning goals. In fact, with a weaker instructor or in a self-paced format, it would have been an expensive failure that damaged careers and confidence.

Let me explain what went wrong, and then I’ll share insights into how I responsibly use AI to create educational experiences.

The Red Flags Appeared Immediately

I always test-drive curriculum before teaching it. I work through every exercise, take every quiz, and trace the learning path from beginning to end. Within minutes of digging in, I had one eyebrow raised. Within hours, I was altering my approach to start planning supplemental and alternative content.

As a responsible AI user, I recognized slop when I saw it.

So, let’s talk about the most glaring issues. This isn’t a comprehensive list, but it shows what can go wrong without proper involvement by subject matter experts and a quality assurance cycle.

Sequencing Issues

Right out of the gate, one of the first group activities was coding a calculator using methods, loops, and conditional logic. This was scheduled right around “Hello, World.” This was the first time I “went rogue” and scrapped the activity entirely. It would have terrified beginners who hadn’t even learned what a variable was yet.

One thing I’ve learned over the years is that learners with engineering mindsets get outright distressed by “magic”. For the best learner experience, you must take care to introduce concepts when they’re ready to understand them. Jumping ahead or introducing concepts that are abstracted creates unnecessary anxiety.

Another example of this was during the object-oriented programming unit the learners were introduced to domain driven design and model-view-controller patterns without having first created anything complex enough to justify using the patterns. In particular, we were still in the terminal, so MVC was completely inappropriate. The result was me having to rein in many of the learners who were mixing patterns and creating a mess.

The most egregious issue was the final capstone instructions. I noted that it called for a React application communicating with a Java back end. However, the course content was completely missing Spring Boot and how to create REST APIs. The final project was literally impossible to complete with the provided curriculum. I spent a weekend building out this content so the learners could succeed.

Technical Hallucinations

Most data shows that AIs hallucinate anywhere between 30 to 70% of the time on tasks and this showed up in the content in a big way. Sample code frequently didn’t compile. It referenced packages that were out of date or didn’t exist. Quiz questions mixed SQL syntax from different database systems. Students frequently would copy code examples and get compiler errors and then assume that they did something wrong.

Even more damning were code snippets and examples where the code provided and the sample output didn’t match. Early stage learners lack the capability to understand whether the code or the output is wrong. This was highly frustrating for the learners. Many of them abandoned the LMS content completely and I frequently had to rewrite exercises so that they made sense.

As an instructor/mentor, I frequently spend time reassuring learners because programming is hard and it does take a lot of time for patterns and understanding to click. When the content is wrong, this increases self-doubt and imposter syndrome. A good mentor can address this with patience and empathy. I want my learners to enjoy the craft as well as building competency. These types of issues in the content actively work against both.

Context Blindness

This was the smoking gun that screamed “AI Generated”. A multi-lesson shopping cart application lost its way completely. Early lessons had students adding data and functionality that later lessons completely ignored. It even went so far as to restructure the application completely mid-sequence. It was as if each lesson was written in isolation, unaware of what had come before or was coming after.

Another notable gap was a lack of fundamental setup and explanations of the coding environment itself. Command line Git? Fine. But IntelliJ really really wants to integrate Git into the project. Students ended up fighting with their own tools. There was little in the way of instructions about IDE configuration, file organization, or where to place resources in a project structure. This made working with file data more difficult than it needed to be.

Outdated Knowledge

Beyond these structural issues, the content itself showed clear signs of being trained on outdated sources. Much of the content used legacy versions of the Spring framework and Python. Useful, modern features like Record types were completely absent. The course failed to cover dependency injection alongside interfaces, which is one of the most critical professional developer patterns.

Playing Hero Ball

By the middle of the first week, I made the decision to only loosely follow the scripted curriculum and hit all the learning objectives from experience.

I estimate that I replaced or supplemented more than half of the planned learning experience. Each week, I’d review the intended learning outcomes, prepare demonstrations that actually made sense, and let students work through the LMS content (or not, as they didn’t find it useful) while focusing on practical and professional patterns that would serve them in real jobs.

I ended up sharing a lot of this content with another instructor teaching a different cohort. They were struggling and didn’t have my background and experience to draw from.

What the Students Reported

The student surveys told the story perfectly. The learners loved the instruction but were critical of the content. Here are some samples when prompted about the experience:

"The exercise instructions were confusing at times or just not available."

"Some of the lessons could be more clear about how some of the output should look like. Also some of the quizzes are marking the wrong answers as correct, like some of the quizzes in Java OOP have questions about MVC when it isn't mentioned anywhere until the next module."

"The material and exercises are very confusing."

But there were also comments like this: "Honestly my instructor has been the best part about the course. His knowledge and experience allows him to pivot or add to the lessons to help us understand the tasks we might face on the job."

These students succeeded in spite of the curriculum, not because of it.

Using AI Responsibly in Education

There are responsible ways to use AI when it comes to education. In fact, I use it every day. It can not replace qualified instructional designers and subject matter experts. However, here are some ways where it does speed things up:

• Outlining Assistance: I compare AI-generated learning objectives with my own drafts to catch gaps (expert blind spot) that I might have missed.

• Language Refinement: If you know me, you know that I can be wordy and use advanced vocabulary. AI is a useful tool in helping me identify these issues so I can adjust to appropriate reading levels and cut some fluff.

• Code Sample Generation: AI can produce boilerplate and beginner friendly code faster than I can type it. However, I always need to review and adjust for consistency. It also frequently needs reined in because it wants to introduce advanced features (e.g. Python list comprehensions) before students have mastered the basics.

  • I’m very much against “magic” in code. My learners can use shortcuts when they fully understand what the shortcut is doing!

• Summarization: AI is genuinely useful at helping me write concise conclusions and key takeaways.

• Knowledge Checks: Given a well-crafted lesson, AI can draft useful flashcards and assessment questions. Again, these must be carefully reviewed and edited. In my specific cases, I toss out about 40% of the output, substantially edit another 40%, and 20% is good as-is.

Between my own creative process and delivering someone else’s AI content, I am confident that AI works as an assistant to expertise, not a replacement for it.

The Hidden Stakes

This year, I’ve seen substantial pushes by big tech to get these tools deeply embedded into education. From a business perspective, it’s a smart move because education providers are often more prone to vendor lock-in than others. As the industry continues to struggle with profitability, capturing education is a compelling path forward.

But here’s what is keeping me up at night: in a self-paced environment or with an instructor who lacked deep subject matter expertise (TA’s in colleges anyone?) the majority of my students would have struggled or failed entirely.

Worse yet, many of them would have blamed themselves for not understanding confusing instructions. They may have gotten stuck on impossible tasks and assumed they just weren’t cut out for software development. Or, perhaps they would have turned to AI tools, which are quite capable of generating beginner level code solutions and then entered the job market without critical comprehension of the fundamentals, setting them up for failure in their first development roles.

The AI-generated content looked professional at a glance. It wasn’t until I started going through the material as if I was a student that the issues emerged. On the whole, it lacked the deep understanding of how people actually learn to code and what skills they need to succeed professionally.

The Bottom Line

AI can accelerate aspects of instructional design, but it cannot replace the human elements that make education work: subject-matter expertise, pedagogical knowledge, and quality assurance by people who understand both the content and the learners.

I was able to save this particular course because I had the experience to recognize the problems and the expertise to fix them on the fly. But that shouldn't be necessary. And it certainly shouldn't be the plan.

So, my message to education providers is to beware. Technical topics like this require expertise to deliver at quality. Don’t fall victim to unscrupulous providers who offer experiences that look reasonable at a glance, but are poorly crafted slop under the hood. Before purchasing any curriculum, I recommend that you have subject matter experts and qualified instructional designers review it.