Same Tools, Same Deadline, Scores From 17 to 81: When Vibe Coding Fails

Twenty-seven people. The same tools, the same deadline, the same amount of help available. The scores ran from 81.4 to 17.3 out of 100.

That spread is the most useful thing I have to offer about vibe coding, because it kills the two comfortable stories at once. It is not that AI makes everyone equally capable. And it is not that only people who could already code got results.

Where the gap opens

Category	Top 7 avg	Bottom 7 avg	Gap
Technical functionality	16.8 / 20	4.7	3.6×
Visual design & UX	22.4 / 30	6.9	3.2×
Responsive & accessible	12.6 / 20	4.8	2.6×
Content & storytelling	18.9 / 30	11.0	1.7×

If you want one sentence to take away: the parts of the work that a model can complete without you being present are the parts where everybody looks similar. Everything else is still yours.

Five patterns in the bottom half

1. Never left one file — 67%. Eighteen of 27 shipped a single HTML page. Not wrong in itself, but it caps what the site can ever become, and it usually means the structure was never designed.
2. Zero accessibility attributes — 41%. Eleven sites had no aria anywhere. Nobody asked for it, so nothing produced it.
3. No responsive rules at all — 33%. Nine had not a single media query. They looked fine on the laptop they were built on, which is the only device that was ever consulted.
4. Thin content — 37%. Ten had under 2,500 characters of body text: a page that looks finished and says nothing.
5. Nothing stored — 44%. Twelve stayed entirely static. Fine as a choice, expensive as a default — and it is what dragged the functionality score down hardest.

Every one of those five is a specification failure, not a model failure. Ask for keyboard operability and you get semantic buttons. Ask for a phone layout and you get media queries. The model did not refuse; it was never asked.

What the top group did differently

I went back through the highest-scoring work looking for a common technique. There is one, and it is duller than anyone wants.

• They asked for behaviour, not appearance. The top site’s brief clearly specified what things had to do. It came back with a 3D model viewer and a working robot simulation on a separate page — because someone asked for those in the first place.
• They kept going after it worked. The bottom half stopped at the first result that rendered. The top half treated that as a draft.
• They built structure. Six pages with a separate module for the message board, rather than one file with everything in it. That is a decision made before the first prompt.
• They tested on something other than their own laptop. This shows up as media queries and as aria, and it is almost perfectly correlated with the final score.

So when does vibe coding actually fail?

• When you cannot tell whether the result is good. This is the real ceiling. You can build far beyond your ability to write code; you cannot build beyond your ability to judge.
• When you stop at the first version that runs. The first draft is the model’s guess at the average of your vague request. Its whole value is as something to react to.
• When nobody wrote down what it must do. Function is the box people skip, and the one the scores punish hardest.
• When it has to be maintained. Code you did not write is code you do not know. In six months you will read it for the first time, in a hurry.

If you are at the bottom of that range

The distance from 17 to 81 sounds like a chasm. In practice it is four habits, and each one is available today:

1. Fill the Function box. Write what it must do before what it must look like. The four boxes are here.
2. Open it on your phone before you call it finished. One minute; recovers most of a category.
3. Ask once more. Whatever came back, say what is wrong with it and get the next version. The gap between one round and three is enormous.
4. Run the five-minute check. Drive it, break it, check what you never asked for.

Frequently asked questions

Were the low scorers just less experienced?

Not in a way that predicted the outcome. The strongest work came from people who could not have written it by hand, and some of the weakest came from people who could have. What predicted the score was how many rounds they went and whether they checked on a second device.

Is a 50-point site a failure?

No — it was the average, and it usually means one category collapsed while the others held. The rubric’s value is that it tells you which one.

Does this mean AI is overrated?

The opposite. Everyone in that room produced a working, deployed website, and 94% of those sites were still online a year later. The floor moved enormously. It is the ceiling that still belongs to the person.

Is vibe coding a skill, or just typing?

The 27 scores answer this. Same tools, same brief, same deadline, and a spread from 17 to 81. If it were only typing, the results would cluster. What the top group had was not talent — it was persistence past the first working version, and specificity in what they asked for.

Is vibe coding hard?

Getting something on screen is easy, and that is the trap. The measured difficulty is all downstream: 44% never got a database working, 41% shipped nothing for accessibility, 33% never made the page survive a phone. None of those are hard problems. They are problems nobody asked the model to solve.

The one line

AI levelled the writing. It did not level the working. The difference between 17 and 81 was not who could code — it was who kept asking, and who checked.

Method: 27 submissions from one university course, scored from source code against a four-part rubric in August 2026. Individual work is not identified, and no student is named.