Are LLMs Terrible at Gauging Efficiency?

by Rian

A theory that I had for sometime, originally not for LLMs, but for those who believe LLMs can write perfect codes. LLMs care little about the environment on which the codes will run on. I know the statement alone makes no sense — how can it be considered coding, if it doesn’t work on the environment? It’s not that it won’t run, it’s just that it will run extremely poorly.

I had several vibe coding projects during the migration from bare bone filesystems to proper indexed databases. I could have just imported them, but it won’t feed any metadata that I have personally built. I was in the process of writing Python scripts to prepare the migration, only to realize I can speed up the process by explaining the algorithm, APIs I want, and how my original metadata DBs were structured to the LLM. After jumping back and forth between Gemini and Claude, I chose the lesser evil of the two: Claude.

Claude is not the perfect AI. Sure, it doesn’t go around silently erasing the old parts of the scripts whilst working on somewhere else. There are paper trails I can look up, which I find crucial. The product of Claude, however, is counterintuitive. I can’t think of a better way to describe Claude’s current behavior but to call it that. The only sensible way to get one full coherent and somewhat efficient code is via writing a huge prompt, or as Claude puts it, Planning mode. This approach burns through tokens. I would have assumed Claude would focus on modularizing the code base instead to keep the token cost minimal. It doesn’t. In fact, in several occasions, Claude pushed back on the idea initially.

Outside of the whole token balancing act, it has no measure of algorithm efficiencies or future-proofing. Those are the suggestions users (or programmers) have to throw in with the their own unique touch. I won’t deny that’s part of the fun. But at the same time, it is beyond bizarre sometimes. Once, I needed to import files over via API to another DB. Instead of doing handshake once, then transfer all the files over, Claude’s version decided to re-do the handshake per file. That was the whole loop. It will work. As the famous saying goes, if AIs are so smart, it may need to turn itself off as the most inefficient system.

From the angle of sci-fi fan, efficiency is probably the most objective value we all share. Nobody wants inefficient codes. It makes me wonder why these LLMs are not trained to write or implement efficient algorithms. Sure, I can remind Claude to do handshake once and continue operations from opened connection on every DIY projects I do. But this is an objective value even an AI model can appraise. Use Big-O notation for all I care.