When the Old Algorithm Wins: AI Architecture, Pricing Wars, and Cancer Diagnosis

A paper resurfacing on Hacker News this week challenges one of modern AI's foundational assumptions. Titled 'Attention Is Not What You Need' — a deliberate inversion of the 2017 transformer paper that underpins GPT, Claude, Gemini, and virtually every major language model — the arXiv preprint from December 2025 argues that the self-attention mechanism has been over-relied upon. The authors propose alternative architectures that, on specific benchmarks, match or exceed transformer performance with significantly lower parameter counts. The HN community was appropriately skeptical — benchmarks can be cherry-picked, and alternative architectures have a long history of promising results that fail to generalize — but acknowledged that even marginal architectural efficiency gains translate to billions of dollars when applied at frontier training scales.

The economic pressure on those frontier labs is already visible in pricing. An analysis published at minimaxir.com drew 324 comments by forensically examining why OpenAI and Anthropic cut LLM prices by up to 80% within a few months. The author identified three simultaneous forces: inference hardware becoming cheaper, model distillation techniques improving so that smaller models can approximate larger ones, and a deliberate race to establish usage habits before the market consolidates. The strategic read is classic platform economics — accept short-term margin compression to build the switching-cost dependency that creates long-term structural advantage.

IBM CEO Arvind Krishna added an enterprise dimension to that picture, telling The Register that open-weight models are now, in his words, 'irresistibly good and cheap on-prem.' The HN thread split predictably: infrastructure engineers largely agreed with the technical premise — models such as Llama variants are capable enough for a wide range of enterprise tasks, and on-premises deployment provides data sovereignty that cloud APIs do not. Skeptics pointed out that total cost of ownership for on-prem AI is far harder to calculate than IBM's framing suggests, noting that hardware amortization, maintenance staffing, security audits, and update cycles often exceed cloud API costs when fully accounted for.

The medical AI finding may be the day's most counterintuitive result. Research from the Lynch Lab showed that a 10-year-old convolutional neural network outperforms frontier large language models at diagnosing skin cancer from images. The CNN was trained specifically on dermatological image data with dermatology-specific labels; the LLMs it was compared against are general-purpose multimodal systems that were not trained for that diagnostic task. The distinction matters: skin cancer diagnosis requires precise categorical judgment with calibrated confidence, not fluent description. The HN thread also raised the regulatory dimension — there is a growing question about whether frontier LLMs that were not cleared as medical devices should be used in clinical contexts at all, regardless of general benchmark performance. The paper provides critics of that approach with concrete empirical ammunition. Perhaps most pointedly: the CNN did not improve. What changed is the comparison point.