"The limits of my language mean the limits of my world" - this iconic quote by Ludwig Wittgenstein has materialized today as a purely technical term in LLMs: the Language Tax.
Who was Wittgenstein?
Ludwig Wittgenstein (1889–1951) was one of the key thinkers of the 20th century who flipped the philosophy of language on its head twice. First, he reduced thinking to rigid mathematical logic, and later, he proved that language is a living system of "language games," where the meaning of any word depends entirely on the context of its use.
It is fascinating how the works of great minds from the past, created decades before silicon processors, suddenly turn out to be highly relevant for understanding today's cutting-edge technologies. Century-old philosophy now literally describes how deep neural networks operate, where the morphology of a specific language physically defines the boundaries of AI logic and the efficiency of its computational graph.
English (analytical): Strict word order, minimal morphology. Most concepts are encoded into exactly 1 token. The model doesn't get distracted by decoding the linguistic shell and directs all its resources toward pure, high-level reasoning.
Synthetic and inflected languages (e.g., German, Russian): Complex word formation, cases, and inflections. Tokenization algorithms (BPE) are forced to split a single word into 3–4 abstract sub-tokens.
As a result, the Attention mechanism wastes its layer capacity just to glue these fragments back together and restore syntactic connections. This directly hurts context window density and the quality of multi-hop reasoning.
Currently, I am continuing my attempts to deeply understand the differences in tokenization across various languages. In practice, the differences in model behavior and resource consumption when processing the exact same business logic in English versus local languages can be absolutely massive.
Colleagues, have you measured this "Language Tax" in your projects? How often do you find yourself thinking that the answers to modern engineering challenges lie in classical philosophy?
#AI #LLMs #SoftwareArchitecture #Philosophy #LanguageTax #PromptEngineering