How would we do that? There's no historic precedent for that. Its fundamentally an information theory thing: what's the max amount of intelligence you can get out of 1 KB/MB/GB? There has to be a limit and I'm not convinced that it's far off.
Once I start a conversation about Postgres query plans, maybe 95% of the knowledge will be almost certainly not needed, and for an inference provider there will be many more concurrent queries with reasonably large overlap. Maybe future architectures will be able to take advantage of this so not all parameters are needed in memory for almost all instances.
If we are talking about all knowledge, then I agree that the compression ratio is very impressive already.
Mixture of Experts is already used by pretty much all modern LLMs to address exactly this phenomenon.
Hopefully, future models can be trained to be even more aware of external knowledge, accessible through web search / RAG / whatever it will be then, and might not need to internalize much knowledge at all.
So far, frontier capability keeps getting shrunk to fit consumer level hardware. The lag time being over a year, though, precludes this from being called SOTA by the time it arrives.
There must be a limit, I agree, but there have been no signs of approaching it yet. The most recent cohort of small models have shown the biggest leap in capability so far.
All the terms are squishy, but being sloppy about it, I think there's intelligence that needs to be in the model, and knowledge that could live in a database. Right now, models are memorizing a lot of stuff they don't need to. We know how to index lots of information on (comparatively slow) SSDs or across a network.
See Karpathy's "Cognitive Core" idea. (I don't have a good link)
2K, 4K, and 8K+ TVs have been around forever, and although 4K has become the norm, it's widely accepted that there isn't much benefit for most people above 1080p
Your example also counters your point: A lot of people do not care about 4k, and even less do care about 8k or above. We reached a point where most people are completely happy with the quality they get.
> Your example also counters your point: A lot of people do not care about 4k, and even less do care about 8k or above. We reached a point where most people are completely happy with the quality they get.
And when (not if) we get to the 4k (or 8k) equivalent of LLMs, they'll just be baked into hardware and we'll have near instant responses while running locally.
There is no future where OpenAI, Anthropic, etc survive with their current business model; at some point we will hit a point where training a new model is done only every 5 years or so, and in that scenario we aren't going to be running models of pricey server GPUs with latencies measured in seconds and full responses measured in minutes.
We'll be running locally with sub-millisecond latencies and responses measured in milliseconds. There is no way for any big company to compete with the current business plan of selling inference or subscriptions.
Don't forget that it's only an assumption that scaling more results in better models. There may be a ceiling to that. If that is hit and the best performing possible model can run in little VRAM, your argument here doesn't hold anymore.
In a way it is actually the same thing that makes us accept AI as working in the first place. It only needs to be good enough for human perception. The same is probably true for compute.
I'd argue we've already hit the ceiling. Can you truly tell the different between SOTA models from 9 months ago and those from today? There are some improvements but they're mostly marginal.
Plus there is a chance the actual scaling that matters is beyond our reach. Think instead of TB models, PB or ZB models. We don't even have that kind of information. Humanity in its entire history hasn't generated 1ZB of information.
> Standard Def TV was plenty for 50 years. But when more was on offer, everyone went for it, and now you can't even buy a 480p TV.
And yet when we hit 4k, that's were people just stopped buying higher res. 8K is still useful, but only when the screen is so large that it doesn't fit in the room :-/
> How would we do that? There's no historic precedent for that. Its fundamentally an information theory thing: what's the max amount of intelligence you can get out of 1 KB/MB/GB? There has to be a limit and I'm not convinced that it's far off.
Sure, it might be very near using the current approach, but... it might also be might be very far off because we are using the wrong approach.
I mean, look at the max amount of intelligence you can get out of a human brain powered by two bananas...
I think this is the future - at least it will be for on-device models. Apple, for instance, will "bake silicon" once a year for their current model, and use that chip in all their devices.
SoTA AI will move well past 1TB+ memory requirements in 1 year or so.
Also, you will be able to play with much more competent models locally. They will still feel like children compared to the adults living in the SoTA region.
The fundamental issue is that for these systems bigger is essentially always better (if affordable). So if we can squash something like Kimi K3 down to run on a 'normal' system, that just incentivizes devs to increase the model size until once again we're at the limit of what can be run.
Are you thinking of fundamental architectural changes (like the Transformer)? Or incremental (like MoE or GQA)? Are there specific neolabs or techs you are following that lead you to this prediction?
Indeed it seems quite possible we are one architecture breakthrough away from existing chip stock driving us all the way to ASI.
Nobody has done AFAIK the information theory to prove it’s not possible.
Yeah, you can directly print the SOTA AI model on the chip. I also believe this can be done on older process nodes. The most important factor would be speed. How fast can you go from new model to new chip?
Hyperscalers are screwed, data center mania couldn’t even be completed during this massive spending spree, all while the people seemingly sitting on the sidelines are working on getting these things baked into the OS and chipsets in a way consumers wouldn’t notice
EU chip industry: best I can do is supply chips that are in 99% of industrial equipment and vehicles. Also supply the only machines in the world that enable leading edge lithography processes.
No comment on the second part, but the first part is wrong. EU doesn't have a moat in process technology anymore (since 2010s). They have a bunch of legacy fabs, like 350, 180 and 130 nm but Taiwan, US, and China (and others like Israel, or Singapore) have these too. EU is severely lacking behind chip design, production and packaging capabilities for industrial, automotive, and space kind of niche applications since the heavy divestment from semiconductors after 2008 financial crisis and 2015 Euro crisis.
On top of this, once European giants like NXP (formerly Philips semiconductors), Infineon (formerly Siemens semiconductors), and ST are actively reducing their footprint in Europe and moving to India or SE Asia.
In short: No, EU has no future in those markets as designer or producer of chips.
EU: Perhaps we should also spend a few billion on attracting american glue manufacturers so they open a factory and a half here so we can catch up in this chip making business.
CXMT uses old Qimonda patents (even though Qimonda's tech is now outdated), and the former CEO of Qimonda is currently building a fab for an FeRAM-based alternative to DRAM in Magdeburg. It's a gamble as FeRAM still suffers from a more limited lifetime than conventional DRAM at this moment but I think there are still somewhat interesting developments happening in Europe.
Shovels have always been the better out of the two investments.
There were rumours about China grabbing hold of one of ASML's machines, but with no additional information on this it's safe to assume they're still hammering out 7nm chips using DUV at piss poor yields.
I have nothing to say about this company's long term potential, but I'm curious to know whether this will become an enormous "tik stok" like SK Hynix became in South Korea. I've watched a short documentary about the investment mania in SK and it's really scary.
Morgan Stanley's target price for SPCX in their report is $300, with $8 being the space component, and the $292 being everything else, like what actually gets put into space. The Morgan Stanley report doesn't say SPCX is worth $8/share. Just wanted to be clear and not have a conversation based on vague implications.
If the golden stick of memory sitting on Trumps desk has its intended effect - and the US allow Apple to buy memory from CXMT then another 500% surge might happen.
Most of the price increase did occur in Q4 2025 and most prices are fairly stable since then. Maybe this is different for LPDDR, GDDR, or HBM, but I would like to see evidence for that.
Why does the US stock market keep having record years while the population doesn't seem to benefit, whereas the opposite is happening in China? Isn't that a failure of US capitalism?
The more interesting question is: why does the rest of the world not increase their production prowess? I am aware of most of the reasons, but I am getting tired of the "China is better than the rest of the world". It is more that China did a lot of things that the others either failed to do or no longer wanted to do. Case in point is Germany: from "our cars are best" to "hey, chinese cars are now better" (or, cheaper, for a comparative quality). This constant complaining is so strange. Would that energy not be better spent to try to become better?
Mostly self-inflicted wounds in form of climate pledges, climate taxes and such. Because of the goal to achieve zero emissions goal and climate neutrality, we abandoned cheap energy sources such as coal and pushed our heavy industries out of Europe to China. We still need those products, so we import them from China while wondering why European industrial segment isn't growing. Even in the clean energy sector, most solar panels come from China.
It is because in the west we suffer from self inflicted obstacles that make it impossible to get out of our own way. The Chinese gov must think they are playing on easy mode.
Because the capitalist west doesn't have strategic view, beside maybe on the defense sector.
Everything else must produce profits next quarter not in the next 3 years.
It's this. Wall Street won't allow american companies to make long term decisions and long term planning that cost money in the short term, because they want all that money immediately now. Like, right now now. If an executive promised they could deliver big profits in 5 years by spending the next two investing their profits, Wall Street would fire them and put in someone who will happily spend all the profit (plus debt) to do stock buybacks and dividend disbursals.
It's even worse, the entire ideology is that industrial policy can't, by definition, be better than what the market would decide without interference. So every kind of actual strategic decision making (i.e. investing a few hundred billions in the industries of importance for the next decade(s), or at least in the infrastructure supporting it) would actually make matters worse. That is essentially what neoliberalism is all about.
This is core reason why the German industry became so irrelevant in the last 2 decades: The rail infrastructure became pseudo privatized and now is in shambles (IIRC only about 60% of trains arrive on time). The PV industry was axed by the conservatives about 15 years ago and now is completely dominated by China. Even now, as the other EU countries demand strategic investments - and again the German conservatives say this ought to happen without any new debt. "We can only spend what we have", to quote Merz. Imagine China saying this about chip production, or even Sam Altman about compute hardware...
> Would that energy not be better spent to try to become better?
The problem is, shareholders have bled Western companies dry in the chase for returns on investments. There is no money left in the companies to retool them to be competitive with China.
Meanwhile, the CCP just pours dollars into their companies.
Unfortunately, there is no punishment in Western societies if you run a company into the ground as long as you pay your debts on the way out.
It takes 2 years in EU to get approval to build a shed. Imagine how long it takes to get approvals for a chip factory which uses a lot of water and very toxic chemicals.
UK has been trying to get approvals for a new runway at Heathrow airport for over 20 years !!! now.
Of course CXMT, Micron, and family will still be profitable, but maybe not 'surge 470% from IPO' profitable.
If we are talking about all knowledge, then I agree that the compression ratio is very impressive already.
Hopefully, future models can be trained to be even more aware of external knowledge, accessible through web search / RAG / whatever it will be then, and might not need to internalize much knowledge at all.
There must be a limit, I agree, but there have been no signs of approaching it yet. The most recent cohort of small models have shown the biggest leap in capability so far.
See Karpathy's "Cognitive Core" idea. (I don't have a good link)
Standard Def TV was plenty for 50 years. But when more was on offer, everyone went for it, and now you can't even buy a 480p TV.
2K, 4K, and 8K+ TVs have been around forever, and although 4K has become the norm, it's widely accepted that there isn't much benefit for most people above 1080p
And when (not if) we get to the 4k (or 8k) equivalent of LLMs, they'll just be baked into hardware and we'll have near instant responses while running locally.
There is no future where OpenAI, Anthropic, etc survive with their current business model; at some point we will hit a point where training a new model is done only every 5 years or so, and in that scenario we aren't going to be running models of pricey server GPUs with latencies measured in seconds and full responses measured in minutes.
We'll be running locally with sub-millisecond latencies and responses measured in milliseconds. There is no way for any big company to compete with the current business plan of selling inference or subscriptions.
In a way it is actually the same thing that makes us accept AI as working in the first place. It only needs to be good enough for human perception. The same is probably true for compute.
Plus there is a chance the actual scaling that matters is beyond our reach. Think instead of TB models, PB or ZB models. We don't even have that kind of information. Humanity in its entire history hasn't generated 1ZB of information.
And yet when we hit 4k, that's were people just stopped buying higher res. 8K is still useful, but only when the screen is so large that it doesn't fit in the room :-/
Sure, it might be very near using the current approach, but... it might also be might be very far off because we are using the wrong approach.
I mean, look at the max amount of intelligence you can get out of a human brain powered by two bananas...
Some groups are baking models into silicone, Deepmind has an example, it gets 18,000 tokens/sec on Llama 3.1, not sure about parameter size
It's like people saying mobile chips are going to be better than PC chips... until they realize PC can be made of mobile chips too if that comes true.
Also, you will be able to play with much more competent models locally. They will still feel like children compared to the adults living in the SoTA region.
Indeed it seems quite possible we are one architecture breakthrough away from existing chip stock driving us all the way to ASI.
Nobody has done AFAIK the information theory to prove it’s not possible.
Their posterior. Or worse, their interest in going to the moon (HODL style, not Artemis style).
Hyperscalers are screwed, data center mania couldn’t even be completed during this massive spending spree, all while the people seemingly sitting on the sidelines are working on getting these things baked into the OS and chipsets in a way consumers wouldn’t notice
On top of this, once European giants like NXP (formerly Philips semiconductors), Infineon (formerly Siemens semiconductors), and ST are actively reducing their footprint in Europe and moving to India or SE Asia.
In short: No, EU has no future in those markets as designer or producer of chips.
Difference between Europe and the others is that it doesn't brag nearly as much.
There were rumours about China grabbing hold of one of ASML's machines, but with no additional information on this it's safe to assume they're still hammering out 7nm chips using DUV at piss poor yields.
i mean, it shouldn't be scary, because their failure is their own, and i doubt there would be collateral damage outside of those high risk investors.
It's scary for _them_, but it's an interesting case study for an outsider.
Is this accurate? It looks like they are no longer rising, at least for DDR5 memory:
https://de.pcpartpicker.com/trends/price/memory/
Most of the price increase did occur in Q4 2025 and most prices are fairly stable since then. Maybe this is different for LPDDR, GDDR, or HBM, but I would like to see evidence for that.
So others did spend energy on becoming better and now you see a different picture when you look at the world.
This is core reason why the German industry became so irrelevant in the last 2 decades: The rail infrastructure became pseudo privatized and now is in shambles (IIRC only about 60% of trains arrive on time). The PV industry was axed by the conservatives about 15 years ago and now is completely dominated by China. Even now, as the other EU countries demand strategic investments - and again the German conservatives say this ought to happen without any new debt. "We can only spend what we have", to quote Merz. Imagine China saying this about chip production, or even Sam Altman about compute hardware...
The problem is, shareholders have bled Western companies dry in the chase for returns on investments. There is no money left in the companies to retool them to be competitive with China.
Meanwhile, the CCP just pours dollars into their companies.
Unfortunately, there is no punishment in Western societies if you run a company into the ground as long as you pay your debts on the way out.
UK has been trying to get approvals for a new runway at Heathrow airport for over 20 years !!! now.
how much more do they need to ship to satisfy you?