AOS for Lemmy.World - A generic Lemmy server for everyone to use.

Technology

A $200 ChatGPT subscription could cost OpenAI $14,000 if you actually used it to its full potential

473116

1d 10h ago by lemmy.today/u/sanitation in technology from www.techspot.com

If you've got a toy project that you want "AI" to give you a hand with, do it now.

Pretty soon all these companies are going to have to pay for all that investment in compute resources they've been busily soaking up over the last few years, and then they're going to have to pay back their investors, and then they're going to have to try and make a profit

This is the golden time for cheap commercial AI. Already the noose is starting to tighten, and it will never again be as cheap as it is now.

25 replies

Sounds like it'll never be worth it.

In five years once this RAM nonsense is over you'll be able to run a comparatively high quality local LLM for very little money. I can't see how these companies will ever make their money back.

If manufacturers are willing to sell components to us in five years that is.

Of course if the colllapse happens before then the story might be different…

I’m slightly optimistic that manufacturers will return to the retail market eventually. Every AI company is racing to hyperscale right now but there will be a point where the infrastructure is built and at that point the growth will slow down quite a bit. In that scenario there will be ongoing demand for components to be replaced as they become obsolete but I can’t imagine the demand will be the same level it is right now as everyone rushes to build.

That’s assuming this all works the way they want it to. If the economics aren’t viable and the bubble bursts…

Their Datacenter buildout doesn't work they want to. Most projects are very much delayed, and those that even started getting built are over budget. OpenAI and Anthropic will collapse in the next years, and this is coming from someone who absolutely sees the good things about the technology itself.

OpenAI and Anthropic will collapse in the next years

Stop, I can only handle so much good news!

There is no way, absolutely NO WAY to recuperate the amount of cash burnt on those two companies, and that is not even counting the amount of AI Startup whose cash is currently flowing towards to those two.

How about bailouts? GPT integration with Visa? IPOs sucked into funds? There's a lot of money they can and will try to vacuum, don't you worry about that. And banks will do their damnest best to help with all of that - just look at SpaceX.

At least they failed at getting into index funds for now.

Nasdaq100? They may not be un S&P, but they are in.

“Hyperscale” is utterly meaningless MBA jargon at this point. Equivalent of verbal slop from industry shills and CNBC/Bloomberg sell side simps.

Sorry if that’s true. I understood the word to mean aggressive growth at any cost to try and shut out competition before they can get established.

🤞

Sounds like price hikes to communicate costs are coming and resources are going to be redistributed to productive uses.

This is the golden time for cheap commercial AI.

I suppose, but small open weight models with more advanced coding frameworks optimized for them are catching up fast and you can do it privately at home on a mostly affordable consumer graphics card.

If you have solar it's basically free, minus the graphics card CapEx you may want for gaming anyway, as well as some setup time and a bit of patience.

What's your setup, if I may ask? I'm using llama.cpp router with vscode kilo.ai and qwen3.6-35B-MoE-MTP as a model mostly. It's surprisingly good as a coding assistant, but I think you have to know what you are doing and know your stuff(aka be an experienced developer) to make it useful. just letting it vibe leads to crap code

just letting it vibe leads to crap code

Yup, vibe is occasionally useful for proof of concept stuff, but disastrous for maintainability, security, readability, or large codebases. Without experience it's still a foot gun for anything even slightly serious.

Best approaches for a learner are to consider it autocomplete that needs research. Look up what it's suggesting, see if it's hallucinating, with luck it'll point you in a useful direction where you can learn a good solution, as it has no idea what that is. Also makes a pretty good rubber duck for hashing out architectural decisions, finding alternative approaches etc, though you'll have to point it at a web search for that. Spin up an e.g. vane instance for this, as small models don't have enough world knowledge. Use it to write (or preferably copy from its system prompt examples) boilerplate and unit tests, perhaps descriptive comments (doublecheck).

One thing to do is put everything you learn about coding style into your system prompt as they're dogshit at consistent style without significant beatings around the head. Finding your own comfortable, consistent style is super useful for future readability. The joke about when I wrote this only God and I understood it, now only God does, will come clear in a month or two. Learn to work around it. Simple beats fancy unless you truly need the speed.

While I do use agent iterative approaches, probably best to approach that organically as you grow, monsters lurk there. If you must, containerize / vm / isolate the hell out of something like opencode to muck around with.

FWIW I still write most of my code by hand, it's simpler and more consistent, but I'm keeping an eye on the development of LLMs, and I will let it write scut code (that I edit later). Code and Mathematics are super structured languages, pretty much ideal for large language models, so I can see them maybe, eventually getting good. More general thought, not so much without significant architectural upgrades.

While this advice is true for all models, when it comes to agentic tasks (add this small feature/write this test harness/find bugs/suggest improvements), open source models are still way behind, vibe code or not.

Claude Fable or even Opus in an editor like Zed have a 1 million token context window and will "think" through the goals of the application, test their changes, work through debugging processes the way a programmer would, stop to ask for clarification, check diagnostic tools and linters, prompt to run test code, etc.

Llama, Gemma and Qwen etc. Do lack a lot of the world knowledge to get the goals of the application, but they also just don't have the debugging skills, won't test their code, don't always tool call correctly, get confused as the context increases and nobody has enough vram to run on large context sizes locally.

They can do autocomplete on small functions but aren't really there for more complex tasks.

On top of that, the biggest problem is that the best open source models are trained and released by the same giant tech conglomerates that have an interest in not competing with their own products. Qwen is Alibaba, Llama is Meta, gpt-oss is OpenAI. Even the more "independent" ones, kimi (Moonshot) and GLM (z.ai) are mostly funded by Alibaba and Tencent. They're released for research and marketing purposes and to please their corporate backers with inflated stock. Almost nobody has the resources to train new models from scratch. People make lots of merges and fine tunes but AI is not democratised the way that traditional programming tools have been.

Maybe some day there will be enough cheap compute for open source communities to pool together resources to build competing models but they're not really there yet :(

Context management is a huge part of making smaller models viable (and likely a big part of making frontier models better). Tricks like structured context libraries for thinking improve things a lot, I like approaches that output things like an Obsidian vault that let you dig in and correct bad assumptions easily, even if it's a bit slower. It's a useful deliverable that can (mostly) be reused with updated models.

Things like 'the debugging skills, won’t test their code, don’t always tool call correctly' are tangibly improving model to model, framework to framework, and are problems that will be solved in time, but yes they need handholding ATM.

Things like

test their changes, work through debugging processes the way a programmer would, stop to ask for clarification, check diagnostic tools and linters, prompt to run test code

are mostly down to framework, not model (except for failing to tool call, which is improving), and falling at a respectable rate.

That said, sure, frontier models get more in one go, personally I'm fine with only a 3-4x force multiplier instead of 10 to keep it local, but YMMV. For a business with resources for a bigger server it'll be more like 8 times. Remember that some businesses handle sensitive data and can't (or damn well shouldn't) use frontier models, so the market is there.

Maybe some day there will be enough cheap compute for open source communities to pool together resources to build competing models but they’re not really there yet :(

Not wrong, decentralized inference is mostly solved (with latency penalties), but without decentralized training true democratization will remain out of reach. Hopefully a breakthrough will ensue, but until then we are dependent on the kindness of corporations (or them rugpulling competitors).

This could also be a part of the RAMpocalypse thing, 'if there's not a moat I'll fucking dig one, damn everyone else' (and damn SamA). I doubt that's sustainable long term, but it might get them through to IPO, more's the pity.

Yes, it's trending in that direction, and I've been experimenting with pretty small models on my PC as I don't really have the hardware to go large. If you've got the coding chops to set it up, it's definitely something to keep an eye on.

There's actually scope for someone to set up / sell local compute hardware+software packages, similar to all those coin miners. Give the end user a way to update models, or push models out to them or something, it seems it would be a good middle ground between manually typing code like a peasant and total corporate AI apocalypse.

There’s actually scope for someone to set up / sell local compute hardware+software packages, similar to all those coin miners.

I think that'll be a viable target in the future, and have little doubt some are jumping on it already. However, I also think it's too much of a moving target currently, a near optimal setup changes almost entirely month to month.

I find myself targeting last months setup, as then there's enough literature out there to get it set up in a day or two and most of the kinks have been worked out. Otherwise, I lose too much coding time to debugging the bleeding edge.

IMO, at the moment, if you're not capable of setting it up yourself you likely don't have the experience to use it reasonably safely nor an adequate understanding of its limitations. You'll find yourself using more time fixing the blunders than you gain, and / or the project will spiral out of control in maintainability, security, readability, and so forth. You could get away with small projects written as 'write only' code ala Perl though, keep the prompts and tests, when it needs to change rebuild with the newest hotness. Inefficient and unsatisfying though.

Dozens of Stanford grads walk out on Google CEO's speech

4h 34m ago by lemmy.today/u/sanitation in technology from www.bbc.com

349362

Microsoft punishes idiots who purchased a "perpetual" license of Office 2019 for Mac by disabling it next month, pirates unaffected

10h 45m ago by sh.itjust.works/u/FoxtrotDeltaTango in technology from www.theverge.com

406813

Mark Zuckerberg Orders His Employees to Start Having Fun Again After Brutal Layoffs Culled Their Colleagues

6h 34m ago by lemmy.today/u/sanitation in technology from finance.yahoo.com

155254

Leaked financial docs show OpenAI is losing billions of dollars a year

2h 33m ago by lemmy.today/u/sanitation in technology from arstechnica.com

110105

Google is testing a new CAPTCHA that asks you to make hand gestures on camera

7h 11m ago by lemmy.today/u/StopTech in technology from cybernews.com

137746

UK Introduces Full Social Media Ban For Under 16s - Including X, YouTube, TikTok

1d 16h ago by lemmy.today/u/sanitation in technology from deadline.com

128457

SpaceX is buying Cursor for $60 billion

10h 35m ago by lemmy.today/u/sanitation in technology from www.businessinsider.com

131358

Trump admin abandons fight against wind energy as clean energy output surges

5h 15m ago by lemmy.world/u/Some_Emo_Chick in technology from arstechnica.com

196249

Stop Killing the Internet: Governments are walling off the open internet. We are a global movement opposing restrictions — and building a better internet.

1d 18h ago by lemmy.ca/u/floofloof in technology from www.stopkillingtheinternet.com

10879510

Microsoft's GitHub capacity crunch sends it to AWS

1d 13h ago by lemmy.zip/u/favoredponcho in technology from runtimewire.com

791311

OpenAI Losses Increased Nearly 8X in 2025, With Spending Hitting $34 Billion

1d 8h ago by piefed.world/u/beep in technology from www.wheresyoured.at

1762012

Fox to Acquire Roku in $22 Billion Deal

1d 14h ago by lemmy.today/u/sanitation in technology from www.hollywoodreporter.com

2195713

Canada wants to join the age verification bandwagon and censor the internet with Bill C-34

1d 20h ago by lemmy.today/u/StopTech in technology from ca.news.yahoo.com

47410714

DOJ claims xAI’s unpermitted gas turbines are a matter of ‘national, economic, and energy security’

33m 49s ago by lemmy.today/u/sanitation in technology from techcrunch.com

20115

EU takes a big first step in weakening digital privacy and in empowering big tech

3h 13m ago by sopuli.xyz/u/smps in technology from www.liberties.eu

44316

ChatGPT's market share slips below 50% for first time

12h 35m ago by lemmy.today/u/sanitation in technology from techcrunch.com

2634417

Mark Zuckerberg Orders His Employees to Start Having Fun Again After Brutal Layoffs Culled Their Colleagues

1d 45m ago by lemmy.world/u/madeindex in technology from futurism.com

57111818

Commodore's Next Hardware Release Is Dumb, And Proud Of It

22h 23m ago by lemmus.org/u/sundray in technology from www.timeextension.com

31311219

Kaspersky says hackers are distributing malware via anime girl wallpapers on Steam Workshop's Wallpaper Engine

1h 42m ago by piefed.world/u/beep in technology from securelist.com

43320

Want your images back? Sure... That'll be $5!

1h 50m ago by feddit.org/u/fusspilz in technology from www.lutr.dev

7521

Quote of the day by Apple CEO Tim Cook: 'If you put a key under the mat for the cops, a burglar can find it, too' — a stark warning on threats to undermine privacy

20h 36m ago by lemmy.today/u/sanitation in technology from www.techradar.com

7926722

Amazing article on mechanical watches.

19h 32m ago by lemmy.world/u/sznowicki in technology from ciechanow.ski

203823

Finland charges Russian captain and crew member of ship suspected of damaging undersea cables — prosecutors claim ship had eight more targets before it was stopped by coast guard

8h 35m ago by lemmy.today/u/sanitation in technology from www.tomshardware.com

1951124

Electronic Arts Announces Plans to Cram Far More Advertising Into Its Games / EA Advertising will serve up real-time dynamic ads inside EA Sports titles

1d 6h ago by lemmy.today/u/sanitation in technology from kotaku.com

1892925

Pentagon boasts of using AI to write reports mandated by Congress

18h 17m ago by sh.itjust.works/u/FoxtrotDeltaTango in technology from arstechnica.com

109926

Judge Rules Blacked.com Can Sue Meta for Scraping Its Porn

1d 21h ago by lemmy.world/u/Some_Emo_Chick in technology from www.404media.co

7706727

Fox Is Buying Roku in $22 Billion Deal

2d 4h ago by lemmy.world/u/Some_Emo_Chick in technology from variety.com

55119028

I Could've Rickrolled the Entire FIFA World Cup. All I Needed Was My ID.

6h 40m ago by piefed.world/u/beep in technology from bobdahacker.com

73629

Tech layoffs 2026: Nearly 153,000 jobs cut at Meta, LinkedIn, Salesforce, Robinhood and more

4h 9m ago by lemmy.world/u/return2ozma in technology from tech.yahoo.com

54430

Nvidia CEO: Society needs new 'new social norms' in the age of AI. I used to play in the streets. When cars came along, you obviously can’t play in the streets now

20h 6m ago by lemmy.zip/u/Valuy in technology from apnews.com

452331

AI and brain-computer interface allow speechless ALS patient to work a full-time job

21h 27m ago by lemmy.world/u/vegeta in technology from www.theregister.com

2274632

TikTok Alternative "Loops" is now on Google Play

1d 5h ago by piefed.world/u/beep in technology from blog.joinloops.org

1662633

UK: Under-16s to be banned from social media, Starmer announces

2d 9h ago by lemmy.world/u/Tarambor in technology from www.bbc.co.uk

50429934

Student Reading Ability Spikes After Removing Tech From Class

1d 2h ago by lemmy.today/u/sanitation in technology from www.yahoo.com

2972535

Microsoft CEO says "YouTube monetizes Xbox better than we do", ahead of expected layoffs | Satya Nadella bemoans performance of Xbox

22h 37m ago by lemmy.today/u/sanitation in technology from www.videogameschronicle.com

691236

W Social, Public Institutions and the Theater of European Digital Sovereignty

7h 6m ago by piefed.world/u/beep in technology from blog.elenarossini.com

16037

Map Clustering is Not My Favorite

5h 50m ago by feddit.org/u/lebkuchen in technology from blog.greg.technology

7038

Exclusive: Tesla presented misleading ‘Full Self-Driving’ safety data to European regulators

2d 6h ago by feddit.uk/u/Wudi in technology from www.reuters.com

4604039

How a Single Rogue BGP Announcement Took Telegram Offline Across Three Continents

11h 45m ago by lemmy.world/u/WPSteam in technology from thecybersecguru.com

66240

Apparently The Real Reason Anthropic’s Models Are Offline: A Six-Year-Old Trump Grudge

22h 17m ago by lemmus.org/u/sundray in technology from www.techdirt.com

1562641

Major publishers sue Meta for copyright infringement over AI training

18h 36m ago by lemmy.today/u/sanitation in technology from www.theguardian.com

330642

Arizona Pauses Data Center Tax Incentives After Governor Hobbs Signs Bill

16h 36m ago by lemmy.today/u/sanitation in technology from news.bloomberglaw.com

110343

Meta's Super Expensive New AI Team Is Already a Complete Catastrophe: "It's literally the gulag."

1d 8h ago by lemmy.world/u/madeindex in technology from futurism.com

2312444

Tesla on Autopilot Smashes Straight Through Garage Door, Driver Says

2d 14h ago by lemmy.today/u/sanitation in technology from autos.yahoo.com

56313245

It Is Trivially Easy to Use Reddit to Manipulate AI Search, Research Suggests / A tiny snippet of user-generated text as short as 13 words long is often enough to manipulate the AI agents that power…

1d 8h ago by lemmy.today/u/sanitation in technology from www.404media.co

139946