That's got to be it. Cloud compute is expensive when you're not being funded in Azure credits. One the dust settles from the AI bubble bursting, most of the AI we'll see will probably be specialized agents running small models locally.

I mean yeah? I wasn’t counting in detail, it’s an estimate.

Previously you got 500 requests a month and then it’d start charging you, even on “auto.” So the current charging scheme seems to be encouraging auto use so they can use cheaper LLMs when they make sense (honestly a good thing).

I was questioning the use of the word "prolly"

it means "probably"

Nah, you should find a new bone to pick.

I'm still running Qwen32b-coder on a Mac mini. Works great, a little slow, but fine.

Imagine the price hikes when they need to get that return on hundreds of billions they've poured into these models, datacenters and electricity.

Ah they're learning from the "unlimited" mobile carriers.

"Unlimited" until you meet your limit, then throttled.

Common People

I'm somewhat tech savvy, how do I run llm locally. Any suggestions?
How to know if my local data is safe

Checkout lm studio https://lmstudio.ai/ and you can pair it with vs continue extension https://docs.continue.dev/getting-started/overview.

The whole industry is projecting something like negative $200B for next years. They know it's not worth the price.