That is exactly what Ollama does too.

Ollama can’t run on Android

That's fair, but I think I'd rather self host an Ollama server and connect to it with an Android client in that case. Much better performance.

Censoring is model dependent so you can select one of the models without the guardrails.

But not everything works in those conditions.

never go online again - they won't be able to monitor anything, even if there's code for that included.

Sounds counter-intuitive on a smart phone where you most likely want to be online again at some point in time.

Yes, that's my setup. But this will be useful for cases where internet connection is not reliable

Enclave on iOS does the trick for the rare times i need a local LLM

You can use it in termux

Didn't know about this. Checking it out now, thanks!

Has this actually been done? If so, I assume it would only be able to use the CPU

Duck.ai doesn't data mine, and has o3 mini which I have found to be very good. Its got some extra functionality like lines to break up text.

Yeah duck is all over bothered with since it came out since you don't even need to login to use it.

Yeah I have it in termux. Ollama is in the package repos for termux. The speed it generates does feel like cpu speed but idk

Just firewall the software or is there anything more fancy i would need to do?

Nice! I saw Mozilla also added an ai chat in the browser recently (not in the phone version that I have seen tho).

It is too bad duck.ai only runs the small models. Gpt4o-mini is not very good, it can be very inaccurate and very inconsistent
I would like to see the 4.1-mini instead, faster and better and got function calling, so it can do web searches for example. O3 can't so it can only know what it knows until 2023.

But thanks for the information I will be looking out for when 4.1 is added!

So trust them. If you don't and want to use this, buy a separate device for it, or VM.

Can't? This is not for you.

I won't gonna use my smartphone as a local llm machine.

Is there any useful model you can run on a phone?