This is all very nuanced and there isn’t a clear cut answer. It really depends on what you’re running, for how long you’re running, your device specs, etc. The LLMs I mentioned in the post did just fine and did not cause any overheating if not used for extended periods of time. You absolutely can run a SMALL LLM and not fry your processor if you don’t overdo it.

Of course that is something to be mindful of, but that’s not what the person in the original comment said. It does run, but you need to be aware of the limitations and potential consequences. That goes without saying, though.

Don’t overdo it and your phone will be just fine.

For me the biggest benefits are:

• Your queries don’t ever leave your computer
• You don’t have to trust a third party with your data
• You know exactly what you’re running
• You can tweak most models to your liking
• You can upload sensitive information to it and not worry about it
• It works entirely offline
• You can run several models

There are a few. There’s Private AI. It is free (as in beer) but it’s not libre (or open source). The app is a bit sketchy too, so I would still recommend doing as the tutorial says.

Out of curiosity, why do you not want to use a terminal for that?

I don’t know that one. Is it FOSS?

Thank you for pointing that out. That was worded pretty badly.

For clarification, you’re not acting as an exit node if you’re running a snowflake proxy. Please, check Tor’s documentation and Snowflake’s documentation.

The person using your proxy knows what your IP address is, however. But, to your ISP, that connection will look like that person is on a call with you on, say, Zoom thanks to WebRTC.

Not true. If you load a model that is below your phone’s hardware capabilities it simply won’t open. Stop spreading fud.

Of course! I run several snowflake proxies across my devices and their browsers.

I didn’t use an LLM to make the post. I did, however, use Claude to make it clearer since English is not my first language. I hope that answers your question.

I have tried on more or less 5 spare phones. None of them have less than 4 GB of RAM, however.

Great explanation, Max!

I would argue there would not be any noticeable differences.

The performance may feel somewhat limited, but this is due to Android devices usually having less processing power compared to computers. However, for smaller models like the ones I mentioned, you likely won’t notice much of a difference when running them on a computer.