Perplexity Just Solved Local AI

As token costs increase and agentic max like this baby becoming the norm, more and more people I think are looking into local AI to reduce the costs. But the big issue is that AI just isn't there when compared to like the frontier stuff. Well, what if you could just have both? And that's where Perplexity's hybrid compute steps in. Command AI. We're proud to be partnering with Perplexity for this feature and I would cover it only for the website for the least. It's [laughter] it's just so I mean their their designers we know they are just I love their brand. I do. I do. I love their brand. You see here they've got this uh Mac Mini and I think they're kind of pushing this to the same sort of people who use Hermes who sort of use Open Claw and that kind of thing where they've got that dedicated machine set up to run aic task to always be on always available. And what they've added is this thing called hybrid compute. Basically what it does is when the when perplexity receives the request it delegates the private sensitive uh information such as name, addresses uh just generally sensitive things. It gives that to the local model so it's all kept on device and whatever and then everything else the planning and the heavyduty stuff it's given to the the remote models. I think that's perfect. I think that's exactly what we need at this point whilst local models tend to be not as you know complex but they they really it's undeniable to have private local data running on local AI models. Do you know what I mean? And then the rest that's not whatever. So, I sent it a task yesterday to um look at the we've got a contract and look at this contract, compare it against our insurance policy documents. So, plenty of juicy private information there. I just sent it to Perplexi app and sure enough it brings up this cool little thing to let you know that it's running a local task and how many tokens uh it it's saved you basically. It pops up with how much it cost you in credits that you you buy from Perplexity and how many tokens it was used locally. And I these two I did two tasks. I did one where it was just remote. So yes, I sent all my private data to the cloud. So you know whatever. and then one hybrid. And the one that was cloud only took about two and a half thousand perplexity credits. It was using Astra, but the point is the local version, the same task. The local version took 700 credits. So that's like a third of the cost running it uh on a hybrid solution. So it's a really good cost-saving thing and of course the privacy thing. So yeah, but it's designed to run on 24 GB uh unified memory max minimum. 32 gig is for the best results. And that is because they give you a choice of three models to pick from. And then the only last thing to say, it's got this PII, this personally identifiable information, which is another model that you can download through it that really just determines the sensitive information, names and things like that, which is again all run locally. So really, really quite nice. And like I say, this comes in clutch when you stick it on a Mac Mini um and you just have it running all of the time sort of thing. Obviously, [snorts] local AI is a little bit slower. I wouldn't be recommending this for tasks that you just need, you know, that instant access, but it's still if you can schedule things, if you can have it run in the background, if it's non-urgent, it's a really good solution. And you don't always have to just do the hybrid thing. You can just send it to the cloud and and whatever. But um yeah, you can, you know, it's it's not you're not locked into one or the other sort of thing. Don't know if you can hear it or not. No, but this is lovely ethereal music, isn't it? Yeah, it's it's just so good. So, this is the shortcut to for triggering Perplexity. So, the double command that you press brings up the what do you call it? Spotlight thing for using Plexity really quickly. You can drag files in all of that good stuff. This is just the Perplexity app. Yeah, it's really nice. Uh their intro is really nice and I can hear it also like um holding the function key. So I don't need uh whisper flow anymore. Well, if I use that. Whisper flow is good for writing your emails. But if you want to trigger your agent, then yeah, you can choose that. So this is what it looks like. This is the perplexi app. This is what it looks like. And under this you can obviously select whatever model you have, whatever model you want. And you've got things like I can see there, Kimmy K3, Grock 4.6, Julian 5.3, Deepseek even, and then obviously Fable, Opus, Sonnet, uh, GBT Soul, GPT, Gutty 6 basically. Um, but then you got this hybrid toggle thing. And I've got the Plexity model there for, you know, the the local model. And then when it needs to send it off, I've got Kimmy K3. And whatever I if you we click computer there whatever I type here will do whatever I ask it whether that is a computer use task go off and get a file on my computer and make it add this information and make it into whatever it is if it's computer use or not it will divvy up those tasks whether they're sensitive information versus whether it needs all the compute that uh the Frontier models Frontier Labs uh provide and that's kind of it and it's pretty Cool. Pricing is like this. You've got basically the $20 a month which gives you access just access to computer use. Right now they're giving you 4,000 bonus credits which is quite nice. But you will have to top up your credits and that will give you sort of that added flexibility. But if you go all out and go Plexity Max, you get reoccurring 10,000 monthly credits. So, um, definitely worth considering the local hybrid compute if you've if if you've got a powerful enough Mac to to actually do that cuz it will save you a lot as I experienced to go with that with the hybrid solution. So, yeah, that's that's the pricing. This is part of a larger conversation on my show, Command AI, which we stream live every single week. We discuss the news and all things related to AI in the world of design and web. Catch us next week and join in the banter. Command AI.