Your Browser Can Do WHAT?! 5 AI Features That Actually Work - DEV Community

Explores WebGPU and hardware limits
Are AI models running directly in the browser finally good enough to use in production? Let's find out! 😉
It's been a week since my talk at FrontKon in Prague. As I mentioned before, it was fantastic! I'm adding a few photos below, just to prove that I actually speak at conferences and don't just make these things up. 😀
My talk was titled "Your Browser Is a Compute Platform (And You're Using It for Forms)", and I showed how we can use WebAssembly and WebGPU to perform advanced computations directly in the browser. I was a little concerned to discover that, out of everyone in the audience, maybe three or four people had even heard of WebGPU! And that's a shame, because WebGPU can accelerate certain types of parallel computations by tens of times compared to CPU-based implementations.
But okay, at least the people at FrontKon know what WebGPU is now. 😁 And if you'd like a talk about this at your conference, feel free to reach out! Especially if your conference happens to take place in a particularly attractive tourist destination. 😀
After a conference, I traditionally prepare a blog post related to my talk. However, since I've already written about WebAssembly and WebGPU many times, today I'd like to approach the topic from a different angle. Namely, let's take a look at the current state of AI models running directly in the browser, which, of course, make extensive use of these technologies. 🙂
Browser-based AI models are a fascinating concept. No inference API calls. No network round trips to an AI backend. No worrying that your users will consume too many tokens, because you're not paying for them! But of course, local inference still takes time, and downloading the model in the first place can take a while.
Another advantage is that these models can work offline once they've been downloaded and cached by the browser, provided the application itself supports offline operation.
And that also brings a significant privacy benefit: inference happens directly on the user's device, so their data doesn't need to be sent to an external AI service.
However, as you can probably imagine, these are relatively small models. We're not going to run a massive frontier-scale model in a typical browser tab.
So the question is: how useful are these smaller models, really?
Some of you may remember that I experimented with browser-based AI models about a year ago. Back then, we were mostly laughing that they were a fun curiosity, but not much more than that.
But I've been following people from Hugging Face who work on Transformers.js on LinkedIn, and every now and then, their demos would appear in my feed. At first glance, some of them looked genuinely impressive. You know, practically spaceships running inside a browser. 😉
So before FrontKon, I decided to take another look at what they had to offer. And it turned out that after upgrading Transformers.js to version 4.3.1, which I used for these experiments, and trying some newer models, the results were genuinely impressive!
Of course, Transformers.js isn't the only option for running AI models in the browser. There are other libraries and runtimes, but I really like Transformers.js, and I think it's a pretty representative example of what this technology can achieve today.
At the same time, I don't work for Hugging Face, and nobody is paying me to sell you a spaceship in your browser. So I'll tell you what browser-based AI models can actually do today, what works well, and what still doesn't.
And even better, you can explore the code and try everything yourself!
GitHub: https://github.com/sylwia-lask/practical-browser-ai
Live demo: https://sylwia-lask.github.io/practical-browser-ai
If you like the project, please give it a star on GitHub. I'm collecting them! 😀
For these demos, I wanted to focus on practical applications. More specifically, I wanted to find out how we could use browser-based AI today, right now, in real production applications.
So is this technology worth paying attention to? In my opinion, absolutely!
I probably wouldn't build a mission-critical system that depends entirely on these models working perfectly. 😉 Instead, I'd treat them as interesting, optional enhancements to existing application features, especially in areas like e-commerce, accessibility, and customer support.
The application itself is nothing particularly exotic, by the way. It's a regular React application written in TypeScript, using Web Workers to run model inference without blocking the main UI thread.
The models are loaded on demand, so you don't need to download all five of them just to open the application. The inference runs locally through Transformers.js, using WebGPU where supported and WebAssembly-based CPU execution where a fallback is available.
No AI backend involved!
Let's see what I managed to build. And at the end, I'll share my thoughts on the limitations of these models.
One more thing: these aren't browser-based AI agents yet. They're simply applications using local AI models for specific tasks. But give me a few more weeks, and you'll see agents running in the browser on my blog too!
One of the things I particularly appreciate about browser-based AI is that it's not limited to text generation. I love working with multimodal AI!
In this demo, a small model called RMBG-1.4, with approximately 44 million parameters, does a really impressive job of separating an object from its background. 🙂 The model itself is only about 170 MB to download in the configuration I'm using.
As you can see, we can then save the result as a transparent PNG, and it works surprisingly well!
I can easily imagine this as an additional feature on auction websites or e-commerce platforms.
Imagine someone trying to sell a product but only having a photo like my example. Instead of requiring them to use an external image-editing service, we could simply offer a background removal button directly in the application.
And the photo doesn't even need