Identify plants right in your browser

As a fun side project, I decided to build a plant identification app. Existing plant ID apps are powerful, but they require us to download apps, send our camera data to remote servers, and are just generally way too clunky. That’s why I created Druid, a plant identifier that runs right in your browser, right away.

The awesome scientists behind Pl@ntnet, a popular plant ID app, make their data public. That allowed me to train my model on PlantNet-300K, which contains 306,146 plant images covering 1081 species. For the model architecture, I used the ResNet-18 implentation from PyTorch. ResNet-18 is great balance between model complexity and accuracy, especially for a dataset of this size. Druid’s neural net should be able to run on any modern device with a proper web browser: not just beefy PCs, but also phones and tablets.

My workflow was:

Train a PyTorch ResNet-18 model using the PlantNet-300K dataset.
Export this model to the ONNX format, an open standard for AI model weights.
Build a simple single-page web app using SvelteKit.
Load the ONNX model into a Typescript object using Microsoft’s open-source ONNX Runtime Web library.
Run predictions with Web-GPU acceleration. Feed the results into a softmax function to get the prediction confidence.
Use Wikipedia’s API to get a description of the predicted species.

I drew the assets on a digital tablet using Inkscape, yet another open-source tool.

Because the model has to fit inside a browser, I can’t train it to identify most plant species (there are over 17,000 native species in the U.S. alone). Instead, I’ve clearly displayed Druid’s confidence in its prediction, and included a warning when its confidence is low.

Above: Druid shows a warning when its prediction confidence is low. This is basil!

Druid is just a fun experiment. Don’t expect it to put the other plant ID apps out of business!

Try it here.

Source code.