Technical note: a Recognition-assisted Camera for Automated Microscopy (RaCAM)
Abstract. Automated microscopy workflows, including image acquisition, processing, and recognition using artificial intelligence (AI) are getting a growing interest from the scientific community in biogeosciences, as more and more research institutes are actively working on building datasets of images to train artificial convolutional neural networks (CNNs) to identify microscopic objects.
Here, we present a new, affordable, AI-assisted, Raspberry Pi-powered camera, with the first, built-in, and fully auto-mated microscopy workflow (including automated image acquisition, processing and recognition) that can fit any microscope equipped with a regular C-mount (or CS-mount) camera thread. This camera is equipped with an integrated Single-Board Computer (Raspberry Pi 5) and high-resolution camera sensor (12.3 mp), attached together using a 3D-printable adaptor. Us-ing a new open-source software (RaCAM user interface), written using the Python language, and freely downloadable too, the camera is capable of performing automated acquisition of field of view images, segmenting each visible object of interest, and identifying them using trained CNN onnx models in a few seconds as part of a whole automated workflow.
The camera is also adapted to on-field tasks such as core description, biostratigraphy or even palaeoenvironmental reconstructions based on microfossils census data or morphometry, as it can operate without the need for a spare computer and run directly on a power bank. Finally, as the RaCAM workflow relies on images directly captured by the camera, applications can also be extended outside of the microscopy and micropaleontology research fields as any picture acquired with this device can virtually be processed by the automated workflow.
The manuscript introduces a camera designed for automated microscopy, utilising Raspberry Pi hardware, a 3D-printed case, and free software. This system acquires images, segments the object of interest within the field of view, and identifies it using a convolutional neural network (CNN). The innovative feature of this solution is that all processing occurs on the Raspberry Pi, requiring only a monitor or screen for operation.
This approach offers several advantages, including a low-cost solution accessible to individuals worldwide who need to count and identify biological objects. The low cost stems from the 3D-printable case and the Raspberry Pi itself. Notably, the image resolution from the high-quality camera sensor (4056x3040, 12 MP) exceeds that of many commercial cameras, such as the Leica ICC50 W/E (2592x1944, 5 MP). Additionally, the segmentation and artificial intelligence software tools provide a significant advantage for anyone looking to take their first steps toward automation.
I believe this manuscript has the potential to be relevant for scientific investigations within the journal's scope. As I followed the individual steps, I encountered some technical issues. Specifically, I got stuck during the segmentation process: the program displays "processing," but nothing happens because AutoDiato_RaCAMx40.ijm is designed for radiolaria, not for pollen. I also attempted to download the radiolarian dataset from Carlsson et al. (2023), but the images were already segmented, preventing me from testing the segmentation and identification.
This issue could arise for anyone, so I would like to request some sample images of Radiolaria that cover the entire field of view and work with the provided codes. Additionally, it would be helpful to include examples of other proxies to support the strong statements in the last sentence of the abstract. This would demonstrate that your tool can handle "any micropaleontological image," not just radiolaria.
- Your RaCAM_software.zip file
- A segmentation macro from ImageJ
- A CNN ONNX model, which must be trained beforehand using the ParticleTrieur program. Alternatively, this model can be created following the last part of your GitHub instructions, which appears below the program screenshot. Please note that this part is not clearly described in the manuscript.
Carlsson V., Danelian T., Tetard M., Meunier M., Boulet P., Devienne P., & Ventalon S. (2023): Convolutional neural network application on a new middle Eocene radiolarian dataset. – Marine Micropaleontology.