This article is contributed. See the original author and article here.
Founded in 2007, Red Panda Network has become a world leader in efforts to protect Red Pandas and their habitat. Using a variety of techniques such as community based conservation and developing anti poaching networks, they are committed to saving every last panda that is currently living in the Himalayan mountains.
Since my first meeting with Sonam Lama, Red Panda Network it was evident that their current workflow lacked any sophistication and revolved heavily around human labour. From manually classifying and retrieving images to making in-field observations using pen and paper, it was clear that the workflow needed a flair of innovation to allow the organisation to work smarter and not harder.
The Red Panda Network’s current workflow to track red panda’s in the mountains is illustrated in the diagram below. The process can be decomposed into two main sub processes : image retrieval from camera traps + classification and data entry + analysis.
Image retrieval and classification
Currently, the organisation has several camera traps in the mountains. These motion activated cameras take pictures when motion is detected in their vicinity and saves the images to the local SD card. As you can imagine, these camera traps can capture several false positive images ( when the motion is thought to be a red panda when it is not ) . Periodically, a member of the red panda network team has to go to the mountains, download the SD card data then classify these images as being red panda or not.
To put things into perspective, the Red Panda Network currently has around 20 cameras that took a total of 55,000 images over approximately 2 months – that is a lot of manual classifying! The images then get sent to the red panda headquarters where a spreadsheet is updated for further analysis later on.
The Red Panda network has a team of volunteers called the forest guardians. Forest Guardians are local people who are paid to monitor and protect red panda habitat, as well as educate communities. They currently do all their data gathering on paper and manually relay the information to the RPN HQ. The data entry of the manual observations is from pencil and paper which is prone to errors, inaccuracies and can unnecessarily take a long time.
As you can probably see, both processes can benefit greatly from a technological solution. I will primarily be focusing my efforts on the first problem for two main reasons. The first is that this problem can make use of an elegant solution that incorporates AI, IoT as well as cloud computing all using the azure platform. The second reason is that by tackling this problem, I will be saving the employees a lot of time which would allow them to focus their time in other conservation efforts that the red panda network is involved in.
Proposed solution and workflow
My proposed solution leverages the following Azure services:
- Azure maps
- Azure App Service
- Azure Active Directory
- Cognitive Services
- Azure Blob Storage
- Azure Table Storage
In the proposed workflow, users will first be authenticated using Active directory. Once authenticated, users will be able to upload images associated with a specific camera trap and view them on an azure map. The web app will be responsive which will also allow for forest guardians to upload images straight from their phone. Once they have uploaded the images, they will be stored in azure blob storage with the relevant metadata stored in an azure table. When the user uploads an image, they will be classified as being “panda” or “not panda” using a pre-trained classifier and display them on the azure map to reflect their classification.
For ease of development and deployment, I will be using a CI/CD pipeline. Using Azure pipelines, it will be extremely easy for me to make changes to the source code, push it to my repository on GitHub then deploy it so that users can see these changes within a few minutes. I can also define tests that need to pass before the production code gets changed which will uphold code correctness and platform availability.
To ensure the design of the app adheres to Don Norman’s “10 usability heuristics for user interface design”, I will be using bootstrap for the styling of all my components. This will ensure the UI is consistent, learnable and promotes recognition rather than recall.
Add a camera
View Individual panda (Click on panda icon)
View images from a camera trap
Upload New Images
Classification report after model inference
Full Video demo
Online classifier limit
Initially, to classify user uploaded images I was using the customvision.ai API to make HTTP requests that returned the classification. This worked fine for a few images however during my testing, I decided to upload a batch of images that consisted of 15 images. Each image took around 3 seconds to get classified which meant that the user would be stuck on the loading screen for far too long. The logical next step would be to make asynchronous calls to this API to drastically reduce the wait time. The wait time was indeed reduced however this optimization exposed another problem about the API and its rate limit – users could make a maximum of 10 calls per second. To try and work around this, I simulated a pause in my code after every 10 asynchronous calls. This seemed like a crude solution whose runtime was still linear.
To improve upon this, I decided that the best way to tackle this problem would be to download the model locally and classify the images on the server. However this brought about its own challenges as I now had to do image preprocessing to ensure that the user uploaded images were compatible with the trained model. To preprocess the images I made use of the “OpenCV” and “Numpy” modules in python which provided a simple API to scale, crop, resize and update orientation of images of any format. I then made asynchronous calls to this local model which yielded excellent results!
The next challenge I am working on is making the classifier improve itself as the user uploads more images ( and corrects misclassifications ) using the system. To do this I will make use of Azure functions that would get triggered when new images have been uploaded to the blob storage. This azure function would then retrain the classifier with these new images as additional training data.
There will also be logic in the azure function that prevents the classifier being re-trained for every new image as this would get costly and tedious. There would also need to be logic to prevent overfitting so if your interested in learning more about the project join me at the following webinar.
Webinar 22nd April 2021
Project 15: Empowering all to preserve endangered species & habitats.
Date: April 22, 2021
Time: 08:00 AM – 09:00 AM (Pacific)
Topic: DevOps and Developer Tools
Join Microsoft Project 15 and students from University College London live on Earth Day (April 22) at 8 am PT to explore some of the ways the tech community is using IoT solutions to further preservation efforts. Get involved – register for the event now.
Brought to you by Dr. Ware, Microsoft Office 365 Silver Partner, Charleston SC.