Week 8 (Jul 22 - Jul 28)

Meeting with Mentors

• I showed the mentors the UI for the inference tab I developed. The major feedback they had was regarding the loading time and the consistency of the output. Sergio let me know that the model inference should take not more than a few seconds.
• Santiago suggested that I note down the time it is taking and also check the device used and if the model is loaded multiple times. David suggested that I give the user information about what is happening in the background actively in the UI. They also suggested that the user should be able to give image size as a parameter in the model config.
• David also talked about the things he planned to do once the UI is done, like extending the metrics and extending for YOLO datasets. But before moving on to that, the main thing that is expected is to have a fully functional UI prototype.
• We just finished the mid-term evaluation this week. They let me know about the things that I can improve on, and the main thing is being able to notice the outputs and evaluate them. Basically, an intuition that I need to slowly develop to be able to look at the output and identify if something is wrong.

To Do for This Week

• Fix the issue with loading time
• Refining the UI
• Working on metrics tab (if time permits)

Progress

• I started with reducing the loading time in the dataset viewer tab. Initially, I noted down all the loading times:
  • App load: 7 secs
  • 12 images load: 11 secs
  • Ground truth load: 10 secs
  • On clicking search button: 8 secs
  • Upon clicking an item in the list + go to image: 28 secs
• App load would take 7 secs to import the libraries and modules required for the application. But the rest of the loading times were too high. At first, I tried caching the annotations for all the images in the dataset. That reduced all the image and ground truth loading times, but the initial dataset loading was taking around 42 secs to cache all the annotations. Later, I realized that fetching the annotations per image isn’t the problem, but the complete annotations being loaded every time was the main problem. Storing the dataset_name in the session state of Streamlit made sure that the same dataset object was not loaded repeatedly, which drastically reduced the loading times on the page. Now the optimized loading times are as follows:
  • Page load: 7 secs
  • Loading dataset initially and storing in session state: 11 secs
  • Ground truth load: 2-3 secs
  • On clicking search button: 2 secs
  • Upon clicking go to image: 4 secs

• Then I moved on to the inference tab. I added debug statements with time information to check which part of my code was slowing the inference down. Looking at those debug points, I understood that the inference() function defined in my TorchImageDetectionModel class was the main culprit. It was taking the same time even in my sample notebook. I tried a bunch of fixes, but none of them worked for me. I reached out to my mentors, and after taking their suggestions, I tried using just the CPU. Turns out the main issue was caused by my Mac MPS. The code architecture or the UI didn’t have any issues that had to be fixed. David also tried it on his machine and didn’t notice any issue either. The inference is loaded in 2-3 secs the first time and less than a second from the second time onwards.

• I then moved on to enhancing the inference page UI. The earlier UI was clumsy. Now I added the option to save the model first, and then it can be reused to run inference on multiple pictures.

What’s Next?

• Work on metrics tab and finish the UI prototype
• Minor fixes/enhancements:
  • Add image size to the model config
  • Check why MPS was very slow
  • Fix issue in dataset tab where after clicking “go to image”, the orange highlight isn’t on the right picture
• Test the UI and fix any issues found