Coding Period Week 1

The first week of the coding period was focused on testing the Cityscapes dataset adapter I started during the community bonding period, improving its implementation based on feedback from my mentors, preparing the tutorial for Cityscapes image segmentation, and exploring the SemanticKITTI dataset to understand how to support it in PerceptionMetrics.

Improving the Cityscapes Adapter

This week, I continued working on the Cityscapes dataset adapter and updated it based on the feedback and merges from last week. The main goal was to make the adapter fit better with the existing PerceptionMetrics dataset interface and ensure that it can be used cleanly from the evaluation pipeline.

I worked on cleaning up the adapter implementation, registering the dataset so that it can be built through the common dataset creation workflow, and fixing the arguments passed through build_dataset. I also restored the required cityscapesscripts dependency, since the adapter uses the official Cityscapes label definitions to build the ontology and class mappings.

Another important part of the work was checking how the adapter handles Cityscapes train IDs and label IDs. This is important because many pretrained segmentation models use train IDs, while the original dataset also provides raw label IDs. Making this configurable should make the adapter easier to use with different evaluation setups.

Cityscapes Tutorial

I also completed preparing a dedicated Cityscapes image segmentation tutorial for PerceptionMetrics. The tutorial shows how to load the Cityscapes dataset, configure the ontology, use a pretrained segmentation model, and export the data in a format that can be evaluated by the existing metrics pipeline.

For this, I experimented with a pretrained SegFormer model fine-tuned on Cityscapes and checked how its predictions align with the ontology generated by the adapter. This helped me verify the adapter beyond just loading files, because the tutorial exercises the complete flow from dataset loading to prediction and evaluation preparation.

A screenshot of the tutorial is shown below, where you can see the predicted segmentation masks overlaid on the original Cityscapes images.

Exploring SemanticKITTI

In parallel, I started exploring the SemanticKITTI dataset to understand what will be needed for future support. Since SemanticKITTI is a LiDAR semantic segmentation dataset, it has a different structure from Cityscapes and requires working with point clouds, binary label files, and sequence-based organization.

I looked at the existing LiDAR utilities in PerceptionMetrics and compared them with the SemanticKITTI data format. This helped me identify which parts of the current codebase can already be reused and which parts will need a new dataset adapter or additional configuration.

David also suggested using the MMDetection3D codebase as the model zoo for SemanticKITTI, since it has many pretrained models and a well-established evaluation pipeline. The adapter will follow a structure similar to the usual LiDAR dataset adapters. This will involve loading the point cloud data, matching it with the corresponding label files, and building an ontology based on the SemanticKITTI class definitions.

The visualization of point clouds and predicted labels using VisPy looked very good, and David suggested comparing both Open3D and VisPy for this part. This will be important for the tutorial, since visualizing point clouds with predicted labels is a key part of understanding the results. David also suggested experimenting with the LiDAR utilities available in torch_segmentation.py.

Below is a screenshot of the beautiful visualizations of SemanticKITTI point clouds and labels using VisPy, which will be a great reference for the tutorial.

Next Steps

This week, David merged both the Cityscapes adapter and the tutorial, which is great progress. In the coming week, I plan to start working on the SemanticKITTI dataset adapter, focusing on loading point cloud data and matching it with the corresponding labels.

I also plan to experiment more with the existing LiDAR utilities, compare Open3D and VisPy for visualization, and start preparing the structure for a future SemanticKITTI tutorial.