This is a follow up to Voxel from Multicam and is part of a series where I try to train models to perform common self driving tasks from scratch.
I’ve previously put together occupancy models for self driving but that’s only one specific perception task.
Another common driving task is semantic segmentation. Semantic segmentation takes in the image and for every pixel predicts a specific class. This can be used to tell things like walls apart from cars or classify different types of lane lines and curbs on a road.
This is a follow up to Monocular Depth Improvements and is part of a series where I try to train models to perform common self driving tasks from scratch.
Background I spent a couple of months optimizing single camera (monocular) depth models before realizing that maybe there’s a better way. One of the biggest improvements I made to the monocular models was adding a 3D geometric constraint to enforce that the model didn’t predict depths below the ground.
This is a follow up to DIY Self Driving.
In the past few months I’ve been iterating on my previous work on creating self driving models. The main goals were initially:
train depth models for each camera generate joint point clouds from the multiple cameras use the fused outputs to create a high quality reconstruction that I can use to label things like lane lines This post lists all the various problems I ran into and some of the mitigations I applied for those issues.