Just turn up the light and wave the camera faster, or, if you can't, maybe reduce fps if you need to conserve space at the time. Using only a covering subset of images with consistent redundancy for depth map reconstruction is easier after initial Structure-from-Motion than finding ways to recover from cases of under sampled geometry.
And with video data the initial SfM isn't nearly as expensive as with unstructured picture collections, and it's even cheaper if you get accelerometer and gyroscope data from the camera to initialize the motion recovery part and help feature matching to skip known-not-overlapping views.
You actually only use ~10 to ~40 degree view angles for depth map reconstruction, because of occlusion and texture distortion degrading results at high angles.
And with video data the initial SfM isn't nearly as expensive as with unstructured picture collections, and it's even cheaper if you get accelerometer and gyroscope data from the camera to initialize the motion recovery part and help feature matching to skip known-not-overlapping views.
You actually only use ~10 to ~40 degree view angles for depth map reconstruction, because of occlusion and texture distortion degrading results at high angles.