Last week, we introduced you to the concept of photogrammetry--using a series of photo images to computationally map a 3D model or space. We discussed the current state of photogrammetry, including what software is available for consumers and what kind of hardware you need. Turns out, photogrammetry is pretty accessible, and you can do a lot even with free tools like Autodesk's 123D Catch and your smartphone camera. Of course, more advanced software, more processing power, and better camera equipment can go a long way to improving your models. But so can the simple act of taking better source photos. Today, I'm going to give you some tips about how to best take your photogrammetry photos to give that computing software the best references to output a clean(ish) 3D mesh.
By far, the biggest impact on the final output file is what happens in the shooting phase. In fact, it is usually easier to reshoot a new series of source photos of your subject then try to save a computed capture that’s not working right away. Take some time, think, try to visualize the computer aligning your photos. Try to think of angles you have missed. When you are done shooting, I recommend loading up the images and see how well the images align as soon as possible. If certain images are off or are confusing the software, re-shoot them while you still have access to your subject. It may be necessary to reshoot multiple times for one model.
Ideal Conditions for Photogrammetry Software
This part isn't too complicated. Your software will want a nice, clean, sharp, evenly lit image, with every surface of your subject visible from three or more angles. The software will also like a good amount of parallax (different positions and angles) between those images to do its calculation. And it'll really be happy if undesired parts of the image, not part of the subject, are masked off, either with a green screen or in an image editor.
Sounds simple, right? Well that's because it's easier to explain what photogrammetry software likes by giving you examples of what kind of imagery it doesn't like. That's because a lot of photography flourishes--depth of field, dramatic lighting, wide-angle distortion, etc.--are actually counter-productive to the task of photogrammetry. Below, I'll go in-depth through the image qualities that will confuse your software and produce bad models.
Image Qualities that Negatively Affect Photogrammetry Software
Good Focus is one of the easiest things you can do to improve your scan. Carefully adjust the focus to include the important parts of the image in your composition.
If you are scanning a person, it is imperative that they hold absolutely still. Have your subject pretend time is frozen, or that they are a statue. If they smile or track you with their eyes as you take pictures, the relative positions of the features on their face will change. When the computer tries to compare those photos, it will either try to blend the feature (eyes come out looking all milky and weird) or think it is at a different depth.
Transparent and Shiny Objects
The problem with clear or shiny objects is that the patterns the computer looks for on the the object, do not move with the surface of the object. On a wineglass, for example, you see the reflection of the room as well as a distorted version of whatever is behind the glass. Current photogrammetry software is not smart enough yet to distinguish between a white dot on on object and the reflection of light--it assumes all surfaces are opaque and non-reflective.
Very Thin Objects
When the software starts working on your photos, it begins by generating a point cloud. That's basically a 3D model made of dots instead of triangles. When an aspect of your subject is very thin, the software cannot place enough dots across that component to figure out its shape. It is also very difficult to place the point in the exact same place along the length of the thin object.
Since the software is tracking points across several images many crossing objects will block many of the points in one photo or another. This is especially bad when the crossing objects look very similar (as in the leaves on a tree), because the software will not be sure if a point is further away or actually a different object entirely.
Very Plain Featureless Textures
If there are no features to track the software cannot rebuild depth, as in a blank wall, white bowl, or plain skin with even make-up.
Very Repetitive Features
While tracking the relative movement of a certain pattern, the software can easily make a “jump” to an identical nearby pattern. It will perceive this jump as movement in your photography and assume the pattern is at a different depth instead of along a plane. Examples include a honeycomb or picket fence.
Blinking or Moving Lights
Televisions playing video, Christmas tree lights, and LCD screens with a very narrow viewing angle will have features that change between your images. It is best to try and make these static during the shoot or mask them out after the fact.
I’ve had people spin in a chair while shooting video or photos burst mode, and then using those sequential frames for photogrammetry. This can work, and make your scan go quickly, but the shutter speed of most cameras isn’t fast enough. Slow shutter can also introduce blur in the direction of the moving object. If a feature is blurred for four pixels, that effectively quarters your resolution in that direction.
Depth of Field
You want as much of your subject in focus as possible, this means you should usually stop down to between f/8 and f/16. Each lens will have a different point where diffraction sets in and the higher f-stop will start to reduce overall sharpness, rather than increase it. Most common lenses can be looked up on a website like DPReview.com to find the best f-stop for your lens.
Generally a lens with a low amount distortion will perform better than something like a fish eye lens that has a large amount of barrel distortion.
Most cameras use a CMOS type sensor that reads each line of pixels at a slightly different time, this means light from a moving object will be read at a slightly different position, leading to a wobbly looking distortion called rolling shutter. This will make the software think the object is changing shape. This is especially prevalent in cell phone and tablet cameras, when there is high frequency vibration. Hold your phone or camera as still as possible when you take each picture.
A highly compressed image will create false features for the software to try and track. A raw image or a hi-quality JPEG will not suffer from this issue, so save your photos at the highest quality possible.
Sometimes you can get away with it, but generally speaking, a static light will give you a more consistent result. A moving shadow will sometimes be tracked as false movement.
Shadows, even fairly subtle ones, usually on the downward side of an object can suffer from excessive noise and track badly, even leaving holes in the model.
Under or Over Exposure
A photograph that is significantly under or overexposed will lose usable detail, and if the exposure is inconsistent, it can cause light and dark patches on the model.
Shooting toward the sun or other bright light can create a lens flare, a halo that will move with the camera and affect the points it crosses.
A Typical Photogrammetry Photo Shoot Setup
Unless you have multiple cameras, there are two standard ways to set up a photo shoot for photogrammetry. The first is to put the camera on a tripod, and rotate your subject using a turntable or office chair. The second is to put the subject in the center, and you then move with the camera around the outside taking pictures. Both techniques have their pros and cons. And both are appropriate for different scenarios. Always shoot extra photos to make sure you have enough angles. Too many images may overwhelm the software (especially if you don't have enough system RAM), but extra images will give you the luxury of picking the best shots after the fact.
Set Up Your Lights
You’ll need soft light to evenly illuminate your subject. You can setup a shoot outside on an overcast day, or diffuse the sun with translucent plastic sheets, and white or aluminium-wrapped foam core to bounce light on the opposite side and underneath. If you are unable to use the sun, set up lights all around your subject, and diffuse them if necessary.
Set Up Your Camera
If you are a using a DSLR with a cropped APS-C sensor, use a lens between 28mm and 100mm. Select the lens based high sharpness and low distortion at about f/8. If you are scanning a person, a 50mm or 85mm prime lens will have minimal distortion, and put you a good distance from the subject (approximately four feet).
There should be a good deal of overlap in each picture, so try to include about 50% of what was visible in the last photo.
As for settings, f/8, ISO 400, shutter speed 1/30 Is a good starting point. If you do not have enough light, increase the ISO to 800 or the shutter speed to 1/15. A tripod makes this slow of a shutter acceptable, and a full-frame camera makes using higher ISOs OK as well.
Pick a place to start. Since the photos are usually processed in order, it is better to start from an angle where you can see a lot of detail, like the front of a face. Take a test shot. Zoom in to 100% on your camera's LCD and make sure that every part of your subject is in focus. Compare the part of your subject that's nearest to you to the farthest, for example the tip of a nose with the back of an ear.Take some test photos to make sure your subject is in focus, and that the ISO doesn't make the image too noisy.
There should be a good deal of overlap in each picture, so try to include about 50% of what was visible in the last photo.
The Walk-Around Method
Position your subject so that you have plenty of room to walk all the way around. Walk around once to make sure that you can see all parts that you want to capture, and that your own body doesn't cast an obvious shadow on the subject. The subject should be high enough to allow you to shoot the underside details (like beneath a chin), but low enough so it is not too difficult to shoot from above. A stepstool can help! Position the camera at a height that will give you a good view of the important features. For a plate of cookies this should be pointing down about 45 degrees. For a person the camera should be around the same height at eye level.
Once you have found your height, check all your camera settings, pause to steady the camera, focus, and take your first shot. Check the shot. If everything looks good, take shots around the subject around every 10 to 15 degrees, pausing to steady yourself between each shot. Once you have done a 360 degree ring around the subject, you should have taken between 24 and 36 pictures. Now look for parts of your subject that are not clearly visible from the first set of pictures. If your subject is a person, the common areas are: behind the ears, under the chin, and the top of the head. Push in closer and take 2-3 pictures of theses areas. Taking additional rings of photographs may be necessary.
The Turntable Method
The turntable method can be very useful if you have time for setup. You’ll need something to rotate your subject, like a Lazy Susan, or an office chair that spins.
You’ll need a tripod for you camera that you can adjust to shoot above and below your subject. Light setup is easier with this method because you only need to light for one angle, and you can put the lights very close. Put your diffused lights to the left and right of your subject just out of frame. Adjust and add light until the light is very even with no obvious shadows or highlights. If you are using a green screen, light it as well. The more even the light on it, the easier it will be to take out later in PhotoShop after.
To set up a green screen, place it about six feet behind the subject and make sure it covers the background for all the angles you want to shoot from. Just as with the other method, position the camera at a height that will give you a good view of the important features. (45 degrees for a plate of cookies, etc).
Shoot between 20 and 60 Photos. Too many photos may overwhelm the software, so review and remove bad and redundant shots. If you can test the footage on your photogrammetry software, now is the time to do it. If not, take another set or two of source photos for safety.
The Turntable method will be easier to setup if you are using artificial lighting. It will also be faster to turn the object than move the camera. It also makes it easier to use a green screen, since it stays in one place.
The Walk-around Method requires less setup if you are shooting outdoors or otherwise don't need to set up lights. If you are scanning a person, they will have an easier time keeping their eyes fixed if they are not spinning around on a chair.
And that's basically it! Remember, as long as you used good technique, you can always reprocess your footage with a newer version or a different piece of software later on. Or even process footage that was not meant for Photogrammetry. Your source photos, moreso than any other element, determine the quality of your photogrammetry-generated 3D model.
Next time, I'll walk you through the complete process of one photogrammetry shoot, from the taking of the photos, to the software processing and 3D printing of a model. See you next time!