Panorama Stitching

Photographers have been shooting panoramas since the development of photography itself. With the advent of digital capture, a number of new avenues of exploration have been opened for panoramic image making. This page discusses the basics of digital capture and processing of panoramic images.

Basics
History
Types of panoramas
Limitations and considerations
Workflow
Shooting panoramas
Stitiching and post-production
HDR panorama photography

Basics

In the age of the digital image, panoramic photography is a technique of shooting multiple images of a single scene and combining them to form a single image with a wide field of view, from a relatively narrow capture of two to three frames along a horizontal to a deep zoom image composed of 30 frames, encompassing far more then the human eye’s natural field of view.

Panoramic images are distinguished from wide angle images by their aspect ratio. An image photographed with a wide angle lens may show a wide field of view (even 180º or more), but contain it within a standard 1:1.33 aspect ratio. Panoramic images typically use wide format ratios – at least 1:2 – to show a scene. Some panoramic images, such as 360º “little planet” projections, may not use an unusually elongated aspect ratio, but will instead show a field of view significantly wider than that of the human eye.

basic panorama
Figure 1 Panoramic images can be as simple as two handheld exposures stitched by hand in Photoshop.

tiny planet
Figure 2 Or they can be constructed from many exposures using complex hardware and software combinations.

History

Panoramic photography as a technique began to appear nearly as soon as the creation of the photographic image itself. Its origins are found in the panoramic painting traditions of the early 19th century. Louis Daguerre, inventor of the daguerreotype, was well known before his photographic revolution for his 1822 Diorama, a Parisian attraction that was a combination of scenic painting and theater. One of the first known dedicated panoramic cameras, patented in 1843 in Austria, was based on the daguerreotype, producing plates up to 24 inches (70 cm) wide with a 150º field of view.

Panoramic cameras became commonplace after the development of flexible film, and a number of dedicated panorama camera models were manufactured, from fixed lens cameras that had flat film planes and stationary lenses, to rotating lens systems with curved film planes, to rotating or “scanning” cameras that captured 360º of a scene.

Due to the cost and difficulty of producing wide-format digital sensors, very few dedicated digital panoramic systems have been developed. Those that are available are very expensive and technically complex compared to similar film-based systems. Some digital point-and-shoot cameras include a “panorama” shooting mode, but this is simply an in-camera crop. Digital photographers are therefore left with the option of stitched panoramas, where the camera captures multiple views of a scene, that is then reconstructed with software.

Types of panoramas

The most basic type of panoramic image can be obtained by cropping a wide angle image into a wide aspect ratio. Though this is a bit of a cheat, it will serve the purpose of producing a wide format image with a minimum of stress.

After basic image cropping, we get into multi-shot “stitched” panoramas. These can be shot handheld or on a tripod, but for maximum accuracy and minimal distortion, specialized hardware must be used to rotate the image around the entrance pupil (often mistakenly referred to as the nodal point) of the camera lens. Rotating the camera around the entrance pupil eliminates parallax errors from frame to frame and allows a panorama to be properly stitched. When the camera is not rotated around the entrance pupil, foreground and background objects’ apparent distances from one another will appear to change as the camera is rotated, thus making proper stitching very difficult or impossible

Single row panoramas

The simplest multi-shot panoramas are comprised of two or more images shot along a single horizontal or vertical axis.

One of the simplest and most fool-proof ways to shoot a simple two-frame panorama is by using a shift lens. The lens can be shifted fully from one side to another, then easily stitched together. Because the lens shifts around its optical center, there are no parallax errors (as long as the camera remains stationary for both exposures), and there is no need for lens distortion or vignetting correction from image to image.

shift lens

Figure 3 A shift lens shifts around its optical center, so there are no parallax errors.

HDR Panorama
Figure 4 This HDR panorama was shot with a shift lens and is constructed from two exposure-bracketed frames.

Using a shift lens to create a panorama also makes shooting HDR panoramas simpler. Since there are only two frames to deal with and no parallax error, stitching is greatly simplified and file management is much simpler than other multi-shot HDR panorama techniques.

Single row panoramas can also be shot with a camera mounting system that rotates the camera and lens around the entrance pupil.

Multi-row panoramas

A multi-row panorama is one where more than a single row of images along an axis is captured.

In order to shoot multi-row panoramas without significant parallax errors, a dedicated camera mounting system capable of rotating the camera around the lens’ entrance pupil should be used.

Pano rig
Figure 5 A rig such as this one from Really Right Stuff must be used to shoot multi-row panoramas without significant parallax errors.

Because more images are being shot in a multi-row panorama, there is a greater need for carefully considered file management. A file naming or metadata convention that identifies images as part of a panorama is advisable, as is separating each set of panorama images into its own folder. Mixing multi-row panoramas and single-frame shots in a single folder of raw files can become very confusing down the road.

360º panoramas

360º panoramas have gained in popularity as the equipment and software for production has become cheaper and easier to use. Very few single-shot 360º camera solutions are available for digital photographers, and those that are available are prohibitively expensive and technically complex. Digital photographers are therefore limited to using compacts or DSLRs and to capture multiple images which are then stitched together.

Depending on the goal and the gear, 360º shots can be captured in relatively few frames...

360 panorama
Figure 6 This 360º pano was constructed from the five images above, shot with a 180º fisheye lens.

...or shot with many, many frames to create complicated deep-zoom images, such as this panorama of Paris

Joiners

While not a seamlessly-stitched panorama image, a joiner has an appeal all its own. The concept has been around for a long time, but its most well known champion is photographer and painter David Hockney, who uses the technique to create images that address motion and time as well as space and dimension.

This is an open-ended technique, but bears mentioning nonetheless.

Limitations and considerations

The restrictions to bear in mind when shooting are the same as many multi-shot techniques, such as HDR capture. In-scene movement, particularly movement in the areas where frames will be stitched, can cause problems.

Scene movement
Figure 7 The boat in this crop from a larger image was moving in the same direction as the camera was panning, and happened to be positioned across a blending plane. It is very important to be aware of moving people and objects in your scenes, otherwise you may end up with someone or something in many places at once.

Some pano stitching software, such as Kolor’s Autopano, can select a particular area of a stitched image and allow the user to determine which frame to include. This is not a guaranteed cure-all, however, and retouching work in Photoshop is almost always inevitable when dealing with movement.

When shooting outdoors, changing light is frequently a problem, especially with multi-row panos. Nothing ruins a pano shoot like getting to the last frame of a scene and realizing that the sun has gone behind a cloud bank. Speed and careful setup become the panorama photographer’s most important allies when shooting outside.

problem flare
Figure 8 Because the sun wasn't in the center of a frame in this Little Planet image, the flare lines don't match up and must be retouched.

Because of the technical, multi-shot nature of digital panorama photography, a significant amount of post-production time is required for almost all panoramic images. Of the images on this page, every one required at least twice the amount of time to produce in the computer as it did to set up and shoot. Post-production is greatly aided by considered planning before the shoot, notes taken during the shoot, and careful file organization when arriving back at the studio.

(Of particular use are notes indicating how many rows and images across comprise a panorama – it can also be extremely helpful to note which frame # begins and ends a pano sequence. This will speed the process of organizing panorama sequences into “shot” folders during editing.)

Workflow

Panoramic photography and high dynamic range (HDR) photography share a lot of workflow similarities. Both have a tendency to create a large number of files, and both often require specialized software, much of which is both more technically complex and less refined (in terms of a GUI) than your usual workflow PIEware.

Considerations

You might be tired of hearing it, but file management is at a premium when shooting panoramas. A “raw files” folder of a shoot that includes three panoramas, two HDR sequences and 50 single-frame shots might not seem like a problem in the days after the shoot, but will inevitably become a frustrating jigsaw puzzle months and years down the road.

Organize your files.

Most panorama stitching software allows the user to save “project files,” which contain the parametric instructions that the stitcher used to blend and create the panoramic projection. Like HDR files, panorama project files are frequently the product of a large amount of work, particularly when a file has been “hand stitched” with custom control points. Project files, like XMP files, are usually small text files, so they won’t eat up hard drive space, and can be a boon down the road.

Shooting HDR panoramas adds another level of complexity. If you are already accustomed to HDR capture, your workflow will be greatly aided by this experience.

 

Shooting panoramas

There are almost as many ways to shoot panoramas as there are to skin cats. Pano shooting can be a simple handheld process or a complex set-up involving dedicated camera mount gear, levels, and lasers (yes, lasers).

All of the techniques we’ll discuss share a few important similarities:

Speed is of the essence
Light will change; people, cars, and clouds will move; security will ask you to get lost.

Plan the shot
It’s important to know where your pano will begin and end. It sounds basic, but it can be very tempting to keep shooting. Before you know it, your three-frame pano has become 30 frames and you have lost track of where you started. Planning is also essential because you need to…

Lock down your camera settings
Know what you’re shooting so you know where to set focus and exposure. Set the camera on manual focus and exposure and leave it there (I go so far as to tape the focus and zoom rings on my lenses in place when I’m shooting). If the focus changes, the panorama won’t blend well and you’ll be doing a re-shoot. Similarly, if the exposure changes, that section of the scene will be darker or brighter than its neighbor frames, and that’s terrible.

Overlap and be aware of your content
In order to stitch properly, frames in a panorama need to have some overlap – 25% on each side is a good rule of thumb. It’s also important to consider where your overlaps will occur. Objects in the scene, small objects in particular, can become distorted during stitching, so if it’s important it should be entirely in a single frame when possible. Additionally, bright objects such as windows, specular reflections, and the disc of the sun, can change depending on the angle of the camera, so be sure that they don’t fall on stitch lines.

Handholding

Handholding is the simplest and one of the most fun ways to create panos. All you have to do is turn and shoot.

A few caveats:

Most stitching software is sophisticated enough to compensate for camera movement that isn’t around the optical center of the lens, but it isn’t necessarily a sure thing. Especially when shoot more than two-frame panos, you will likely need to spend some time in your stitcher hand-assigning control points or else some quality time with Photoshop’s clone tool.

Remember our important points from before: shoot fast, plan ahead, set the camera to full manual and overlap. I generally shoot handheld panos while I’m shooting mostly single frames and the inspiration for a pano strikes. It’s easy to forget to change the camera settings and pre-visualize. Constant vigilance.

 

handheld panorama
Figure 9 This HDR panorama was shot handheld on a 200mm lens and stitched (relatively painlessly) with PTGui Pro. Click for a larger version.

 

Shift lens panos

If you shoot with shift lenses (aka "perspective control" lenses) then you already have a panorama shooting kit in your camera bag. Line up your shot, shift the lens fully to one side of frame, capture, then shift fully to the other side. Because there is a minimum of distortion, the two shots will line up easily and will have the appropriate amount of overlap.

Shooting around the optical center

To create the most precise panoramas – be they single row, multi row, or 360º – your camera needs to be mounted on a tripod in such a way that it will rotate around the lens’ optical center or entrance pupil. Also known as the “no parallax point,” this is the point at which objects in a scene will maintain their relative positions to one another when the camera rotates.

 

Parallax Error example
Figure 10 This image shows what happens when the camera is not rotated around the entrance pupil. As the camera rotates on the tripod, objects do not maintain their same apparent distances. Mouse over to see the same image taken with the camera mounted over the lens’ entrance pupil.

 

To do this we need to have a tripod system that allows the camera to be positioned off the center of the tripod head.

 

Single Row
Figure 11Notice the off-centre camera position

 

Before finding the entrance pupil, there are a few important issues to be aware of, depending on your lens. The easiest arrangement for shooting a panorama would be to shoot with a prime lens, which has a fixed entrance pupil. Zoom lens’ entrance pupils change depending upon the focal length and focus, so it is important to be sure the focus ring does not move. If you are shooting with a fisheye lens (perhaps to shoot 360º scenes), your lens’ entrance pupil is not a single point, but is instead cone shaped and will vary depending upon the angle at which the light enters the lens. The best strategy for dealing with this variable entrance pupil is to determine the correct tripod setting for the areas of the image which will be blended.

Finding the entrance pupil

Here are a few techniques for finding your system’s entrance pupil:

The basic way:

  1. In your viewfinder, line up a vertical line in the foreground with a vertical line in the background. (A lightstand in the foreground and a doorframe work well, as does a piece of tape on a window)
  2. Pan the camera to the left. If the foreground object moves camera right in relation to the background object, then the camera is mounted too far forward. If the foreground object moves left, the camera is too far back. When the camera is properly set to rotate around the entrance pupil, both objects will maintain their positions relative to one another.

 

The grid:

From: http://www.outline.be/quicktime/tuto/TheGrid.pdf

This method is based on “the basic way,” but uses a grid in front of the camera in lieu of two vertical lines. A piece of metal grating with a regular grid can be used, as can a piece of plexiglass that has lines drawn on it.

Set your grid in front of the camera and use it as you would the foreground object in the previous method. The presence of more lines and points for checking proper alignment makes this a slightly more precise technique, and because it includes horizontal lines, the grid can be used to check proper alignment for multi-row panorama systems as well.

The sight:

From: http://www.johnhpanos.com/epcalib.htm

Both of the previous methods are very useful ways to find a lens’ entrance pupil. An alternate approach is to build an “entrance pupil sight” which can be used very precisely to determine if the camera is properly aligned.

 

Pin Sight
Figure 12 A simple sight can be made in a couple of minutes with a needle and a strip of cardboard.

 

  1. With the camera in front of a bright window, lens facing you, set the lens at a small stop – f16-22 is ideal.
  2. Depress the depth-of-field preview button. Looking into the lens you will see a small, bright spot. This is the entrance pupil. Look down the sight and line it up with the entrance pupil.
  3. Rotate the camera on its axis clockwise. If the entrance pupil moves left, then the camera is positioned too far forward. If it moves right, the camera is too far back. Adjust until the entrance pupil remains stationary in your sight, and you’ve found the entrance pupil.

 

Pin Sight 1
Figure 13 When the entrance pupil is positioned in front of the point of rotation, the dot will move in the same direction as the camera body.
Pin Sight 2
Figure 14 When the entrance pupil is positioned behind the point of rotation, the dot will move in the opposite direction from the camera body.
Pin Sight 3
Figure 15 Perfect!

 

This technique can also be used to check that the camera is precisely centered above the tripod base on multi-row systems:

  1. Line up the entrance pupil in the sight, as before.
  2. Rotate the mount on the horizontal axis 180º, so that the camera is facing away from you, then flip the camera over 180º on the vertical axis. If the entrance pupil is in the centered over the tripod, your alignment is correct.

 

Pin Sight 4
Figure 16 Just right.

 

An interesting upshot of this technique is that it will give you a good idea of the horizontal field of view of your lens. The entrance pupil will only be visible as long as the sight is in the lens’ field of view. If you have a pano head with degrees marked on it you can note for how many degrees from left to right the entrance pupil is visible. This is useful information for shooting 360º panoramas.

Getting it all: 360º panoramas

One of the most interesting aspects of panorama photography is the ability to shoot 360º images. Using a multi-row camera mount system means that the entire sphere of a scene can be captured. Combine this with HDR imaging’s ability to record the total range of light in a scene and you can create some very striking images.

Depending on your lens, a 360 can be captured in as few as six frames – four images looking around horizontally (ie north, south, east, and west), and a zenith and nadir (up and down) – or as many as you have patience to shoot. The longer the focal length of your lens, the more exposures will be needed. The basic formula for estimating how many exposures you will need is…

shots required = 1.3 x (hFOV of your panorama ÷ hFOV of your lens)

...where hFOV = horizontal field of view. However, before taking this as written, it is important to consider the content of your scene. It is always advisable to avoid placing objects, particularly small objects, on seams, so you may wish to add additional sectors to your panorama to avoid this. In addition, if your composition includes the disc of the sun, it is essential to place it in the middle of an image. You will almost always have to deal with some amount of lens flare and other optical flaws (like all of us, no lens is perfect), but placing the sun in the center of the frame that includes it will minimize this as much as possible (An important note: always be extremely careful when shooting into the sun. Looking directly at the sun can cause irreversible damage to both your eyes and your camera’s sensor. Don’t look directly at the sun, even through your viewfinder, and keep exposures as short as possible).

Photographing the zenith and nadir

Shooting a 360º introduces several additional difficult technical problems: the sky, and the nadir. Every 360º image will include a zenith and nadir: the shot taken with the camera pointing straight up and the shot pointing straight down.

The zenith is not terribly complex to shoot: the camera points straight up. However, if you are shooting on a lens whose field of view when looking straight up does not include any part of the ground or tall objects in the scene, you will have a very difficult time stitching the image as blue sky and clouds don’t offer very good opportunities for stitching software to create control points. For this reason it is usually good practice to put the camera in portrait orientation when in its shooting mount. This allows for the maximum vertical FOV.

The nadir image poses a particularly difficult puzzle for shooting. When the tripod is normally set, the nadir shot will be predominantly of the base of the panorama shooting rig. Unless you’re a particular fan of self-reflexivity in your photography, you probably need a way to shoot the ground below the camera. If you are shooting at fast shutter speeds it may be possible to take the camera off its mount and handhold it in approximately the correct position to shoot the nadir. Most stitching software can compensate for slight variations from frame to frame, but this is not an ideal situation, and is not possible with slow shutter speeds. What doesn’t stitch perfectly is then left to be cloned in Photoshop.

The most precise method for shooting the nadir is to precisely mark the position of the camera when it is pointing straight down, and then reposition the tripod so that it is no longer directly beneath the camera. A laser pointer attached to the hotshoe can be used to mark a point directly below the camera to record the distance from the sensor plane mark to the ground. An offset arm replaces the panorama mount on the tripod and, using the point marked with the laser pointer, the camera can be carefully repositioned.

 

Stitching and post-production

Most modern panorama stitching programs are based more or less on Panorama Tools by Helmut Dersch, and most share a number of similarities. 

There is a wide pantheon of panorama stitching software, including dedicated panorama heavyweights PTgui Pro, Kolor’s Autopano Pro, and Autodesk Stitcher.

PTGui Pro is the most commonly used and most powerful, but each offers different benefits. Autopano Pro has a simple, streamlined workflow and is able to control motorized panorama heads, such as those made by GigaPan. Autopano is also able to recognize different types of pano sequences in a folder of images automatically, a very handy tool. Autodesk Stitcher has very useful ghost removal tools. All of these programs are able to manage and blend HDR panorama files, and handle 360º images.

The amount of fine tuning and retouching needed after stitching a panorama depends on the quality of the shooting and the stitching itself. Minor stitching errors are par for the course, as are “ghosts” and lens artifacts such as flares and chromatic aberration.

Projections

After shooting any panorama and stitching it, what you are left with is either part or all of a spherical image. Unfortunately your inkjet printer probably won’t print on a sphere, so we must “unwrap” the image. All software tools that handle panoramas are able to redraw a spherical image using several different types of projections. These are closely related to cartography, where a map of a spherical planet must be recreated on a flat surface. Each type of projection creates different types of distortion, so it is up to the photographer to decide what projection is appropriate for a given image.

There are three main types of projection: flat (or rectilinear), cylindrical, and spherical (or equirectangular).

Flat projections have minimal distortion of straight lines, and are frequently useful for architecture. This is the type of projection that looks most natural to us, as it is the type of image produced by our normal camera lenses. However, its practical field of view is limited to a 90ºx90º maximum field of view before the image begins to distort wildly, so it cannot reproduce a full 360º image.

Cylindrical projections are similar to flat projections along the north-south axis, but do not curve horizontal lines on the “equator” of the spherical image. The upshot of this is that 360º can be projected horizontally, and up to 120º vertically with distortion of horizontal lines. The practical field of view of a cylindrical projection is limited to around 360ºx90º. It is not possible to project the poles (nadir and zenith) of the image unless you happen to have an infinitely tall roll of printer paper.

Spherical projections are able to reproduce the entire 360ºx180º field of a sphere using nonlinear math. The poles of these images become wildly distorted, but they are there, nonetheless. These images appear similar to cylindrical projections around the middle latitudes, but become increasingly distorted towards the poles. Spherical projections for print (or anywhere they are displayed as a single image) are frequently cropped at the poles as the distortions become fairly extreme. This is the most common projection used for interactive panoramas on the web; because we are looking only at a small portion at a time, the sections that are distorted can be constantly changed as the area being viewed changes.

Other types or projection are available, depending on the stitching software. Mercator projections are based on cylindrical projections, but don’t stretch the view as much along the outside vertical edge. Vedutismo projections preserve all straight diagonal lines through the center of the image (think about landscape paintings with a distinct vanishing point). This is most useful for images with a distinct vanishing point, such as a river leading off into the horizon. Stereographic projection is a way of projecting the image sphere onto a flat plane as it would be seen from the pole. This creates “little planet” images, where the scene seems to float in the sky.

HDR panorama photography

If you’re already going so far as shooting panoramas with a dedicated camera mount and stitching software, you might as well be shooting HDR, right?

All the workflow and file management considerations associated with HDR image capture apply to panorama photography. Please do take some time and read the HDR section of our site.

The major issue with HDR pano production is when to create the HDR file. The “traditional way” would be to create a stitched panorama for each exposure level in the stitching software, then use HDR tonemapping software to create the HDRI file. But we’re living in the modern age. All of the panorama stitchers mentioned above are able to handle both the stitching and creation of HDR files in one fell swoop. It’s that easy.

It is important when exporting the stitched HDR panorama to consider how large the final image will need to be. Panorama files can be very big – after all, they can have a 360ºx180º field of view. Even a six-frame pano from an eight megapixel camera could approach a 36MP image after stitching, if not sized down. While this might not seem excessive, think about the file that would be created from a 30-frame panorama taken with a 22MP camera. Most tonemapping software simply can’t handle files that size. It is necessary to test the file size limits of your tonemapping software and computer system to determine what its file size limit is. However, there is hope; as 64-bit operating systems become the standard and tonemapping software is rewritten to use a 64-bit core, this limitation will be minimized.

Up to main Image Editing page
Back to HDR Image Editing

feedback icon
Last Updated September 22, 2015