Revised version of the tutorial "How to make a spherical 3D panorama with a single camera and a fisheye lens"

[Wim.Koornneef]

MODERATOR NOTE:
Many of the links are dead however Wim has provided us the source files to the images referenced in this thread.
To download these files, unzip and view click:
http://www.nodalninja.com/forum/3D_r...gle_camera.zip
***********************************
***********************************



Hello Forum,

In this tutorial I will give some general instructions and explain step by step how to;
- shoot the images,
- make a left and right eye panorama with PTGui 9,
- make an anaglyph panorama,
- make an interactive 3D panorama,

I also tell a bit about the shape of the mask for PTGui and at the end of the tutorial is a Links section for the download of software, the tutorial with screenshots, and some examples.
I suggest that you read all steps of the workflow and download the tutorial with screenshots before you start making your own 3D panorama.

----------------------------------------------------------------------------------

General instructions:

Use a fisheye lens with a FOV of 180 degree with the camera in portrait position.
Take care that there are no objects in the scene closer then 1 m to the lens.
Only shoot outdoors when there is no wind at all.
Avoid dynamic scenes, you really need a static or "frozen" scene.
Use a sturdy tripod, a good panohead and level the panohead carefully.
If your rotator has a little wobble, or your tripod or pole isn't really stable, then the tilt and roll can vary between shots and then you have to use a different method for optimizing then described in this tutorial, this is because the method I describe is based on a linked tilt and roll for all images for optimizing.

----------------------------------------------------------------------------------

How to shoot the images:

1) Set the upper rail of your panohead in the horizontal position (zero tilt) and shift your camera forwards on the upper rail out of the NPP of the lens.
When shooting in a room with objects relatively close by then shift approx. 30 mm forwards, when shooting in a larger space then shift between 30 and 60 mm and when shooting outside shift 60 mm or more.

I suggest that you start with a small shift to get familiar how to make 3D panoramas and try out larger shifts later when you are more experienced.
The more shift you use the more 3D depth you will get in your panorama. Having to much 3D depth in a panorama is always a bad thing, on the opposite, when having not as much as is possible 3D depth this will be accepted by most people.

2) Set your rotator to the proper number of images.
Use the table below as a guide for the number of shots needed for a given lens shift, using to much images is never a problem, using not enough images will cause stitching errors.
When shooting closer then 2 meter to objects you really need more images then stated in the table to avoid stitching errors, f.i. when shooting at 1 meter distance limit the forward shift to 4 cm and shoot 50 instead of 25 images.

shift 2 cm > 15 images
shift 3 cm > 20 images
shift 4 cm > 25 images
shift 5 cm > 30 images
shift 6 cm > 35 images
shift 7 cm > 40 images
shift 8 cm > 45 images
shift 9 cm > 50 images
shift 10 cm > 55 images

3) Set the camera in M mode for a fixed exposure of all images, set the white balance to a fixed setting and the focus of the lens to Manual with a fixed distance setting.
Shoot all images around and take care that you don't move the gear by accident (use a remote control).

Process the images for removing CA, enhancing sharpness and all other stuff you normally do with your fisheye images.

----------------------------------------------------------------------------------

How to make a left and right eye panorama (equirectangular) with PTGui 9:

4) In PTGui Pro 9 you load all images, apply a lens calibrated template and setup your template for advanced use.
Let PTGui automatically place the cp's.

Optimize for lens shift parameters "d" and "e" and image parameters yaw, pitch and roll in steps with linked roll and pitch for all images and remove all cp's with errors of 8 px or higher.
Avoid optimizing for the image parameter pitch and the vertical lens shift parameter "e" in the same step when you start optimizing as it will give issues with high and wrong values.
When the cp errors are all below 8 px and the pitch and the vertical lens shift parameter are more or less normal then the optimizing is done.

5) Open the Mask window and the Panorama Editor window, select the first image in the Mask window and draw a red mask with a large unmasked part that looks like a half moon shape in the right half of the fisheye image.
You can see the result of your drawing in real time in the Panorama Editor, please keep in mind that the shape of the mask is more important then fine tuning of the edges of the mask.
When the mask looks fine then save the mask as "left_mask"

Instead of drawing your own mask from scratch you can also download the default mask that I used for this tutorial (see the Links section at the end of this tutorial), resize it to the image size of your camera and modify the shape of the mask in a graphic application (f.e. Photoshop), after saving the mask you load it in the first image.

6) Open to the Source Images tab.
Control-click (right mouse click) on the first image and copy the mask with the mouse menu.
Select all images and paste the copied mask to all images with the mouse menu.

Important, if there are moving objects in the scene you have to manually mask them out of the image, this can be a lot of work as you have to check -every- image for moving cars, people, birds, etc.

7) Go to the Create Panorama tab and set the settings just as you normally do for your camera and lens.
Now save the template and name it whatever you like as long as the word "left" is in it.
Create the panorama, this will be the left eye image (equirectangular).

8) Open the mask that you saved earlier in step 5 in a graphic application, flip the mask horizontal and save the mask as "right_mask" in PNG format.
BTW, you can re-use the masks for other 3D projects as long as you use the same camera and lens (and the same zoom setting if you are using a zoom fisheye lens).

9) Go back to PTGui's Mask window, select the first image again and load the "right_mask" image.
Repeat step 6 and 7 but this time you save the template and the image with the word "right" in it.
Create the panorama, this will be the right eye image.

----------------------------------------------------------------------------------

How to make an anaglyph panorama (equirectangular):

10) For making a red/cyan anaglyph you use the application StereoPhoto Maker (Windows only - free), for making an amber/blue anaglyph you use ColorCode 3D Editor (Windows only - not free).
On OSX you can run StereoPhoto Maker with Wine for OSX (free) but for ColorCode 3D Editor on OSX you need a emulator, f.e. Parallels or Bootcamp
See the Links section for download links.

To avoid stereo window violation the left and right eye images must be to set to the proper zero parallax point, this is the point where you will not see a colored edge at the left and right side of an object when viewing the anaglyph without a viewer.
The setting of the zero parallax point is done by aligning the left and right eye images to objects in the foreground of the scene, the aligning procedure is different for StereoPhoto Maker and ColorCode 3D Editor and therefore described in different steps..

Continue with step 11 for making an amber/blue anaglyph panorama with ColorCode 3D Editor.
Skip step 11 and go to step 12 for making a red/cyan anaglyph panorama with StereoPhoto Maker.

11) Aligning and making of an amber/blue anaglyph with ColorCode 3D Editor.

When using ColorCode 3D Editor you have to align the left and right eye images in a graphic application, this is because ColorCode 3D Editor is designed to work with rectangular images and can not wrap-shift images over the 0/360 border when aligning a 360 degree panorama.

When you use Photoshop for aligning then open the left and right eye equirectangulars, copy one of the images on top of the other one in a separate layer and reduce the Opacity of the new layer to get both layers visible.
Select the background layer (make sure that it is highlighted !) and with the menu option "Filter", the option "Others" and the sub menu "Offset" you align the background layer in such a way that objects close to nadir are on top of each other (only shift in a horizontal direction !).
After deleting the copied layer and after flatten the image you save the aligned panorama.

In ColorCode 3D Editor you open the aligned left and right eye images and set the gamma (try 1.2 for a start).
Don't use any other option unless you know what you are doing, the reason for this is that most options of ColorCode 3D Editor can't be used for making 360 degree anaglyph's !
Inspect the anaglyph with your ColorCode 3D viewer on and save the output as stereo image in PNG format when you are happy with the result.
Continue with step 13.

12) Aligning and making of a red/cyan anaglyph with StereoPhoto Maker.
In StereoPhoto Maker you open the left and right eye images with the File menu option "Open left/right images, choose in the View menu the option "Panorama Mode (360 degree)" and the option "Fit panorama height to screen".

With the menu option "Stereo", the option "Color Anaglyph" and the submenu "Ghost-reduced Anaglyph" you get a window with 2 histograms. Set the slider of the Contrast of the upper histogram to -20 [Lab] and the slider of the Contrast of the lower histogram to +10 [RGB] and apply the settings.

Now the images must aligned in such a way that objects close by in nadir don't have colored edges.
Important, don't put your red/cyan viewer on (otherwise you can't see the colored edges) and only align the images in a horizontal direction with the left and right arrow keys on your keyboard.

After aligning the images you put on your red/cyan viewer and check the overall brightness of the anaglyph.
If the image is a little to dark then you need to compensate the gamma, you do this with the menu Adjust and option Color adjustment, adjust the gamma for the left and right image with the same numbers.

After this you save the anaglyph panorama with the "File" menu option "Save Stereo Image …".
Important, do not save the anaglyph panorama as a normal JPEG image because this will cause color ghosting in the anaglyph, only save as PNG image (preferred) or as special ghost reduced JPEG file.

----------------------------------------------------------------------------------

How to make an interactive 3D panorama (Flash, HTML5, WebGL):

13) Use the anaglyph panorama as input in your panorama software and process the image just as you do with any other panorama.

Important, if your pano software have an option to disable color subsampling for JPEG tiles then use this option to avoid color ghosting in the output.
When using Pano2VR v3.1 (or later version) color subsampling is automatically disabled when setting the compression quality of the JPEG tiles to 90% or higher.
If your pano software dousn't have an option to disable color subsampling of the JPEG tiles then use the workaround as described in the tutorial "How to avoid color ghosting in anaglyph panoramas" (see Links section).

----------------------------------------------------------------------------------

About the shape of the mask for PTGui:

The default mask with a simple large half moon shape will do fine in most situations but to get more or less 3D depth or to get 2D in nadir and zenith you can use small shaped masks (see the Links section for examples).
Be careful when using a different mask then the default mask, with small shaped masks it can happen, depending on the width of the mask and the contrast of objects in the scene, that you get stains, brightness or contrast errors in the output caused by the blender of PTGui (PTGui blender, Enblend and Smartblend all have the same issues).

When you have blender issues and really needs to use a small shaped mask it is best to output the panorama as layered Photoshop image (ignore the 2 or 4 GB warning of PTGui), open the image in Photoshop, enlarge the canvas approx. 10% at one side (left or right side), copy and shift the layers to the enlarged part to get a seamless image, blend the panorama with Auto-blend, flatten the image, reduce the canvas to the original size and shift the image if you want to get the original position (see step 11 if you need to know how to shift the image over the 0/360 border).
The enlarging of the canvas and the copying of the layers is needed to avoid a visible 0/360 seem as Auto-blend is not capable to wrap-blend over the 0/360 degree borders of the image.
Although the result with Auto-blend is most times very good even then it still can happen that the output has blender artifacts, if this is the case then you have to modify your masks or manually retouch the output.

Bottom line, first try the default mask before you use a smaller shaped mask.

----------------------------------------------------------------------------------

Links section (all files with short URL's are stored on my website):

The 3D panorama I used for this tutorial:
red/cyan http://tinyurl.com/3b3zu2z
amber/blue http://tinyurl.com/3kzxuew

Photos of the setup:

http://tinyurl.com/3djmrhg

http://tinyurl.com/3qsqkex

The tutorial "How to make a spherical 3D panorama with a single camera and a fisheye lens" (PDF 2,9 MB):

http://tinyurl.com/3mbsx9c

The tutorial "How to avoid color ghosting in anaglyph panoramas" (PDF 1 MB):

http://tinyurl.com/3oen2e5

The left eye mask I used for the tutorial:

http://tinyurl.com/3gb2ctc

Example of a mask with more 3D depth:

http://tinyurl.com/3q4xnn6

Example of a mask with less 3D depth:

http://tinyurl.com/3ef9zm4

Example of a smart mask for a 3D panorama with a 2D part in zenith and nadir:

http://tinyurl.com/3ty3zla

ColorCode 3D:

http://www.colorcode3d.com

StereoPhoto Maker:

http://stereo.jpn.org/eng/stphmkr

Wine with Winebotler for OSX (only for StereoPhoto Maker, not for ColorCode 3D)

http://winebottler.kronenberg.org

Here is a video of my talk at the PanoTools Meeting 2011 in Vienna about the making of 3D panoramas with approx. the same info as this tutorial.
Beware it is a 50 minute talk ;-)
http://blip.tv/panotools-meetings/pt...wk-sm2-6133442

----------------------------------------------------------------------------------

I appreciate your feedback about this tutorial, if anything is unclear then just let me know by using the Reply button of this posting.

Success,
Wim

[Hugh]

Thanks Wim!

Thanks for sharing your information with us.
I am ready to explore the possibilities of 3D Panoramas once more and hope that the new functionality in PTGui will make the job a little less labour intensive and time consuming.
I think the results are great when the final panorama is made, but I have found it to be a "long road".

All the best, Hugh.

[tonesh]

Hello Wim,
I have some things to understand:
1) Is the number of images unrelated to the the focal length ? And what type of lens are you suggesting?
2) When you say "Optimize for d e y p r" what do you mean? I know that a,b,c parameters are for the lens correcting, and I guess that I do not have to change them,
so it remains the d and e parameters that are for vertical and horizontal shift, so what are the y,p,r parameters?
3) Do you think I can use "Anaglyph Workshop Software" that runs on Mac instead StereoPhoto Maker?

Thanks & ciao

Toni
StudioArgento

[Wim.Koornneef]

Hello Toni,

Good to hear from you :-)

About your questions:
1) Is the number of images unrelated to the the focal length ? And what type of lens are you suggesting?

Any lens and camera combination that can be used to make a spherical pano with a single row of images will do fine.
It don't make any difference if you shoot with a crop size sensor camera and a 8mm fisheye lens or a 10-12mm fisheye lens on a full frame sensor camera, the number of images will be the same.

2) When you say "Optimize for d e y p r" what do you mean? I know that a,b,c parameters are for the lens correcting, and I guess that I do not have to change them, so it remains the d and e parameters that are for vertical and horizontal shift, so what are the y,p,r parameters?

Thanks for bringing this up, I have replaced some of the abbreviations for the parameters in the tutorial to avoid confusion.
You got it right, the d and e parameters are for correcting the lens shift and the y p r parameters are for correcting the yaw (y), pitch (p) and roll (r) of the images.

3) Do you think I can use "Anaglyph Workshop Software" that runs on Mac instead StereoPhoto Maker?

I am not familiar with "Anaglyph Workshop Software" but if you can open a left and a right image, align the images over the 0/360 border and can output as an anaglyph image then the application is suited for the job.
If the application is made for "normal" 3D images then I guess the feature to shift the image over the 0/360 border is missing and then you have to do the aligning in Photoshop, just as you have to do when using the ColorCode 3D Editor.
Most times the shift for aligning the left and right images is very small so you could try to output the anaglyph without the aligning step, there is a good chance that the result is still fine.

Success and if you encounter any problem then please feel free to ask for assistance.

Best,
Wim

[tonesh]

Thanks Wim,
now you got me into the 3D pano experimenting hell.
Hope it doesn't burn too much !! :-)

Cheers

Toni

[Wim.Koornneef]

Hello Forum, I have good news :-)

With the release of Pano2VR 3.1 it is now possible to make color ghosting free anaglyph panoramas.
All you have to do is to set the compression quality of the JPEG tiles to 90 or higher and then automatically the color subsampling of JPEG images is disabled.

As you can see the tutorial is changed a bit, it is now divided in sections and I added some new items, f.i. about how to avoid color ghosting and a PDF tutorial about this so I hope that it now will be even easier to make your own 3D panorama.

Best,
Wim

[casanova007]

Wim,

Thank you very much for this wonderful tutorial.
There is only one part of the tutorial that I didn't understand, you wrote:

"Optimize for lens shift parameters "d" and "e" and image parameters yaw, pitch and roll in steps with
linked roll and pitch for all images and remove all cp's with errors of 8 px or higher.
Avoid optimizing for the image parameter pitch and the vertical lens shift parameter "e" in the same
step when you start optimizing as it will give issues with high and wrong values.
When the cp errors are all below 8 px and the pitch and the vertical lens shift parameter are more or
less normal then the optimizing is done."

I even looked at the pdf file but still, it is not clear.

Could you please explain more about what you mean by "steps" and "Avoid optimizing for the image parameter pitch and the vertical lens shift parameter "e" in the same
step"?. If it is not a problem, could you put these steps in bullet point (numbers) and also describe which parameter(s) must be optimized in each step?

Also I had one more question,
Does your tutorial apply to circular fish-eye lenses such as sigma 4.5 mm on a cropped sensor?

Best Regards,

Ali


PS. - Also I wanted to ask if the shape of the mask (that you provided a link for) must be modified if it is going to be applied on a circular fish-eye image, because I noticed that your mask also covered edges of the frame on the opposite side of the fully covered area of the frame. Is it enough to only cover half of the circle on a circular fish-eye image or also parts of the edges on the opposite side also should be covered?

[Wim.Koornneef]

Hello Ali,

Optimize a 3D project just as you normally do but to avoid that the optimizer of PTGui will calculate crazy numbers for pitch (the "p" image parameter) and the vertical lens shift parameter (the "e" parameter) it is best to not optimize both in the same step.

This is how it goes, you first optimize the project for all parameters just as you normally do but without the "e" parameter.
Remove bad CP's, optimize again, and when you don't get it better then disable optimizing for "p" and enable optimizing for "e" and run the optimizer again.
By doing this you have optimized both "p" and "e" in 2 (or more) different steps, never together but always seperate.

Only for learning purposes, save the optimized (in steps) project, note the values for "p" and "e", then enable them both and run the optimizer again.
Depending on where the CP's are placed in the images (horizon, foreground, closeby objects) there is a big chance that after optimizing the values for "p" and "e" are changed a lot from the original values !
This proves that it is better to avoid optimizing "p" and "e" in one step..

BTW, it is always best to start with a lens calibrated template, this makes optimizing a project a lot easier.
This is not only true for a 3D pano but most times also for a "normal" pano......

Using a full circular image, a cropped circular image or a full frame fishe eye image makes absolutely no difference in the method so yes, you can use the tutorial also for a sigma 4.5 mm on a cropped size sensor camera.

About the shape of masks.
When using one of my image masks you have to resize the mask to make sure that width and height are exactly the same as your images, otherwise it can happen that parts of the mask are covering the opposite half of the image, this will not harm the stitched quality of the pano but it will reduce the 3D depth....

In my example I use a half moon shape mask, this shape gives a more or less vertical stripe in the Panorama Editor window when applied on a cropped circular image but I am sure it will also give a good result on your images, just give it a try.

Success,
Wim

[casanova007]

Hi Wim!

Thank you for taking the time to reply. I will definitely follow these steps and will give that mask a try.
I had already experimented with another method that another member of this forum posted which involved shooting two series of images and rotating the camera along its lens axis, but the result had a lot of ghosting (possibly because panoramas are different from rectilinear images in their projection). I also saw another method involving a rig containing 2 cameras installed side by side and a motorized custom made head, still its result had some ghosting.

But from viewing your samples, I can tell that your method is very accurate and results are excellent. Thank you again for the explanation.

Best,


Ali

[GregoryA]

My first try to make a spherical 3D panorama

Hello Wim!

Thanks for sharing your information with us.
This is my first try:

http://www.arbitphoto.com/3D_pano_03262014/tour.html

I used Canon 5D MIII + Nikon 10.5 mm with adapter+NN5

All the best,
Gregory Arbit

https://www.arbitphoto.com

[Wim.Koornneef]

Gregory,

Your 3D panos looks great, hard to believe this is a first try :-)

Success,
Wim

[Hugh]

Beautiful Gregory,

I have only made one anaglyph 360 panorama using Wim's great guidance so appreciate the effort that goes into a result like yours.

All the best, Hugh.

[GregoryA]

Hi Hugh,

Thank you for your remark, but I see big room for improvement.
1. I am using the NN5, so I can not shift more than 30mm.
2. I have to find the optimal shape for mask.

All the best,
Gregory

[Hugh]

Hello Gregory,

Hmm.
I think your colour looks good while wearing anaglyph glasses and the 3D effect is just about right so like the result as it is.
I can hardly see any red/blue shift when viewing your panos without anaglyph glasses.
I used an NN 3 so was limited to 24 shots round and my pano has quite a bit of red/blue shift when viewed without the anaglyph glasses, so think you actually have achieved a great result.

All the best, Hugh.

[GregoryA]

Hello Hugh,

Today I did one more test with 7.5 degree interval, so it gave me 50 shots.
I will show the result shortly.

Have a nice weekend,
Gregory