Voodoo Camera Tracker: A tool for the integration of virtual and real scenes

Version 1.2.0 beta for Linux and Windows
Copyright (C) 2002-2012 Laboratorium für Informationstechnologie

This non-commercial software tool is developed for research purpose at the Laboratorium für Informationstechnologie, University of Hannover. Permission is granted to any individual or institution to use, copy, and distribute this software, provided that this complete copyright and permission notice is maintained, intact, in all copies and supporting documentation. University of Hannover provides this software "as is" without express or implied warranty.

Contents

[Overview] [Download] [Usage] [Troubleshooting] [Camera Model] [PNT Files] [Bug Report]

Overview

The Voodoo Camera Tracker estimates camera parameters and reconstructs a 3D scene from image sequences. The estimation algorithm offers a full automatic and robust solution to estimate camera parameters for video sequences. The results are useful for many applications like film production, 3D reconstruction, or video coding. The estimated parameters can be exported to the 3D animation packages: 3D Studio Max, Blender, Lightwave, Maya, and Softimage.

The Voodoo Camera Tracker works very alike to commercial available camera trackers (also called match movers). Please consider buying a commercial product, if you need a camera tracker with professional support.

The estimation method consist of five processing steps:

1) Automatic detection of feature points

The feature points are detected with sub-pixel accuracy using a corner detector. The detected feature points in the current image are marked with green crosses. Feature points that are detected as outliers during the processing are marked with red crosses.

08sr3_286_pnt.jpg

2) Automatic correspondence analysis

The points can be matched from one image to the next by choosing matches which have the highest cross-correlation of image intensity for regions surrounding the points. The paths of the feature points are drawn in yellow here.

08sr3_286_corr.jpg

3) Outlier elimination

Due to misalignments or moving object in the scene some of the correspondences may be incorrect. To achieve a robust matching, a random sampling algorithm is employed to detect bad correspondences, called outliers.

4) Robust estimation of camera parameters

The camera parameters are estimated in an incremental fashion by using optimization technique applied to the good correspondences, called inlier.

5) Final refinement of the camera parameters

Finally, a refinement step is applied to all camera parameters of the sequence. This step tries to distribute the estimation error evenly over the sequence. Afterwards, virtual objects can be integrated into the real image sequence.

08sr3_286_comp.jpg

Download

The Voodoo Camera Tracker is availabe for free download from our download page as a binary file for Linux and Windows.

Usage

A typical camera tracking session includes the following steps

Main Tracking Window

After typing the command >voodoo the Main Tracking window is shown (Fig. 1).

Figure 1. Main Tracking Window

The menu bar of this window has the following structure:

File → Load → Sequence

By choosing this item from the menu the Sequence Select window opens (Fig. 2).

Figure 2. Sequence Select window

You can load an image sequence that is stored in several image files. The Voodoo Camera Tracker is supporting Targa 2.0 (.tga), Portable Network Graphics (.png), and JPEG (*.jpg) image formats.

  • Click Browse, a file browser opens.
  • Select the file name of the image sequence you want to track.
    For example, if you want to open the sequence: "./sample_image_sequence/uni.##.tga", you first choose the file folder named "sample_image_sequence", and then click on a file out of the sequence, e.g. "uni.00.tga". Then click Open.
    The Voodoo Camera Tracker tries to detects all other images from the sequence automatically by replacing any number in the file name with a "#". Then it replaces the "#" with any possible number and checks if there is such a file in the directory. The Voodoo Camera Tracker can handle numbering with and without leading zeros. If you use numbering without leading zeros, please choose a file with the largest number of digits in the file browser.
  • The Voodoo Camera Tracker fills the detected start and end sequence number as well as step number in the Start, End and Step text field. You can modify these numbers according to your needs. With step number 2 for example every second image will be skipped.
  • In the field Deinterlace you can check one of three options. With not interlaced the selected image sequence will not be deinterlaced. The original data from the file is used for tracking. With the selection maintain even lines or maintain odd lines the image sequence will be deinterlaced accordingly.
  • In the field Motion Type you can select two options. With the selection rotation the camera motion model takes only rotation and zoom operations into account. This motion model is valid for sequences taken with a camera on a tripod. If you select free move the camera must include a translational motion.
  • Click OK and the selected sequence is loaded.

File → Load → Initial Camera

By choosing this item from the menu the Initial Camera Parameters window opens (Fig. 3).

Figure 3. Initial Camera Parameter window

You can set the initial camera parameters manually or load a camera file.

  • Set the parameters in the text fields manually
    • The desired camera parameters can be filled in the text fields. Only the text field with white background color can be edited, the others with dark gray background are read only.
    • In general, it is not necessary to fill in all text fields. The two most important parameters are the size of the Film Back and the Focal Length, which are located in the widget called General. The absolute size of the Film Back is arbitrary, important is the resulting image aspect ratio. The most common aspect ratios are 4:3 = 1.33333 or 16:9 = 1.77777. An easy way to adjust the size of the Film Back is to use the Type combo box, where the type of the used camera can be selected. Please check if the resulting aspect ratio is correct. If you do not know the Focal Length it does not matter, the Voodoo Camera Tracker is configured to estimate it by default. If you know the Focal Length enter it here and change the settings in the Main Control Panel not to estimate it.
    • In the widget called Advanced some additional parameters like Principal Point and Radial Distortion can be set. Leave these parameters to default values if you are not familiar with their meaning.
    • The widget called Position hold the translation and rotation of the camera. The rotation is given by the angles pan, tilt and roll, which correspond to three successive rotations around the Y-, X- and Z-axes, respectively.
    • For each image of the sequence the Voodoo Camera Tracker holds one camera parameter set that is linked to this image. By clicking OK all camera parameter sets of the image sequence are set according to the parameters of the initial camera.
  • Load a CAHV camera file
    • The Voodoo Camera Tracker uses the CAHV model introduced by Yakimovski and Cunningham. The parameter of these model are read from files which have the extension ".cam". Here you find more information on the CAHV camera model.
    • To read a CAHV camera file, click the Load button and select a ".cam" file.
    • The loaded parameters are shown in the widget.
    • For each image of the sequence the Voodoo Camera Tracker holds one camera parameter set that is linked to this image. By clicking OK all camera parameter sets of the image sequence are set according to the parameters of the initial camera.

File → Load → CAHV Cameras

For each image of the sequence the Voodoo Camera Tracker holds one camera parameter set that is linked to this image. The parameter sets can be read from files which have the extension ".cam". The parameters are loaded for all the camera views in the image sequence by choosing this menu item. There must be one file for each image and the files must use the same numbering as the image sequence.

File → Load → Feature Points

For each image of the sequence the Voodoo Camera Tracker holds a set of feature points. The feature points can be read from files which have the extension ".pnt". The feature points are loaded for all images of the sequence by choosing this menu item. There must be one file for each image and the files must use the same numbering as the image sequence.

File → Load → User Primitives

This menu item loads a user primitive in STL (stereolithography) format that can be placed into the scene by using the Modelling Tools. The Voodoo Camera Tracker can read STL files in binary or ASCII format.

File → Save → 3D Studio Max (MAX Script)

By choosing this item from the menu the Voodoo Camera Tracker exports the estimated camera parameters to the 3D animation package 3D Studio Max using a Max Script file. This file has the extension ".ms". To run the script go to [MAXScript→Run Script] in 3D Studio Max. The image sequence can be displayed as an environment map in 3D Studio Max:

  • in the perspective viewport right-click "Perspective" and select [Views→voodoo_cam]
  • select [Rendering→Environment]
  • click the "Environment Map" button and select "bitmap" from the list
  • enter the file name of the image sequence
  • activate the "Sequence" checkbox
  • select [Views→Viewport Background]
  • activate the "Use Environment Background" and the "Display Background" checkbox and select "voodoo_cam" in the "Viewport" popup list

We tested the import with 3ds Max Version 6 as well as the 2007-2010 versions.

Figure 4. Export camera parameters to 3D Studio Max

File → Save → Blender Python Script

By choosing this item from the menu the Voodoo Camera Tracker exports the estimated camera parameters to the 3D animation package Blender using a Python Script file. This file has the extension ".py". Please select in the file dialog if you want to export to "Blender 2.4x and lower" or "Blender 2.5x and higher".
Load the python script into Blender's text editor and execute the script with ALT-P. Select the "voodoo_render_cam" camera and press Ctrl-Numpad 0 to activate it.
To display the image sequence as a backbuffer in Blender 2.4x, go to [Buttons Windows→Scene→Render Buttons→Output] and enter the file name of the image sequence in the "backbuf" field. A '#' at the end of the file name is replaced by the current (four-digit) frame number. We tested the import with Blender Version 2.42a.
To display the image sequence in the viewport in Blender 2.5x and higher, go to [3D View→View→Properties→Background Images->Add Image]. Select "Camera" in the "Axis" field and "Sequence" in the "Source" field. You also must set the correct resolution and aspect ratio in [Properties→Render→Dimensions→], e.g., choose the "TV PAL 4:3" render preset for the provided examples. We tested the import with Blender Version 2.53 beta, 2.57, and 2.62.

File → Save → LightWave 3D

By choosing this item from the menu the Voodoo Camera Tracker exports the estimated camera parameters to the 3D animation package Lightwave 3D using a LightWave Scene file. This file has the extension ".lws". To open the file, go to [File→Load→Load Scene] in Lightwave. The image sequence can be displayed as a background image in Lightwave:

  • select [Image Editor→Load]
  • enter the file name of the image sequence and activate "Sequence" in the "Image Type" popup list
  • open [Scene→Backdrop→Compositing] and select the image sequence in the "Background Image" popup list

We tested the import with LightWave 3D 7.5d Discovery Edition.

File → Save → Maya Script

By choosing this item from the menu the Voodoo Camera Tracker exports the estimated camera parameters to the 3D animation package Maya using a Maya Script file. This file has the extension ".ma". To run the script go to [File→Open Scene] in Maya. The image sequence can be displayed as an environment map in Maya:

  • select [Panels→Perspective→VoodooCam]
  • select [View→Camera Attribute Editor]
  • open the "Environment" field and create an image plane
  • the "Image Plane Attributes" window opens and you can enter the file name of the image sequence
  • activate "Use Image Sequence" checkbox and adjust the "Frame Offset"
  • in the "Placement" window select fit "To Size"

We tested the import with Maya Personal Learning Edition Version 8.5.

File → Save → Softimage 3.9 | XSI (3.0)

By choosing this item from the menu the Voodoo Camera Tracker exports the estimated camera parameters to the 3D animation package Softimage 3D using XSI File Format V3.0. This file has the extension ".xsi" and can be imported to Softimage 3.9 or Softimage XSI. To import the file go to [Tools→Import→Objects→XSI Import] in Softimage 3.9 and in XSI go to [File→Import→Input .xsi].

Figure 5. Export camera parameters to Softimage 3.9

File → Save → Textfile

This menu item exports the estimated camera parameters and 3D feature points into a text file. We suggest you parse this file if you want to write your own import filter for an application that is currently not supported by the Voodoo Camera Tracker.

File → Save → CAHV Cameras

By choosing this item from the menu the Saving Camera Files window opens (Fig. 6).

Figure 6. Saving Camera Files window

For each image of the sequence the Voodoo Camera Tracker holds one camera parameter set that is linked to this image. These parameters can be stored in a CAHV camera file (extention ".cam"). There is one file for each image.

  • The Voodoo Camera Tracker suggests a file template. Any "#" in the file template will be automatically replaced with the frame number of the image.
  • If you want to change the destination directory you can use the Browse button.
  • The Voodoo Camera Tracker fills the start and end sequence number as well as step number in the Start, End and Step text field with the values used for tracking.
  • By clicking OK the estimated camera parameter of the image sequence are saved.

File → Save → Feature Points

For each image of the sequence the Voodoo Camera Trackerholds a set of 2D and 3D feature points. These feature points can be stored in ".pnt" files. There is one file for each image.

  • The Voodoo Camera Tracker suggests a file template. Any "#" in the file template will be automatically replaced with the frame number of the image.
  • If you want to change the destination directory you can use the Browse button.
  • The Voodoo Camera Tracker fills the start and end sequence number as well as step number in the Start, End and Step text field with the values used for tracking.
  • By clicking OK the estimated feature points of the image sequence are saved.

File → Save → User Primitives

This menu item saves user primitives that are placed into the scene with the Modelling Tools.

File → Exit

By choosing this item from the menu the Voodoo Camera Tracker will be closed.

All data, that is not saved, will be lost.

View → Controls

By choosing this item from the menu the Main Control Panel window opens (Fig. 7).

Figure 7. Main Control Panel window with selected Harris detector

The camera parameter estimation (camera tracking) consists of five basic steps: Detection, Correspondence Analysis, Outlier Elimination, Estimation, and Final Estimation. For some steps the Voodoo Camera Tracker offers different algorithms. You are free to choose different combination of algorithms for every step. For example you can use the Harris , Foerstner, or Susan corner detector in the detection step. Some algorithms are in development state and therefore disabled in this beta version of the Voodoo Camera Tracker. If you are not familiar with the algorithms you can leave everything to the default configuration or play around with some parameters. In case of any trouble feel free to contact us for support.

With the Main Control Panel you can choose the combination and parameter of the algorithms you want.

  • An algorithm is selected if a dot is putted in the box behind the respective algorithm name and the Apply or OK button is pressed.
  • If you click once on the algorithm name the parameter widget of the algorithm is shown on the right side.

The default algorithm in the Correspondence Analysis step is the KLT Tracking method. It uses its own feature point detection algorithm and therefore you can not select an algorithm in the Detection step if the KLT Tracking method is enabled. The two most important parameters that sometimes need adjustment are modified with the "Max. Corners" and the "Win. Size for Local Max." text fields. The "Max. Corners" field limits the number of feature points per image. For sequences that are easy to track, you may want to reduce the number of feature points to speed up the computation. The "Win. Size for Local Max." can be used to evenly distribute the feature points in the image. A larger number increases the spread of the feature points. A nice distribution of feature points in the images is very important for good tracking results. Please also make sure that there are sufficient feature point tracks in the foreground and the background of the scene.

You can also skip a step of the estimation process by clicking on the Enable/Skip field next to each step. The Voodoo Camera Tracker doesn't execute this step now, but uses the results already in memory for this step, if you press the Track button. This can be very useful, especially, if you edit, add, or remove feature points manually (see for Modelling Tools details). If you don't choose the Skip for the Detection and Correspondence Analysis step all your edited feature points will be overwritten.

View → Draw Options

By choosing this item from the menu the Draw Options Deck window opens (Fig. 8).

Figure 8. Draw Options Deck window

The settings in this window influence the 2D and 3D drawings which are overlayed in the actual image in the Main Tracking Window and in the 3D Scene Viewer. By clicking on the colored square, it is possible to select a color for each overlay. It is also possible to turn specific overlay elements on or off.

In the following some important terms are defined

  • Inlier: Feature points or correspondences which are found reliably in the outlier elimination step. These feature points or correspondences are used in the final estimation step.
  • Outlier: Feature points or correspondences which are found unreliably in the outlier elimination step. These feature points or correspondences are neglected in the final estimation step.
  • FPoints: Abbreviation for feature points.
  • FCorrs: Abbreviation for (feature) correspondences.
  • Search Win: Search window for the establishment of the correspondences between two consecutive image use in the Correspondence Analysis step
  • Polar Raster: The polar raster is a very useful tool to evaluated the estimated camera parameters if the pure rotational camera motion model is applied, which is valid for sequences taken with a camera on a tripod.

Fig. 9 shows an example of tracking results. Inlier are marked as green crosses and outlier as red crosses. The paths of feature points are drawn in yellow. One feature point is selected, marked as blue cross and the belonging search window is drawn in orange.

Figure 9. Detected feature points and their paths

View → Camera Parameter

By choosing this item from the menu the Camera Parameter window opens (Fig. 10.).

Figure 10. Camera Parameter window

This window has the same text fields as the Initial Camera window expect the Load button. For each image of the sequence the Voodoo Camera Tracker holds one camera parameter set that is linked to this image. By changing the displayed image in the Main Tracking Window the parameter shown in the Camera Parameter window change accordingly. Hence you have access to every parameter set of the sequence and can change the parameters according to your needs. Changes are applied to the stored camera parameter set by pressing the OK or Apply button.

View → 3D Scene Viewer

By choosing this item from the menu the 3D Scene Viewer window opens (Fig. 11.).

Figure 11. 3D Scene Viewer window

In this viewer a 3D scene is displayed with a global 3D coordinate system. One unit of the coordinate system raster equals the focal length given by the actual camera parameter set. By pressing a key and then dragging the mouse while holding down a mouse button you can perform to following movements to explore the scene

key + mouse buttonmovement
o + left free orbit OpenGL camera
o + middle up/down orbit OpenGL camera
o + right fast dolly OpenGL camera
u + left translate left/right up/down OpenGL camera
u + middle free orbit OpenGL camera
u + right translate straight OpenGL camera
p + left dolly OpenGL camera
p + middle fast dolly OpenGL camera
p + middle very fast dolly OpenGL camera along z-axes
i + left translate OpenGL camera interest along x-axes
i + middle translate OpenGL camera interest along y-axes
i + right translate OpenGL camera interest along z-axes
s put OpenGL camera interest on CAHV camera
a reset OpenGL camera position
d put OpenGL camera interest on selected 3d feature point
f flip the coordinate system

The Options menu of the 3D Scene Viewer window contains a Draw Options window where you can choose whether a 3D object is drawn or not and what color is used for drawing.

View → Modelling Tools

By choosing this item from the menu the Modelling Tools toolbox is displayed (Fig. 12.).

Figure 12. Modelling Tools toolbox

With these modelling tools you can quickly place simple primitives into the scene and test your estimated camera parameters.

First create a primitive by pressing a tool button in the Create toolbox.

You can place a primitive easily by using the 2d anchor tool button. If the 2d anchor tool button is selected and you move the mouse over a vertex of the primitive, it will be highlighted (see Fig. 12). You can now move the vertex to the correct position in the image by clicking and dragging with the left mouse button. The translation, rotation and scale of the primitive is changed automatically as the vertex is moved. With the (T)-, (R)-, and (S)- buttons, which will appear when you select the 2d anchor tool, you can (de)activate the estimation of the primitive's (T)ranslation, (R)otation, and (S)cale parameters. To place the vertex more exactly you can use the zoom tool button. If you want to delete an object press the DELETE key.

In Fig. 13 you can see the result of two fitted cubes. You can download this sequence from here, and try it yourself.

Figure 13. Fitted Cubes

With the selection tool button you can select the active primitive. A primitive can be selected by clicking on one of its vertices. The selection tool can also be used to select feature points. One feature point is selected by left click (see FPoint Editor) and multiple feature points are selected by right clicking and dragging the mouse over the feature points.

To improve the estimation results it is often very useful to add manual feature point tracks. This can be done by using the create feature points button. Left click creates a new feature point at the position of the mouse pointer. The feature point is automatically selected. Now the user can go to the next consecutive image and add another feature point to the selected track by pressing the right mouse button. A correspondence between a selected feature point in the previous image and an existing feature point in the actual image can be generated by clicking on the existing feature point with the middle mouse button. Please note that the mouse wheel can be used to go quickly to the next image of the sequence. It is important to generate long tracks in the background as well as in the foreground and to distribute the feature points well in the images. Manually added feature points are marked with a golden rectangle. They will never be detected as outliers in the Outlier Elimination step.

After you have added manual feature point tracks, you can try to estimate or re-estimate the camera parameters by pressing the Track button. Please make sure that you have selected Skip in the Detection and Correspondence Analysis step or all your edited feature points will be overwritten (see Main Control Panel for details). Sometimes it make also sense just to redo the Final Estimation step with the new manually added feature points. However, this should be only done, if your camera estimation result in memory is already very good and you added just a few manual points to make it perfect. If you want to execute the Final Estimation step on the camera estimation result in memory, Skip must be activated for all previous steps (see Main Control Panel for details).

You should save the CAHV camera parameter and feature points before you try to add manual feature point tracks. If you have saved the CAHV camera parameters and feature points, you can always load them again and have another try if something went wrong.

Another option to improve the estimation results is to remove wrong tracks that are not automatically detected as outliers. Use the selection tool to select a feature point and delete it using the FPoint Editor. If multiple feature points are selected by right click, you can delete them by pressing the DELETE key. After you have removed wrong feature point tracks you can try to improve the camera parameters by pressing the Track button and re-estimate the camera parameters (activate Skip in the Detection and Correspondence Analysis step).

Fig. 14 shows a sequence with some manually added feature point tracks. The manually added feature points in the foreground are marked with golden rectangles around them. By manual adding and removing feature point tracks it is even possible to estimated camera parameters for footage with low image quality and many moving objects, like the one shown in Fig. 14 : "Trip Down Market Street Before the Fire (1905)", public domain footage supplied by archive.org

Figure 14. Manually added feature point tracks

View → FPoint Track Editor

During automatic tracking, feature point tracks often get lost because of perspective distortion of the tracked image patch, or because tracks in the background are cut by foreground objects, or because tracks get out of the camera's field of view and appear again later. The FPoint Track Editor allows to merge those disconnected tracks manually. This is usually done after you have estimated the camera parameters successfully and want to refine your solution.

By choosing FPoint Track Editor from the menu the FPoint Track Editor window opens (Fig. 15).

Figure 15. FPoint Track Editor window

Select a feature point in the first part of the disconnected track using the selection tool button of the Modelling Tools. Then click the left From selection button in the FPoint Track Editor. The left two images in the FPoint Track Editor now display the images where the first part of the disconnected track starts and ends. Then move to another image and select a feature point in the second part of the disconnected track and click the right From selection button. Check carefully, if now in all four images the cross marks the same point in 3D and press the Merge button. After you have merged some of the broken tracks you can try to refine you estimation results by running a Final Estimation step (activate Skip in all estimation steps except the Final Estimation step and press the Track button).

You should save the CAHV camera parameter and feature points before you try to refine the results manually. If you merge a wrong track this can have unpredictable results. If you have saved the CAHV camera parameter and feature points you can always load them again and have another try.

If you press the Automatic Search button in the "Automatic Search for Merge Candidates" group, disconnected track candidates are searched automatically. Use the "Previous" and "Next" button to browse through the candidates and press the Merge button to accept the current candidate. Please note that it is usually not necessary to go through all the candidates. Just try to merge a few, e.g. 50, and check if this already sufficiently improves your results.

The FPoint Track Editor can also be used to remove unwanted feature point tracks. Use the combo box in the "Remove FPoint Tracks" group to select what you want to remove. Then press the remove button.

Help → Help

By choosing this item from the menu this manual opens.

Help → Notify Level

By choosing this item from the menu the Notify Level window opens (Fig. 16).

Figure 16. Notify Level window

By choosing the notify level you can change the amount of information printed to the console, while the program is running.

FPoint Editor

Figure 17. FPoint Editor window

The FPoint Editor window displays the parameters of a feature point. This window is displayed, if you select a feature point by using the "selection" tool button of the Modelling Tools.

Troubleshooting

  • Avoid motion blur.
  • Avoid fast motions. The displacement of a tracked feature point from frame to frame should be smaller than approx. 30 pixels.
  • Avoid very long sequences, e.g. not more than 200-400 frames.
  • Usually, it is more robust to estimate the camera parameters for a camera on a tripod (Motion Type = Rotation).
  • If your camera is not on a tripod (Motion Type = Free Move), record your footage with constant focal length.
  • Record your footage with a large field of view (small focal length).
  • It is very important to generate long tracks in the background as well as in the foreground and to distribute the feature points well in the images. The main reason for failure are insufficient feature point tracks. If the estimation fails, change the parameters of the automatic Correspondence Analysis step in the Main Control Panel, or add, edit, or delete some feature point tracks manually with the Modelling Tools.
  • The camera motion is estimated in relation to a static scene. Make sure moving objects do not take up large areas in the images.
  • Voodoo Camera Tracker does currently not allow to use a tracking mask to mask out areas with moving object in the images. However, you can manually delete feature point tracks in certain areas of the image with the Modelling Tools.
  • It is not an advantage to use a dolly (except for artistic reasons), because Voodoo Camera Tracker does not constrain the camera motion to a straight line or curve.

CAHV Camera Model

Parameters of the CAHV camera model:

(Cx, Cy, Cz)    : CameraPosition [mm]
(Ax, Ay, Az)    : RotationAxis2  [unit vector]
(Hx, Hy, Hz)    : RotationAxis0 [pel] (including FocalLength, PixelSizeX, and Principal Point offset)
(Vx, Vy, Vz)    : RotationAxis1 [pel] (including FocalLength, PixelSizeY, and Principal Point offset)
(K3, K5)        : Radialdistortion  K3 [1/(mm)^2]  K5 [1/(mm)^4]
(sx, sy)        : PixelSize	[mm/pel]
(Width, Height) : ImageSize	[pel]

Internally used parameters:

(ppx, ppy)      : Principal Point offset [pel]
f               : Focal Length [mm]
fov             : Horizontal Field of View [degree] = (2*atan(0.5*Width*sx/f)*180/PI;
(H0x, H0y, H0z) : RotationAxis0 [unit vector]
(V0x, V0y, V0z) : RotationAxis1 [unit vector]

Image coordinates and 3D coordinates:

(x, y)          : image coordinates [pel]
(X, Y, Z)       : 3D coordinates [mm]

The projection of 3D coordinates in the camera image can be done by

 [ x' ] =  [ Hx Hy Hz ] [ 1 0 0 -Cx] [ X ]
 [ y' ] =  [ Vx Vy Vz ] [ 0 1 0 -Cy] [ Y ]
 [ z' ] =  [ Ax Ay Az ] [ 0 0 1 -Cz] [ Z ]
                                     [ 1 ]

or

 [ x' ] =  [f/sx 0    ppx] [ H0x H0y H0z ] [ 1 0 0 -Cx] [ X ]
 [ y' ] =  [0    f/sy ppy] [ V0x V0y V0z ] [ 0 1 0 -Cy] [ Y ]
 [ z' ] =  [0    0    1  ] [ Ax  Ay  Az  ] [ 0 0 1 -Cz] [ Z ]
                                                        [ 1 ]

then x = x'/z' and y = y'/z' , if the origin of the image coordinates is in the center of the image,
or x = x'/z' + 0.5*(Width-1) and y = y'/z' + 0.5*(Height-1) , if the origin of the image coordinates is in the upper left corner.

PNT File Format

Saving the feature points with the Voodoo Camera Tracker generates one .pnt file for each input image.
Each line in this text file represents a 2D feature point:

x y manual type3d px py pz ident hasprev pcx pcy support

x : x position of 2D feature point in that image
y : y position of 2D feature point in that image

manual : manual feature point (1 yes, 0 no)

type3d : 0 for no 3D point, 1 for Spherical, 2 for Cartesian

px : x position of 3D feature point
py : y position of 3D feature point
pz : z position of 3D feature point

ident : unique identifier for a 3D feature point

hasprev : has a previous feature point (1 yes, 0 no)

pcx : x position of 2D feature point in the previous image
pcy : y position of 2D feature point in the previous image

support : inlier / outlier bitmask

If the feature points have the same ident in different .pnt files they are on a common track.

Bug Report

The Voodoo Camera Tracker is developed for research purpose and has therefore less features than similar commercial products. Please consider buying a commercial product if you need a camera tracker with professional support. However, your experience report is important to us in order to improve the software according to your needs. Please send bugs and hints to Thorsten Thormählen (http://www.tnt.uni-hannover.de/~thormae) or Hellward Broszio (http://www.tnt.uni-hannover.de/~broszio). Do not hesitate to contact us in case of any trouble with this software.

Voodoo Camera Tracker