You are here: start » tutorials » matchmoving

Differences

This shows you the differences between two versions of the page.

--- tutorials:matchmoving [2008/06/21 16:29]
sympodius
+++ tutorials:matchmoving [2010/06/18 05:16] (current)
@@ Line 5: / Line 5: @@
 <object width="425" height="350"><param name="movie" value="http://www.youtube.com/v/bpfjyWHczjM"></param><embed src="http://www.youtube.com/v/bpfjyWHczjM" type="application/x-shockwave-flash" width="425" height="350"></embed></object>
 </html>
@@ Line 10: / Line 11: @@
 ===== Introductory Disclaimer =====
-As with all our tutorials, we do not pretend to know what we are doing. These instructions are based on our own experiences and a lot of trial and error. We are not, by any stretch of the imagination, experts. We may get things wrong… often. If you find anything incorrect in this tutorial (or anything else on our website) then please [[john@fictionality.co.uk|let us know]].
+As with all our tutorials, we do not pretend to know what we are doing. These instructions are based on our own experiences and a lot of trial and error. We are not, by any stretch of the imagination, experts. We may get things wrong... often. If you find anything incorrect in this tutorial (or anything else on our website) then please [[john@fictionality.co.uk|let us know]].
-Also, this tutorial is for Windows since that was the system we were using when we figured all this stuff out. We will also assume that you have version 2.46 of Blender. You can download it from [[http://download.blender.org/release/Blender2.46|blender.org]].
+Also, this tutorial is for Windows since that was the system we were using when we figured all this stuff out. We will also assume that you have version 2.46 of Blender. You can download it from [[http://download.blender.org/release/Blender2.46|blender.org]]. Thanks to amdbcg and rubicon for their suggestions.
 Thanks,
@@ Line 31: / Line 32: @@
 This last technique is the one we plan to use in our film and is the one we will attempt to demonstrate in this tutorial. We're going to use a fairly simple example to demonstrate the technique as most of the things we are planning do not involve especially complex interactions.
@@ Line 38: / Line 43: @@
 The largest problem we face is tracking the camera's movements. If the camera is stationary, then it is fairly trivial to put CGI elements into a shot (in terms of technology at least). While there may be one or two shots in our film that use this technique, most will involve a moving camera. However, the best shots can be achieved by limiting the camera's movements to simple panning. That is, the camera is on a tripod and does not change location, just where is it pointing. However, if you wish to use a completely stationary camera then you may still find other aspects of this tutorial useful.
-We made a simple panning shot of a desk in John's bedroom, leaving a nice space on the desk where we were planning on adding a virtual object. If you want to follow along with the tutorial using this same footage, you can download it from us.
+We made a simple panning shot of a desk in John's bedroom, leaving a nice space on the desk where we were planning on adding a virtual object. If you want to follow along with the tutorial using this same footage, you can [[http://www.fictionality.co.uk/downloads/CamTrackSource.mpg|download it from us]].
 The first thing we did when we got the footage from the camera was to convert it to a series of Targa (.tga) image files (one for each frame). This is so that it can be used as background images in Blender and so that it can be processed for the camera motion tracking.
@@ Line 44: / Line 49: @@
 Some common mistakes at this stage can be that you do not know the image size of the shots that were taken, their aspect ratio, their frame rate and the focal length of the lens. Make sure that you know these details when you create your source material as it will improve the results.
-Blender can be used to create the series of Targa files quite easily. Simply open Blender and delete everything in the “3D View” window by selecting everything (hit “A” key until everything is highlighted in pink) and hitting the “X” key. Once everything is deleted, you will need to add a camera back so that we can render out the Targa files. Just hit the space bar while your mouse is over the the 3D View window and select, “Add -> Camera.”
+Blender can be used to create the series of Targa files quite easily. Simply open Blender and delete everything in the “3D View” window by selecting everything (hit “A” key until everything is highlighted in pink) and hitting the “X” key.
-Go into the “Video Sequence Editor” window. Then select “Add -> Movie.” Choose the source video file. Once you click “Select Movie,” you will be in a grab mode with the movie sequence. Move it so that it is on track 1 at frame 1. The number of frames of the movie will be the first thing on it's label. To make sure we render the whole sequence to Targa files, we must set the animation length to match.
+{{tutorials:blenderdelete.png|Blender Delete Selection}}
-Press “F10” to get to the scene panel. Make sure that the start frame is set to “1” and that the end frame is set to the number of frames in your movie. In our case, this was 725. Make sure you select the “Do Sequence” option and change the output to what is needed. In our case, we want to set up Blender to match the same dimensions as the original source footage. Still in the Scene Buttons Panel, change the SizeX and SizeY of the Format tab to match the original video file. Our source file was 1440 pixels wide and 1080 pixels high. However, the format of the original MPEG video had a 16:9 dimension ratio. A little bit of maths shows us that 1440:1080 is not a 16:9 ratio:
+Once everything is deleted, you will need to add a camera back so that we can render out the Targa files. Just hit the space bar while your mouse is over the the 3D View window and select, “Add -> Camera.”
+{{tutorials:blenderaddcamera.png|Blender Add Camera}}
+Go into the “Video Sequence Editor” window. Then select “Add -> Movie.”
+{{tutorials:blenderselectvse.png|Blender Select Video Sequence Editor Window}}
+{{tutorials:blendervseaddmovie.png|Blender Video Sequence Editor Add movie}}
+Choose the source video file. Once you click “Select Movie,” you will be in a grab mode with the movie sequence. Move it so that it is on track 1 at frame 1. The number of frames of the movie will be the first thing on it's label. To make sure we render the whole sequence to Targa files, we must set the animation length to match.
+{{tutorials:blendervsepositionmovie.png?600|Blender Video Sequence Editor Position Movie Strip}}
+Press “F10” to get to the scene panel. Make sure that the start frame is set to “1” and that the end frame is set to the number of frames in your movie. In our case, this was 725. Make sure you select the “Do Sequence” option and change the output to what is needed.
+{{tutorials:blenderinitialframesettings.png|Blender Frame Settings}}
+In our case, we want to set up Blender to match the same dimensions as the original source footage. Still in the Scene Buttons Panel, change the SizeX and SizeY of the Format tab to match the original video file. Our source file was 1440 pixels wide and 1080 pixels high. However, the format of the original MPEG video had a 16:9 dimension ratio. A little bit of maths shows us that 1440:1080 is not a 16:9 ratio:
 <code>
@@ Line 56: / Line 79: @@
 The two results would be the same if the ratios matched. However, we can simply adjust the aspect ratios of the Blender output to correct this so that we end up with a 16:9 ratio output. We just need to set the AspX and AspY values to match the results of the above equations. For AspX, we use the result of the second equation because the width needs to be compensated by the ratio of the height. Thus, we set AspX to 120. Similarly, we set AspY to the result of the first equation as the height needs to be compensated for by the ratio for the width. Thus, we set AspY to 90. Incidentally, this will not affect the aspect ratio of image files, but since we plan on outputting a video file later, we may as well set it up now. Also remember to set the output format to Targa and set the quality to 100.
+{{tutorials:blenderinitialrendersettings.png|Blender Initial Render Settings}}
 You can select the output for the Targa files by changing the “/tmp\” field in the output tab of the scene panel to the desired directory. Then just hit the “ANIM” button to produce the Targa files. They will be saved in the directory you specified.
-We have noticed that Blender will occasionally output blank frames at the end of a clip instead of the ones from the source video. You can see this before rendering the output by playing the file in Blender's preview first. To do this, change the view type of the Video Sequence Editor to “Image Preview” instead of “Sequence.” You may have to zoom out a little to see the video properly. Now just hit “Alt+A” while your mouse is over the Image Preview window to play through the source video. The simplest solution to this is to cut off the frames that are causing the problems. To do this, go back into the “Sequence” view and right click on the right arrow of the video strip and drag it left to a point where you expect the video to be fine. You can test your estimation by previewing the video again. Once no black frames show up and you are happy with the video, you can see how many frames the video strip has become (it's shown on the far right on the video strip) and change the length of the output video accordingly. In our case, we reduced our video length to 715 frames.
+{{tutorials:blendersetrenderdirectory.png|Blender Set Render Directory}}
+We have noticed that Blender will occasionally output blank frames at the end of a clip instead of the ones from the source video. This appears to me due to some sort of memory issue. We've found the best way to deal with it is to change the start frame to the number of the first blank Targa file and then re-render the blank frames. If it continues to output blank frames, try restarting your computer before the re-render. If that fails, you will have to remove the offending frames. The simplest solution to this is to cut off the frames that are causing the problems. To do this, go back into the “Sequence” view and right click on the right arrow of the video strip and drag it left to a point where you expect the video to be fine. Once no black frames show up and you are happy with the video, you can see how many frames the video strip has become (it's shown on the far right on the video strip) and change the length of the output video accordingly. In our case, we reduced our video length to 715 frames.
+{{tutorials:blendervsenewframelength.png|Blender Video Sequence Editor New Frame Length on Strip}}
 Make sure that you save the Blender file somewhere as we will be using it again later.
-The next step is to take the Targa files and put them into a Camera Motion Tracking software package. Although there is a discontinued program called “Icarus” that is somewhat popular in the Blender community, we decided to use one called “Voodoo” because it seems easier to use and is still under development (which means it will continue to improve). Both programs are free to use, but Icarus can only be used for educational purposes. Voodoo can at least be used to make movies you might plan on making money from. Sadly, neither program is open source, but we live in hope.
+The next step is to take the Targa files and put them into a Camera Motion Tracking software package. Although there is a discontinued program called “Icarus” that is somewhat popular in the Blender community, we decided to use one called “Voodoo” because it seems easier to use. Both programs are free to use, but Icarus can only be used for educational purposes, while Voodoo can be used for all non-commercial purposes. Sadly, neither program is open source, but we live in hope.
+When we originally wrote this tutorial, Voodoo was available to be used for commercial purposes as well. However, one of our readers has now informed us that this policy has changed. We find this rather sad news as we no longer know of a good, free camera motion tracking package that can be used for commercial purposes. There is always a lot of talk in the community about including Camera Motion Tracking as a feature of Blender, but as far as we know no one has ever attempted this. In any case, we thank Robert Hamilton for informing us of the change in Voodoo's terms and conditions.
 You can get Voodoo from, http://www.digilab.uni-hannover.de/download.html
@@ Line 77: / Line 108: @@
 {{tutorials:finalestimation.png?600|Final Estimation Voodoo Command Prompt Window}}
-Click “File -> Save... -> Blender Python Script...” and save the information in a file somewhere (we called ours “cammotion.py”). This will be a Python file that we can use in Blender. Make sure you select “Export all” in the next window. Our own file is available to download.
+Click “File -> Save... -> Blender Python Script...” and save the information in a file somewhere (we called ours “cammotion.py”). This will be a Python file that we can use in Blender. Make sure you select “Export all” in the next window. Our own file is [[http://www.fictionality.co.uk/downloads/cammotion.py|available to download]].
 {{tutorials:exportallpoints.png|Export All Points from Voodoo}}
@@ Line 84: / Line 115: @@
 We're going to assume you know Blender a bit because this is quite an advanced tutorial anyway. Load up Blender and open the file we created earlier. Go to the 3D View and make sure you delete everything in the scene by selecting everything (hit “A” key until everything is highlighted in pink) and hitting the “X” key.
-{{tutorials:blendereraseselectedimages.png?400|Blender Select All and Erase}}
 Once the scene is empty, change to Blender's Text Editor window and open the “cammotion.py” file (or whatever you called it). Then run it and you're 3D View should now have a camera and a bunch of dots on it. Hit “NUMPAD 0” to see the camera's viewpoint.
@@ Line 95: / Line 124: @@
 Press “F10” to get to the scene panel and change the start in the animation tab to 1. Find out how many Targa files your animation produced to get the frame number for the end. In our case, we had 715 Targa files so we had 715 as our end frame. Change the Buttons Window's current frame to 1.
-You can now see the tracking of the camera in Blender. In the 3D View Window, hit “ALT+A” to see the animation from the camera's viewpoint (Hit “ESC” to stop it). You should see that the little dots match the green crosses that were seen in the Voodoo Tracker. The virtual camera movement should match the original camera quite well. The dot's are not in their correct 3D position, however, but will look that way when viewed through the camera. If you change to a different view you will see that the dots are probably in a surprising position. If it bothers you that the camera is at a funny angle, there is an empty in the same position as the camera. Right click near the camera until you see a dot selected instead of the camera. This empty is grouped to everything else created by the python script, so you can rotate it to suit you.
+You can now see the tracking of the camera in Blender. In the 3D View Window, hit “ALT+A” to see the animation from the camera's viewpoint (Hit “ESC” to stop it). You should see that the little dots match the green crosses that were seen in the Voodoo Tracker. The virtual camera movement should match the original camera quite well. The dot's are not in their correct 3D position, however, but will look that way when viewed through the camera. If you change to a different view you will see that the dots are probably in a surprising position. If it bothers you that the camera is at a funny angle, there is an empty in the same position as the camera. Right click near the camera until you see a dot selected instead of the camera. This empty is grouped to everything else created by the python script, so you can rotate it to suit you. You can also use this empty to match the scene with shots that have a worth match move. It is a quick way to change everything in the scene to match the virtual elements with the original footage.
 To make things easier still for us we need to set up Blender a little further. We want to set up the virtual camera to show a background image that will help us sync things up. The dimensions of the virtual camera should still be set up correctly from before.
@@ Line 102: / Line 131: @@
 {{tutorials:viewbackgroundimage.png|Blender View Background Image Menu Option}}
+{{tutorials:blenderbackgroundimagesettings.png|Blender Background Image Settings}}
 If you find that the dots do not perfectly match the background image, you may be using incorrect values for the AspX and AspY. This can sometimes occur if you chose the wrong initial camera type in Voodoo. To correct this, you'll need to fiddle about with these two values until things line up better.
@@ Line 107: / Line 138: @@
 At this point you should be able to add your virtual elements and match them up with the original scene by checking the view in the camera. Remember that the dots in the scene are not in their correct 3D position, but will show the rough focal distance of the original camera. So, elements that should be in focus in your shot should probably be placed at the same distance from the virtual camera as the dots. You may have to play about with the camera a bit to match the lens better. You can do this by selecting the camera and hitting “F9”. It's important that you set your objects at a good distance from the camera if you want to get a good perspective on them. This is largely a trial and error process.
-Once you have your scene all set up, you'll need to tell Blender to use the original footage for the background during the render (you may not always want to do this but this is a nice quick way for this test). If you go back to the Video Sequence Editor you should see that the original video is still there from earlier on. Select the video and delete it. We're going to use the Targa files we created instead as this will ensure a perfect frame rate match with the Blender scene. Add the Targa files by selecting “Add -> Images.” now navigate to the directory with your Targa files and hit “CTRL+A” to select them all. You can right-click to deselect any files that are not part of the image sequence. Move the video strip to track 1 at frame 1. You can then add in the scene by selecting, “Add -> Scene -> Name of Scene.” In our case the “Name of Scene” was “scene.” Add the scene strip on track 2 at frame 1. First select the scene strip and then hold shift and select the original film strip as well. Now select, “Add -> Effect -> Alpha Under.” A new strip will be created. Place it on track 3 at frame one.
+Once you have your scene all set up, you'll need to tell Blender to use the original footage for the background during the render (you may not always want to do this but this is a nice quick way for this test). If you go back to the Video Sequence Editor you should see that the original video is still there from earlier on. Select the video and delete it. We're going to use the Targa files we created instead as this will ensure a perfect frame rate match with the Blender scene. Add the Targa files by selecting “Add -> Images.” now navigate to the directory with your Targa files and hit “CTRL+A” to select them all. You can right-click to deselect any files that are not part of the image sequence. Move the video strip to track 1 at frame 1. You can then add in the scene by selecting, “Add -> Scene -> Name of Scene.” In our case the “Name of Scene” was “scene.” Add the scene strip on track 2 at frame 1. First select the scene strip and then hold shift and select the original film strip as well. Now select, “Add -> Effect -> Alpha Under.” A new strip will be created. Place it on track 3 at frame 1.
+{{tutorials:blenderaddalphaunder.png|Blender Video Sequence Editor Add Alpha Under}}
 You can now select the output format you want to use for the video in the Format tab of the Scene Panel (F10). We choose FFMpeg with an MPEG 2 output at 100 quality. We kept our frame rate at 25 fps to match our original footage. Then just hit the “ANIM” button to produce your render (“Do Sequence” should still be selected from earlier, as well as the resolution and aspect ratio). It will be saved in the directory you specified earlier.
+{{tutorials:blendervsefinalrendersettings.png?600|Blender Final Render Settings}}
@@ Line 120: / Line 155: @@
 <object width="425" height="350"><param name="movie" value="http://www.youtube.com/v/hKIpjz0n2uw"></param><embed src="http://www.youtube.com/v/hKIpjz0n2uw" type="application/x-shockwave-flash" width="425" height="350"></embed></object>
 </html>
@@ Line 127: / Line 163: @@
 The first issue (which you may have realised for your selves) with our shot is that there were no moving elements. Match moving largely relies on the idea that there are a lot of elements in the shot that are stationary and that it is mainly the camera that is moving. However, you can still include moving things in the shot (like actors and such), but you will probably get results that are not quite as good. Also, if you choose not to use a tripod shot and use a free moving camera, you may find the results will be way off. Some of the tests we did with this weren't very good. Our recommendation for good results is to limit the number of active elements in the original shot and try to stick to tripod-based panning.
-You may also find that the perspective in Blender does not line up with the original shot. This usually stems from keeping the object at an incorrect distance from the camera. Generally, moving the object to different distances from the camera and resizing it to compensate can solve this problem. However, you might find that adjusting the focal length settings in Voodoo can also improve these perspective issues.
+You may also find that the perspective in Blender does not line up with the original shot. This usually stems from keeping the object at an incorrect distance from the camera. Generally, moving the object to different distances from the camera and resizing it to compensate can solve this problem. However, you might find that adjusting the focal length settings in Voodoo can also improve these perspective issues. The ideal situation is for the original camera, the voodoo settings and the Blender virtual camera to all have exactly the same settings. If you can achieve this then you will get the best results.
 This tutorial has not covered shadows, reflections, lighting and when real elements move in front of the virtual elements. We hope to cover all of these in a later tutorial.