4 \section{Proxy}%
5 \label{sec:proxy}
6 \index{proxy}
8 Proxies was introduced to allow for a smoother timeline experience. Full HD and everything from 4K size up are usually very large in file size. In addition, some commonly used codecs (e.g. h264/5) are very compressed and interframe type, so they are more stressful for timeline playback. These files cause performance problems on weaker PCs. As a solution the developers have introduced proxies to reduce the file size. Reduced means to minimize the resolution/dimension, as usually not the full resolution is needed, because the compositor usually takes only a fraction of the computer screen. Therefore the scaling factor, which indicates how much the original resolution should be reduced.
9 There is also the \textit{proxy 1:1} \index{proxy!1:1} that maintains the original resolution but still allows you to act on the codec (lowering the bit rate, for example). See also next section: \ref{sec:transcode}.
10 When rendering, the original setting will be used to avoid loss of image quality.
12 To use the proxy in a classic way, i.e. as a scaling, select \texttt{Settings $\rightarrow$ Proxy settings} and change the Scale factor from Original size to your downsized choice.  You can choose ffmpeg as the File Format \index{file format} and a choice of various codecs associated with that.  A good choice is the default of mpeg which can usually be quite fast.  In addition, to modify values for that codec, click on the wrench icon.
14 We can use the \textit{Proxy 1:1} in two ways: setting "1" as \textit{Scale factor}, or check the \textit{Rescaled to project size (FFMPEG only)}  button. In this mode we don't have scaling, i.e. downsize, but only variations of the codec parameters that allow to maintain the original resolution. The advantage is that you can use filters that require the original size to work well. NB: if we set any scaling, by activating the \textit{Rescaled to project size (FFMPEG only)} button we automatically lose scaling and enter Proxy 1:1 mode.
16 When you have completed your choices, just click OK, and then the video tracks will be rendered. This may take some time, but previous proxy renders will be reused.
18 The proxy videos will be added to your assets in a separate Proxy folder \index{proxy!folder}, and the video track edits will use the proxies.
19 The assets in both the Media folder and Proxy folder will look proxied when dragged to the Viewer although the scale may be different.
20 Proxy downsizing renders all loaded tracks, but only work on the $1^{st}$ video layer of any multi-layer media.  Rendered proxy media is saved in the same directory as the original media.
21 However, if you proxy your session, the clips do not get proxied to the Proxy folder, but if you Drag and Drop the clip from the Clip folder to the Viewer or the Timeline, you will see that it too is proxied.
22 As usual, you can delete proxy files from the project or disk in the Resources window if you no longer want to retain them.
23 And you can save your project either as proxied or not.
25 Because Proxy works on the original media in all circumstances, when you resize the
26 media in the Resources window, it does not actually resize the media but puts it in
27 a \textit{buffer} in that format. This might lead to some confusion for the user who
28 expects this resizing to be maintained when disabling Proxy for that media. However,
29 that does not happen because the resize is only in a buffer and not in the original
30 media. This result is different than in the case of media \textit{edits}, like cuts or adding plugins because this information is not in a buffer but rather part of the
31 copy \textit{Editing Decision List} (EDL).
33 You can also nest clips while in a proxied state, but you can not drag the proxied nested clips to the viewer or the timeline.
589 files named \texttt{Nested\_\#\#\#.idx} and you may want to
590 periodically delete any index files which are no longer in use.
592 \subsection{Nested Clips}%
593 \label{sub:nested_clips}
594 \index{nested clips}
596  It is also possible to create
597 \textit{clips} and convert them to \textit{nested edl}.  This is
598 done by first creating a clip using the standard cut, clipboard,
599 paste, and/or edit panel buttons.  Now, using the resources
600 \textit{clip} folder, select a clip to be nested, and use the right
601 mouse button to select a clip.  This activates the clip popup menu.
602 Select the \textit{Nest to media} menu item, and the clip will be
603 converted to a \textit{Nested: Clip} and put in Media
604 folder. Conversely, you can select a \textit{Nested: Clip}, use the
605 \textit{EDL to clip} menu item, and the clip will be reverted to a
606 \textit{Clip}.  This works similarly to the group / un-group editing
607 features of many graphic design editing programs, but in this case
608 the groups are rendered compositions (figure~\ref{fig:nesting}).
610 Nested clips can be proxied and when they are, the resulting files
611 are placed in the user's \texttt{\$HOME/Videos} directory by default. This 612 can be modified by changing 614 \texttt{Settings$\rightarrow$Preferences$\rightarrow$Interface} 615 tab, Nested Proxy Path. 617 \begin{figure}[htpb] 618 \centering 619 \includegraphics[width=1.0\linewidth]{nesting.png} 620 \caption{Nested clips in Timeline and Resources window} 621 \label{fig:nesting} 622 \end{figure} 624 \paragraph{Notes:} creating nested clips is a complicated internal process for \CGG{}, especially for managing audio tracks. Therefore you need to make sure you have your project set up as you expect before you start nesting with the \texttt{Nest to Media} command. Below are several things to remember. 626 \begin{itemize} 627 \item The creation of the nested clip is based on the settings in \texttt{Settings$\rightarrow$Format}. Be sure that the number of audio tracks and channels is the same as we want to have in the nested clip. In addition, it is better to make each audio track independent from the others (which could be linked in more than one channel) using the option: \texttt{Audio$\rightarrow$Map 1:1} 628 \item You can do any editing on the nested clip once it is loaded to the timeline. For example we can start an OpenEDL \index{openEDL} session to make the changes. Once out of the OpenEDL session it may be that the changes made are not visible in the original nested clip. You can fix this by doing RMB on the nested clip in the \textit{Resources/Media} window and choosing \texttt{Rebuild index}. This process may take some time depending on the size of 629 the media; because of the additional time required to do this, it is not done automatically in order to allow the user to make the decision as to when they are done editing enough and need to see the exact results 630 in order to save time. 631 \item Because of the way the timeline thumbnails are displayed, after editing using OpenEDL of a 632 nested clip, the thumbnails of the video track may be redrawn and may look different and out of sync with the 633 way they did originally with respect to the audio waveforms. This is simply a result of the redrawing of the 634 video track which is independent from the redrawing of the audio tracks. The playback is still synchronized. 635 \end{itemize} 637 Workaround cases for the alpha channel not working as intended when using an EDL inserted as a reference 638 or when an EDL is nested, is described here. 640 \paragraph{Case 1 using the Projector on the EDL within a Master Project:} follow these next steps. 641 \begin{itemize} 642 \item Open your EDL with \textit{Open EDL} by using the middle mouse button on the Nested edit on its track. 643 \item In this nested EDL, insert a new Video track below the last video track. 644 \item Add the Alpha plugin to the new Video track for the entire length of the nested EDL. 645 \item Change the value of the Alpha to 0.00. 646 \item Close the EDL. 647 \end{itemize} This solves the alpha channel not working as expected and the Master Project now looks correct. 649 \paragraph{Case 2 using FadeIn/Out with autos:} 650 In the Master Project for the Videeo track containing the nested EDL, in its Patchbay change 651 the \textit{Overlay Mode} from \textit{Normal} to \textit{PorterDuff->SrcOver}. 654 \paragraph{Usage Examples of Nested Clips} 656 \begin{description} 657 \item[Example 1:] You want to make a flashback/rewind at the end 658 of your video that represents a quick summary of the entire video in 659 black and white. On he timeline, you have 60 seconds of edits with 660 clips, cuts, zoom in, zoom out and any other edits. Now you want to 661 get this 60 seconds \textit{compressed} to 10 seconds, play in 662 reverse, and in black and white at the end of your video. You would 663 copy the 60 seconds in a clip, nest the clip in the Clip folder of 664 the Resources window and drag it to the timeline. You will see only 665 a clean clip without all of the edits that were used to create it 666 because nesting display a clip without having to actually use the 667 Render menu. Now you can add a Reverse effect, Color3way plugin for 668 black and white, and use the Speed auto to get the 60 seconds down 669 to only 10 seconds. 670 \item[Example 2:] You are working on a complex project with a 671 team in a separate location. You create some sub projects, i.e.\ 672 sequences, that you or the team will use in the Master project to 673 merge the sequences in the right order and to make the final color 674 correction steps. 675 \end{description} 677 In each of the examples you can see the benefit of nesting to create 678 clean looking timelines because of the automatic rendering 679 capability of nesting. 681 \section{Multi-Camera / Mixer}% 682 \label{sec:multicamera_mixer} 683 \index{multi-camera} 684 \index{mixers} 686 Use the Mixer Viewer to see multiple media playing simultaneously in re-sizable mini-viewers. This can be used in various ways and is useful to edit videos shot by multiple cameras from different viewpoints that were simultaneously recorded in order to create a single good video. Everything will have to be initially synced so you can decide which one of the camera angles is best suited at any time. 688 The number of cameras/mixers you can have is generally limited to the available resources on your computer. Currently, the number of File Descriptors available in the OS limits cameras to about 50. If you have many \textit{mixer viewers} you will probably want to use proxy mode whenever possible. Also, in the \texttt{Settings$\rightarrowPlayback A} tab \textit{Video Out} section, uncheck \textit{play every frame} and choosing a Video Driver of \textit{X11} with \textit{use direct X11 render if possible} checked, will provide better performance. 690 Figure~\ref{fig:multicam01} shows 9 media sources in the left corner, the composed video in the right corner, the timeline with the top video track with pieces of the 9 overwrites, and the choice in Resources of Mixed. 692 \begin{figure}[htpb] 693 \centering 694 \includegraphics[width=1.0\linewidth]{multicam01.png} 695 \caption{Using Mixer capability in Cin for multiple cameras} 696 \label{fig:multicam01} 697 \end{figure} 699 \subsubsection*{Easiest Method to Getting Started}% 700 \label{ssub:easiest_method_started} 702 \begin{enumerate} 703 \item This method assumes all of your media or cameras are aligned the way you want them already. 704 \item From the \textit{File} pulldown, create a \textit{New project} with the desired format for Audio and Video output (or you can just use the default). 705 \item \texttt{File\rightarrow$Load} the media files you want to work with using \textit{Create new resources only}. 706 \item In the Resources window, with the Media folder, highlight the list of media you want to \textit{Mix}. This is done using a ctrl or shift mouse button press as you would in a standard listbox selection. 707 \item Right click the mouse on the media selection and choose \textit{open mixers} \index{mixers!open mixers}. This opens multiple mixer viewer \index{mixers!viewer} windows, one for each media item that was highlighted. You can 708 do them 1 at a time instead. This also adds the source media tracks to the main window. 709 \item Now use the timeline \index{timeline} to play and you will see all viewers/cameras playing. Stop when you get to the 710 end of the \textit{good} camera playback. 711 \item Simply double click the \textit{good} mixer viewer and from where you first started playing to the playback insertion pointer is the source section, which will be pasted in the destination video/audio tracks at the top of the new project. 712 \item Repeat steps 6-7. Start playing again, stop when you want, double click the desired mini-viewer! 713 \end{enumerate} 715 \subsubsection*{Some Hints and Caveats}% 716 \label{ssub:hints_caveats} 718 \begin{itemize} 719 \item You can easily overwrite a section of the new track by \textit{selecting} a section on the timeline, then double click on one of the mini-viewers to overwrite/replace that section. 720 \item If you edit the output tracks, it only edits output tracks, and the input tracks may no longer be lined up. 721 \item You can add a silent section by selecting past a section and start overwriting that section from then on. 722 \item If you use the cursor hairline to create the selection endpoint, it must be past the end of the destination. 723 \item The compositor shows composed media. This is the media that will be rendered. 724 \item The program always uses overwrite as the paste operation. 725 \item Use the timeline edit handles to move the start and end points of that section. 726 \item Only middle mouse drag handle operations should be used normally. 727 \item Other drags will displace the media source/destination timeline correspondence. 728 \item To re-tile the mixer windows after you have resized and moved them around, you can use the Window 729 pulldown of \textit{Tile mixers} or the shortcut of Alt-t. 730 \end{itemize} 732 \subsubsection*{But, I want to use only the first set of audio tracks\dots}% 733 \label{ssub:but_use_only_first_audio} 735 There are many cases where you may want to compose using media from several different tracks while using the the same audio tracks as associated from a specific viewer. Since mixer source tracks can be updated any time by using a mixer toggle \index{mixers!toggle}, this makes it possible to do this. 737 Procedure to update the mixer audio source track list: 739 \begin{enumerate} 740 \item Single click to highlight the mixer window you want to re-associate to the audio track. 741 \item In that audio track’s patchbay click the expand toggle, the arrow on the right side. 742 \item In the expanded pane that appears, there is another arrow on the left side. This icon has the tooltip \textit{Mixer}. Click this and because in step \#1 you highlighted the mixer window, it will now be toggled on. Once you click the mixer icon it will then point up. 743 \item Now, disassociate any audio that is unwanted by expanding its patchbay and toggling off the mixer. 744 \end{enumerate} 746 \subsubsection*{Expert Usage}% 747 \label{ssub:expert_usage} 749 When you double click a mixer viewer window, it operates an \textit{overwrite} paste operation. This moves \textit{src} (source) track edits to \textit{dst} (destination) track edits over the same selected timeline region. 751 \begin{itemize}[noitemsep] 752 \item \textit{Src tracks} should be not playable and not armed in the main window patchbay gui. 753 \item \textit{Dst tracks} should be playable and armed in the main window patchbay gui. 754 \end{itemize} 756 Each mixer viewer maintains a list of the tracks which will be used as src. This list is made visible selecting the window with the left mouse button. When the mixer viewer is selected, a highlight is drawn around the media image. All track patchbay \textit{mixer} toggles are updated to reflect the src tracks included in the selected viewer src track list. The \textit{play track} patchbay toggles can be used to manage the list. 758 \begin{itemize}[noitemsep] 759 \item \textit{Turning on} a toggle (pointing up) includes the track in the src track list. 760 \item \textit{Turning off} a toggle (pointing right) removes the track from the src track list. 761 \end{itemize} 763 New Mixer viewers can be created using the main menu \texttt{Window$\rightarrow$Mixer Viewer}, or with a shortcut of \texttt{Shift-M}. When a new viewer is created, the currently enabled patchbay \textit{mixer} toggles are used to create the viewer source track list. The toggles are cleared after the window is created. This is to improve the work flow. Use the following list of steps to create individual mixer viewers. 765 To create a list of mixer viewers: 767 \begin{enumerate} 768 \item Setup the session \texttt{settings$\rightarrow$format}, width, height, frame rate, color model, aspect ratio. 769 \item Create dst tracks using the a/v track pulldowns (or use shortcuts "t" / "T"), armed and playable. 770 \item Append src tracks using \texttt{file$\rightarrow$open$\rightarrow$append tracks}, or the resource window using pasting. 771 \item Using the track patchbay, disarm editing and disable playback of the audio/video src tracks. 772 \item Using the track patchbay, mark the new tracks as \textit{mixer} source to be added to the viewer. 773 \item Create a mixer viewer using the main menu pulldown, or the shift "M” shortcut. 774 \item Repeat steps$3-6$for each mixer viewer needed for the session editing. 775 \end{enumerate} 777 When you single click a mixer window, it becomes selected and highlighted and all of the patchbay mixer toggles are updated to reflect the state of the viewer’s src tracks. Tracks that will be src are shown as enabled. If you change a toggle, the src tracks for the selected window will be modified. This means you can associate or dis-associate any media track to any mixer window. 779 When you double click a mixer window, an overwrite paste is invoked. The mixer viewer’s src tracks are overwritten to the dst tracks. The timeline region for both the source and destination are the same for the overwrite paste function. The selection region is used if it is active. If the selection is empty, that is it is a hairline, the selection region is from the end of the destination playable edits to the selection cursor hairline. The hairline must be past the end of the playable edits on the destination tracks. 781 The mixer viewer configuration is saved with the session data. When a saved session is loaded in \textit{replace project} or \textit{replace project and concatenate tracks}, the mixer viewer will be reopened. 783 \subsubsection*{Using Proxy with \textit{Open Mixers}}% 784 \label{ssub:using_proxy_open_mixers} 786 The best way to use proxy \index{proxy} with your multiple cameras is to follow the steps below: 788 \begin{enumerate} 789 \item Load media with insertion strategy of \textit{create resources only}. 790 \item Highlight the media in the Resources window and right click on this to choose \textit{open mixers}. 791 \item Use the \texttt{Settings$\rightarrow$Proxy settings}\dots to bring up the proxy menu. 792 \item Choose the size and other options you want and click the checkmark OK. If you choose the option \textit{Beep when done} you will hear a short beep if all media is already proxied or a longer beep when all proxies have been created. 793 \item When your editing is complete, use \textit{Settings} pulldown and proxy to \textit{original size}. 794 \end{enumerate} 795 Instead of Open Mixers, you can Insert Mixers \index{mixers!insert mixers} with new tracks at the timeline insertion point. 797 \subsubsection*{Options available in the \textit{Mixer Windows}}% 798 \label{ssub:options_available_with_mixers} 799 \index{mixers!viewer RMB options} 801 There are several options you will see in each Mixer viewer \index{mixers!viewer} that help with using them. These 802 are shown in figure~\ref{fig:mixer-playable}. To use these, RMB in the desired mixer window, 803 and choose one of the options as described here. 805 \begin{figure}[htpb] 806 \centering 807 \includegraphics[width=1.0\linewidth]{mixer-playable.png} 808 \caption{A Mixer window showing available options with the white X designating no playing} 809 \label{fig:mixer-playable} 810 \end{figure} 812 \begin{enumerate} 813 \item Fullscreen / Windowed - will bring up a fullscreen display of that window or revert to the original size. 814 \item Resize Window - allows for resizing the window to a choice of different sizes. 815 \item Tile Mixers \index{mixers!tile mixers} - makes it easy to get all of the mixers nicely tiled to a standard size. 816 This is also available in the \textit{Window} pulldown. 817 \item Playable - enabled by default so that you will see a checkmark next to it in the 818 popup. The benefit of making a mixer window not playable is to save cpu time. When a specific 819 mixer window is unchecked, so not Playable, a white cross appears in the upper left hand corner 820 of the mixer. To switch back to playing again, RMB and choose Playable and this will turn 821 playing back on. The program automatically updates the mixer image when the timeline cursor 822 stops or when it is moved to a different spot even when play is disabled. You can see the 823 white cross designating not playable in figure~\ref{fig:mixer-playable}. Statistics that show 824 the savings are as follows on a 16 cpu laptop: 825 \begin{verbatim} 826 5 Mixers all playing -- cpu usage = 1275 827 1 only playing -- cpu usage = 405 828 2 only playing -- cpu usage = 800 829 \end{verbatim} 830 \end{enumerate} 832 \subsubsection*{Mixers pulldown}% 833 \label{ssub:mixers_pulldown} 835 From Program windows menu we can access the \texttt{windows$\rightarrowmixers...} there are several entries available (see figure~\ref{fig:mixers-pulldown}): 837 \begin{figure}[htpb] 838 \centering 839 \includegraphics[width=0.5\linewidth]{mixers-pulldown.png} 840 \caption{Mixers... pulldown} 841 \label{fig:mixers-pulldown} 842 \end{figure} 844 \begin{description} 845 \item[Mixer Viewer:] \index{mixers!viewer} (Shift-M) opens a new empty \textit{Mixer 1} window, which will then be associated with a track. We can open as many Mixers as we like. 846 \item[Drag Tile mixers:] \index{mixers!drag tile mixers} (Alt-t) the default size of a mixer window is related to the std size of the \textit{Viewer} window. If we want to modify it at our convenience, we can build a window of the desired size with the \textit{Drag Tile mixers} function and then automatically order our mixers within it with the right click of the mouse. 847 \item[Align mixers:] \index{mixers!align mixers} see \nameref{sub:audio_video_sync_waveform} 848 \item[Mix masters:] \index{mixers!mix masters} this feature makes it very easy to get into the multi-camera mixer mode after tracks have already been set up and edited. Before this addition, you could only \textit{Open Mixers} from original media assets in the \textit{Resources} window. 849 \item[Entries:] If the mixers of several sources have been opened, the entries of each assets will appear, useful for recalling a specific mixer window. 850 \end{description} 852 \subsection{Recover Mixer Windows}% 853 \label{sub:recover_mixer_windows} 854 \index{mixers!recover mixer window} 856 It is a hazard that you might accidentally \textit{undo} (\texttt{z or Ctrl-z}) too far and lose your mixer windows. Here are the steps to recover. It is recommended that you make a backup of your project before performing the recovery steps just in case there are other problems. 858 In the patchbay box to the left of the main timeline, there is a right pointing arrow on the right side. This is called the \textit{Expander} (figure~\ref{fig:mixer-patchbay01}). When you click on each expander, another line appears below that expander arrow and the timeline track height is slightly increased. If you Shift-click on a single expander, the patchbay will expand for all of the tracks. 860 \begin{figure}[htpb] 861 \centering 862 \includegraphics[width=0.35\linewidth]{mixer-patchbay01.png} 863 \caption{Mixer patchbay} 864 \label{fig:mixer-patchbay01} 865 \end{figure} 867 \begin{enumerate} 868 \item Expand all of the patchbay lines, either one by one, or Shift-click on one to do them all. This is so you can see the \textit{mixer} right pointing arrow on the second expanded line. 869 \item Use the \textit{Window} pulldown and choose \textit{Mixer Viewer} to bring up a new mixer window. Now you will be making an association between the mixer viewer and the track’s video. 870 \item Click on the new mixer window to make sure it is highlighted with a white border. This designates it as the \textit{in use} mixer viewer. 871 \item Set your play to the beginning of the video using the \textit{Home} key or \textit{Home} transport button. 872 \item In the patchbay for a video track click on the \textit{mixer} arrow on the expanded2^{nd}line which is a right facing arrow. Now the arrow will point up. If there are audio tracks with that video, click on each of its audio tracks \textit{mixer} arrow until they point up also. 873 \item Next move your insertion pointer on the timeline where there is video. Some of the time this just helps so that the new mixer viewer window gets redrawn and you can see that the images appear; but the image may not appear until the program does a redraw later. Now the mixer viewer should be 874 correctly associated. Note if you have large video, give it some time to update. You may have to click on the mixer viewer window if the image does not show. You can always start over with that mixer if you encountered any problems. 875 \item Click the arrows that are pointing up in that video and its audio so they go back to pointing right. That mixer viewer is complete so you need to do this to make sure the \textit{mixer} arrows are off. 876 \end{enumerate} 878 Repeat steps 2 through 7 for each of the mixer viewers you need going down the patchbay starting on step 2 first with Mixer 1, then 2 to 7 steps for Mixer 2, then again run 2 to 7 for Mixer 3 and so on. 880 Sometimes the association does not stick initially. If not, highlight the mixer viewer with the problem, change the mixer arrows to point up, and reassociate. 882 % For html, make sure using below is different from via in subsection 883 \section{Audio/Video sync using Waveforms/Timecodes}% 884 \label{sec:audio_video_sync} 885 \index{audio/video sync} 887 \subsection{Audio/Video sync via Waveform}% 888 \label{sub:audio_video_sync_waveform} 889 \index{audio/video sync via waveform} 891 Multi-camera \index{multi-camera} footage of a single event can have various shots starting and ending at different times. So when the footage start times are different, you can use the mixer audio to synchronize the clips on the timeline. The program algorithm attempts to find and align automatically the waveforms of the media. 893 Synchronizing multiple camera videos based on audio tracks can be done with \CGG{} easily enough with the \texttt{Window\rightarrow$Mixers$\dots\rightarrowAlign mixers} option. Align mixers brings up a window displaying your mixers, the currently selected Master Track, and a list of all of the Audio Tracks (figure~\ref{fig:mixer-align01}). There is a limit of 32 audio tracks per each mixer (that should be enough!) 895 \begin{figure}[htpb] 896 \centering 897 \includegraphics[width=0.9\linewidth]{mixer-align01.png} 898 \caption{Align Mixers window} 899 \label{fig:mixer-align01} 900 \end{figure} 902 Two different methods of aligning the audio for mixers are available. They are most easily referred to by the button that is pressed -- \textit{Match} and \textit{Match All}. There are also 2 methods of activating the alignment -- \textit{Apply} and \textit{Move}. 904 \textit{Match} consists usually of the next set of steps to take advantage of this feature: 906 \begin{enumerate} 907 \item Load your camera media with insertion strategy of \textit{resources only} 908 \item Highlight in the Resources window, all of the media you want to mix. 909 \item Right mouse button on one of the media and click on Open Mixers; all mixer windows come up. 910 \item To arm the created mixer tracks and related audio tracks. They are automatically placed on the timeline disarmed. This is right for multi-cam but not for audio sync. 911 \item Provide a small target audio pattern on the Master Track for syncing by marking with the In/Out points ([ and ]). NOTE: When choosing between the audio tracks to align the one that will have to work as Master, we must rigorously use the one with the right-most synchronization peak. This is because the Master track will be started at the beginning of the timeline. If during alignment the other tracks should move further to the left than the Master, they will be truncated at 00:00:00:000, finding no more space available, and will be shortened with loss of footage for the next multi-cam 912 \item Make a selection on the timeline in which to look for the pattern. Left mouse click, then drag select and highlight a search time domain. 913 \item Use the \texttt{Window\rightarrow$Mixers$\dots\rightarrowAlign mixers} to bring up its dialog window. 914 \item Highlight in the first listbox, the Mixer number you want to align. Click on \textit{Match}. This will take a few seconds so watch the rendering time percentage on the lower right hand side zoom panel. The buttons will be ghosted out until finished. Now note the changed values in the Audio Tracks listbox. 915 \item If you are satisfied with the calculated Nudge values -- that is they are very close to1.0-- in the Audio Tracks listbox and the audio track selected as the Master Track in the Master Track listbox, hit the \textit{Apply} button. 916 \item If you plan on performing more alignment tasks, click on Checkpoint so you can go back to a previous step in case you make a mistake. 917 \item Last, click on the OK green checkmark or to cancel click on the red X. Or just close the gui. 918 \item To disarm the mixer tracks and related audio tracks for the next multi-cam operations. 919 \end{enumerate} 921 \paragraph{Reset} is used to start over with the current session data, not an undo. This means you can use the match repeatedly to refine alignments. All of the Audio Tracks listbox values will be reset. 923 \paragraph{Checkpoint} provides a method to create checkpoints that save the current state. This is especially helpful while learning or doing more complicated operations where you might make a mistake or do not like the results and need to get back to a previous state. 925 \paragraph{Undo} is used to put the media back to a previous state on the timeline. If you choose \textit{start over} the session will reload with the original, before any changes were applied. You can also go back to any of your previous checkpoints that you created earlier which are listed there, such as \textit{chkpt 1}. 927 \paragraph{Match All}is used when you have several mixers, instead of only picking 1 to match, it picks the best match for EACH of the mixer tracks based on a single master track. So when you hit Apply, each track might move differently. You do not set In/Out points but you have to make a selection within which to match. 929 \paragraph{Apply} button will apply the nudges that were generated during the Match or Match All execution (figure~\ref{fig:mixer-align02}). 931 \begin{figure}[htpb] 932 \centering 933 \includegraphics[width=0.9\linewidth]{mixer-align02.png} 934 \caption{Aligned mixer window after "Match". Note the Nudge amounts above.} 935 \label{fig:mixer-align02} 936 \end{figure} 938 \paragraph{Move} is very handy when you are using mixers, if you have an edit somewhere that you need to fix specifically without moving any of the other track pieces. In this case you have to select a section (like you do a group, but do not make a group), generate a match, and then you can just \textit{Move} that set only -- everything else stays where it is at its current location (figure~\ref{fig:mixer-align03}). 940 \begin{figure}[htpb] 941 \centering 942 \includegraphics[width=1.0\linewidth]{mixer-align03.png} 943 \caption{Match setup for aligning by audio. Note that [ ] are set over a sample waveform highlighted selection that includes that.} 944 \label{fig:mixer-align03} 945 \end{figure} 947 More detailed information follows about how this all works and the information in the dialog window. It is important to know that the result of the calculation is \textit{best match} but you can still override the selections if you decide there is a better one. The dialog window is split into 3 sections: 949 \begin{enumerate} 950 \item \textit{Mixers} lists the mixers that are active by highlighting them all initially. You can decide that you do 951 not want 1 or more mixers to be used in the correlation calculation by un-highlighting the one(s) that should not be used. In some cases you have to have at least 2 in order to align audio. 952 \item \textit{Master Track} lists each of the audio tracks currently loaded for all of the mixers. You can decide to highlight a different audio track to be used as the master for correlation, but only 1 can be used. Remember to use the track with the right-most synchronization peak as the Master. 953 \item \textit{Audio Tracks} lists each of the mixer audio tracks. Again, you can highlight a different set of which 954 mixer audio tracks that you want to use for the waveform correlation. Any audio tracks that are not 955 highlighted, that is \textit{turned off}, will not be considered in the correlation calculation. 956 \end{enumerate} 958 The corresponding input position is determined by track input correlation. 960 The letter "R" in the Audio Tracks listbox represents the correlation value.R=1.0$designates that if both the pattern and the matching section were in the highlighted area, they are completely correlated -- this is a good self-test to check. 961$Nudge=0.0means just that. 963 The header \textit{pos} stands for the timeline position. When the \textit{Apply} button is pressed, only the Mixers listbox is relevant at that time. 965 The Mixer with the master track generally does not move, everything else will be lined up (figure~\ref{fig:mixer-align04}). 967 \begin{figure}[htpb] 968 \centering 969 \includegraphics[width=1.0\linewidth]{mixer-align04.png} 970 \caption{An audio Match is complete. Note the waveform is aligned.} 971 \label{fig:mixer-align04} 972 \end{figure} 974 \textit{Match All} option basically consists of the following steps: 976 \begin{enumerate} 977 \item Highlight the Mixer to use in the Mixer listbox. 978 \item Highlight the Master Track you want to use in the Master Track listbox. 979 \item On the timeline, mark your selection on the Master Track. 980 \item Click on the \textit{Match All} button. 981 \item Note the nudge values to see if they make sense, and if so, press \textit{Apply}. 982 \end{enumerate} 984 \textit{Match} option basic steps (just for comparison with Match All): 986 \begin{enumerate} 987 \item Set the In/Out points [ ] of the target. 988 \item On the timeline, mark your selection. 989 \item Click on the \texttt{Match} button. 990 \item Note the nudge values to see if they make sense, then press \textit{Apply} (or \textit{Move} when doing a group). 991 \end{enumerate} 993 \subsection{Align Timecodes}% 994 \label{sub:align_timecodes} 995 \index{audio/video sync via timecodes} 996 \index{align timecodes} 998 Align Timecodes is especially useful in the case where you create video with multiple cameras capable of recording a timecode in the metadata of each file. Let’s say we have recorded three videos at the same time at a concert with each camera set up at unique positions and at different angles. All of the cameras start recording at various times but were synchronized with the same master clock (\textit{Timecode Sync Generator} or \textit{Jam Sync timecode}) so that the recordings are timestamped with that synchronized time. For example, the 3 cameras have embedded metadata as follows: \quad \textbf{camera 1} = 00:00:00:01 \quad \textbf{camera 2} = 00:00:10:07 \quad \textbf{camera 3} = 00:00:17:22 1000 \begin{figure}[ht] 1001 \centering 1002 \includegraphics[width=1.0\linewidth]{timecode-01.png} 1003 \caption{Before sync} 1004 \label{fig:timecode-01} 1005 \end{figure} 1007 Timecode synchronization is performed by the program when you choose the option \textit{Align Timecodes} and works in 2 steps. The first automatic step is to locate the earliest timecode \index{timecode} and the second step is to align the edits on the armed tracks using that time on the timeline. 1009 When you load in the three files to different tracks, they'll be placed on a timeline that starts from00:00:00:00as usual. But, by middle mouse clicking (MMB) on any of the video tracks, you can view the timecode embedded at that point in that file. 1011 To make it easy to align the videos onto the timeline manually, set the \textit{origin point}, that is the leftmost time index, of the timebar to the timecode of the earliest video. For the 3 aforementioned cameras this would be camera 1 with a timeecode of00:00:00:01$. By using the \textit{clapboard} button on the main timeline, you can set$00:00:00:01$as the start timecode. Then positioning the camera 1 video at the start of the timeline will line its frames up with the times on the timebar. 1013 \begin{figure}[ht] 1014 \centering 1015 \includegraphics[width=1.0\linewidth]{timecode-02.png} 1016 \caption{Clapperboard button} 1017 \label{fig:timecode-02} 1018 \end{figure} 1020 Similarly, using the middle mouse button to view the other 2 camera’s timecodes, you can position them so their timecodes line up and now all three tracks will be in sync. You can do this manually or simply do it automatically with the option under: \texttt{Tracks$\rightarrow$Align Timecodes}. See figure~\ref{fig:timecode-03} 1022 \begin{figure}[ht] 1023 \centering 1024 \includegraphics[width=1.0\linewidth]{timecode-03.png} 1025 \caption{After sync} 1026 \label{fig:timecode-03} 1027 \end{figure} 1029 Timecode synchronization in \CGG{} is based on the inclusion of an actual \textit{timecode} saved in 1030 the media as can be seen using the \texttt{ffprobe} or \texttt{mediainfo} command line. Examples are shown here: 1032 \begin{lstlisting}[style=sh] 1033 # ffprobe tut1.mov 1034 Metadata: 1035 handler_name: VideoHandler 1036 timecode: 00:00:20:01 1038 # mediainfo mtb.mp4 1039 Time code of first frame: 11:05:49:41 1040 Time code, striped: Yes 1041 Title: GoPro AVC 1042 \end{lstlisting} 1044 Many newer cameras and media in a MOV container have valid timecodes often with SMPTE 1045 compliant start Timecode in Quicktime format, whereas others do not. Most low end cameras do not. 1046 When no timecode is present, various clock time values may be used to synchronize instead. 1048 There are various types of timecodes and they are written in metadata in various ways. Because it is not clear which timecode in the file is the one to use or there is none, the precedence as described next is followed. The contexts, packets, streams, and data refer to the corresponding ffmpeg data or metadata objects: 1050 \begin{enumerate} 1051 \item if the format context has a timecode use: \texttt{fmt\_ctx$\rightarrow$timecode - 0} 1052 \item if the layer/channel has a timecode use: \texttt{st$\rightarrow$timecode - (start\_time -nudge)} 1053 \item find the first program stream with video stream, and if the video stream has a timecode use: 1054 \texttt{st$\rightarrow$timecode - (start\_time-nudge)} 1055 \item find timecode in any stream use: \texttt{st$\rightarrow$timecode} 1056 \item read 100 packets, save \texttt{ofs=pkt.pts*st$\rightarrow$time\_base - st$\rightarrow$nudge}: 1057 \begin{itemize} 1058 \item decode frame for video stream of the first program 1059 \item if \texttt{frame$\rightarrow$timecode} has a timecode use:\texttt{ frame$\rightarrow$timecode - ofs} 1060 \item if side\_data has gop (group of pictures) timecode use: \texttt{gop$\rightarrow$timecode - ofs} 1061 \item if side\_data has smpte timecode use: \texttt{smpte$\rightarrow$timecode - ofs} 1062 \end{itemize} 1063 \item if the \texttt{filename/url} scans \texttt{*date\_time.ext} use: \texttt{date\_time} (yr, month, day, hr, min, sec, nanosecond) 1064 \item if stat (display file status) works on the \texttt{filename/url} use: \texttt{mtime} (modification time) 1065 \item finally, if none of the above are usable, then return \texttt{failure} 1066 \end{enumerate} 1068 Operation of Align Timecodes includes the following options to help in your setup and review (figure~\ref{fig:timecode-04}): 1070 \begin{figure}[ht] 1071 \centering 1072 \includegraphics[width=1.0\linewidth]{timecode-04.png} 1073 \caption{Timecode via MMB on track and via Resources windows/Info} 1074 \label{fig:timecode-04} 1075 \end{figure} 1077 \begin{itemize} 1078 \item \textit{Align Timecodes} under the \textit{Tracks} pulldown 1079 \item \textit{Clapperboard} icon on the main timeline to set a timecode. 1080 \item Timecode option added to timebar choices (in timecode$hh:mm:ss:ff) with use of the session frame 1081 rate as a time standard. 1082 \item Time Code Start in the Resources window on the bottom of \textit{Info} for the media if the timecode for that 1083 asset has been scanned and is known. You can scan the asset’s Timecode by using the middle mouse 1084 button on its track which then displays the timecode for that on the timebar. 1085 \end{itemize} 1087 \paragraph*{Notes} 1089 \begin{itemize} 1090 \item \textit{Align Timecodes} is a good first step to quickly sync media up either perfectly or roughly. 1091 \item Currently this is not known to function for syncing audio recorded on a separate audio device, 1092 because no test material was provided to ensure that this works. 1093 \item Disarmed tracks will not be affected by using \textit{Align Timecodes}. 1094 \item No edits or sections will get cut off on the left or elsewhere, unless the moved edits overlap. 1095 \item You can add a timecode to video using an \texttt{ffmpeg} command if you want to align a particular video 1096 which has no timecode to ones that do. For example: 1097 \end{itemize} 1099 \begin{lstlisting}[style=sh] 1100 ffmpeg -i originalfile.mp4 -metadata timecode="14:36:08:29" -c copy newfile.mp4 1102 # where "-i" is followed by the input filename 1103 # where "-metadata timecode" is followed by the timecode key 1104 # where "-c copy" just copies the video/audio to the following output filename 1105 \end{lstlisting} 1107 \section{Subtitles}% 1108 \label{sec:subtitles} 1109 \index{subtitles} 1111 \CGG{} can create subtitles directly in the timeline with the Subtitle tool. Subtitles are added by using the main window pulldown \texttt{File\rightarrow$Subtitle} (Alt-y) which brings up a window allowing you to type the filename of a previously generated text file containing the desired words/lines, the script. After entering the filename, click \texttt{Load} to read in your script. By creating a script file ahead of time, it lets you easily add dialog that was already written out and carefully edited for spelling and proper grammar. 1113 The file must be plain text; a .srt or .sub can also be used, but only the text lines will be used and not the timecodes or comments. The format of the script/text input file has specific requirements as listed below: 1115 \begin{itemize} 1116 \item Lines can be any length but they will be broken up to fit according to some criteria below. 1117 \begin{itemize} 1118 \item Running text used as script lines will be broken into multiple lines. 1119 \item The target line length is 60 characters. 1120 \item Punctuation may be flagged to create an early break. 1121 \item Single carriage return ends an individual script line. 1122 \item Double carriage return indicates the end of a entry and helps to keep track of where you are. 1123 \end{itemize} 1124 \item The "\textit{=}" sign in column 1 indicates a comment seen in the script text to assist you in location. 1125 \item An "\textit{*}" at the beginning of the line is a comment and not a script line. 1126 \item \textit{Whitespace} at either the beginning of a script line or the end will be removed. 1127 \end{itemize} 1129 Figure~\ref{fig:subtitle01} shows the Subtitle window you will see. 1131 \begin{figure}[htpb] 1132 \centering 1133 \includegraphics[width=0.8\linewidth]{subtitle01.png} 1134 \caption{Subtitle window} 1135 \label{fig:subtitle01} 1136 \end{figure} 1138 To put the subtitles onto your media, first add a subtitle track via the pulldown \texttt{Tracks$\rightarrow\$ Add subttl} (Shit-Y). In the Subtitle window, note that there are 2 major textboxes.  There is the \textit{Script Text} textbox showing the current entry of text from your input file and there is the \textit{Line Text} textbox showing the currently active text.  In your subtitle track, select a timeline region (in/out or drag select with hairline cursor/highlight or via labels or the \textit{selection start/length/end time} textboxes in the Zoom Panel) to indicate the region where you want the active Line Text to be pasted.  Then click the \texttt{Paste} button in the Subtitle window to paste the line onto the subtitle track.  Silence will be added to the subtitle track in the places in the media where there are gaps.
1140 Editing in the Line Text box can be used to change the active script line. By double clicking the timeline over the subtitle track, you can reselect the active script line.  The subtitle text will be reloaded into the Line Text box and can be edited and re-pasted as the new active subtitle text.  You can also highlight multiple lines in the Script Text box and paste them (using the usual window paste methodology) into the Line Text box.  After pasting to the timeline, the Line Text box will be updated with the next script line.  In addition, if you triple click a line in the \textit{Script Text} box, it will automatically become the current line in the \textit{Line Text} box.
1142 When you are finished, before clicking on \textit{Save}, you can specify the output format using the \textit{Format} drop-down button. You can choose between the classic \texttt{.udvd} (micro DVD) and the more universally supported \texttt{.srt} (subrip) and \texttt{.sub} (subviewer). The next step is to provide a legitimate filname in the \textit{Path} box; your current directory will be used if only a filename but no directory path is supplied.  The filename used will automatically have a "--" after it followed by the \textit{track label} and then \textit{udvd/srt/sub} extension added; any extension in the filename will be removed..  If you click OK before saving, the subtitle script position is saved with the session.  This is convenient for continuing where you left off.
1144 To reposition the script, use the slider or tumbler buttons:
1146 \textit{Slider} bar to move through the text entries quickly. \\
1147 \textit{Prev} or \textit{Next} buttons to go to the previous or next script line.
1149 Figure~\ref{fig:subtitle02} shows what the pasted subtitle script looks like in a portion of the main window.
1151 \begin{figure}[htpb]
1152         \centering
1153         \includegraphics[width=1.0\linewidth]{subtitle02.png}
1154         \caption{Subtitles on timeline}
1155         \label{fig:subtitle02}
1156 \end{figure}