4 \section{\CGG{} Quick Start Guide}%
5 \label{sec:cin_quick_start_guide}
6 \index{quickstart guide}
8 \CGG{} is a software program NLE, Non-Linear Editor, that provides a way to edit, record, and play audio or video media on Linux. It can also be used to color correction, retouch photos, motion tracking, watch TV, and create DVDs.
10 \subsection{Install the Software}%
11 \label{sub:install_software}
13 On the internet, click on the Download page at:
15 {\small \url{https://cinelerra-gg.org/downloads/}}
17 Here you will see several Operating System distro packages that are already built for you to download. Click on your preference and read the specific instructions for usage.
21 \includegraphics[width=1.0\linewidth]{packages.png}
24 However, if you want to get going as quickly as possible, just do this so that everything is in 1 place:
26 \begin{itemize}[noitemsep]
27 \item Download your Operating System’s tar file from {\small \url{https://cinelerra-gg.org/download/tars/}} to \texttt{/tmp}.
28 \item Key in: cd /name-of-directory-where-you-want-the-software (for example, \texttt{cd /software})
29 \item Key in: \texttt{mkdir cin}
30 \item Key in: \texttt{cd cin}
31 \item Key in: \texttt{tar -xJf /tmp/cinelerra-5.1-*.txz} (if you put the tar in \texttt{/tmp} AND replace * with full name)
34 \subsection{Start \CGG{}}%
35 \label{sub:start_cinelerra_gg}
37 Depending on how you installed the software, you can log in as root or as a user if you used a package.
39 \begin{itemize}[noitemsep]
40 \item Key in: \texttt{/your-software-directory-path/bin/cin}
41 \item Or if you installed using the pkg method, click on the \textit{Cin icon}.
46 \includegraphics[width=1.0\linewidth]{Fenstergrundposition-en.png}
47 \caption{Clockwise: Viewer; Compositor; Resources and Main/Program/Timeline}
50 You will now see 4 separate windows appear. The top 2 windows from left to right are the Viewer which is most useful for previewing clips and media and the Compositor which displays the current working frame at the timeline position. The bottom 2 windows are the \CGG{} Program, also called the timeline, which is where the real work gets done and the Resources window showing a selection of media or effects.
52 Any of these windows can be resized to better suit your needs. Note that if your system’s native language is not English, some of the words you see on the screen will be correctly translated for you, others will be in english, and some will have not very good translations.
54 It is important to know that \CGG{} does not directly change your media. It writes all changes to what is called the EDL, Edit Decision List. This way you original media remains completely intact.
56 Before you get startedi here is a note about \textit{Context Help}. If you need more
57 detailed information on a window, menu, button or other particular GUI element than
58 is shown in the tooltip, press Alt/h hotkey and the HTML page in your configured web
59 browser will display the documentation for the item currently under the mouse.
61 \subsection{Load Media}%
62 \label{sub:load_media}
64 On the main timeline program window are many pulldowns, the first of which is \textit{File}.
68 \includegraphics[width=1.0\linewidth]{load.png}
69 \caption{Load media window -- note the icons top right for more options}
73 \item Click on \textit{File} for a list of available options and note that in the right hand column are shortcuts for
74 many of the options that will come in handy if you use \CGG{} often.
75 \item Next click on the second one down -- \textit{Load files$\dots$} -- which brings up the Load menu.
76 \item Below \textit{Select files to load} on the top left side is a textbox and if you look all the way to the right
77 side of the textbox, there is a down arrow which you use to navigate your file system. Highlight the
78 desired file system and you will see that directory name appear in the textbox and the files below.
79 \item Scroll to the media file you would like to work on and highlight that file. When you do, you will
80 see that filename also appear in the textbox below the listing of files. You could have directly
81 keyed in that file in that textbox instead.
82 \item On the bottom of the Load menu, is a box called \textit{Insertion strategy}. For getting started the
83 default of \textit{Replace current project} is sufficient. But you can click on the down arrow to see what
84 is available for future use.
85 \item Now click on the green colored checkmark on the bottom left hand side to actually load the file
86 and see it appear on the timeline in \CGG{}’s main window and a single frame in the Compositor.
87 The first track will most likely be video thumbnails and the next tracks may be audio waveforms.
88 \item Press the space bar in the main Program window and your video will start playing and press the
89 space bar again to stop the play. While playing, you should see the video in the Compositor
90 window in the upper right hand side of your screen and if you have your audio hooked up, you
91 will hear the sound. To get back to the beginning of the video, hit the home key on your keyboard.
94 \subsection{Choose Output Format}%
95 \label{sub:choose_output_format}
97 You can skip this step if you want the format of your output to be the same as your input. However, to create output media that is widely viewable on many platforms, to include phones and television, you should set your format accordingly.
100 \item On the main timeline, use the \textit{Settings} pulldown (about the $7^{th}$ pulldown from the left side top) and
101 click on \textit{Format} which is the first option in that list.
102 \item A \textit{Set Format} menu will appear that shows what the current format is for your loaded media in an
103 Audio and a Video tab. In the United States, the Video Frame rate is usually expected to be 29.970 and usually the Color model is only changed if you have a personal preference.
104 \item The \textit{Canvas size} is probably the only thing you will want to change here in order to get to the
105 most commonly viewable settings. On the right hand side of the Width parameter is a down arrow.
106 Left click the down arrow to see your options.
107 \item Highlight $1280\times720$ HD for a good common option.
108 \item Click OK to have this option take effect. When you do, the Compositor window may change to fit
109 this option and may look wrong sized.
114 \includegraphics[width=0.7\linewidth]{set-format.png}
115 \caption{Format menu to change settings}
118 \begin{enumerate}[resume]
119 \item If the video now looks too small or too large in the Compositor, you will want to \textit{autoscale} it to
120 look correct when the new media is created. To do this, mouse over to the Resources window in the
121 lower right hand corner and under the word Visibility, highlight \textit{Video Effects} to see some
123 \item Mouse over the \textit{Auto Scale} icon, left click to highlight the words underneath the icon, and mouse
124 drag the icon to the timeline video track. When you see a white colored outline show on that track,
125 drop the Auto Scale icon there and you will see that the video may now automatically scale to a
126 new value. Click on the magnifying glass icon on the brown colored line beneath the main timeline
127 video which opens a new window. In that window, again use the down arrow to choose $1280\times720$
128 HD, then dismiss this window.
129 \item If not needed, to remove the Auto Scale plugin, right mouse on the brown line and choose \textit{Detach}.
134 \includegraphics[width=0.8\linewidth]{magnifier.png}
135 \caption{Effect blue bar with magnifier}
138 \subsection{View and Listen}%
139 \label{sub:view_listen}
143 \includegraphics[width=1.0\linewidth]{pulldown_button.png}
144 \caption{Menu pulldowns at the top with Transport buttons below. Note the colored tooltips too.}
148 \item On the second line, below the pulldowns, are transport buttons to move back and forth on the
149 timeline and play forward or reverse, fast or slow, or a single frame. When you mouse over one of
150 these buttons, a colored tooltip appears to tell you its function along with a key shortcut
151 inside of parenthesis. When you left click the mouse on the transport button it starts the play and
152 click again to stop it. As you use these buttons, watch the Compositor to watch your video.
153 \item On the timeline, you only see thumbnails and not every single picture. You may want to
154 use your keyboard’s \textit{down arrow} to expand the thumbnails and the \textit{up arrow} to unexpand them -- on
155 United States keyboard, the arrow keys are generally together on the lower right hand side of the
156 keyboard, a little to the right of the space bar. This is a more cpu intensive operation and for very
157 large video can be time-consuming.
160 If you need more detailed information on a button or other particular GUI
161 element than is shown in the tooltip, press Alt/h hotkey and the HTML page
162 in your configured web browser will display the documentation for the item
163 currently under the mouse.
165 \subsection{Edit/Compose}%
166 \label{sub:edit_compose}
168 There may be sections of your media that you want to delete, or audio that is hard to hear and needs to be enhanced, or there is a need for a descriptive title that you want to add. Here are a few basics. But first be sure that you are in \textit{cut and paste} mode (this is the default) by checking to verify that you see a gold color around the “I” i-beam mode icon as in the figure above. If the arrow to the left is gold, you are in \textit{drag and drop} mode so switch to \textit{cut and paste} by clicking on the “I” instead.
172 \includegraphics[width=1.0\linewidth]{some_editing.png}
173 \caption{From left to right: Audio 1 is disarmed -- BandSlide transition in Video 1 -- A highlighted section.}
177 \item You should look at the \textit{Edit} pulldown - $2^{nd}$ from the upper left on the main timeline to see the
178 most common options to use. The first option in the list is \texttt{Undo} followed by a terse comment of
179 the last operation that you performed that can be undone.
180 \item To delete a section of video/audio is described next. Various ways to do that are available but the
181 easiest is to move your mouse and left click at the beginning of the section you want to delete on the
182 timeline and while holding down the left mouse button, drag to the end of the section to be deleted.
183 When you do this, a white colored highlighted section becomes visible. Use the \textit{Edit} pulldown and
184 choose the \textit{split/cut} option to cut out the highlighted area (note the shortcut of "x"). Remember if you
185 cut the wrong thing out you can always use the Edit pulldown to Undo that.
186 \item To add a transition where there is deleted section which may make your video look disjointed, do
187 the following. Go back to the Resources window in the bottom right hand corner. Change to
188 \textit{Video Transitions} by highlighting that underneath the word \textit{Visibility}. Highlight a transition like
189 \textit{BandSlide} with the left button mouse click, hold down and drag to the video track and when you see
190 a white colored box around the area that you deleted above, drop the icon. Right mouse click the
191 icon on the track to vary some parameters like length.
192 \item To insert another clip from a different video, first you have to load the other video on another track.
193 Go to \textit{File} pulldown again and choose the \textit{Load files} option. Type in a directory at the top again and
194 erase any specific file that you may have chosen previously in the bottom 2 textboxes. It is very
195 important to now change the Insertion strategy to \textit{Append in new tracks} or you will write over
196 your current work. But if you make this mistake, you can use the Edit pulldown and Undo that!
198 \item Once the new video is on the track below your current work, you want to work with only this new
199 track, so disarm your other tracks by looking to the left of each track’s timeline and click the $2^{nd}$
200 button beneath the track name, for example Video 1 or Audio 1. The track name textbox will turn
201 red to remind you that the track has been disarmed. The boxed area is called the patchbay.
202 \item Move to the area you want to make a clip of on your newly loaded track, hold down the left mouse
203 button and drag the area to be made into a clip which will turn the color white. Remember, you
204 disarmed the other tracks so only this track is relevant at this time. On the second line of the main
205 window to the right of the transport buttons, are action buttons and as you mouse over them a
206 colored tooltip explains its purpose. Find the one that says \textit{To clip} which is on the right
207 hand side of the right bracket symbol.
208 \item Click on \textit{To clip} and a small window comes up which you can comment in, but you do not have
209 to, so just click on the green checkmark and now you will have a clip.
210 \item Disarm that new track and re-arm your original tracks so you can go back to working on them
211 \item Move your cursor to the spot in your original video where you want to insert the clip. Make a
212 \textit{Split} with the \textit{Split | Cut} option.
213 \item Go to the Resources window and under the word Visibility, highlight \textit{Clips} so you can see your
214 recently created clip in the box to the right. Highlight that clip and drag it to where you did the
215 blade cut and drop it in.
221 \includegraphics[width=1.0\linewidth]{title_color.png}
222 \caption{Compositor + Title menu for setting parameters + the Color Picker.}
225 \begin{enumerate}[resume]
226 \item To add a Title or any wording you will use the \textit{Title} plugin. In the Resources window, under the
227 word \textit{Visibility}, highlight \textit{Video Effects}. In the box to the right, many plugin icons appear. Scroll
228 to the right using the scroll bar at the bottom of the Resources window to locate Title. Highlight the
229 \textit{Title} icon and drag/drop to your video track. By now your video track may be in sections as you
230 deleted, added blade cuts, and inserts so where you drop the Title icon will be surrounded with a
231 white colored box. It will take effect in that entire area so you may want to highlight a section as
232 usual with left mouse click on the timeline and drag to the end of the desired area.
234 \item Right click on the brown colored bar that appeared below your video track to get to options and
235 then left click on \textit{Show} to get the Title window to appear.
236 \item Now for the fun part. First type in some words in the bottom large text box just to see what it
237 does. There are so many variable parameters here and they are a lot of fun to play around with.
238 \item You can dismiss the Title window when finished BUT be sure to leave the brown colored Title bar
239 on the track. And if you enabled the \textit{drag} feature, you should disable it so you do not forget.
240 \item Right mouse click on the bottom Text box to see many more interesting parameters.
244 \subsection{Back up your work}%
245 \label{sub:backup_your_work}
247 At this time, or even earlier if you think you might make a mistake or if you are concerned about computer crashes, you should save your work. Use the \textit{File} pulldown, and you can use \textit{Save as} to designate a directory and filename. Then click the green checkmark. You are saving the EDL which is the set of changes that you have made -- this file is separate from your original media.
249 \subsection{Create your new media}%
250 \label{sub:create_new_media}
253 \item Once again in the main Program window, click on the \textit{File} pulldown and highlight/click the \texttt{Render}
254 option which is about the $9^{th}$ option down from the top of the list. A \CGG{} Render menu will
256 \item First key in the first textbox the file to render to under \textit{Select a file to render to}.
257 \item For the \textit{File Format}, click on the down arrow and select FFMPEG (because this is the most
258 commonly used format; later you may want to experiment with others). To the right of that box,
259 click on the down arrow and highlight \texttt{mp4} -- again because this is common. When you click on
260 mp4, notice that if there is an extension to your filename in the \textit{file to render to} above, it may
261 change it to mp4 and if there is none, it will add \texttt{.mp4} because that is what is expected.
262 \item Make sure there is a red colored checkmark next to the words Audio and Video right below if you
263 have/want both audio and video. To the left of that checkmark box, is a symbol that looks like a
264 wrench. Click on this for Audio just to see the default Preset options which are just fine so dismiss
265 the menu. Then click on the wrench for Video and check Pixels by using the down arrow to the
266 right to be yuv420p -- this is most commonly usable option. And click on the green checkmark.
267 \item Check the Insertion Strategy in the Render Menu window. You might want to change that to
268 a different strategy than the default of \textit{Append in new tracks}. If not, then when the Render is done,
269 your new video will automatically be loaded in another set of tracks below your work tracks. Click
270 on the green checkmark in the lower left corner to start the render.
271 \item As the render is running, you will see the video play by in the Compositor. Rendering is usually
272 slow, especially with plugins added.
275 \subsection{Play your new media}%
276 \label{sub:play_your_new_media}
278 The file you created in the Render step should now be playable. You can test this in \CGG{} most easily by going to the Resource window in the lower right corner, clicking on the Media folder, and dragging and dropping the last video to the Viewer window. There is a separate set of transport buttons on the bottom on that screen to use for playing.
280 \section{Overview on Formats and Codecs}%
281 \label{sec:overview_formats}
285 Here is an overview of the formats (also called containers) and codecs that are used in \CGG{}, by ffmpeg and the internal engine. Roughly speaking these are divided into uncompressed codecs (or codecs with \textit{Intraframe} compression, which can be lossy or lossless) and compressed codecs of \textit{Interframe} type (LongGOP, almost always with lossy compression). The All-I (intraframe) codecs are suitable for editing because a cut or other operation on the timeline corresponds to the exact frame on which you are operating. The interframe types use Groups of Pictures (GOP) and a cut or other operation is accurate (and requires no further calculation) only if it coincides with the beginning of the GOP, and not with an internal frame. There is also color compression: Color Space \textit{bit-depth} and \textit{Chroma-Subsampling} for YUV models. In addition, heavy compression requires the system to do more encoding/decoding work on the timeline. High quality codecs have high bit rates and bit depths but this also affects the performance of the system, not to mention the increased disk space usage. Some formats implement both audio and video streams, others audio only or video only.
287 \subsection{Video FFmpeg Formats}%
288 \label{sec:FFmpeg_video}
290 FFmpeg supports hundreds of codecs and formats. Some are proprietary and cannot be implemented in FFmpeg or can be voluntarily compiled as non-free; others are proprietary but their use is free; finally there are the Open formats/codecs, fully supported and well documented. We are only describing here a selection of the most well-known and most frequently used ones.
292 \subsubsection{High Quality}
293 \label{ssub:ffmpeg_video_high_quality}
295 High quality formats are also called Mezzanine codecs, Digital Intermediate, Preservation codecs or Editing codecs. These have no compression or intraframe lossless or near-lossless compression and are suitable for editing, post-processing, mastering and archiving. They are also used for the interchange of files between different programs. They take up a lot of disk space and require a powerful system.
298 \item[MKV] Open, highly configurable and extensively documented. Can have seeking problems. Belongs to the Matroska family.
299 \newline Presets: \textit{ffv1, ffvyuv}
300 \item[MXF] Created by Avid. It is probably the best and most advanced container for editing.
301 \newline Presets: \textit{DNxHR, ffv1, AVC\_Intra\_100}
302 \item[MOV] Created by Apple. It is a suitable format for editing because it organizes the files within the container into hierarchically structured \textit{atoms} described in a header. This brings simplicity and compatibility with various software and does not require continuous encoding/decoding in the timeline.
303 \newline Presets: \textit{DNxHR, ffv1, CineformHD, huffyuv}
304 \item[PRO] Different extension, but it is still mov. Prores is proprietary and there are no official encoders except the original Apple one. The engine used by ffmpeg is the result of reverse engineering and, according to Apple, does not guarantee the same quality and performance of the original\protect\footnote{https://support.apple.com/en-us/HT200321}. Option \textit{vendor=apl0} is used to make it appear that the original Apple engine was used.
305 \newline Presets: \textit{ProRes; ProRes\_ks}
306 \item[QT] Different extension, but it is always mov.
307 \newline Presets: \textit{DNxHD, magicyuv, raw, utvideo}
308 \item[MP4] mostly used for General Purpose. It belongs to the large MPEG family.
309 \newline Presets: \textit{AVC\_Intra\_100}
310 \item[RGB] Raw format.
311 \newline Presets: \textit{raw}
312 \item[YUV] Raw format.
313 \newline Presets: \textit{raw}
314 \item[AVI] Old and limited format (no multi streams, no subtitles, limited metadata) but with high compatibility.
315 \newline Presets: \textit{ffv1}
318 \subsubsection{General Purpose}
319 \label{ssub:ffmpeg_video_general_purpose}
321 These are also called Delivery codecs. They are the most used and widespread being suitable for streaming, video sharing, watching TV, smartphones, plus more. Because of lossy compression type Interframe, they produce smaller files with variable quality. They are not suitable for editing, compositing and color correction. Further rendering of these formats worsens the quality exponentially. The most used codecs have hardware support (vaapi, vdpau, nvenc) that make them more efficient.
324 \item[MOV] Created by Apple. It is a suitable format for editing because it organizes the files within the container into hierarchically structured "atoms" described in a header. This brings simplicity and compatibility with various software and does not require continuous encoding/decoding in the timeline.
325 \newline Presets: \textit{Presets: mov}
326 \item[QT] Different exstension, but it is always mov.
327 \newline Presets: \textit{mjpeg, DV, Div, CinePack}
328 \item[MP4] The most popular. Many other formats belong to this family (MPEG);
329 \newline h264 is actually x264, open, highly configurable and documented; h265/HEVC is actually x265, open, highly configurable and documented. x264-5 is for encoding only.
330 \newline Presets: \textit{h265, h265, mjpeg, mpeg2, obs2youtube}
331 \item[WEBM] Open; similar to mp4 but not as widespread (it is used by YouTube). It belongs to the Matroska family. In \CGG{} there are specific Presets with \texttt{.youtube} extension, but they are still webm.
332 \newline Presets: \textit{VP8, VP9, AV1}
333 \item[MKV] Open, highly configurable and widely documented. It might have seeking problems. It belongs to the Matroska family.
334 \newline Presets: \textit{Theora, VP8, VP9}
335 \item[AVI] Old and limited format (no multistreams, no subtitles, limited metadata) but with high compatibility.
336 \newline Presets: \textit{asv, DV, mjpeg, xvid}
337 \item[MPG] Parent of the MPEG family, to which MP4 also belongs. Mpeg is used by \CGG{} as default for proxies and mpeg-2 is the standard for Video DVDs.
338 \newline Presets: \textit{mpeg, mpeg2}
341 \subsubsection{Note on Matroska (mkv) container}
342 \label{ssub:note_mkv_container}
345 Matroska is a modern universal container that is Open Source so there is lots of ongoing development with community input along with excellent documentation. Also derived from this format is the \textit{Webm} container used by Google and YouTube, which use the VP8-9 and AV1 codecs. Although using in \CGG{} is highly recommended, you may have seeking problems during playback. The internal structure of matroskas is sophisticated but requires exact use of internal keyframes (I-frame; B-frame and P-frame) otherwise playback on the timeline may be subject to freeze and drop frames. The mkv format can be problematic if the source encoding is not done well by the program (for example, OBS Studio). For an easy but accurate introduction of codecs and how they work see: {\small\url{https://ottverse.com/i-p-b-frames-idr-keyframes-differences-usecases/}}.
347 To find out the keyframe type (I, P, B) of your media you can use ffprobe:
349 \begin{lstlisting}[numbers=none]
350 $ ffprobe -v error -hide_banner-of default=noprint_wrappers=0 -print_format flat -select_streams v:0 -show_entries frame=pict_type input.mkv
353 \textbf{-v error -hide\_banner:} serves to hide a blob of information that is useless for our purposes.
355 \textbf{-of:} is an alias for \textit{-print\_format} and is used to be able to use \textit{default=noprint\_wrappers =0}.
357 \textbf{-default=noprint\_wrappers=0:} is used to be able to show the information from the parsed stream that we need.
359 \textbf{-print\_format flat:} is used to display the result of ffprobe according to a \textit{flat} format (you can choose CSV, Json, xml, etc).
361 \textbf{-select\_streams v:0:} is used to choose the first stream (0) in case there are multiple audio and video streams (tracks, in \CGG{}).
363 \textbf{-show\_entries:} shows the type of data collected by ffprobe that we want to display (there are also types: \texttt{\_streams}, \texttt{\_formats}, \texttt{\_packets}, and \texttt{\_frames}. They are called \textit{specifiers}).
365 \textbf{-frame=pict\_type:} within the chosen specifier indicates the data to be displayed; in this case \textit{pict\_type}, that is, the keyframe type (I, P, B) of the frame under consideration.
367 \textbf{input.mkv:} is the media to be analyzed (it can be any container and codec).
369 (see {\small\url{https://ffmpeg.org/ffprobe.html}} for more details)
371 We thus obtain a list of all frames in the analyzed media and their type. For example:
373 \begin{lstlisting}[numbers=none]
374 frames.frame.0.pict_type="I"
375 frames.frame.1.pict_type="P"
376 frames.frame.2.pict_type="B"
377 frames.frame.3.pict_type="B"
378 frames.frame.4.pict_type="B"
382 There are also 2 useful scripts that not only show the keyframe type but also show the GOP length of the media. They are zipped tars with readme's at: \newline
383 {\small\url{https://cinelerra-gg.org/download/testing/getgop_byDanDennedy.tar.gz}} \newline
384 {\small\url{https://cinelerra-gg.org/download/testing/iframe-probe_byUseSparingly.tar.gz}}
386 We can now look at the timeline of \CGG{} to see the frames that give problems in playback. Using a codec of type Long GOP, it is probably the rare I-frames that give the freezes.
387 To find a solution you can use MKVToolNix ({\small\url{https://mkvtoolnix.download/}}) to correct and insert new keyframes into the mkv file (matroska talks about \textit{cues data}). It can be done even without new encoding. Or you can use the \texttt{Transcode} tool within \CGG{} because during transcoding new keyframes are created that should correct errors.
389 \subsubsection{Image Sequences}
390 \label{ssub:ffmpeg_image_sequences}
392 The image sequences can be uncompressed, with lossy or lossless compression but always Intraframe. They are suitable for post-processing that is compositing (VFX) and color correction. Note: even if \CGG{} outputs fp32, exr/tiff values there are normalized to 0-1.0f.
395 \item[DPX] Film standard; uncompressed; high quality. \textit{Log} type.
396 \item[PNG] Uncompressed or lossless compression. Supports alpha channel.
397 \item[WEBP, TIFF, GIF, JPEG, ...] Variable compression, size and quality.
400 \subsubsection{Old Pro Formats}
401 \label{ssub:ffmpeg_old_pro_formats}
403 Some formats, though used in the past in the pro field, are disappearing with the evolution of technologies. DVD is becoming more and more niche, while Bluray is still widespread (also as a backup); DV/HDV remains only as a support for old Camcorders with magnetic tapes. DV is still a quality format, with intraframe compression; HDV is mpeg-2 compressed.
406 \item[AVI] old and limited format but with high compatibility.
407 \newline Presets: \textit{DV\_pal, DV\_ntsc, mjpeg}
408 \item[QT] belongs to the Apple mov family.
409 \newline Presets: \textit{DV, mjpeg}
410 \item[M2TS] format for Bluray (mpeg4). Bluray player devices need a standard Bluray disc structure (bdwrite) for playback\protect\footnote{\CGG{} offers specific functionality for creating DVDs/Blurays}.
411 \newline Presets: \textit{AVC422, Lossless, Bluray, hevc}
412 \item[MP4] Belongs to the MPEG family. Motionjpeg has jpeg compression, then Intraframe, so it maintains good quality and fluidity in editing. It is now an old and limited codec.
413 \newline Presets: \textit{mjpeg}
416 \subsection{Audio FFmpeg Formats}%
417 \label{sub:FFmpeg_audio}
419 Audio formats and codecs take much less resources and space than video ones, so they are often used without compression for maximum quality. However these are compressed formats and codecs widely used in streaming and sharing.
421 \subsubsection{High Quality}
422 \label{ssub:ffmpeg_audio_high_quality}
425 \item[FLAC] Open; used for storing music. It has lossless compression.
426 \newline preset: \textit{flac}
427 \item[PCM] Raw format that encodes the signal with \textit{modified pulse modulation} (pcm). FFmpeg does not support pcm audio if you use mp4 as a container.
428 \newline Presets: \textit{s8, s16, s24, s32}
429 \item[WAV] Raw format created by Microsoft. 32-bit addressing leading to the 4 GB recording limit. It is a widely used standard.
430 \newline Presets: \textit{s24le, s32le}
431 \item[W64] Wave format created by Sony to override the 4GB recording limit. Poorly supported.
432 \newline Presets: \textit{s16le, s24le, s32le}
433 \item[MKA] Open, highly configurable and documented. It belongs to the Matroska family. Uncompressed pcm type.
434 \newline Presets: \textit{s16le, s24le, s32le}
435 \item[ALAC] Apple's codec, free to use but not open source. It is lossless and of high quality but is slower than other similar codecs.
436 \newline Presets: \textit{m4a, mkv, qt}
439 \subsubsection{General Purpose}
440 \label{ssub:ffmpeg_audio_general_purpose}
443 \item[MP3] Belongs to the MPEG family. The most widely used in streaming and sharing.
444 \newline preset: \textit{mp3}
445 \item[OGG] Open, highly configurable and documented. It belongs to the Matroska family. Flac has lossless compression; opus is compressed but modern and of good quality, superior to mp3. Vorbis is compressed and dated, but lightweight and compatible.
446 \newline Presets: \textit{flac, opus, vorbis}
447 \item[PRO] Created by Apple; compressed audio codec, competing with mp3.
448 \newline Presets: \textit{aac256k}
451 \subsection{\CGG{} Internal Engine}%
452 \label{sub:internal_engine}
454 FFmpeg is the default engine, but you can also use its internal engine, which is limited in supported formats but efficient and of high quality.
456 \subsubsection{Video general purpose}
457 \label{ssub:internal_general_purpose}
460 \item[RAW DV] supports the DV standard.
461 \newline Presets: \textit{dv}
462 \item[MPEG Video] highly configurable. Extension \texttt{.m2v}.
463 \newline Presets: \textit{mpeg1, mpeg2}
464 \item[OGG Theora/Vorbis] Open, easily configurable. Theora for video, Vorbis for audio.
465 \newline Presets: \textit{theora, vorbis}
468 \subsubsection{Image Sequences}
469 \label{sub:internal_image_sequences}
471 There are quite a few formats available. Note: even if \CGG{} outputs fp32, exr/tiff values there are normalized to 0-1.0f.
474 \item[EXR Sequence] OpenEXR (Open Standard) is a competing film standard to DPX, but \textit{Linear} type.
475 \item[Ppm Sequence] is RGB Raw.
476 \item[Tga Sequence] is RGB(A) compressed or uncompressed.
477 \item[Tiff Sequence] is RGB(A) or RGB(A)-Float with various compression types.
478 \item[Jpg, gif Sequences] lossy compressed and limited formats.
481 \subsubsection{Audio general purpose}
482 \label{sub:internal_audio_general_purpose}
485 \item[AC3] widely used multichannel standard (Dolby Digital). Format with lossy compression.
486 \newline Presets: \textit{ac3}
487 \item[Apple/SGI AIFF] Created by Apple; is an uncompressed format (pcm type) or with 32/64-bit floating point compression.
488 \newline Presets: \textit{aif}
489 \item[Sun/Next AU] created by Sun and used in Unix environment, now in disuse. It can be of pcm type or with lossy compression.
490 \newline Presets: \textit{au}
491 \item[Flac] Open, lossless compression, very good quality.
492 \newline preset: \textit{flac}
493 \item[Microsoft WAV] created by Microsoft. It can have 16-24-32-bit linear or float compression.
494 \newline Presets: \textit{wav}
495 \item[MPEG Audio] Very widespread standard. Extension \texttt{.mp3}.
496 \newline Presets: \textit{mp3}
499 \section{Overview on Color Management}%
500 \label{sec:overview_color_management}
501 \index{color!management}
503 \CGG{} does not have support for ICC color profiles or global color management to standardize and facilitate the management of the various files with which it works. But it has its own way of managing color spaces and conversions; let's see how.
505 \subsection{Color Space}%
506 \label{sub:the_color_spaces}
508 A color space is a subspace of the absolute CIE XYZ color space that includes all possible, human-visible color coordinates (therefore makes human visual perception mathematically tractable). CIE XYZ is based on the RGB color model and consists of an infinite three-dimensional space but characterized (and limited) by the xyz coordinates of five particular points: the Black Point (pure black); the White Point (pure white); Reddest red color (pure red); Greenest green color (pure green); and Bluest blue color (pure blue). All these coordinates define an XYZ matrix. The color spaces are submatrices (minors) of the XYZ matrix. The absolute color space is device independent while the color subspaces are mapped to each individual device. For a more detailed introduction see: \small\href{https://peteroupc.github.io/colorgen.html}{https://peteroupc.github.io/colorgen.html}
509 \normalsize A color space consists of primaries (\textit{gamut}), transfer function (\textit{gamma}), and matrix coefficients (\textit{scaler}).
512 \item[Color primaries]: the gamut of the color space associated with the media, sensor, or device (display, for example).
513 \item[Transfer characteristic function]: converts linear values to non-linear values (e.g. logarithmic). It is also called Gamma correction.
514 \item[Color matrix function] (scaler): converts from one color model to another. $RGB \leftrightarrow YUV$; $RGB \leftrightarrow Y'CbCr$; etc.
517 The camera sensors are always RGB and linear. Generally, those values get converted to YUV in the files that are produced, because it is a more efficient format thanks to chroma subsampling, and produces smaller files (even if of lower quality, i.e. you lose part of the colors data). The conversion is nonlinear and so it concerns the "transfer characteristic" or gamma. The encoder gets input YUV and compresses that. It stores the transfer function as metadata if provided.
522 A color management system (CMS) describes how it translates the colors of images/videos from their current color space to the color space of the other devices, i.e. monitors. The basic problem is to be able to display the same colors in every device we use for editing and every device on which our work will be viewed. Calibrating and keeping our hardware under control is feasible, but when viewed on the internet or DVD, etc. it will be impossible to maintain the same colors. The most we can hope for is that
523 there are not too many or too bad alterations. But if the basis that we have set up is consistent, the alterations should be acceptable because they do not result from the sum of more issues at each step. There are two types of color management: \textit{Display referred} (DRC) and \textit{Scene referred} (SRC).
526 \item \textbf{DRC} is based on having a calibrated monitor. What it displays is considered correct and becomes the basis of our color grading. The goal is that the colors of the final render will not change too much when displayed in other hardware/contexts. Be careful to make sure there is a color profile for each type of color space you choose for your monitor. If the work is to be viewed on the internet, be sure to set the monitor in \textit{sRGB} with its color profile. If for HDTV we have to set the monitor in \textit{rec.709} with its color profile; for 4k in \textit{Rec 2020}; for Cinema in \textit{DCP-P3}; etc.
527 \item \textbf{SRC} instead uses three steps:
529 \item The input color space: whatever it is, it can be converted manually or automatically to a color space of your choice.
530 \item The color space of the timeline: we can choose and set the color space on which to work.
531 \item The color space of the output: we can choose the color space of the output (on other monitors or of the final rendering).
533 \textit{ACES} and \textit{OpenColorIO} have an SRC workflow. Please note that the monitor must still be calibrated to avoid unwanted color shifts.
534 \item There is also a third type of CMS: the one through the \textbf{LUTs}. In practice, the SRC workflow is followed through the appropriate 3D LUTs, instead of relying on the internal (automatic) management of the program. The LUT combined with the camera used to display it correctly in the timeline and the LUT for the final output. Using LUTs, however, always involves preparation, selection of the appropriate LUT and post-correction. Also, as they are fixed conversion tables, they can always result in clipping and banding.
537 \subsection{Display}%
540 Not having \CGG{} a CMS, it becomes essential to have a monitor calibrated and set in sRGB that is just the output displayed on the timeline of the program. You have these cases:
543 \begin{tabular}{|l|l|p{8cm}|}
545 \textbf{Timeline} & \textbf{Display} & \textbf{Description} \\
547 sRGB & sRGB & we get a correct color reproduction \\
548 sRGB & Rec.709 & we get slightly dark colors, because gamma \\
549 sRGB & DCI-P3 & we get over-saturated dark colors, because gamma and bigger gamut \\
554 \subsection{Pipeline CMS}%
555 \label{sub:pipeline_cms}
557 INPUT $\rightarrow$ DECODING/PROCESSING $\rightarrow$ OUTPUT/PLAYBACK $\rightarrow$ DISPLAY $\rightarrow$ ENCODING
560 \item[Input] color space and color depth of the source file; better if combined with an ICC profile.
561 \item[Decoding] how \CGG{} transforms and uses the input file (it is a temporary transformation, for usage of the internal/ffmpeg engine and plugins).
562 \item[Output] our setting of the project for the output. In \CGG{} such a signal is \texttt{8-bit sRGB}, but it can also be 8-bit YUV in \textit{continuous playback}.
563 \item[Display] as the monitor equipped with its color space (and profiled with ICC or LUT) displays the signal that reaches the user and what we see. The signal reaching the display is also mediated by the graphics card and the operating system CMS, if any.
564 \item[Encoding] the final rendering stage where we set not only formats and codecs but also color space, color depth, and color range.
567 \subsection{How \CGG{} works}%
568 \label{sub:how_cingg_works}
571 \item[Decoding/playback:] Video is decoded to internal representation (look at \texttt{Settings /Format/Color model}). Internal format is unpacked as 3 color values + one alpha value every pixel. \CGG{} has 6 internal pixel formats (RGB(A) 8-bit; YUV(A) 8-bit and RGB(A)\_FLOAT 32-bit (see Color Model in \nameref{sec:video_attributes}). The program will configure the frame buffer for your resulting video to be able to hold data in that color model. Then, for each plugin, it will pick the variant of the algorithm coded for that model.
572 \CGG{} automatically converts the source file to the set color model (in a buffer, the original is not touched!). Even if the input color model matches what we set in \texttt{Settings/Format/Color model}, there will always be a first conversion because \CGG{} works internally (in the buffer) at 32-bit in RGB. For playback \CGG{} has to convert each frame to the format acceptable by the output device, i.e. sRGB 8-bit. In practice, the decoded file follows two separate paths: conversion to FLOAT for all internal calculations in the temporary (including other conversions for plugins, etc.) and simultaneously the result in the temporary is converted to 8-bit sRGB for on-screen display. See also \nameref{sec:conform_the_project}. To review, a \textit{temporary} is a single frame of
573 video in memory where graphics processing takes place.
574 \CGG{} use X11 and X11 is RGB only and it is used to draw the \textit{refresh frame}. So single step is always drawn in RGB. Continuous playback on the other hand can also be YUV for efficiency reasons.
575 \item[Color range:] One problem with the YUV color model is the \texttt{YUV color range}. This can create a visible effect of a switch in color in the Compositor, usually shown as grayish versus over-bright. The cause of the issue is that X11 is RGB only and it is used to draw the \textit{refresh frame}. So single step is always drawn in RGB. To make a YUV frame into RGB, a color model transfer function is used. The math equations are based on Color\_space and Color\_range. In this case, color\_range is the cause of the \textit{grayish} offset. The \textit{YUV MPEG color range} (limited or TV) is 16..235 for \textbf{Y}, 16..240 for \textbf{UV}, and the color range used by \textit{YUV JPEG color range} (full or HDTV) is 0 to 255. The cause is that 16-16-16 is seen as pure black in MPEG, but as gray in JPEG and all playback will come out brighter and more grayish. This can be fixed by forcing appropriate conversions via the ColorSpace plugin. See \nameref{sec:color_space_range_playback}
576 \item[Plugins:] On the timeline all plugins see the frames only in internal pixel format and modify this as needed (\textit{temporary}). Some effects work differently depending on colorspace: sometimes pixel values are converted to float, sometimes to 8-bit for an effect. In addition \textit{playback single step} and \textit{plugins} cause the render to be in the session color model, while \textit{continuous playback} with no plugins tries to use the file’s best color model for the display (for speed). As mentioned, each plugin we add converts and uses the color information in its own way. Some limit the gamut and depth of color by clipping (i.e. \texttt{Histogram}); others convert and reconvert color spaces for their convenience; others introduce artifacts and posterization; etc. For example, the \texttt{Chroma Key (HSV)} plugin converts any signal to HSV for its operation.
577 If we want to better control and target this color management in \CGG{}, we can take advantage of its internal ffmpeg engine: there is an optional feature that can be used via \texttt{.opts} lines from the ffmpeg decoded files. This is via the \texttt{video\_filter=colormatrix=...}ffmpeg plugin. There may be other good plugins (lut3d...) that can also accomplish a desired color transform. This \texttt{.opts} feature affects the file colorspace on a file by file basis, although in principle it should be possible to setup a \texttt{histogram} plugin or any of the \texttt{F\_lut*} plugins to remap the colortable, either by table or interpolation.
578 \item[Conversion:] Any conversion is done with approximate mathematical calculations and always involves a loss of data, more or less visible, because you always have to interpolate an exact value when mapping it into the other color space. Obviously, when we use floating point numbers to represent values, these losses become small and close to negligible. So the choice comes down to either keeping the source color model even while processing or else converting to FLOAT, which in addition to leading to fewer errors should also minimize the number of conversions, being congruous with the program's internal one. The use of FLOAT, however, takes more system resources than the streamlined YUV. Color conversions are mathematical operations; for example to make a YUV frame into RGB, a color model matrix function is used. The math equations are based on color\_space and color\_range. Since the majority of sources are YUV, this conversion is very common and it is important to set these parameters to optimize playback speed and correct color representation.
579 \item[Encoding:] Finally, the encoding converts to colorspace required by the codec.
582 \subsection{Workflow}%
585 Let us give an example of color workflow in \CGG{}. We start with a source of type YUV (probably: YCbCr); this is decoded and converted to the chosen color model for the project, resulting in a \textit{temporary}. Various jobs and conversions are done in FLOAT math and the result remains in the chosen color model until further action. In addition, the temporary is always converted to sRGB 8-bit for monitor display only. If we apply the \texttt{ChromaKey (HSV)} plugin, the temporary is converted to HSV (in FLOAT math) and the result in the temporary becomes HSV. If we do other jobs the temporary is again converted to the set color model (or others if there is demand) to perform the other actions. At the end of all jobs, the obtained temporary will be the basis of the rendering that will be implemented according to the choice of codecs in the render window (\textit{Wrench}), regardless of the color model set in the project. If we have worked well the final temporary will retain as much of the source color data as possible and will be a good basis for encoding of whatever type it is.
587 For practical guidelines, one can imagine starting with a quality file, for example, \textit{10-bit YUV 4.2.2}. You set the project to \texttt{RGBA-FLOAT}; the \texttt{YUV color space} to your choice of Rec709 (for a FullHD) or BT 2020NCL (for UHD) and finally the \texttt{YUV color range} to JPEG. If the original file has the MPEG type color range then you convert to JPEG with the \texttt{ColorSpace} plugin. If you want to transcode to a quality intermediate you can use \textit{DNxHR 422}, or even \textit{444}, and maybe do the editing step with a \textit{proxy}. For rendering you choose the codec appropriate for the file destination, but you can still generate a high-quality master, for example \textit{ffv1 .mov} with lossless compression.
591 \includegraphics[width=1.0\linewidth]{color_01.png}
592 \caption{Color settings (Settings $\rightarrow$ Format / Settings $\rightarrow$ Preferences)}
projects / goodguy / cin-manual-latex.git / blob