\chapter{Overlays}%
\label{cha:overlays}
+\index{overlays}
-The purpose of the Overlay Modes is to control the foreground and background stacking and use blending to reshape image object boundaries. It normally makes use of a binary type alpha blending system for all in or all out. To use the available operations in Cinelerra GG, follow these steps:
+The purpose of the Overlay Modes is to control the foreground and background stacking and use blending to reshape image object boundaries. It normally makes use of a binary type alpha blending system for all in or all out. To use the available operations in \CGG{}, follow these steps:
\begin{enumerate}
\item In the main window, look at the Patchbay on the far left.
\begin{figure}[htpb]
\centering
- \includegraphics[width=0.99\linewidth]{images/overlay-01.png}
+ \includegraphics[width=0.99\linewidth]{overlay-01.png}
\caption{Patchbay pulldown with Porter Duff and Graphic Art overlays expanded}
\label{fig:overlay-01}
\end{figure}
Conceptually, when the foreground color is completely opaque, the resulting blended color will be the foreground color. If it is transparent, the blended color will be the color of the background. When the value of the alpha channel is $1$, the image is all there, if it is $0$, there is no image at all, otherwise it is only partially there. In other words, the alpha value goes from $0$ to $1$, where full transparency is $0$ and opaque is represented by $1$. Alpha blending models opacity.
-When blending source and destination shapes (Dst and Src), the shape boundaries can be changed with the alpha blending effects. There are a total of 10 standard Porter-Duff operators, but there are 30 possible overlay modes used in Cinelerra-GG. Each is characterized by its value in the four regions: source, destination and both, with the \textit{neither} region always being blank. The source and destination regions can either be blank or filled with the source or destination colors. A specific compositing math formula is used to calculate effect. This is only applicable to RGB; some effort has been made to accommodate YUV, but the effects are not as predictable, and may not be useful.
+When blending source and destination shapes (Dst and Src), the shape boundaries can be changed with the alpha blending effects. There are a total of 10 standard Porter-Duff operators, but there are 30 possible overlay modes used in \CGG{}. Each is characterized by its value in the four regions: source, destination and both, with the \textit{neither} region always being blank. The source and destination regions can either be blank or filled with the source or destination colors. A specific compositing math formula is used to calculate effect. This is only applicable to RGB; some effort has been made to accommodate YUV, but the effects are not as predictable, and may not be useful.
-Below, in figure~\ref{fig:normal}, are the results of utilizing the 30 available operations within Cinelerra as listed on a following page. Src is the solid green rectangle and Dst is the solid red rectangle. There are better illustrations of what alpha blending can do, however for consistency sake, these are the results when using standards.
+Below, in figure~\ref{fig:normal}, are the results of utilizing the 30 available operations within \CGG{} as listed on a following page. Src is the solid green rectangle and Dst is the solid red rectangle. There are better illustrations of what alpha blending can do, however for consistency sake, these are the results when using standards.
+For an alternative set of source video illustrations, see \nameref{sec:altviews}.
\begin{figure}[htpb]
\centering
- \includegraphics[width=0.84\linewidth]{images/normal.png}
+ \includegraphics[width=0.84\linewidth]{normal.png}
\caption{Normal and Arithmetic overlays}
\label{fig:normal}
\end{figure}
\begin{figure}[htpb]
\centering
- \includegraphics[width=0.84\linewidth]{images/porter-duff.png}
+ \includegraphics[width=0.84\linewidth]{porter-duff.png}
\caption{Porter Duff overlays}
\end{figure}
\begin{figure}[htpb]
\centering
- \includegraphics[width=0.84\linewidth]{images/logical.png}
+ \includegraphics[width=0.84\linewidth]{logical.png}
\caption{Logical overlays}
\end{figure}
\begin{figure}[htpb]
\centering
- \includegraphics[width=0.84\linewidth]{images/graphic-art.png}
+ \includegraphics[width=0.84\linewidth]{graphic-art.png}
\caption{Graphic Art overlays}
\end{figure}
The implementation math forms are subsequently listed, where:
\vspace{2ex}
-\begin{lstlisting}[language=bash, numbers=none]
+\begin{lstlisting}[style=sh]
Legend:
D = Destination
S = Source
a = alpha
c = chroma (color)
-|| = OR (logical operator);
+|| = OR (logical operator);
? : = if (true/false) ... then (conditional ternary operator)
\end{lstlisting}
The previous math forms are the only truly accurate description of each blending operation, but short descriptions are below where \textit{Source} is the output from the next track and \textit{Destination} is the output from the lower track stacking. Blending starts with new Source and combines it with the previous render stack output, which is referred to as Destination. The new output becomes the next Destination and the next up stack level becomes the new Source. Source is above; Destination is below.
-Note: the Graphic Art group operates principally on color, and the others operate principally on alpha.
+Some notable items that affect the results are listed here.
+\begin{enumerate}
+ \item So as not to be surprised by the results, set your Video Driver to X11, not X11-OpenGL, because when rendering your video no graphics library is used. Rendering is handled in software and there is sometimes anomaly differences when using the OpenGL driver.
+ \item The Graphic Art group operates principally on color, and the others operate principally on alpha.
+ \item When using the Title plugin with text, blending or changes in the text will only take affect in Subtract, Divide, and Difference. This matches the implmentation of the original Cinelerra version.
+ \item If comparing results with the original Cinelerra version for Subtract and Divide, you may have to switch the order of the source and destination tracks to see the same output. If you do not want to switch tracks, you can use the \textit{overlay} plugin as shared and change the Layer order and/or Output layer. The reason for having to switch tracks to get equivalent output is because of differences in the math formulas used. Subtract exactly matches the original Cinelerra version if you switch the 2 tracks; Divide matches in one order only.
+ \item Instead of using the patchbay overlays, you may prefer to use the \textit{overlay} plugin instead.
+ \item If you see undesirable results, always check under Settings->Format and check your Color Model as that will vary or interfere with the look.
+\end{enumerate}
-\subsection*{Normal}%
+% Leave the word Group in due to latex2html dual name conflict
+\subsection*{Normal Group}%
\label{sub:normal2}
+\index{overlays!normal}
\begin{description}
\item[Normal:] Normal mode is the default layer mode. The result color is the source color. The layer on top covers the layers below it. If you want to see anything below the top layer when you use this mode, the layer must have some transparent areas. It is \textit{stacked on top}. Math formula used is different than that used by Gimp; there is no SVG equivalent.
\subsection*{Arithmetic Group:}%
\label{sub:arithmetic_group}
+\index{overlays!arithmetic}
Standard numerical operations.
\begin{description}
- \item[Addition:] The source is added to the destination and replaces the destination. Addition mode is very simple - the pixel values of the upper and lower layers are added to each other. The resulting image is normally brighter. The equation can result in color values greater than $255$, so some of the light colors may be clipped to the maximum value of $255$. Math formula is the same as that used by SVG but different than Gimp.
- \item[Subtract:] Subtract mode reduces the pixel values of the upper layer by the pixel values of the lower layer. The resulting image is normally darker. You might get a lot of black or near-black in the resulting image. The equation can result in negative color values, so some of the dark colors may be clipped to the minimum value of $0$. Math formula used is different than that used by Gimp; there is no SVG equivalent.
+ \item[Addition:] The source is added to the destination and replaces the destination. Addition mode is very simple - the pixel values of the upper and lower layers are added to each other. The resulting image is normally brighter. The equation can result in color values greater than $255$, so some of the light colors may be clipped to the maximum value of $255$. Math formula is the same as that used by SVG but slightly different than Gimp.
+ \item[Subtract:] Subtract mode reduces the pixel values of the upper layer by the pixel values of the lower layer. The resulting image is normally darker. You might get a lot of black or near-black in the resulting image. The equation can result in negative color values, so some of the dark colors may be clipped to the minimum value of $0$. Math formula used is different than that used by the Gimp "Default" version, but you can get the same as the "Legacy" version if you switch the 2 tracks or use the \textit{overlay} plugin and switch to Top first. There is no SVG equivalent.
\item[Multiply:] The source color is multiplied by the destination color and replaces the destination. The resulting color is always at least as dark as either the source or destination color. Multiplying any color with black results in black. Multiplying any color with white preserves the original color. Math formula is the same as used by SVG and Gimp.
- \item[Divide:] Divides source color by destination color. If the source color is white, the result color is the underlying color. The resulting image is often lighter. Math formula used is different than that used by Gimp; there is no SVG equivalent.
- \item[Replace:] Replace mode will cause any existing destination to be replaced by the source media. Mathematical formula used is the same as used by Gimp; there is no SVG equivalent.
+ \item[Divide:] Divides source color by destination color. If the source color is white, the result color is the underlying color. The resulting image is often lighter. Math forumula is different than that of Gimp, but by changing the order of the Source and Destination tracks you can get the same results in some cases (this is true for the original Cinelerra version also). There is no SVG equivalent.
+ \item[Replace:] Replace mode will cause any existing destination to be replaced by the source media. Math formula used is the same as used by Gimp; there is no SVG equivalent.
\end{description}
\subsection*{Porter-Duff Group}%
\label{sub:porter-duff_group}
+\index{overlays!Porter-Duff}
Industry standard compositing operators.
\subsection*{Logical Group}%
\label{sub:logical_group}
+\index{overlays!logical}
\begin{description}
\item[Min:] The output color is the component-wise minimum value of the source and destination colors. There is no SVG or Gimp equivalent math formula.
\subsection*{Graphical Art Group}%
\label{sub:graphical_art_group}
+\index{overlays!graphical art}
Typical operations from popular \textit{paint} packages.
\item[Softlight:] Darkens or lightens the colors, dependent on the source color value. If the source color is lighter than 0.5, the destination is lightened. If the source color is darker than $0.5$, the destination is darkened, as if it were burned in. The degree of darkening or lightening is proportional to the difference between the source color and $0.5$. If it is equal to $0.5$, the destination is unchanged. Using pure black or white produces a distinctly darker or lighter area, but does not result in pure black or white. The effect is similar to shining a diffused spotlight on the destination. A layer with pure black or white becomes markedly darker or lighter, but does not become pure black or white. Soft light is not related to “Hard light” in anything but the name, but it does tend to make the edges softer and the colors not so bright. Math formula is the same as used by Gimp; SVG formula differs.
\end{description}
+\section{Alternative Views of Overlay Modes}%
+\label{sec:altviews}
+
+An alternative set of source videos used to provide a different real world view is shown here to help illustrate alpha blending. They use a landscape photo as one and the other is half of a color and half of a black and white gradient photo. First is shown the Aritmetic, Logical, and Graphic-Art views.
+After that the Porter Duff overlays are shown using the same source videos but
+with each of the Source videos offset in a different direction so that the
+generated output is more obvious.
+
+\begin{figure}[H]
+ \centering
+ \includegraphics[width=1.0\linewidth]{normal2.png}
+ \caption{Normal and Arithmetic overlays}
+\end{figure}
+
+\begin{figure}[H]
+ \centering
+ \includegraphics[width=1.0\linewidth]{logical2.png}
+ \caption{Logical overlays}
+\end{figure}
+
+\begin{figure}[H]
+ \centering
+ \includegraphics[width=1.0\linewidth]{graphic-art2.png}
+ \caption{Graphic Art overlays}
+\end{figure}
+
+\begin{figure}[H]
+ \centering
+ \includegraphics[width=1.0\linewidth]{porter-duff2.png}
+ \caption{Offset source videos to illustrate Porter Duff overlays}
+\end{figure}
+
+\begin{figure}[H]
+ \centering
+ \includegraphics[width=1.0\linewidth]{porter-duff3.png}
+ \caption{Porter Duff overlays}
+\end{figure}
+
+\section{Problems with transparencies}%
+\label{sec:transparencies}
+
+Often in applying overlays, or some other type of blend, we get unexpected results that can look wrong. Sometimes the cause may be the different formula used by CGG compared to those in Gimp or other software. Other times they are a result of setup or order as listed next.
+Keep in mind that blends can be applied not only from the Patchbay but also as an Overlay plugin (see \nameref{sub:overlay}) and the \textit{Blend Algebra} plugin see \nameref{sub:blend_algebra}.
+Here are some factors that can lead to unexpected results:
+
+\begin{enumerate}
+ \item The colors of the 2 tracks that will be blended; we often consider only these channels to guess the final result.
+ \item The Alpha channel of the 2 tracks that can interact with each other in ways that are intuitively (but not mathematically) unexpected.
+ \item The choice of Source track and Destination track (\textit{top} or \textit{bottom}).
+ \item Overlays effect generally depends on the \textit{colorspace} of the project - YUV can differ from RGB. With the \textit{Blend Algebra} plugin this dependency is addressed; the necessary colorspace conversion can be done automatically.
+ \item Overlays can depend on implicit clipping. In \texttt{RGBA-FLOAT} no clipping is done; in \texttt{RGBA8888} clipping to 8-bit unsigned cannot be avoided per definition. The \textit{Blend Algebra} plugin addresses this dependency also, clipping for float colorspace can be switched on explicitly, and then the results for float and RGBA8888 should be identical.
+ \item Unclipped overlays can be displayed differently depending on the graphic driver (\textit{X11}, \textit{X11-Xv}, \textit{X11-OpenGL}, etc.). The OpenGL driver could be more robust in these cases; it runs perhaps under control of some {\CGG} shader and seems to clip the bounds correctly. However, all rendering is done in software without the use of OpenGL so be sure to check results without OpenGL.
+ \item The presence of a third black background track, (or even the color of the \texttt{Composer Background Color}, which in {\CGG} is black by default but whose color can be varied manually) which can interfere with viewing the blend in the Compositor and show unexpected results (see \href{https://www.cinelerra-gg.org/bugtracker/view.php?id=559}{MantisBT \#559}).
+ \item If the footage material in a track has transparency by itself, it might be modified by overlay formula and develop some color hidden by transparency before overlay was applied.
+ \item Overlays can be applied at two places: via the \textit{Overlay} plugin or the \textit{Blend Algebra} plugin
+ in which sequence in the video processing pipeline plugins are processed, is defined; and in the \texttt{Patchbay} in which sequence in the video processing pipeline plugins are processed, is not defined.
+ \item There may be \textit{masks}. The mask can be applied either after plugins, or before them. Results of the \textit{Overlay} plugin depend on it.
+ \item There is \texttt{fader} control. Probably, fader is applied after all plugins and overlays, but this order has not been verified.
+ \item There are two buttons in the patchbay, \texttt{Play track} (can be switched off) and \texttt{Don't send to output} (can be switched on). In both cases video from the affected track disappears, but the background left is different in both cases.
+ \item Effects from some other plugins can have transparent parts (\textit{Titler}, for example).
+\end{enumerate}
+
+Ultimately, if we get a result that looks wrong to us, we need to look at the formula for that blend and calculate our mathematical result. Then compare our calculation with the result we see on the screen.
+In principle, you could attach \textit{Blend Algebra} in addition to \textit{Overlay} plugin, switch one of them on, another off, change the formula in \textit{Blend Algebra} and see how the modifications affect the result, and even print and analyze the intermediate color values. This can be most helpful if you use some specially drawn regular color patterns instead of your working footage.
+Another possibility is to attach the \textit{Blend Program} plugin which modifies any given tracks directly.
+
+%%% Local Variables:
+%%% mode: latex
+%%% TeX-master: "../CinelerraGG_Manual"
+%%% End:
projects / goodguy / cin-manual-latex.git / blobdiff