These same steps have been verified to work for creating Dailymotion videos -- however, the created files must be renamed before uploading to change the youtube extension to webm instead for Dailymotion.
-\subsection{VP9 parameters\protect\footnote{credit Frederic Roenitz}}%
+\subsection{VP9 parameters}%
\label{sub:vp9_parameters}
\index{rendering!VP9 parameters}
-\textsc{VP9} is a video codec licensed under the BSD license and is
+\textsc{VP9}\protect\footnote{credit Frederic Roenitz} is a video codec licensed under the BSD license and is
considered open source,
% Sisvel Announces AV1 Patent Pool, March 10, 2020
% https://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=139636
\label{sec:batch_rendering}
\index{batch rendering}
-Batch Rendering automates the rendering of audio/video files in that
+Batch Rendering as implemented in \CGG{} is considered to be more of
+an advanced feature and careful usage is advised. It automates the
+rendering of audio/video files in that
you can establish a set of job parameters, save them, and use them
-repeatedly. It also allows for \CGG{} to be run by external
-programs, with no need for the user to manually interact with the
-user interface (figure~\ref{fig:batch01}).
+repeatedly (figure~\ref{fig:batch01}). It also allows for \CGG{} to
+be run by external programs, with no need for the user to manually
+interact with the user interface.
\begin{figure}[htpb] \centering
\includegraphics[width=1.0\linewidth]{batch01.png}
understand certain concepts.
\begin{enumerate}
- \item You must define a list of Batches (\textit{Job} \index{job}) before starting the rendering. This is created using the \textit{New} button and displayed in \textit{Batches to Render} dialog.
+ \item You must define a list of Batches (\textit{Job} \index{job}) before starting the rendering. This is created using the \textit{New} button and displayed in \textit{Batches to Render} dialog.
\item Each batch consists of a source project already created in \CGG{}, e.g. \texttt{aaa.xml}, to which we assign the rendering parameters.
\begin{itemize}
\item to associate \texttt{aaa.xml} to the batch we use the \textit{EDL Path} input field.
parameters for a single batch; a batch is simply a pairing of a
project file with a choice of output file and render settings.
-Set the \textit{Output path}, \textit{File format}, \textit{Audio},
+It may be advisable to start with a \textit{Delete} so you don't have any problems. Set the \textit{Output path}, \textit{File format}, \textit{Audio},
\textit{Video}, and \textit{Create new file at each label}
parameters as if you were rendering a single file. These parameters
apply to only one batch. In addition to the standard rendering
job.
\item[Elapsed:] the amount of time taken to render the batch if
finished. If field is empty, it did not run.
-\end{description} To start rendering from the first enabled batch,
+\end{description}
+
+The \texttt{File $\rightarrow$ Batch Render} pulldown brings up the
+Batch Render window to be used for batch rendering as well as DVD/BD
+creation. There are some additional buttons that can save time and
+mistakes. These are described next.
+
+\begin{description}
+ \item[Save Jobs] when you have set up the batch jobs the way you
+ want and you think you may have to run them more than once, it is
+ beneficial to save the jobs for later use so you easily run them
+ again. It is recommended to use a filename with .rc as the extension
+ so that it is obvious that it is a list of batch jobs to be run.
+ \item[Load Jobs] reload a previous set of saved jobs. This can come
+ in handy if you did not have the time to render them when you
+ originally set them up, if you need to rerun, or if you got
+ interrupted.
+\end{description}
+
+To start rendering from the first enabled batch,
hit \textit{Start}. Once rendering, the main window shows the
progress of the batch. After each batch finishes, the elapsed column
in the batch list is updated and the next batch is rendered until
You can automate \CGG{} batch renders from other programs. In the
batch render dialog, once you have created your list of batch render
jobs, you can click the button \textit{Save Jobs} and choose a file
-to save your batch render list to. Once you have created this file,
+to save your batch render list to. It is recommended that you use
+a filename with the extension of .rc in order to make it obvious that
+this is a list of batch jobs to render. Once you have created this file,
you can start up a batch render without needing to interact with the
\CGG{} user interface. From a shell prompt, from a script, or other
program, execute:
\begin{lstlisting}[style=sh]
-{path_to_cinelerra}/cin -r batchjob.xml
+{path_to_cinelerra}/cin -r batchjob.rc
\end{lstlisting} substituting your actual filename for
-\texttt{batchjob.xml}. When invoked with these parameters, \CGG{}
-will start up and perform the rendering jobs in that list, without
-creating its usual windows.
-
-\subsection{Command Line Rendering}%
-\label{sub:command_line_rendering}
-\index{rendering!command line}
-
-The command line rendering method consists of a way to load the
-current set of batch rendering jobs and process them without a
-GUI\@. This is useful if you want to do rendering on the other side
-of a low bandwidth network and you have access to a high powered
-computer located elsewhere. Setting up all the parameters for this
-operation is somewhat difficult. That is why the command line aborts
-if any output files already exist.
-
-To perform rendering from the command line, first run \CGG{} in
-graphical mode. Go to \texttt{File $\rightarrow$ Batch
- Render}. Create the batches you intend to render in the batch window
-and close the window. This saves the batches in a file. Set up the
-desired render farm attributes in \texttt{Settings $\rightarrow$
- Preferences} and quit out of \CGG{} if you want to use the Render
-Farm capability. These settings are used the next time command line
-rendering is used to process the current set of batch jobs without a
-GUI\@.
-
-On the command line run:
-
-\begin{lstlisting}[style=sh]
-cin -r
-\end{lstlisting}
+\texttt{batchjob.rc}. \textbf{Warning} this file will be modified
+so if you use any filename that is not a legitimate list of batch jobs to
+render, that file will be overwritten and its previous contents destroyed.
+When invoked with these parameters, \CGG{} will start up and run the
+rendering jobs in the list contained in that file starting at the defined
+\textit{active region}, without creating its usual windows. If you do not
+specify a filename, the default will be \$HOME/.bcast5/batchrender.rc.
+Possible messages you might see where you started up the job are as follows.
+\begin{description}
+\item[The following files exist: - filename - Won't overwrite existing files] that batch job will not run in order to prevent writing over previous run.
+\item["filename" No such file or directory] the specified batch job file does not exist.
+\item["filename": Permission denied] the specified batch job file does not have write permission so can not be updated.
+\item[Render::run: filename] the batch job with the name of filename will be processed.
+\item[** rendered 0 frames in 0.000 secs, 0.000 fps] either you used a file that is not a list of batch jobs or the batch jobs within the file were not enabled.
+\end{description}
-\subsection{More about Save/Use EDL and Save/Load Jobs}%
-\label{sub:more_save_use_edl_jobs}
+\subsection{Advanced features}%
+\label{sub:advanced_features}
\index{batch rendering!more options}
-The \texttt{File $\rightarrow$ Batch Render} pulldown brings up the
-Batch Render window to be used for batch rendering as well as DVD/BD
-creation. There are some additional buttons that can save time and
-mistakes. These are described next.
+\textbf{Warning}: \textit{Save to EDL path} overwrites the current EDL thus destroying the original contents.
+
+Although the operation of Batch Rendering in \CGG{} is similar to that of other NLEs, there is one big difference that we need to take into account. The render setup is not done on a project-by-project basis, which are then brought into the Batch window to be rendered automatically. The setup must be done in the Batch rendering window, where various projects are loaded and set up. In the case of similar projects, derived from a single EDL with some variation, this mode offers the possibility of altering the projects without having to open each individual project, make the changes, set up the rendering, save and import into the Batch window. The procedure is to select the batch we want to modify in the Batches to render window; operate on the currently open timeline (even if it does not correspond to the one we want to modify) making the desired changes and then press the \textit{Save to EDL path} button. Thus the chosen batch, while retaining its original name, will now contain the modified project. Since this possibility destroys the original EDL overwriting it with the modified one, you must be very careful. This procedure is convenient in case the batches are similar, i.e. they are variations of the same EDL, where we want to experiment with other effects, other output formats or when trying out various cuts of a DVD/BD before the final production. It might also be useful to use an \textit{active region} of the timeline, so as to speed up rendering times but still have an indicative result for comparison. Instead operating on different projects, we can do a \textit{save as...} of the project on the timeline to have a new EDL with a new name and then replace it with the batch selected in the joblist using the \textit{Use Current EDL} button. The new project (with its name) overwrites the original project.
The \textit{Save to EDL Path} and \textit{Use Current EDL} buttons
can be valuable tools for advanced usage or for developers doing
testing. Description of how you can expect them to work will help
-to illustrate how to take advantage of their capabilities.
+to illustrate how to take advantage of their capabilities (figure~\ref{fig:batch-advanced}):
+
+\begin{figure}[htpb] \centering
+ \includegraphics[width=0.7\linewidth]{batch-advanced.png}
+ \caption{New Buttons with Unsafe GUI in batchrender}
+ \label{fig:batch-advanced}
+\end{figure}
+
\begin{description}
-\item[Save to EDL Path] if you have made a change to the EDL, use
+\item[Save to EDL Path] Warning: this function overwrites the contents of the original EDL with new data, keeping the name of the original. If we don't know exactly what we're doing we may lose the original project. If you have made a change to the EDL, use
this button to save the changes so that they will be used in the
render operation. Although you can get the same results by using
\texttt{File $\rightarrow$ Save\dots}, this capability was initially
However, it is important to note that the result will be saved with
the name \texttt{original\_name} – that is, the new content from
\texttt{new.xml} but with the old name of
- \texttt{original\_name.xml}.
-\item[Use Current EDL] if you are working on media and still testing
+ \texttt{original\_name.xml}. To have this functionality we have to enable the checkbox in \texttt{Settings $\rightarrow$ Preferences $\rightarrow$ Appearance} tab; section \textit{Dangerous:} and unchecked (default) \textit{Unsafe GUI in batchrender}.
+\item[Use Current EDL] Warning: this function overwrites the contents of the original EDL with new project. If we don't know exactly what we're doing we may lose the original project. if you are working on media and still testing
out the results, you can take advantage of this click-box to quickly
get results. Basically, you change the media, save that change with
another name (in order to preserve the original name in case you
listbox will be automatically updated to the \texttt{new\_name.xml}
and the current existing highlighted job will be replaced with the
\texttt{new\_name.xml} in the EDL column.
-\item[Save Jobs] when you have set up the batch jobs the way you
- want and you think you may have to run them more than once, it is
- beneficial to save the jobs for later use so you easily run them
- again.
-\item[Load Jobs] reload a previous set of saved jobs. This can come
- in handy if you did not have the time to render them when you
- originally set them up, if you need to rerun, or if you got
- interrupted.
-\item[Warn if Jobs/Session mismatched] After you set up your render
+\item[Warn if Jobs/Session mismatched] Warning: It is better to keep this function unchecked because it is only needed in case of changes on the original EDL. By default it is hidden and is shown only if we enable the checkbox in \texttt{Settings $\rightarrow$ Preferences $\rightarrow$ Appearance} tab; section \textit{Dangerous:} and checked \textit{Unsafe GUI in batchrender}. After you set up your render
and press Start, the program checks to see if the current EDL
session matches your Batch Render job. If the EDL has been changed
since the batch job was created, it warns you so that you have the
Otherwise, you can dismiss that warning box, disable the warning
message by unchecking the box and use the original values. If you
never want to be warned about the mismatches, leave the box
- unchecked (figure~\ref{fig:batch02}).
+ unchecked (figure~\ref{fig:batch02}). It is advisable to keep it unchecked because it can cause problems.
\end{description}
\begin{figure}[htpb] \centering
- \includegraphics[width=1.0\linewidth]{batch02.png}
- \caption{Batch render with the 4 ghosted buttons on the right side
- + the Warning message below}
- \label{fig:batch02}
+ \includegraphics[width=0.9\linewidth]{batch02.png}
+ \caption{Batch render with the 4 ghosted buttons on the right side
+ + the Warning message below}
+ \label{fig:batch02}
\end{figure}
+A very clear tutorial on these features can be found \href{https://linuxvideoediting.blogspot.com/2021/01/save-edl-path-use-current-edl-in-cinelerra-gg.html}{here}\protect\footnote{credit Igor Vladimirsky}; in Russian but easily translatable with DeepL or similar.
+
+\subsection{Command Line Rendering}%
+\label{sub:command_line_rendering}
+\index{rendering!command line}
+
+The command line rendering method consists of a way to load the
+current set of batch rendering jobs and process them without a
+GUI\@. This is useful if you want to do rendering on the other side
+of a low bandwidth network and you have access to a high powered
+computer located elsewhere. Setting up all the parameters for this
+operation is somewhat difficult. That is why the command line aborts
+if any output files already exist.
+
+To perform rendering from the command line, first run \CGG{} in
+graphical mode. Go to \texttt{File $\rightarrow$ Batch Render}.
+Create the batches you intend to render in the batch window and
+close the window. This automatically saves the batches in a file
+with the name of \$HOME/.bcast5/batchrender.rc. Set up the
+desired render farm attributes in \texttt{Settings $\rightarrow$
+ Preferences} and quit out of \CGG{} if you want to use the Render
+Farm capability. These settings are used the next time command line
+rendering is used to process the current set of batch jobs without a
+GUI\@. It is important to remember that the rendering will begin at
+the defined \textit{active region} saved when the project was saved.
+
+On the command line run:
+
+\begin{lstlisting}[style=sh]
+cin -r
+\end{lstlisting}
+
\section{Background Rendering}%
\label{sec:background_rendering}
\index{background rendering}
path. Since hundreds of thousands of image files are usually
created, ls commands will not work in the background rendering
directory. The browse button for this option normally will not work
- either, but the configuration button for this option works.
+ either, but the configuration button for this option works. The
+ default value will be /tmp/brender .
\item[File format] The file format for background rendering has to
be a sequence of images. The format of the image sequences
determines the quality and speed of playback. JPEG generally works
- well.
+ well and is the default.
\end{description}
+Tip: If you have rendered your whole project with \textit{File format}
+set to JPEG and there are no missing numbers in the sequence, you can
+create a video from that sequence outside of \CGG{}.
+For example, if using the default output so that your files are named
+/tmp/brender000000, /tmp/brender000001, ... in a window, you would type:
+
+\begin{lstlisting}[style=sh]
+ffmpeg -f image2 -i /tmp/brender0%5d -c:v copy brender.mov
+\end{lstlisting}
+which would create the video file brender.mov - be sure to delete
+existing brender files before creating a new sequence to ensure there
+are no missing numerical values in the sequence.
\section{Render Farm Usage}%
\label{sec:render_farm_usage}
video is rendered in the background, to speed up rendering
significantly. Because rendering is memory and cpu intensive, using
multiple computers on a network via a render farm is a significant
-gain. With \CGG{} installed on all nodes, the master node and the
-clients communicate via a network port that you specify.
-
+gain. With \CGG{} installed on all nodes, the master node and the clients communicate via a network port that you specify.
+The Master node is the one of the instance of \CGG{} that we normally start with its gui; the other nodes are the instances of \CGG{} that we decide to start in parallel from the terminal to create the Render farm (clients).
\CGG{} can distribute the rendering tasks over the network to the
-other computers of the Render Farm. The render farm software tries
+other computers of the Render Farm; or among all threads of a multicore CPU. The render farm software tries
to process all of the rendering in parallel so that several
computers can be used to render the results. The \textit{Total jobs
- to create} in the setup or labels on the timeline are used to divide
+to create} in the setup or labels on the timeline are used to divide
a render job into that specified number of tasks. Each background
job is assigned a timeline segment to process and the jobs are sent
to the various computer nodes depending upon the load balance. The
The following steps are just a guideline to start your render farm.
It is assumed that you already have the master and client nodes
communication, shared filesystem, permissions and usernames synched.
+Let's take the example of a network with 2 PCs: the master and the client. On the client we set 5 tasks; on the master we set 2 tasks.
\begin{enumerate}
\item On the master computer, use \texttt{Settings} $\rightarrow$
\item in the \textit{Hostname} box, keyin your hostname or ip
address such as 192.168.1.12 or \textit{localhost};
\item enter in a port number such as 401--405 (only a root user
- can use privileged ports) or $1025$ and click on \textit{Add Nodes};
+ can use privileged ports) or $10650...$ for non-root and click on \textit{Add Nodes}. To find a range of free ports to use you can look at the file \texttt{/etc/services};
\item you will see something like the following in the Nodes
listbox to the right:\newline
\begin{tabular}{lllc} On & Hostname & Port & Framerate
\\\midrule
- X & 192.168.1.12 & 401 & 0.0 \\
- X & 192.168.1.12 & 402 & 0.0 \\
- X & 192.168.1.12 & 403 & 0.0 \\
- X & 192.168.1.12 & 404 & 0.0 \\
- X & 192.168.1.12 & 405 & 0.0 \\
- X & localhost & 406 & 0.0 \\
- X & localhost & 407 & 0.0 \\
+ X & 192.168.1.12 & 10650 & 0.0 \\
+ X & 192.168.1.12 & 10651 & 0.0 \\
+ X & 192.168.1.12 & 10652 & 0.0 \\
+ X & 192.168.1.12 & 10653 & 0.0 \\
+ X & 192.168.1.12 & 10654 & 0.0 \\
+ X & localhost & 10655 & 0.0 \\
+ X & localhost & 10656 & 0.0 \\
\end{tabular}
- \item set the Total number of jobs to create;
+ \item set the Total number of jobs to create. This number only pertains to client nodes, so we do not need to consider the master node;
\item click OK on the bottom of the Preferences window.
\end{itemize}
-\item Now we must join the nodes created to instances of \CGG{}. On the client computers ($192.168.1.12$), on the terminal, start 5 background \CGG{} tasks via:
+\item For external network nodes, now we must join the nodes created to instances of \CGG{}. On the client computers ($192.168.1.12$), on the terminal, start 5 background \CGG{} tasks via:
\begin{lstlisting}[style=sh]
cd {path_to_cinelerra}
-cin -d 401 cin -d 402
+cin -d 10650 cin -d 10651
...
-cin -d 405
+cin -d 10654
\end{lstlisting}
-\item Similarly, on the terminal, we must join the local nodes created to instances of \CGG{}. On the master node (localhost), start the 2 background \CGG{} tasks via:
+In practice, at each instance that we start, the cursor will be positioned in a new line ready to enter the next command, without exiting the task. If we have several PCs on the network, repeat these steps for each new client (with its own ip address).
+\item Similarly, on the terminal, we must join the local nodes (internal to the Master node) created to instances of \CGG{}. On the Master node (localhost), start the 2 background \CGG{} tasks via:
\begin{lstlisting}[style=sh]
cd {path_to_cinelerra}
-cin -d 406
-cin -d 407
+cin -d 10655
+cin -d 10656
\end{lstlisting}
+Similar to the previous point, the cursor positions itself in a new line ready to enter the next command, without exiting the task.
\item When your video is ready, setup a render job via \texttt{File
$\rightarrow$ Render} or \texttt{File $\rightarrow$ Batch Render}
and check OK.
Scripts icon.
\item If you plan on doing more rendering, you can just leave the
master/client jobs running to use again and avoid having to restart
- them. Or you can kill them when you no longer are using them.
+ them. You can also close the terminal, but the jobs will remain active until you turn off the PC. Or you can kill them when you no longer are using them.
\end{enumerate}
\subsection{Render Farm Menu and Parameter Description}%
existing node or enter a new node.
\item[Port] keyin the port number of an existing or new node here.
You can also type in a range of port numbers using a hyphen, for
- example $1501-1505$ when you need to add many.
+ example $10650-10799$ when you need to add many.
\item[Apply Changes] this will allow you to edit an existing node
and to then commit the changes to hostname and port. The changes
will not be committed if you do not click the OK button.
\end{lstlisting} As it completes its jobs, you will should see:
\begin{lstlisting}[style=sh]
RenderFarmClientThread::run: Session finished
-\end{lstlisting} A quick way to start a sequence of clients is to
- use:
-\begin{lstlisting}[style=sh,mathescape]
-for n in `seq 1501 1505`; do
- cin -d $\$$n
-done
\end{lstlisting}
\item[Render Using Render Farm] After you have followed the
preceding steps, you are ready to use the render farm. Click on
\item[Assemble the Output Files] Once all of the computer jobs are
complete, you can put the output files together by using the shell
script, \textit{RenderMux} (from the menubar \textit{scripts} button
- just above FF), if the files were rendered using ffmpeg, or you can
- load these by creating a new track and specifying concatenate to
+ just above FF), if the files were rendered using ffmpeg (see \nameref{sec:menu_bar_shell_commands}).
+
+ If you want to remain within the open project in \CGG{}, you can load these files by creating a new track and specifying concatenate to
existing tracks in the load dialog in the correct numerical order.
File types which support direct copy can be concatenated into a
single file by rendering to the same file format with render farm
disabled as long as the track dimensions, output dimensions, and
asset dimensions are equal.
+
+ Finally if you want to use ffmpeg from terminal externally to \CGG{} you can go to the directory where the rendered files are, it creates a text file \texttt{list.txt} containing the list of all our files:
+ \begin{lstlisting}[style=sh]
+ file 'name.webm001'
+ file 'name.webm002'
+ ...
+ file 'name.webm00n'
+ \end{lstlisting}
+ and finally give the command
+ \begin{lstlisting}[style=sh]
+ ffmpeg -f concat -i list.txt -c copy output.webm
+ \end{lstlisting}
\end{description}
\subsection{Quick and Easy Render Farm Setup – The Buddy System
add the plugins, such as a Title, etc.
\item Check for a set of unused ports in \texttt{/etc/services}
file, if username is root usually $401-425$ are available; if
- non-root, then $1024-1079$.
+ non-root, then $10650-10799$.
\item On the master computer, in \texttt{Settings $\rightarrow$
Preferences, Performance} tab:
\begin{itemize}
/{cinelerra_pathname}/cin -d port#
\end{lstlisting}
\item On the master, bring up the render menu and name the output
- files, for example \texttt{/tmp/myoutput.mp4}.
+ files, for example \texttt{/tmp/myoutput.webm}.
\item The client nodes output results will be on their local
\texttt{/tmp} filesystems so you will have to again use
\textit{rsh/ftp} or \textit{usb sneaker net} to move them over to
the main computer. File names will be the render job output file
name with port number tacked on
- (e.g. \texttt{/tmp/hb.mp4001...mp4005}).
+ (e.g. \texttt{/tmp/hb.webm001...webm005}).
\item Load the files by concatenate to existing track on the master
node or use RenderMux shell script.
\end{enumerate}
If you are lucky enough to have a computer with a large cpu core
count, setting up a render farm can really take advantage of using
-all of the cpus. This is much faster than the default automatic
+all of the cores at 100\%. This is much faster than the default automatic
threading capability. Since you don’t need to communicate with other
computers, you will not have to be concerned about TCP communication
-or shared disks/files; only localhost nodes. On the terminal, we will open many instances of \CGG{} by connecting them to the jobs created. The number of such jobs can be the total number of CPU threads or not. When you are going to be doing other work
+or shared disks/files; only localhost nodes. On the terminal we will open many instances of \CGG{} by connecting them to the jobs created. The number of such jobs can be the total number of CPU threads (-1) or not. When you are going to be doing other work
simultaneously while rendering a large job, you will want to leave
some of the cpus available for that. Be sure to set \textit{Project SMP
cpus} in the \texttt{Settings $\rightarrow$ Preferences, Performance} tab to your CPU
section of the video so keep that in mind when designating port
numbers.
\item If not running as root, a port number in the higher range of
- $1024$ and above must be used instead of the $400+$ range.
+ $10650$ and above must be used instead of the $401+$ range.
\item The master and client jobs on the ports do not go away so if
you want to stop them, you will have to kill them via: \texttt{kill
PID\#}.
time. You can adjust the timer in \texttt{Settings $\rightarrow$
Preferences, Performance} tab.
\item When you get the message \texttt{RenderFarmClient::main\_loop:
- bind port 400: Address already in use}, use a different port.
+ bind port 400: Address already in use}, use a different port. See \texttt{/etc/services} for free ports.
\item A message of \texttt{RenderFarmServerThread::open\_client:
unknown host abcompany} means that the hostname of abcompany is not
in \texttt{/etc/hosts} so you will have to add it or use the ip
projects / goodguy / cin-manual-latex.git / blobdiff