\section{How Builds Really Work and Even More Options}
\label{sec:builds_really_work_more_options}
+\index{build!more options}
This section describes how builds really work if you want to know more about making changes or understanding the process; probably only for a developer or system administrator.
\section{Configuration Features}
\label{sec:configuration_features}
+\index{build!configuration}
A listing of the current configuration features as of January 11, 2020:
\section{Thirdparty Parallel Build}
\label{sec:thirdparty_parallel_build}
+\index{build!thirdparty}
The Makefile in the thirdparty build directory employs a set of macros used to create a build rule set of thirdparty library build dependencies. The standard build sequence of [source, config, build] is used to prepare thirdparty products as static libraries. Build package dependency can be specified in the Makefile std-build macro call. These Makefile macro calls define the rules used for each thirdparty build. The expanded rule definitions may be viewed by using:
24 cpus & = & 2 mins\\
\end{tabular}
+\section{Using the very latest Libraries}
+\label{sec:latest_libraries}
+\index{build!use latest library}
+
+Using the most current libraries can be a challenge for some of the Operating System distros that use
+stable compilers, assemblers, and their own libraries. Because they are stable, they frequently do
+not keep up with the very latest of thirdparty libraries. Consequently, some program constructs may
+not yet be implemented. \CGG{} tries to maintain stability since it is better to have less features
+and no crashes. The goal is to make \CGG{} widely available on many platforms rather than
+dependent on advanced tools that are not supported on some distros.
+
+\CGG{} attempts to upgrade to the latest releases of many thirdparty libraries about every 3-4
+months. But it is often difficult to keep some of these thirdparty libraries up to date as their
+developers switch from tried and true standard tools to newer, less standard tools. As a result
+in order to build \CGG{} on 2-3 versions of any distro rather than only the most current version,
+some thirdparty libraries can not be kept up to date and may even be to the point of no further
+updates. In a lot of cases, the updated releases provide little new capabilities but rather
+are bug fixes that may not even be relevant to \CGG{}'s use.
+
+So as a computer savvy user or a developer, if you would like to build \CGG{} with the latest
+thirdparty releases for some of the packages here are a few suggestions based on other
+developer's feedback and experimentation.
+
+\textbf{dav1d}
+\begin{description}[noitemsep]
+ \item Status - currently \CGG{} is staying at 0.5. This is disappointing because there
+may be speed gains in version 0.6 that would be beneficial. However, it is usable for decoding
+whereas libaom is a lot slower. Unfortunately, it has no effective encoder.
+ \item Problem - 0.6 dav1d requires NASM 2.14 and uses instructions like vgf2p8affineqb,
+not exactly an "add" instruction. It also uses meson which is not widely available on all
+distros. The only distros that are built for \CGG{} that are at 2.14 are the latest version
+of Arch, Debian(10), Gentoo, Tumbleweed, and Fedora(31). The rest are at 2.12 and 2.13 including
+the most widely used Ubuntu. The NASM requirement apparently provides for using AVX-512
+instructions (like vgf2p8affineqb, which is more like a whole routine than a simple instruction).
+ \item Workaround already in use by \CGG{} - a Makefile was generated to replace Meson usage
+but has to be continuously updated for new releases. Dav1d 0.5 requires NASM 2.13 so at this level
+the newer distros mostly work. The availability of meson and nasm are a significant problem on
+many systems which are still in wide use.
+ \item Your workaround - Because a request to dav1d developers to consider changes to
+ensure their library is more widely usable does not appear to be in their future, since it works
+for them, you can upgrade NASM to 2.14 to stay up to date. Of course, install meson also.
+\end{description}
+
+\textbf{OpenExr}
+\begin{description}[noitemsep]
+ \item Status - currently at latest version
+ \item Problem - the OpenExr tarball is not a single package but is 2 packages instead
+ \item Workaround already in use by \CGG{} - reworked the packages so that it looks like
+one package with 2 stubs
+ \item Your workaround - perhaps use the same workaround
+\end{description}
+
+\textbf{OpenCV}
+\begin{description}[noitemsep]
+ \item Status - 2 different versions specific for O/S but none for Ubuntu 14, 32 or 64 bit
+ \item Problem - There are really 2 problems here. The first is OpenCV is not really
+"Open" in that Surf is patented/non-free and there is no actual source available for certain
+capabilities. The second is that cmake 3.5.1 is required for OpenCV 4.2.
+ \item Workaround already in use by \CGG{} - using 3.4.1 for older distros and 4.2 for newer
+ \item Your workaround - upgrade cmake to 3.5.1 for upgrade to 4.2; add non-free to the
+compile; and use binaries that you do not know what they contain since no source code to compile.
+Look into opencv4/opencv2/core/types.hpp:711;27
+\end{description}
+
+\textbf{webp}
+\begin{description}[noitemsep]
+ \item Status - currently at version 1.1.0
+ \item Problem - requires cmake 3.5
+ \item Workaround already in use by \CGG{} - leaving out of Ubuntu14, Ubuntu, Centos7
+ \item Your workaround - upgrade on those systems to cmake 3.5
+\end{description}
+
+\textbf{libaom}
+\begin{description}[noitemsep]
+ \item Status - currently at version 3.0.0 for older O/S and 3.1.1 for newer O/S
+ \item Problem - requires cmake 3.5 at 3.0.0 and 3.6 for 3.1.1
+ \item Workaround already in use by \CGG{} - leaving out of Ubuntu14, Ubuntu, Centos7
+ \item Your workaround - upgrade on those systems to cmake 3.6
+\end{description}
+
+\textbf{x10tv}
+\begin{description}[noitemsep]
+ \item Status - this is the x10 TV remote control
+ \item Problem - INPUT\_PROP\_POINTING\_STICK not defined error on older distros
+ \item Workaround already in use by \CGG{} - leaving out of Ubuntu14, Ubuntu, Centos7
+ \item Your workaround - look into /usr/include/linux/input-event-codes.h
+\end{description}
+
+\textbf{libvpx}
+\begin{description}[noitemsep]
+ \item Status - currently at version 1.8.1
+ \item Problem - when decoding a test file, it failed to correctly load to the timeline
+ \item Workaround already in use by \CGG{} - not upgrading to 1.8.2
+ \item Your workaround - no analysis for a solution has been performed yet
+\end{description}
+
\section{Find Lock Problems with Booby Trap}
\label{sec:find_lock_problems_booby_trap}
+\index{build!booby trap}
-A Booby Trap is used in CinGG for setting a trap to catch lock problems that might have been missed. It will trap boobies only if compile by adding \textit{-{}-with-booby} on the configuration command line. This is the default if you compile using \texttt{./bld.sh} from the GIT repository. It should not interfere with normal execution.
+A Booby Trap is used in \CGG{} for setting a trap to catch lock problems that might have been missed. It will trap boobies only if compile by adding \textit{-{}-with-booby} on the configuration command line. This is the default if you compile using \texttt{./bld.sh} from the GIT repository. It should not interfere with normal execution.
If you have the time and inclination, enable \textit{-{}-with-booby} and send any trap output that you find. Maybe you will catch some boobies and if you do, send a snapshot of any boobies you find.
\section{Valgrind Support Level}
\label{sec:valgrind_support_level}
+\index{build!valgrind}
Valgrind is a memory mis-management detector. It shows you memory leaks, deallocation errors, mismanaged threads, rogue reads/writes, etc. \CGG{} memory management is designed to work with Valgrind detection methods. This assists in developing reliable code. Use of Valgrind points out problems so that they can be fixed. For example, when this version of \CGG{} shuts down, it deallocates memory instead of just stopping, thus making memory leak detection possible.
\texttt{CFLAGS=-ggdb make -j8 rebuild\_all}
\end{enumerate}
+If you frequently make mods and changes in \CGG{} or the thirdparty libraries, do not depend on those
+compiled versions to have the headers updated; so be sure to fully rebuild as shown in the previous 2
+steps before running valgrind or you will get false errors.
+
Now your \CGG{} obj has all of the debug stuff. Next run valgrind as root for the most useful results:
\hspace{2em}\texttt{cd /path/cinelerra-5.1/cinelerra}
This runs \CGG{} under the control of valgrind, and produces a log file in /tmp which will list information about any leaks, usually clearly identifiable. Be sure to Quit out of \CGG{} normally instead of Ctrl-C or a SEGV otherwise the program does not have a chance to cleanup and there will be some false alarms. But it runs very slowly, and is basically single threaded, which means that race conditions may be impossible to catch$\dots$ like one thread deletes memory that another thread is currently using. But overall it is a big help and if you test any new features, please email the log output. A lot of effort when writing the code was put into trying to be sure that all of the object constructors have matching destructors so that the leaks can be identified. There are already several libraries that create predictable memory leaks and valgrind does a good job for most of these.
-It is impossible to test everything with valgrind because some things are just too big and slow for a practical test. Occasionally you can find a leak or an illegal memory access. There are several false alarms that are difficult to avoid \textit{Conditional jump} messages, and \textit{unhandled DW\_OP\_}, but anything with the word \textit{illegal} in the message is important. Memory leaks that originate in \CGG{} are good to find and fix, but are usually not deadly.
+It is impossible to test everything with valgrind because some things are just too big and slow for a practical test. Occasionally you can find a leak or an illegal memory access. There are several false alarms that are difficult to avoid \textit{Conditional jump} messages, and \textit{unhandled DW\_OP\_}, but anything with the word \textit{illegal} in the message is important. Memory leaks that originate in \CGG{} are good to find and fix, but are usually not
+deadly. The listing of the memory leaks can be quite voluminous so locating the \textit{LEAK SUMMARY} section
+towards the end of the report is most useful.
+
\section{CFLAGS has -Wall}
\label{sec:cflags_has_-wall}
\section{Prof2 -- A Profiler}
\label{sec:prof2_profiler}
+\index{build!prof2}
Frequently there is a problem with a program running slow and you do not know why. You need a thumbnail analysis of where the program is spending most of its time without all of the overwhelming details. This is when a Profiler comes in handy.
Key in: \qquad \texttt{make clean all install}
-Later, if you wanttitle to remove this from the system,
+Because \textit{smap} may have to be found in the system if \textit{How to use} below does not work, you will have to do the following:
+\newline
+\newline
+Key in: \qquad \texttt{cp -a smap /usr/local/bin}
+
-key in: \qquad \texttt{make uninstall}
+Later, if you want to remove this from the system,
+
+Key in: \qquad \texttt{make uninstall}
\subsection{How to use}
\label{sub:how_to_use}
\textbf{10.200t 0.001u+0.000s 21.566r 47.3\%}\\
\textit{--- profile end ---}
-The summary line above in Bold represents the User time, System time, Real time and the percentage is how much Timer time elapsed over Real time so in this case the measurement covers 47.3\% of time.\\
+The summary line above in Bold represents the User time, System time, Real time and the percentage is how much Timer time elapsed over Real time so in this case the measurement covers 47.3\% of time.
+
So why use a Profiler? Because it is the ``ls'' for executable functions!!
+
+\section{How to Create a new Theme}
+\label{sec:how_create_theme}
+\index{theme!create new theme}
+
+A \textit{Theme} is a base class object that is created and customized as \textit{ThemeName}.
+It is constructed during program initialization in a theme plugin
+\texttt{PluginTClient},
+defined in \texttt{plugins/theme\_name} source directory.
+
+\texttt{theme\_name.C} and \texttt{theme\_name.h} are derived \textit{Theme} class object constructors.
+
+A \textit{Theme} is constructed during initialization in \texttt{init\_theme} (\texttt{mwindow.C}). The theme plugin is accessed using the \textit{name} from preferences and then the theme plugin is loaded which contains the code to construct the theme. A \textit{Theme} object has functions and data that \CGG{} uses to do a variety of customizations, such as \texttt{default\_window\_positions}, and it can modify GUI defaults like \\
+\texttt{default\_text\_color} when it is initialized.
+
+The theme plugin contains a \textit{new\_theme} function that allocates and constructs a
+\textit{ThemeName} object with base classes of \textit{BC\_Theme} (gui setup), \textit{Theme} (\CGG{} defaults), and \textit{ThemeName}, with definitions and overrides that create the custom theme. To create a new theme, a new plugin is needed:
+
+\begin{lstlisting}[numbers=none]
+ #include "header files.h"
+ PluginClient* new_plugin(PluginServer *server)
+ {
+ return new NameMain(server);
+ }
+
+ NameMain::NameMain(PluginServer *server)
+ : PluginTClient(server)
+ {
+ }
+
+ NameMain::~NameMain()
+ {
+ }
+ const char* NameMain::plugin_title() { return N_("Name"); }
+
+ Theme* NameMain::new_theme()
+ {
+ theme = new ThemeName;
+ extern unsigned char _binary_theme_name_data_start[];
+ theme->set_data(_binary_theme_name_data_start);
+ return theme;
+ }
+
+ Name::Name()
+ : Theme()
+ {
+ }
+
+ Name::~Name()
+ {
+ delete stuff;
+ }
+\end{lstlisting}
+
+When a theme is constructed by \texttt{NameMain::new\_theme()}, it sets a pointer to a
+block of data created in the plugin build that contains all of the png data
+files in the \texttt{plugins/theme\_name/data} directory. These images may define or override the appearance of gui images, such as \textit{ok.png} (the ok button). There are usually a large number of images that need to be defined. The theme plugin adds them to the theme image data in the \texttt{theme $\rightarrow$ initialize()} function. The best list of theme image setup is probably in SUV (\texttt{plugins/theme\_suv/suv}).
+
+The easy way to create a new theme is to copy an existing theme and change
+its name to \textit{ThemeName}, change \texttt{plugin\_title()} to the new name, and then tweak the definitions until you are happy with the results. The file
+names and Makefile also need to be updated to the new theme name. The source
+can by manually rebuilt by invoking \textit{make} in the \texttt{plugins/theme\_name}
+directory.
+
+Once the new theme is built into the plugin library, it will automatically be discovered by the plugin probe
+and it will become an available theme in \textit{Preferences}.
+
+If you are ready to add it to the main build, then \textit{theme\_name} should be
+included in the DIRS targets of the \texttt{plugins/Makefile}, and \texttt{plugin\_defs} needs \textit{theme\_name} in the themes list.
+
+Themes usually require considerable time to create from scratch. For
+example, the SUV theme has over 800 lines in the initialize function, and has over
+500 png images in the data directory. Most of these images and data values are
+required to be initialized by the custom theme constructor; very tedious and
+time consuming work. Creating a new theme is usually a lot of work.
+
+\section{How Context Help works in the Program Code}
+\label{sec:context_help_coding}
+\index{context help}
+
+All class methods related to context help start with the common pattern \texttt{context \_help} in their names. It is easy to get all occurences in the code with the following command (example):
+
+\begin{lstlisting}[style=sh]
+ grep -F context_help `find . -name '*.[Ch]' -print`
+\end{lstlisting}
+
+The base functionality is defined in several \textit{BC\_WindowBase} class methods in \\ \texttt{guicast/bcwindowbase.C} (search on \texttt{context\_help}). All BC\_Window's and BC\_SubWindow's inherit these methods.
+
+For the simplest case of context help definition, it is sufficient to add the call to \texttt{context\_help\_set\_keyword()} in the most inclusive widget constructor. If \texttt{Alt/h} is pressed with the mouse over this widget's window, its \texttt{keypress\_event()} method (inherited from BC\_WindowBase) will catch this hotkey, fetch the keyphrase defined by \texttt{context\_help\_set\_keyword()} and call the \texttt{doc/ContextManual.pl} script with this keyphrase. Then ContextManual.pl script does the whole processing of the keyphrase given and calls web browser to display the found HTML manual page. The browser is called in the background to prevent from blocking the calling \CGG{} thread.
+
+An example from \textit{cinelerra/zoombar.C}:
+
+\begin{lstlisting}[style=sh]
+ ZoomBar::ZoomBar(MWindow *mwindow, MWindowGUI *gui)
+ : BC_SubWindow(...)
+ {
+ this->gui = gui;
+ this->mwindow = mwindow;
+ context_help_set_keyword("Zoom Panel");
+ }
+\end{lstlisting}
+
+If \texttt{Alt/h} is pressed with the mouse over some subwindow (arbitrary depth) of the \texttt{context\_help\_set\_keyword()} caller, the \texttt{keypress\_event()} method of that subwindow catches the hotkey. Its own context help keyword, probably, will be empty. In this case the whole widget hierarchy is traced up to \textit{top\_level} widget, their context help keywords are checked, and the first nonempty keyword is used for context help.
+
+This approach allows us to define the help keyword common to the whole dialog window with a bunch of diverse buttons with a single call to \texttt{context\_help\_set \_keyword()}, without placing such a call into each button constructor. And at the same time, this approach allows to assign different help keywords to GUI elements belonging to the same window but documented in different manual pages.
+
+\subsubsection{An example with several different help keywords from cinelerra/mwindowgui.C:}%
+\label{ssub:exemple_different_help_keywords}
+
+\begin{lstlisting}[style=sh]
+ MWindowGUI::MWindowGUI(MWindow *mwindow)
+ : BC_Window(_(PROGRAM_NAME ": Program"), ...)
+ {
+ this->mwindow = mwindow;
+ ...
+ context_help_set_keyword("Program Window");
+ }
+ ...
+ FFMpegToggle::FFMpegToggle(MWindow *mwindow, MButtons *mbuttons, int x, int y)
+ : BC_Toggle(...)
+ {
+ this->mwindow = mwindow;
+ this->mbuttons = mbuttons;
+ set_tooltip(get_value() ? FFMPEG_EARLY_TIP : FFMPEG_LATE_TIP);
+ context_help_set_keyword("FFmpeg Early Probe Explanation");
+ }
+ ...
+ StackButton::StackButton(MWindow *mwindow, int x, int y)
+ : BC_GenericButton(x, y, mwindow->theme->stack_button_w, "0")
+ {
+ this->mwindow = mwindow;
+ set_tooltip(_("Close EDL"));
+ context_help_set_keyword("OpenEDL");
+ }
+ ...
+ ProxyToggle::ProxyToggle(MWindow *mwindow, MButtons *mbuttons, int x, int y)
+ : BC_Toggle(...)
+ {
+ this->mwindow = mwindow;
+ this->mbuttons = mbuttons;
+ ...
+ context_help_set_keyword("Proxy");
+ }
+ \end{lstlisting}
+If the widget we wish to add context help to, overloads keypress\_event() of the base class with its own method, then it is necessary to introduce a small addition to its keypress\_event() handler: the call to \texttt{context\_help\_check\_and\_show()} has to be added in a suitable place in the handler.
+
+An example with \texttt{context\_help\_check\_and\_show()} from \textit{cinelerra/mbuttons.C}:
+\begin{lstlisting}[style=sh]
+ int MButtons::keypress_event()
+ {
+ int result = 0;
+ if(!result) {
+ result = transport->keypress_event();
+ }
+ if(!result) {
+ result = context_help_check_and_show();
+ }
+ return result;
+ }
+\end{lstlisting}
+
+All the keypress handlers are not equal, therefore we provide different variants of the methods context\_help\_check\_and\_show() and context\_help\_show() (the latter for explicit checking on hotkey).
+
+An example with context\_help\_show() from cinelerra/editpanel.C:
+\begin{lstlisting}[style=sh]
+ int EditPanelTcInt::keypress_event()
+ {
+ if( get_keypress() == 'h' && alt_down() ) {
+ context_help_show("Align Timecodes");
+ return 1;
+ }
+ ...further processing...
+ return 1;
+ }
+\end{lstlisting}
+
+A single example of much more sophisticated keypress\_event() handler can be found in \texttt{cinelerra/trackcanvas.C} (search on context\_help). The problem here was to track the particular object drawn on the canvas to figure out whose context help is to be requested. Another rare example of difficulty may appear when context help is desired for some GUI object which is not subclassed from BC\_WindowBase.
+
+\textit{ContextManual.pl} looks for occurence of keyphrases in the following order:
+
+\begin{enumerate}
+ \item In \textit{Contents.html}
+ \item In \textit{Index.html}
+ \item In all the \textit{CinelerraGG\_Manual/*.html} files via grep
+\end{enumerate}
+
+Keyphrase matching is tried first exact and case sensitive, then partially and case insensitive. The special keyword \texttt{TOC} shows Contents, \texttt{IDX} shows Index. If the keyword starts with \texttt{FILE:}, the filename after \textit{FILE:} is extracted and that file is shown. You can look in the ContextManual.pl script text.
+
+For debugging purposes the script can be called from command line. For this to work, the environment variable \texttt{\$CIN\_DAT} has to be set to the parent directory of doc.
+
+
+\subsubsection{Providing context help for plugins}%
+\label{ssub:providing_context_help_plugins}
+
+In the simplest case, nothing has to be done at all. The plugin context help functionality introduced in \texttt{cinelerra/pluginclient.C} automatically assigns context help keyword from the plugin's title for all plugins subclassed from \textit{PluginClient}. For this to work, the manual page documenting the plugin must be entitled identically to the programmatic title of the plugin. A special treatment can be necessary in the following cases:
+
+\begin{enumerate}
+ \item \textit{Rendered effects} are not subclasses of PluginClient and require an explicit context help definition via \texttt{context\_help\_set\_keyword().}
+ \item Only the main plugin dialog inherits context help functionality from the parent class. If a plugin opens additional dialog windows, they probably require explicit context help definitions.
+ \item If the plugin title differs from its subsection title in the documentation, either its help keyword or the corresponding title in the manual should be renamed. Another possibility can be an addition to the \textit{\%rewrite table} in \texttt{doc/ContextManual.pl}.
+ \item If the plugin title contains some characters special for HTML and/or Perl regular expression syntax, the special characters have to be escaped. For example, the keyphrase corresponding to the title \textit{Crop \& Position (X/Y)} should be converted to:
+
+ \begin{lstlisting}[style=sh]
+ Crop & Position \\(X\\/Y\\)
+ \end{lstlisting}
+
+ Such a rewriting can be done either in the C++ code or in ContextManual.pl.
+\end{enumerate}
+
+One additional explanation about previous user information when plugin tooltips are off. Help for the \textit {selected plugin} is shown because in this case it would be difficult to figure out, without a visual feedback, which particular item is currently under the mouse.
+
+\subsection{Manual Maintenance is now Tightly Coupled with the Program Code}
+\label{sub:manmaintain}
+
+If some section of the \CGG{} manual gets renamed and is used for Context Help, the corresponding help keywords should be adjusted too. All the keywords used can be listed via the following example command:
+
+\begin{lstlisting}[style=sh]
+ grep -F context_help `find . -name '*.C' -print` | grep -F '"'
+\end{lstlisting}
+----------------
+
+Note that the keyword does not have to exactly match the section title; it can also be a case insensitive substring of the section title.
+
+If some new \CGG{} window or dialog is created and documented, the inclusion of context\_help\_set\_keyword() calls and/or adaptation of keypress\_event() handler (if any) should be included in the program code.
+
+If some new \CGG{} plugin is created, it is best to document it in the section entitled exactly equal to that plugin's title. Then probably its context help will work out of the box. Otherwise, some adaptation to its keypress\_event() (if any) or to the %rewrite table in ContextManual.pl may be necessary.
+
+Why the local copy of \CGG{} manual should be used?
+
+\begin{enumerate}
+ \item For context help keyphrases matching, the local copy of \textit{Contents.html} and \textit{Index.html} is necessary anyway.
+ \item Grepping \textit{CinelerraGG\_Manual/*.html} files of the remote manual from the website cannot work per definition.
+ \item The local copy usage ensures exact matching of the version of the manual to the version of \CGG{}. Otherwise, if one uses for some reason an older version of \CGG{} with the help fetched from the newer version of the website manual, various incompatibilities can be expected.
+ \item Packing the manual into AppImage, the current method of \CGG{} packaging, should be much easier than merging two different git branches when building from source packages, as was done earlier.
+\end{enumerate}
+
+What about Localization?
+
+For now, \CGG{} context help is not localized at all. There is no complete \CGG{} manual in any language except English. But even should the manual be translated to other languages, the context help keywords must remain unlocalized literal string constants. First, because the set of languages known to \CGG{} itself is not the same as the set of languages the manual will be translated to. If some "translated keyword" is fed to the help system, while the manual in this language does not exist, keyword matching cannot succeed. Second, such a help localization with the translation of all keywords can be done and then maintained much easier and much more reliably inside the ContextManual.pl script rather than in the \CGG{} binary.
+
+More about ContextManual.pl file?
+
+The bin/doc/ContextManual.pl script can be configured further. Look in the script text. You can define your preferable web browser, or redefine it to 'echo' for debug purposes. There are also some predefined HTML pages for Contents and Index, and several explicitly rewritten keyphrases.
projects / goodguy / cin-manual-latex.git / blobdiff