1 \chapter{Developer's Section}%
2 \label{cha:developer's_section}
4 \section{How Builds Really Work and Even More Options}
5 \label{sec:builds_really_work_more_options}
7 This section describes how builds really work if you want to know more about making changes or understanding the process; probably only for a developer or system administrator.
9 Builds occur in 4 basic steps:
11 \begin{enumerate}[nosep]
12 \item unpack/patch source code
14 \item make build targets
18 So, an example of what happens in 4 steps for a single-user build would be as follows:
20 \begin{enumerate}[nosep]
21 \item unpack/patch source code: \\
22 \texttt{git clone --depth 1 ``git:/{\dots}/target'' cinelerra5} \\
24 \item configure build:\\
25 \texttt{./configure --with-single-user}
26 \item make build targets:\\
27 \texttt{make 2 > \&1 | tee log}
32 A lot of things can be tweaked to change the results. Mostly these changes are parameters to the configure step, which can change important build related items, like the application name, or where and what the target system directories should be. This makes it possible to have several versions at the same time on the same computer if needed. To see what it is that the makefiles use to build \CGG{}, look at the resulting top-level global\_config file which is created by the ./configure step.
34 Building \CGG{} requires many thirdparty libraries, and it is recommended that you use the static build version included in the git repo. Some of them are patched, and fix significant bugs. It is important to note that because system installation historically has been with as many shared objects as possible, the defaults are that any system library detected during configuration setup will be used, when the package is built \textit{-{}-without-single-user}, which is the default build. To build with static thirdparty libraries for system install to the system /usr area, use:
36 \hspace{2em}\texttt{.configure -{}-enable-static-build --prefix=/usr}
38 Sometimes, additional package parameters and variables are needed during thirdparty builds. These optional values occur before and after the \textit{configure} and \textit{make} commands during a build. A presentation of the format of the package qualified variable names and how they appear in the build procedure are:
41 \begin{tabular}{@{}ll}
42 pkg.cfg\_vars & prepended to configure\\
43 pkg.cfg\_params & appended to configure\\
44 pkg.mak\_vars & prepended to make\\
45 pkg.mak\_params & appended to make\\
46 pkg.cflags & added as CFLAGS+=\$(cflags) to pkg.vars\\
47 pkg.cppflags & added as CPPFLAGS+=\$(cppflags) to pkg.vars\\
50 These steps are done for EACH of the packages in the thirdparty build:
52 \hspace{2em}\texttt{<pkg.cfg\_vars> ./configure <pkg.cfg\_params>}
54 \hspace{2em}\texttt{<pkg.mak\_vars> make <pkg.mak\_params>}
56 The thirdparty Makefile has a set of default vars and params used to build each of the needed thirdparty packages, but you can specify new or overriding values for these Makefile substitutions. These thirdparty build config changes are specified in the top-level file: \textit{cin\_config}. By using this file, you can save the configuration changes made for the current build to use the next time you do a new build. For example, to add an include file path to the giflib build, add this line to \textit{cin\_config}:
58 \hspace{2em}\texttt{giflib.cflags := -I/usr/local/include/giflib5}
60 To have a param/var change apply to all thirdparty builds, use:
62 \hspace{2em}\texttt{CFG\_VARS, CFG\_PARAMS, MAK\_VARS, MAK\_PARAMS}
64 CFLAGS, CXXFLAGS and LDFLAGS are forwarded to the thirdparty build environment via:
66 \hspace{2em}\texttt{CFLAGS=-ggdb ./configure -{}-with-single-user}
68 However, there is no guarantee that the thirdparty build will honor the environmental flags.
70 Finally, there are build controls, which enable/disable and set build features.
72 A few of the more useful ./configure -{}-parameters are:
75 -{}-with-jobs=n & where n=number of build jobs; defaults to 1.5*cpus+2\\
76 -{}-enable-static-build & build all 3rd party libs; defaults to yes if single-user, else no\\
77 -{}-with-single-user& build installs to <build\_path>/bin; no system installation\\
81 The ./configure command builds \textit{global\_config}. The global\_config is read by the thirdparty/Makefile to create the recipes and definitions used by the thirdparty build.
83 There are a lot of different options. Thirdparty library build control is available in the configure step of the build. Thirdparty libraries are built on a demand basis. So if you use:
85 \begin{tabular}{l p{11cm}}
86 -{}-enable-libname=auto & the static-build enable, or the lack of a system library causes a build\\
87 -{}-enable-libname=yes & this forces the thirdparty build\\
88 -{}-enable-libname=no & this forces no thirdparty build\\
91 FFmpeg is a \textit{strongly connected} component in the build linkage and widely influences the \CGG{} library demands. It is possible to make small additions to the ffmpeg configuration step using the environment variable \texttt{FFMPEG\_EXTRA\_CFG}. For example, to eliminate the use of libvdpau (an nvidia support library) in the ffmpeg configuration step after you have determined that it is causing an error, use:
93 \begin{itemize}[label={},nosep]
94 \item \texttt{make clean}
95 \item \texttt{autogen.sh}
96 \item \texttt{export FFMPEG\_EXTRA\_CFG=" -{}-disable-vdpau"}
97 \item \texttt{./configure} {\dots}
100 Specific information on using the current ffmpeg GIT repository follows. You have to supply the actual URL location of the ffmpeg git as you can see in this example \texttt{?bld.sh?} script:
102 \begin{lstlisting}[numbers=none]
104 () ./autogen.sh\newline
105 ./configure --with-single-user --with-booby --with-git-ffmpeg=https://git.ffmpeg.org/ffmpeg.git
106 make && make install ) 2>1 | tee log
107 mv Makefile Makefile.cfg
108 cp Makefile.devel Makefile
111 Since the procedure for obtaining the latest ffmpeg version is not always kept up-to-date and the line numbers will always change, you may have to create that patch first. Generally those line numbers are only updated by a developer when a new stable version with worthwhile features is actually included in the \CGG{} build. FFmpeg is constantly changing and many times the git version is not as stable as desired.
113 \section{Configuration Features}
114 \label{sec:configuration_features}
116 A listing of the current configuration features as of January 11, 2020:
119 \fontsize{10pt}{12pt}\selectfont
122 `configure' configures \CGG{} to adapt to many kinds of systems.
124 Usage: ./configure [OPTION]... [VAR=VALUE]...
126 To assign environment variables (e.g., CC, CFLAGS...), specify them as
127 VAR=VALUE. See below for descriptions of some of the useful variables.
129 Defaults for the options are specified in brackets.
132 -h, --help display this help and exit
133 --help=short display options specific to this package
134 --help=recursive display the short help of all the included packages
135 -V, --version display version information and exit
136 -q, --quiet, --silent do not print `checking ...' messages
137 --cache-file=FILE cache test results in FILE [disabled]
138 -C, --config-cache alias for `--cache-file=config.cache'
139 -n, --no-create do not create output files
140 --srcdir=DIR find the sources in DIR [configure dir or `..']
142 Installation directories:
143 --prefix=PREFIX install architecture-independent files in PREFIX
145 --exec-prefix=EPREFIX install architecture-dependent files in EPREFIX
148 By default, `make install' will install all the files in
149 `/usr/local/bin', `/usr/local/lib' etc. You can specify
150 an installation prefix other than `/usr/local' using `--prefix',
151 for instance `--prefix=$HOME'.
153 For better control, use the options below.
155 Fine tuning of the installation directories:
156 --bindir=DIR user executables [EPREFIX/bin]
157 --sbindir=DIR system admin executables [EPREFIX/sbin]
158 --libexecdir=DIR program executables [EPREFIX/libexec]
159 --sysconfdir=DIR read-only single-machine data [PREFIX/etc]
160 --sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com]
161 --localstatedir=DIR modifiable single-machine data [PREFIX/var]
162 --libdir=DIR object code libraries [EPREFIX/lib]
163 --includedir=DIR C header files [PREFIX/include]
164 --oldincludedir=DIR C header files for non-gcc [/usr/include]
165 --datarootdir=DIR read-only arch.-independent data root [PREFIX/share]
166 --datadir=DIR read-only architecture-independent data [DATAROOTDIR]
167 --infodir=DIR info documentation [DATAROOTDIR/info]
168 --localedir=DIR locale-dependent data [DATAROOTDIR/locale]
169 --mandir=DIR man documentation [DATAROOTDIR/man]
170 --docdir=DIR documentation root [DATAROOTDIR/doc/cinelerra]
171 --htmldir=DIR html documentation [DOCDIR]
172 --dvidir=DIR dvi documentation [DOCDIR]
173 --pdfdir=DIR pdf documentation [DOCDIR]
174 --psdir=DIR ps documentation [DOCDIR]
177 --program-prefix=PREFIX prepend PREFIX to installed program names
178 --program-suffix=SUFFIX append SUFFIX to installed program names
179 --program-transform-name=PROGRAM run sed PROGRAM on installed program names
182 --disable-option-checking ignore unrecognized --enable/--with options
183 --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
184 --enable-FEATURE[=ARG] include FEATURE [ARG=yes]
185 --enable-silent-rules less verbose build output (undo: "make V=1")
186 --disable-silent-rules verbose build output (undo: "make V=0")
187 --enable-dependency-tracking
188 do not reject slow dependency extractors
189 --disable-dependency-tracking
190 speeds up one-time build
191 --enable-a52dec build a52dec (yes)
192 --enable-djbfft build djbfft (yes)
193 --enable-audiofile build audiofile (no)
194 --enable-encore build encore (no)
195 --enable-esound build esound (no)
196 --enable-ffmpeg build ffmpeg (yes)
197 --enable-fftw build fftw (auto)
198 --enable-flac build flac (auto)
199 --enable-giflib build giflib (yes)
200 --enable-ilmbase build ilmbase (auto)
201 --enable-lame build lame (auto)
202 --enable-libavc1394 build libavc1394 (auto)
203 --enable-libraw1394 build libraw1394 (auto)
204 --enable-libiec61883 build libiec61883 (auto)
205 --enable-libdv build libdv (auto)
206 --enable-libjpeg build libjpeg (auto)
207 --enable-opus build opus (auto)
208 --enable-openjpeg build openjpeg (auto)
209 --enable-libogg build libogg (auto)
210 --enable-libsndfile build libsndfile (auto)
211 --enable-libtheora build libtheora (auto)
212 --enable-libuuid build libuuid (yes)
213 --enable-libvorbis build libvorbis (auto)
214 --enable-mjpegtools build mjpegtools (yes)
215 --enable-openexr build openexr (auto)
216 --enable-tiff build tiff (auto)
217 --enable-twolame build twolame (auto)
218 --enable-x264 build x264 (auto)
219 --enable-x265 build x265 (auto)
220 --enable-libvpx build libvpx (auto)
221 --enable-lv2 build lv2 (auto)
222 --enable-sratom build sratom (auto)
223 --enable-serd build serd (auto)
224 --enable-sord build sord (auto)
225 --enable-lilv build lilv (auto)
226 --enable-suil build suil (auto)
227 --enable-libaom build libaom (auto)
228 --enable-dav1d build dav1d (auto)
229 --enable-libwebp build libwebp (auto)
230 --enable-ffnvcodec build ffnvcodec (auto)
231 --enable-static-build build static ([auto])
232 --enable-x264_hidepth build x264 10bit ([no])
233 --enable-x265_hidepth build x265 10bit ([no])
236 --with-PACKAGE[=ARG] use PACKAGE [ARG=yes]
237 --without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no)
238 --with-jobs parallel build jobs (auto)
239 --with-exec-name binary executable name (cin)
240 --with-single-user to install cin in bin (no)
241 --with-ladspa-build build ladspa library (yes)
242 --with-lv2 lv2 library support (yes)
243 --with-cinlib cinelerra library path (auto)
244 --with-cindat cinelerra share path (auto)
245 --with-plugin-dir plugin install dir (auto)
246 --with-ladspa-dir ladspa install dir (auto)
247 --with-config-dir .bcast config dir ($$HOME/.bcast5)
248 --with-nested-dir nested proxy dir ($$HOME/Videos)
249 --with-browser cin_browser path (firefox)
250 --with-git-ffmpeg git ffmpeg using url (no)
251 --with-noelision use noelision/libpthread (auto)
252 --with-booby window lock trace booby trap (no)
253 --with-libzmpeg build libzmpeg (yes)
254 --with-commercial enable commercial capture (yes)
255 --with-thirdparty use thirdparty build (yes)
256 --with-shuttle shuttle device (yes)
257 --with-wintv usb 2040:826d wintv device (yes)
258 --with-x10tv usb 0bc7:0004 X10 remote device (yes)
259 --with-vaapi video acceleration api (yes)
260 --with-vdpau video decode+presentation api for unix (yes)
261 --with-nv nvenc/nvdec ffnvcodec api (yes)
262 --with-cuda nv cuda plugins (auto)
263 --with-clang use clang instead of gcc/g++ (no)
264 --with-gl use opengl (auto)
265 --with-oss use OSS audio (auto)
266 --with-xft use libXft (auto)
267 --with-alsa use libasound/alsa (auto)
268 --with-firewire use firewire (auto)
269 --with-dv use dv (auto)
270 --with-dvb use dvb (auto)
271 --with-video4linux2 use v4l2 (auto)
272 --with-xxf86vm use xf86vmode (auto)
273 --with-esound use esd (auto)
274 --with-shuttle shuttle dev support (auto)
275 --with-shuttle_usb use libusb-1.0 (auto)
276 --with-lv2 use lv2 (auto)
277 --with-cuda build cuda plugins (auto)
278 --with-dl system has libdl (auto)
279 --with-opencv opencv=sys/sta/dyn,git/tar=url (auto)
280 --with-numa system has libnuma (auto)
281 --with-openexr use openexr (auto)
283 Some influential environment variables:
284 CC C compiler command
285 CFLAGS C compiler flags
286 LDFLAGS linker flags, e.g. -L<lib dir> if you have libraries in a
287 nonstandard directory <lib dir>
288 LIBS libraries to pass to the linker, e.g. -l<library>
289 CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if
290 you have headers in a nonstandard directory <include dir>
291 CCAS assembler compiler command (defaults to CC)
292 CCASFLAGS assembler compiler flags (defaults to CFLAGS)
293 CXX C++ compiler command
294 CXXFLAGS C++ compiler flags
296 Use these variables to override the choices made by `configure' or to help
297 it to find libraries and programs with nonstandard names/locations.
299 Report bugs to <mail@lists.cinelerra-gg.org>.
304 \section{Thirdparty Parallel Build}
305 \label{sec:thirdparty_parallel_build}
307 The Makefile in the thirdparty build directory employs a set of macros used to create a build rule set of thirdparty library build dependencies. The standard build sequence of [source, config, build] is used to prepare thirdparty products as static libraries. Build package dependency can be specified in the Makefile std-build macro call. These Makefile macro calls define the rules used for each thirdparty build. The expanded rule definitions may be viewed by using:
309 \hspace{2em}\texttt{make -C thirdparty rules}
311 Individual package libraries can be rebuilt, via:
313 \hspace{2em}\texttt{make -C thirdparty <pkg>-clean; make -C thirdparty <pkg>}
315 The rule targets create the set of thirdparty packages which are built from local source archive copies of thirdparty source code and patches, if needed. The build rule set of dependencies allows for compiling multiple thirdparty programs simultaneously using maximum computer resources. This parallel build speeds up the process considerably. For example, these are full static build timings on the production build machine (full build includes building all thirdparty programs as well as all of \CGG{}):
318 \begin{tabular}{@{}rcr}
319 1 cpu & = & 61 mins\\
320 12 cpus & = & 7.5 mins\\
321 24 cpus & = & 2 mins\\
324 \section{Using the very latest Libraries}
325 \label{sec:latest_libraries}
327 Using the most current libraries can be a challenge for some of the Operating System distros that use
328 stable compilers, assemblers, and their own libraries. Because they are stable, they frequently do
329 not keep up with the very latest of thirdparty libraries. Consequently, some program constructs may
330 not yet be implemented. \CGG{} tries to maintain stability since it is better to have less features
331 and no crashes. The goal is to make \CGG{} widely available on many platforms rather than
332 dependent on advanced tools that are not supported on some distros.
334 \CGG{} attempts to upgrade to the latest releases of many thirdparty libraries about every 3-4
335 months. But it is often difficult to keep some of these thirdparty libraries up to date as their
336 developers switch from tried and true standard tools to newer, less standard tools. As a result
337 in order to build \CGG{} on 2-3 versions of any distro rather than only the most current version,
338 some thirdparty libraries can not be kept up to date and may even be to the point of no further
339 updates. In a lot of cases, the updated releases provide little new capabilities but rather
340 are bug fixes that may not even be relevant to \CGG{}'s use.
342 So as a computer savvy user or a developer, if you would like to build \CGG{} with the latest
343 thirdparty releases for some of the packages here are a few suggestions based on other
344 developer's feedback and experimentation.
347 \begin{description}[noitemsep]
348 \item Status - currently \CGG{} is staying at 0.5. This is disappointing because there
349 may be speed gains in version 0.6 that would be beneficial. However, it is usable for decoding
350 whereas libaom is just too slow. Unfortunately, it has no effective encoder.
351 \item Problem - 0.6 dav1d requires NASM 2.14 and uses instructions like vgf2p8affineqb,
352 not exactly an "add" instruction. It also uses meson which is not widely available on all
353 distros. The only distros that are built for \CGG{} that are at 2.14 are the latest version
354 of Arch, Debian(10), Gentoo, Tumbleweed, and Fedora(31). The rest are at 2.12 and 2.13 including
355 the most widely used Ubuntu. The NASM requirement apparently provides for using AVX-512
356 instructions (like vgf2p8affineqb, which is more like a whole routine than a simple instruction).
357 \item Workaround already in use by \CGG{} - a Makefile was generated to replace Meson usage
358 but has to be continuously updated for new releases. Dav1d 0.5 requires NASM 2.13 so at this level
359 the newer distros mostly work. The availability of meson and nasm are a significant problem on
360 many systems which are still in wide use.
361 \item Your workaround - Because a request to dav1d developers to consider changes to
362 ensure their library is more widely usable does not appear to be in their future, since it works
363 for them, you can upgrade NASM to 2.14 to stay up to date. Of course, install meson also.
367 \begin{description}[noitemsep]
368 \item Status - currently at latest version
369 \item Problem - the OpenExr tarball is not a single package but is 2 packages instead
370 \item Workaround already in use by \CGG{} - reworked the packages so that it looks like
371 one package with 2 stubs
372 \item Your workaround - perhaps use the same workaround
376 \begin{description}[noitemsep]
377 \item Status - 2 different versions specific for O/S but none for Ubuntu 14, 32 or 64 bit
378 \item Problem - There are really 2 problems here. The first is OpenCV is not really
379 "Open" in that Surf is patented/non-free and there is no actual source available for certain
380 capabilities. The second is that cmake 3.5.1 is required for OpenCV 4.2.
381 \item Workaround already in use by \CGG{} - using 3.4.1 for older distros and 4.2 for newer
382 \item Your workaround - upgrade cmake to 3.5.1 for upgrade to 4.2; add non-free to the
383 compile; and use binaries that you do not know what they contain since no source code to compile.
384 Look into opencv4/opencv2/core/types.hpp:711;27
388 \begin{description}[noitemsep]
389 \item Status - currently at version 1.1.0
390 \item Problem - requires cmake 3.5
391 \item Workaround already in use by \CGG{} - leaving out of Ubuntu14, Ubuntu, Centos7
392 \item Your workaround - upgrade on those systems to cmake 3.5
396 \begin{description}[noitemsep]
397 \item Status - currently at version 1.0.0
398 \item Problem - requires cmake 3.5
399 \item Workaround already in use by \CGG{} - leaving out of Ubuntu14, Ubuntu, Centos7
400 \item Your workaround - upgrade on those systems to cmake 3.5
404 \begin{description}[noitemsep]
405 \item Status - this is the x10 TV remote control
406 \item Problem - INPUT\_PROP\_POINTING\_STICK not defined error on older distros
407 \item Workaround already in use by \CGG{} - leaving out of Ubuntu14, Ubuntu, Centos7
408 \item Your workaround - look into /usr/include/linux/input-event-codes.h
412 \begin{description}[noitemsep]
413 \item Status - currently at version 1.8.1
414 \item Problem - when decoding a test file, it failed to correctly load to the timeline
415 \item Workaround already in use by \CGG{} - not upgrading to 1.8.2
416 \item Your workaround - no analysis for a solution has been performed yet
419 \section{Find Lock Problems with Booby Trap}
420 \label{sec:find_lock_problems_booby_trap}
422 A Booby Trap is used in CinGG for setting a trap to catch lock problems that might have been missed. It will trap boobies only if compile by adding \textit{-{}-with-booby} on the configuration command line. This is the default if you compile using \texttt{./bld.sh} from the GIT repository. It should not interfere with normal execution.
424 If you have the time and inclination, enable \textit{-{}-with-booby} and send any trap output that you find. Maybe you will catch some boobies and if you do, send a snapshot of any boobies you find.
426 There are 2 potential traps:
428 \begin{itemize}[nosep]
429 \item If you try to unlock a lock when it is not locked
430 \item If you execute a drawing operation without holding the window lock
433 The trap prints the following in the controlling terminal window:
435 \hspace{2em} \textit{BOOBY! \qquad <backtrace>}
437 An example backtrace is below along with a hint below on how to analyze:
439 \begin{lstlisting}[numbers=none]
440 /home/cin5/bin/./cin(_Z5boobyv+0x3f) [0x557069fa9b2f] /home/cin5/bin/./cin(_ZN13BC_WindowBase9draw_lineEiiiiP9BC_Pixmap+0x3b)0x557069fb9a9b]
441 /home/cin5/bin/./cin(\_ZN10BC_ListBox11draw_borderEi+0x73)[0x557069f7dc73]
442 /home/cin5/bin/./cin(+0x9707fb) [0x557069f7e7fb]
443 /home/cin5/bin/./cin(\ZN10BC\ListBox16center\selectionEv+0x4e)[0x557069f7f2ae]
444 /home/cin5/bin/plugins/video/sketcher.plugin(_ZN17SketcherCurveList6updateEi+0x1a0)[0x7f1b8002a4c0]
445 /home/cin5/bin/plugins/video/sketcher.plugin(_ZN18SketcherCurveColor17handle_done_eventEi+0x76)[0x7f1b8002a5f6]
446 /home/cin5/bin/./cin(_ZN15BC_DialogThread3runEv+0xd8)[0x557069f6fb78]
447 /home/cin5/bin/./cin(_ZN6Thread10entrypointEPv+0x45)[0x557069fc5995]
448 /usr/lib/libpthread.so.0(+0x7a9d) [0x7f1b91b4ea9d]
449 /usr/lib/libc.so.6(clone+0x43) [0x7f1b90accb23]
452 To see which routine is reporting the booby key in:
454 \hspace{2em} \texttt{c++filt}
456 And then the $2^{nd}$ line in the backtrace above:
458 \hspace{2em}\texttt{\_ZN13BC\_WindowBase9draw\_lineEiiiiP9BC\_Pixmap}
460 It comes back with the routine as:
462 \hspace{2em}\texttt{BC\_WindowBase::draw\_line(int, int, int, int, BC\_Pixmap*)}
464 \section{Valgrind Support Level}
465 \label{sec:valgrind_support_level}
467 Valgrind is a memory mis-management detector. It shows you memory leaks, deallocation errors, mismanaged threads, rogue reads/writes, etc. \CGG{} memory management is designed to work with Valgrind detection methods. This assists in developing reliable code. Use of Valgrind points out problems so that they can be fixed. For example, when this version of \CGG{} shuts down, it deallocates memory instead of just stopping, thus making memory leak detection possible.
469 The best way to compile and run valgrind is to run the developer static build. This takes 2 steps and you must already have gdb and valgrind installed:
471 \begin{enumerate}[nosep]
472 \item The standard static build:\\
473 \texttt{cd /path/cinelerra-5.1}\\
474 \texttt{make clean}\\
476 \item run the incremental rebuild for debug objs:\\
477 \texttt{CFLAGS=-ggdb make -j8 rebuild\_all}
480 Now your \CGG{} obj has all of the debug stuff. Next run valgrind as root for the most useful results:
482 \hspace{2em}\texttt{cd /path/cinelerra-5.1/cinelerra}
484 \hspace{2em}\texttt{valgrind -{}-log-file=/tmp/log -{}-leak-check=full\\
485 -{}-num-callers=32 ./ci}
487 This runs \CGG{} under the control of valgrind, and produces a log file in /tmp which will list information about any leaks, usually clearly identifiable. Be sure to Quit out of \CGG{} normally instead of Ctrl-C or a SEGV otherwise the program does not have a chance to cleanup and there will be some false alarms. But it runs very slowly, and is basically single threaded, which means that race conditions may be impossible to catch$\dots$ like one thread deletes memory that another thread is currently using. But overall it is a big help and if you test any new features, please email the log output. A lot of effort when writing the code was put into trying to be sure that all of the object constructors have matching destructors so that the leaks can be identified. There are already several libraries that create predictable memory leaks and valgrind does a good job for most of these.
489 It is impossible to test everything with valgrind because some things are just too big and slow for a practical test. Occasionally you can find a leak or an illegal memory access. There are several false alarms that are difficult to avoid \textit{Conditional jump} messages, and \textit{unhandled DW\_OP\_}, but anything with the word \textit{illegal} in the message is important. Memory leaks that originate in \CGG{} are good to find and fix, but are usually not deadly.
491 \section{CFLAGS has -Wall}
492 \label{sec:cflags_has_-wall}
493 When compiling \CGG{} Infinity a CFLAGS option used is \textit{Wall} where the "W" represents warnings and "all" means all. This causes the compile to check for simple mistakes that can be detected automatically and issue warnings when the code is questionable. It can also detect situations where the compiler will generate incorrect code, like type-punned pointer. By turning on this flag, when new code is vetted for predictable mistakes, the code can be corrected before becoming manifested in the application.
495 \section{Prof2 -- A Profiler}
496 \label{sec:prof2_profiler}
498 Frequently there is a problem with a program running slow and you do not know why. You need a thumbnail analysis of where the program is spending most of its time without all of the overwhelming details. This is when a Profiler comes in handy.
500 There are many different profilers available -- this particular one does not do anything special and is fairly ordinary in that it just characterizes frequency of execution of various regions in the program. However, it has some pretty good strengths as listed next.
502 \begin{enumerate}[nosep]
503 \item It takes very little or no preconditioning, i.e. setup
504 \item The effect it has on the running program is minimal
505 \item It runs almost at full speed, which is really nice
506 \item It is not particularly thread aware
507 \item Reports where it is executing at intervals of 100 times a second
513 This profiler works on x86\_64 systems and you must be root to compile and run it. Also, you must install your operating system's \textit{iberty} -- for example, which would be binutils-devel for Fedora or libiberty-dev for Ubuntu 18.
515 Go to the top level \CGG{} directory.
517 Key in: \qquad \texttt{prof2}
519 Key in: \qquad \texttt{make clean all install}
521 Because \textit{smap} may have to be found in the system if \textit{How to use} below does not work, you will have to do the following:
524 Key in: \qquad \texttt{cp -a smap /usr/local/bin}
527 Later, if you want to remove this from the system,
529 Key in: \qquad \texttt{make uninstall}
531 \subsection{How to use}
532 \label{sub:how_to_use}
534 The program you are profiling should be compiled debuggable with stack frame and symbols enabled.
536 To see help, key in: \qquad \texttt{prof -h}
538 usage: -h [-o path] [-d] [-e] [-p libpath] [-012] [-u \#] cmd args...
541 \begin{tabular}{@{}ll}
542 -o & profile output pathname, -=stdout, -{}-=stderr\\
543 -d & debug otitleutput enabled\\
544 -e & child debug output enabled\\
545 -p & specify path for libprofile.so\\
546 -0 & usr+sys cpu timer intervals (sigprof)\\
547 -1 & usr only cpu timer intervals (sigvtalrm)\\
548 -2 & real time timer intervals (sigalrm)\\
549 -u & profile timer interval in usecs\\
552 To execute the profiler, key in:
554 \hspace{2em}\texttt{prof -o /tmp/prof\_list.txt ./cin}
556 where \texttt{/tmp/prof\_list.txt} is the output file and in this case \texttt{cin} is the \CGG{} binary file. The pid of this command will be displayed on the startup window. This comes in handy in the use case where there is a lot of initial load and possible configuration setup inside of \CGG{} and you want to profile plugins and not necessarily all of the setup steps. Then you can use the following command in another window to continue running \CGG{} and obtain the more useful information:
558 \hspace{2em}\texttt{kill -USR1 pid}
560 Running this command refreshes the memory maps used for profiling. When you are profiling a plugin, you want to run this AFTER the plugin loads.
565 There are 3 sections in the resulting output file of this stochastic quick analysis.
567 \begin{enumerate}[nosep]
568 \item The first section is a histogram of the timer intervals of that sample set. Each function occupies a region of addresses. One hundred times a second, the profiler samples the program address in the region of execution.
569 \item In the second section, there is another histogram of the cumulative frequency of all things in the call stack and this should point out who is the culprit. For example, a codec calls a bunch of subroutines, the cost of the subroutines is accumulated to the codec parent caller. This makes the actual cpu user much more apparent.
570 \item The last section is for the library spaces. Each library occupies a region and the profiler adds up the time spent in each of the libraries by frequency.
573 On the very bottom is a 1 line summary which shows you if there is a bad guy and who it is.
575 \subsection{Sample output}
576 \label{sub:sample_output}
578 \textit{--- profile start ---}
580 \textbf{1020 ticks 43 modules 81412 syms}\\
581 \begin{tabular}{@{}rrp{\linewidth-6em}}
582 0.010s & 0.1\% & Autos::copy(long, long, FileXML*, int, int) /mnt0/build5/cinelerra-5.1/bin/cin\\
583 0.010s & 0.1\% & BinFolders::copy\_from(BinFolders*) /mnt0/build5/cinelerra-5.1/bin/cin\\
584 0.010s & 0.1\% & cstrdup(char const*) /mnt0/build5/cinelerra-5.1/bin/cin\\
585 0.010s & 0.1\% & XMLBuffer::encode\_data(char*, char const*, int) /mnt0/build5/cinelerra-5.1/bin/cin\\
586 0.010s & 0.1\% & XMLBuffer::encoded\_length(char const*, int) /mnt0/build5/cinelerra-5.1/bin/cin\\
587 0.010s & 0.1\% & PluginClient::send\_configure\_change() /mnt0/build5/cinelerra-5.1/bin/cin\\
588 0.010s & 0.1\% & UndoVersion::scan\_lines(UndoHashTable*, char*, char*) /mnt0/.../cin\\
589 0.010s & 0.1\% & UndoStackItem::set\_data(char*) /mnt0/build5/cinelerra-5.1/bin/cin\\
590 0.010s & 0.1\% & UndoStack::load(\_IO\_FILE*) /mnt0/build5/cinelerra-5.1/bin/cin\\
591 0.010s & 0.1\% & BC\_Bitmap::cur\_bfr() /mnt0/build5/cinelerra-5.1/bin/cin\\
592 0.010s & 0.1\% & YUV::init\_tables(int, int*, int*, int*, int*, int*, int*, int*, int*, int*, int*, int*, int*, int*, int*) /mnt0/build5/cinelerra-5.1/bin/cin\\
596 \textit{--- profile calls ---}
598 \begin{tabular}{@{}rrl}
599 0.010s & 0.1\% & AutoConf::save\_xml(FileXML*) 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
600 0.010s & 0.1\% & Automation::copy(long, long, FileXML*, int, int) 1.0 /mnt0/.../cin\\
601 0.010s & 0.1\% & AWindow::run() 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
602 0.010s & 0.1\% & Canvas::stop\_single() 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
603 0.010s & 0.1\% & ColorPicker::new\_gui() 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
604 0.010s & 0.1\% & ColorWindow::create\_objects() 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
605 0.010s & 0.1\% & PaletteWheel::draw(float, float) 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
606 0.010s & 0.1\% & PaletteHex::update() 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
607 0.010s & 0.1\% & CWindowGUI::draw\_status(int) 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
608 0.010s & \textbf{0.1\%} & CWindowCanvas::status\_event() 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
612 \begin{tabular}{@{}rrl}
613 0.990s & \textbf{9.7\%} & BC\_Xfer::xfer\_slices(int) 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
614 1.880s & \textbf{18.4\%} & DirectUnit::process\_package(LoadPackage*) 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
615 1.880s & \textbf{18.4\%} & DirectUnit::rgba8888() 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
616 3.910s & \textbf{38.3\%} & \_\_init\_array\_end 1.1 /mnt0/build5/cinelerra-5.1/bin/cin\\
617 5.450s & \textbf{53.4\%} & LoadClient::run() 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
618 7.890s & \textbf{77.4\%} & Thread::entrypoint(void*) 1.0 /mnt0/build5/cinelerra-5.1/bin/cin\\
619 7.890s & \textbf{77.4\%} & start\_thread 1.0 /lib64/libpthread-2.28.so\\
623 \begin{tabular}{@{}rrl}
624 0.010s & 0.1\%/ 0.0\% & /lib64/libm-2.28.so\\
625 0.010s & 0.1\%/ 0.0\% & /lib64/libexpat.so.1.6.8\\
626 0.020s & 0.2\%/ 0.1\% & /lib64/libXext.so.6.4.0\\
627 0.020s & 0.2\%/ 0.1\% & /lib64/libXft.so.2.3.2\\
628 0.020s & 0.2\%/ 0.1\% & /lib64/libxcb.so.1.1.0\\
629 0.040s & 0.4\%/ 0.2\% & /lib64/ld-2.28.so\\
630 0.050s & 0.5\%/ 0.2\% & /lib64/libpng16.so.16.34.0\\
631 0.130s & 1.3\%/ 0.6\% & /lib64/libX11.so.6.3.0\\
632 0.180s & 1.8\%/ 0.8\% & /lib64/libz.so.1.2.11\\
633 0.200s & 2.0\%/ 0.9\% & /lib64/libfontconfig.so.1.12.0\\
634 0.380s & 3.7\%/ 1.8\% & /lib64/libpthread-2.28.so\\
635 1.660s & 16.3\%/ 7.7\% & /lib64/libc-2.28.so\\
636 7.480s & 73.3\%/34.7\% & /mnt0/build5/cinelerra-5.1/bin/cin\\
639 \textbf{10.200t 0.001u+0.000s 21.566r 47.3\%}\\
640 \textit{--- profile end ---}
642 The summary line above in Bold represents the User time, System time, Real time and the percentage is how much Timer time elapsed over Real time so in this case the measurement covers 47.3\% of time.
644 So why use a Profiler? Because it is the ``ls'' for executable functions!!