version update, bld tweaks, resize track booby, 10fps for images

[goodguy/cinelerra.git] / cinelerra-5.1 / libzmpeg3 / docs / index.html

<TITLE>LibMPEG3</TITLE>

<CENTER>

<FONT FACE=HELVETICA SIZE=+4><B>Using LibMPEG3 for advanced MPEG
decoding</B></FONT><P>

<TABLE>
<TR>
<TD>
<CODE>
Author: Heroine Virtual Ltd. (Motion picture solutions for Linux without a warranty)<BR>
Harassment: broadcast@earthling.net<BR>
Homepage: heroinewarrior.com<P>

libmpeg3 is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2, or (at your option) any
later version.<P>

libmpeg3 is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.<P>


</CODE> </TD> </TR> </TABLE> </CENTER>


<H1>Table of contents</H1><P>
<A HREF="#INTRO">Intro</A><BR>
<A HREF="#BUILDING">Building the library</A><BR>
<A HREF="#USAGE">Usage</A><BR>
<A HREF="#TOC">Using tables of contents for editing</A><BR>
<A HREF="#SUBTITLES">Decoding subtitles</A><BR>
<A HREF="#UTILITIES">Using the utilities</A><BR>








<P>

<A NAME=INTRO>

LibMPEG3 decodes several MPEG standards into uncompressed data suitable
for editing and playback.<P>

libmpeg3 currently decodes:<P>

<BLOCKQUOTE>MPEG-2 video<BR>
MPEG-1 video<BR>
mp3 audio<BR>
mp2 audio<BR>
ac3 audio<BR>
MPEG-2 transport streams<BR>
MPEG-2 program streams<BR>
MPEG-1 program streams<BR>
IFO files<BR>
</BLOCKQUOTE><P>

The video output can be in many different color models and frame
sizes.  The audio output can be in twos compliment or floating point. 
Frame accurate seeking, normally impossible in transport streams, is
possible in libmpeg3 through the use of a <B>table of contents</B>. 
MPEG-2 video in YUV-422 colorspace is decodable.  Digital TV broadcasts
and DVD's can be edited using libmpeg3.  Libmpeg3 takes what is
normally a last mile distribution format and makes it editable.<P>

Because of these and other features libmpeg3 is not intended for
consumer applications but serves users who are interested in high
quality editing and footage acquisition.<P>







<A NAME=BUILDING>
<FONT FACE=HELVETICA SIZE=+4><B>Building the library</B></FONT><P>

libmpeg3 depends on the CFLAGS environment variable to get optimization
flags.  You should set it to <P>

<TT>-O3 -march=i686 -fmessage-length=0 -funroll-all-loops
-fomit-frame-pointer -malign-loops=2 -malign-jumps=2
-malign-functions=2</TT><P>

You must run <B>make</B> to build the library and should be using
Kernel 2.4.9 or later.  The makefile automatically determines
appropriate parameters and puts the library in i686/libmpeg3.a. 
Several utilities are also built.  Install the utilities by running
<B>make install</B>.<P>

Unfortunately libmpeg3 excercizes the
system more aggressively than a consumer library and this brings out
different bugs in each kernel version.<P>


2.4.9: ext3 filesystem failure<BR>

2.4.10: memory management failure when running mpeg3toc<BR>

2.4.17: memory management failure after 5 hours of decoding video<P>


As libmpeg3 is not one of the standard MPEG decoding
libraries, these utilities are unlike any you've ever seen before. 
Remember a utility is only as illegal or legal as the guy who runs
it.<P>




<A NAME=USAGE>
<H1>Usage</H1><P>


<FONT FACE=HELVETICA SIZE=+4><B>STEP 1: Verifying file compatibility</B></FONT><P>

Programs using libmpeg3 must <CODE>#include "libmpeg3.h"</CODE>.<P>

Call <CODE>mpeg3_check_sig</CODE> to verify if the file can be read by
libmpeg3.  This returns a 1 if it is compatible and 0 if it isn't.<P>











<FONT FACE=HELVETICA SIZE=+4><B>STEP 2: Open the file</B></FONT><P>

You need an <CODE>mpeg3_t*</CODE> file descriptor:<P>
<CODE>
mpeg3_t* file;
</CODE>
<P>

Then you need to open the file:<P>

<CODE>file = mpeg3_open(char *path);</CODE><P>

<CODE>mpeg3_open</CODE> returns a NULL if the file couldn't be opened
for some reason.  Be sure to check this.  Everything you do with
libmpeg3 requires passing the <CODE>file</CODE> pointer.<P>

Another way of opening a file is <P>

<CODE>mpeg3_open_copy(char *path, mpeg3_t *old_file)</CODE><P>

You need to open multiple copies of a file in realtime situations
because only one thread can access a mpeg3_t structure at a time.  The
audio and video can't read simultaneously.  The solution is not to
repeatedly call mpeg3_open but to call mpeg3_open_copy for every file
handle after the first one.  This copies tables from the first file to
speed up opening.<P>












<FONT FACE=HELVETICA SIZE=+4><B>STEP 3: Set optimization strategies</B></FONT><P>

Call <CODE>mpeg3_set_cpus(mpeg3_t *file, int cpus)</CODE> to set how
many CPUs should be devoted to video decompression.  LibMPEG3 can use
any number.<P>

Call <CODE>mpeg3_set_mmx(mpeg3_t *file, int use_mmx)</CODE> to set if
MMX is used for video.  Disabling MMX is mandatory for low bitrate
streams since it is very lossy.  By the way, lately the compiled MMX
output has been producing corrupted video.  This is a change in the way
modern compilers and CPU's handle MMX from the way it was done 4 years
ago but since modern CPU's are so fast, you're better off not using MMX
at all.<P>







<FONT FACE=HELVETICA SIZE=+4><B>STEP 4: Query the file.</B></FONT><P>

There are a number of queries for the audio components of the stream:<P>

<CODE><PRE>
int mpeg3_has_audio(mpeg3_t *file);
int mpeg3_total_astreams(mpeg3_t *file);             // Number of multiplexed audio streams
int mpeg3_audio_channels(mpeg3_t *file, int stream);
int mpeg3_sample_rate(mpeg3_t *file, int stream);
long mpeg3_audio_samples(mpeg3_t *file, int stream); // Total length
</PRE></CODE>

The audio is presented as a number of <B>streams</B> starting at 0 and
including <CODE>mpeg3_total_astreams</CODE> - 1.  Each stream contains a
certain number of <B>channels</B> starting at 0 and including
<CODE>mpeg3_audio_channels</CODE> - 1.

The methodology is first determine if the file has audio, then get
the number of streams in the file, then for each stream get the number
of channels, sample rate, and length.<P>

There are also queries for the video components:<P>

<CODE><PRE>
int mpeg3_has_video(mpeg3_t *file);
int mpeg3_total_vstreams(mpeg3_t *file);            // Number of multiplexed video streams
int mpeg3_video_width(mpeg3_t *file, int stream);
int mpeg3_video_height(mpeg3_t *file, int stream);
float mpeg3_frame_rate(mpeg3_t *file, int stream);  // Frames/sec
long mpeg3_video_frames(mpeg3_t *file, int stream); // Total length
int mpeg3_colormodel(mpeg3_t *file, int stream);
</PRE></CODE>

The video behavior is the same as with audio, except that video has no
subdivision under <B>streams</B>.  Frame rate is a floating point
number of frames per second.<P>

<TT>mpeg3_colormodel</TT> returns either MPEG3_YUV420P or
MPEG3_YUV422P.  MPEG3_YUV422P is only encountered in high quality video
not available in any consumer distribution medium.<P>





<FONT FACE=HELVETICA SIZE=+4><B>STEP 5: Seeking to a point in the file</B></FONT><P>

Each audio stream and each video stream has a position in the file
independant of each other stream.  A variety of methods are available
for specifying the position of a stream: <B>byte offset, frame,
sample</B>.  Which method you use depends on whether you're seeking
audio or video and whether you have a table of contents for the
stream.<P>

The preferred seeking method if you're writing a player is:<P>

<CODE><PRE>
int mpeg3_seek_byte(mpeg3_t *file, int64_t byte);
int64_t mpeg3_tell_byte(mpeg3_t *file);
</PRE></CODE>

This seeks all tracks to an absolute byte offset in the file.  The
byte offset is from 0 to the result of:<P>

<CORE><PRE>
mpeg3_get_bytes(mpeg3_t *file)
</PRE></CODE>


The alternative to byte seeking is <B>frame or sample seeking</B>. 
Frame seeking is only possible if a <B>table of contents</B> exists. 
The <B>mpeg3toc</B> that comes with libmpeg3 creates tables of contents
from MPEG 1 & 2 streams.  Sample seeking is only possible if the stream
is fixed bitrate audio.  The audio seeking is handled by:<P>

<CODE><PRE>
int mpeg3_set_sample(mpeg3_t *file, long sample, int stream);    // Seek
long mpeg3_get_sample(mpeg3_t *file, int stream);    // Tell current position
</PRE></CODE>

and the video seeking is handled by:<P>

<CODE><PRE>
int mpeg3_set_frame(mpeg3_t *file, long frame, int stream); // Seek
long mpeg3_get_frame(mpeg3_t *file, int stream);            // Tell current position
</PRE></CODE>


You can either perform percentage seeking or absolute byte seeking but
not both on the same file handle.  Once you perform either method, the
file becomes configured for that method.<P>

If you're in byte seeking mode and you want the current time stamp in
the file you can't use mpeg3_get_frame or mpeg3_get_sample because you
don't know the total length in the desired units.  The
<CODE>mpeg3_audio_samples</CODE> and <CODE>mpeg3_video_frames</CODE>
commands don't work in percentage seeking either.  Instead use

<CODE><PRE>
double mpeg3_get_time(mpeg3_t *file);
</PRE></CODE>

which gives you the last timecode read in seconds.  The MPEG standard
specifies timecodes being placed in the streams.  Now you know the
absolute byte position in the file and the current time stamp, enough
to update a progress bar or a text box.<P>

Finally, there is a way to seek to the previous frame of video:


<CODE><PRE>
int mpeg3_previous_frame(mpeg3_t *file, int stream);
</PRE></CODE>

Because MPEG 1 & 2 are really hairy, the set commands won't do much
good for playing backwards.  mpeg3_previous_frame does some tricks to
seek to the previous frame.  Next you have to call a read_frame
command to read it.
<P>





<FONT FACE=HELVETICA SIZE=+4><B>STEP 6: Read the data</B></FONT><P>

<I>To read <B>audio</B> data use:</I><P>

<CODE><PRE>
int mpeg3_read_audio(mpeg3_t *file, 
                float *output_f,      // Pointer to pre-allocated buffer of floats
                short *output_i,      // Pointer to pre-allocated buffer if int16's
                int channel,          // Channel to decode
                long samples,         // Number of samples to decode
                int stream);          // Stream containing the channel
</PRE></CODE>

This decodes a buffer of sequential floats or int16's for a single
channel, depending on which *output... parameter has a nonzero
argument.  To get a floating point buffer pass a pre-allocated buffer
to <CODE>output_f</CODE> and NULL to <CODE>output_i</CODE>. To get an
int16 buffer pass NULL to <CODE>output_f</CODE> and a pre-allocated
buffer to <CODE>output_i</CODE>.  Alternatively you can pass NULL to
both buffer arguments and the decoder won't render anything.<P>

After reading an audio buffer, the current position in the one stream
is advanced.  Remember that if you're using percentage seeking you
can't call <TT>mpeg3_set_sample</TT> to rewind and read every channel. 
How then, do you read more than one channel of audio data?  Use

<CODE><PRE>
mpeg3_reread_audio(mpeg3_t *file, 
                float *output_f,      /* Pointer to pre-allocated buffer of floats */
                short *output_i,      /* Pointer to pre-allocated buffer of int16's */
                int channel,          /* Channel to decode */
                long samples,         /* Number of samples to decode */
                int stream);
</PRE></CODE>

to read each remaining channel after the first channel.<P>







<I>To read <B>video</B> data there are two methods.  RGB frames or YUV
frames.  To get an RGB frame use:</I> <BR>

<CODE><PRE>
int mpeg3_read_frame(mpeg3_t *file, 
                unsigned char **output_rows, // Array of pointers to the start of each output row
                int in_x,                    // Location in input frame to take picture
                int in_y, 
                int in_w, 
                int in_h, 
                int out_w,                   // Dimensions of output_rows
                int out_h, 
                int color_model,             // One of the color model #defines given above.
                int stream);
</PRE></CODE>

The video decoding works like a camcorder taking copies of a movie
screen.  The decoder "sees" a region of the movie screen defined by
<CODE>in_x, in_y, in_w, in_h</CODE> and transfers it to the frame
buffer defined by <CODE>**output_rows</CODE>.  The input values must be
within the boundaries given by <CODE>mpeg3_video_width</CODE> and
<CODE>mpeg3_video_height</CODE>.  The size of the frame buffer is
defined by <CODE>out_w, out_h</CODE>.  Although the input dimensions
are constrained, the frame buffer can be any size.<P>

<CODE>color_model</CODE> defines which RGB color model the picture
should be decoded to and the possible values are given in
<B>libmpeg3.h</B>.  The frame buffer pointed to by
<CODE>output_rows</CODE> must have enough memory allocated to store the
color model you select.<P>

<B>You must allocate 4 extra bytes in the last output_row.</B>  This is
scratch area for the MMX routines.<P>

<CODE>mpeg3_read_frame</CODE> advances the position in the one stream by 1 frame.<P>

<I>To read YUV frames use one of two methods:</I><BR>

<CODE><PRE>
int mpeg3_read_yuvframe(mpeg3_t *file,
                char *y_output,
                char *u_output,
                char *v_output,
                int in_x,
                int in_y,
                int in_w,
                int in_h,
                int stream);
</PRE></CODE>

The behavior of in_x, in_y, in_w, in_h is identical to mpeg3_read_frame
except here you have no control over the output frame size.  <B>You
must allocate in_w * in_h for the y_output, and in_w * in_h / 4 for the
u_output and v_output.</B>

While <B>mpeg3_read_yuvframe</B> allows cropping of letterbox it still
requires one memcpy.  A faster alternative is:<P>

<CODE><PRE>
int mpeg3_read_yuvframe_ptr(mpeg3_t *file,
                char **y_output,
                char **u_output,
                char **v_output,
                int stream);
</PRE></CODE>

This redirects a *y_output, *u_output, and *v_output pointer to the
scratch buffer that decoding took place in.  Since MPEG is temporal
compression, there is always a buffer containing the last decoder
output.<P>




For professional use the library can decode YUV 4:2:2 video in addition
to YUV 4:2:0.  This variable is determined at encoding time, won't
affect the usage of <B>mpeg3_read_frame</B> but you do need an extra
function call in order to use <B>mpeg3_read_yuvframe</B>.  To determine
the encoding of the video stream use

<CODE><PRE>
mpeg3_colormodel(mpeg3_t *file, int stream)
</PRE></CODE>

This returns either <B>MPEG3_YUV420P</B> or <B>MPEG3_YUV422P</B>.  The
output buffers and the YUV to RGB conversion for
<B>mpeg3_read_yuvframe</B> must be adjusted for the higher sampling of
MPEG3_YUV422P.  When using <B>mpeg3_read_yuvframe_ptr</B> you don't
need to adjust any output buffers.<P>

<FONT FACE=HELVETICA SIZE=+4><B>Synchronizing video with audio</B></FONT><P>

To synchronize video with audio in realtime you need to sometimes delay
the video and sometimes drop frames.  It's easy to calculate the number
of frames to drop but if you're using percentage seeking you can't
calculate the exact percentage to seek forward by.  Instead call <P>

<CODE>mpeg3_drop_frames(mpeg3_t *file, long frames, int stream);</CODE><P>

This skips <CODE>frames</CODE> frames from the current position whether
in percentage seeking or absolute seeking.<P>


<FONT FACE=HELVETICA SIZE=+4><B>STEP 7: Close the file</B></FONT><P>

Be sure to close the file with <CODE>mpeg3_close(mpeg3_t *file)</CODE>
when you're done with it.









<P>
<A NAME=TOC>
<FONT FACE=HELVETICA SIZE=+4><B>Using tables of contents for editing</B></FONT><P>

In 1985 everyone watched Smurfs but one guy watched Robotech.  In 1990
everyone watched Teenage Mutant Ninja Turtles but one guy watched
Transformers.  In 1995 everyone watched Pokemon but one guy watched
Behind the Scenes.  Now everyone wants handheld organizers but one guy
wants MPEG editors.  For the wierdos who always looked at the camera
rig instead of the celebrity, libmpeg3 supports a way of seeking to an
exact frame or sample in any kind of MPEG encapsulation format for
editing.<P>

A table of contents must be built for any footage to be edited with
libmpeg3.  Run <TT>mpeg3toc &lt;mpeg stream> &lt;output table of contents></TT><P>


For editing DVD footage, the mpeg stream argument should be the ifo
file belonging to the title set to be edited.  This utility reads
through every file comprising the mpeg stream and records the offset of
every 65536th sample and every keyframe so it can be pretty slow.<P>

The resulting table of contents file should be passed to mpeg3_open
and mpeg3_open_copy just like a normal file.  The only difference is
frame seeking of video is available.<P>










<A NAME=SUBTITLES
<FONT FACE=HELVETICA SIZE=+4><B>Decoding subtitles</B></FONT><P>

Subtitles are encoded in DVD Program streams as 4 color bitmaps.  They
require invoking mpeg3_open with the IFO file corresponding to the VOB
files in order to play the VOB files.  The IFO file contains the color
palette.




<A NAME=UTILITIES
<FONT FACE=HELVETICA SIZE=+4><B>USING THE UTILITIES</B></FONT><P>