|
前言FFmpeg的优秀在于它的功能强大和良好的系统框架，而滤镜就是其中之一。ffmpeg的自带滤镜不但能对视频进行裁剪，添加logo，还能将多个滤镜组全使用。 更妙之处在于它还可以方便地添加自己定义的各种滤镜。这种可扩展性对于实际应用来说就颇有价值了。 闲言少述，书归正传！ 本文第一部分是我对wiki上的一篇教程的翻译和解释，但是它并没有讲解如何将写好的滤镜添加到ffmpeg中编译并运行。 第二部分是我自己实践了的如何将滤镜添加进ffmpeg中进行编译和运行(版本ffmpeg-0.8.5)。 最后一部分附上示例用的滤镜源码(版本ffmpeg-0.8.5)。 Chapter 1 : FFmpeg filter HOWTO (本章节引自http://wiki./index.php?title=FFmpeg_filter_howto) This page is meant as an introduction of writing filters for libavfilter. This is a work in progress, but should at least point you in the right direction for writing simple filters. Contents 1. Definition of a filter 1.1 AVFilter 1.2 AVFilterPad 2. Picture buffers 2.1 Reference counting 2.2 Permissions 3. Filter Links 4. Writing a simple filter 4.1 Default filter entry points 4.2 The vf_negate filter Definition of a filter AVFilter All filters are described by an AVFilter structure. This structure
gives information needed to initialize the filter, and information on
the entry points into the filter code. This structure is declared in libavfilter/avfilter.h: 滤镜的数据结构体(包括需要初始化滤镜变量，滤镜的函数入口点等)定义在libavfilter/avfilter.h中:
The query_formats function sets the in_formats member of connected output links, and the out_formats member of connected input links, described below under AVFilterLink. query_formats 函数设置滤镜输入，输出图像的格式,如YUV420,YUV422等。 Let's take a quick look at the AVFilterPad structure, which is used
to describe the inputs and outputs of the filter. This is also defined
in libavfilter/avfilter.h: AVFilterPad结构体用于描述滤镜的输入和输出。
The actual definition in the header file has doxygen comments describing
each entry point, its purpose, and what type of pads it is relevant
for. These fields are relevant for all pads:
Fields only relevant to input pads are:
Fields only relevant to output pads are:
All pictures in the filter system are reference counted. This means
that there is a picture buffer with memory allocated for the image data,
and various filters can own a reference to the buffer. When a
reference is no longer needed, its owner frees the reference. When the
last reference to a picture buffer is freed, the filter system
automatically frees the picture buffer. 滤镜系统中所有的图像都是有计数的引用。意即，有一个分配了存储空间的图像buffer,而每个滤镜可以有自己的引用指向这个图像buffer。当这个引用不再需要时，拥有这个引用的滤镜将释放这个引用。当图像buffer的最后一个引用也被释放时，滤镜系统将自动释放这个图像buffer。 The upshot of multiple filters having references to a single picture
is that they will all want some level of access to the image data. It
should be obvious that if one filter expects to be able to read the
image data without it changing that no other filter should write to the
image data. The permissions system handles this. 使用组合滤镜的结果就是会有多个引用指向同一个图像buffer，而且它们都希望对buffer中的图像有相同的访问权限。这时就会产生冲突啦，例如，当一个滤镜有读取buffer中图像数据的权限时，肯定不希望有别的滤镜同时在改定buffer中的图像数据。解决这种冲突就需要授权系统来处理。 In most cases, when a filter prepares to output a frame, it will
request a buffer from the filter to which it will be outputting. It
specifies the minimum permissions it needs to the buffer, though it may
be given a buffer with more permissions than the minimum it requested. 通常，当滤镜准备输出一帧数据时，它会向滤镜系统请求一个输出buffer,这时，滤镜系统会返回一个buffer结该滤镜并将这个buffer授予最小的权限。 When it wants to pass this buffer to another filter as output, it
creates a new reference to the picture, possibly with a reduced set of
permissions. This new reference will be owned by the filter receiving
it. 当当前滤镜要将图像buffer输出给下一个滤镜时，该滤镜会创建一个指向图像buffer的新引用，这个buffer的授权可能会减小。而这个新的引用会被下一个滤镜接收。 So, for example, for a filter which drops frames if they are similar to the last frame it output, it would want to keep its own reference to a picture after outputting it, and make sure that no other filter modified the buffer either. It would do this by requesting the permissions AV_PERM_READ|AV_PERM_WRITE|AV_PERM_PRESERVE for itself, and removing the AV_PERM_WRITE permission from any references it gave to other filters. 这段没搞明白什么意思 The available permissions are:
A filter's inputs and outputs are connected to those of another filter through the AVFilterLink structure:
The src and dst members indicate the filters at the source and destination ends of the link, respectively. The srcpad indicates the index of the output pad on the source filter to which the link is connected. Likewise, the dstpad indicates the index of the input pad on the destination filter. The in_formats member points to a list of formats supported by the source filter, while the out_formats member points to a list of formats supported by the destination filter. The AVFilterFormats structure used to store the lists is reference counted, and in fact tracks its references (see the comments for the AVFilterFormats structure in libavfilter/avfilter.h for more information on how the colorspace negotiation is works and why this is necessary). The upshot is that if a filter provides pointers to the same list on multiple input/output links, it means that those links will be forced to use the same format as each other. When two filters are connected, they need to agree upon the dimensions of the image data they'll be working with, and the format that data is in. Once this has been agreed upon, these parameters are stored in the link structure. The srcpic member is used internally by the filter system, and should not be accessed directly. The cur_pic member is for the use of the destination filter. When a frame is currently being sent over the link (ie. starting from the call to start_frame() and ending with the call to end_frame()), this contains the reference to the frame which is owned by the destination filter. The outpic member is described in the following tutorial on writing a simple filter. Because the majority of filters that will probably be written will take exactly one input, and produce exactly one output, and output one frame for every frame received as input, the filter system provides a number default entry points to ease the development of such filters.
Having looked at the data structures and callback functions involved, let's take a look at an actual filter. The vf_negate filter inverts the colors in a video. It has one input, and one output, and outputs exactly one frame for every input frame. In this way, it's fairly typical, and can take advantage of many of the default callback implementations offered by the filter system. First, let's take a look at the AVFilter structure at the bottom of the libavfilter/vf_negate.c file:
Here, you can see that the filter is named "negate," and it needs sizeof(NegContext) bytes of data to store its context. In the list of inputs and outputs, a pad whose name is set to NULL indicates the end of the list, so this filter has exactly one input and one output. If you look closely at the pad definitions, you will see that fairly few callback functions are actually specified. Because of the simplicity of the filter, the defaults can do most of the work for us. Let us take a look at the callback function it does define.
This calls avfilter_make_format_list(). This function takes as its
first parameter the number of formats which will follow as the remaining
parameters. The return value is an AVFilterFormats structure
containing the given formats. The avfilter_set_common_formats()
function which this structure is passed to sets all connected links to
use this same list of formats, which causes all the filters to use the
same format after negotiation is complete. As you can see, this filter
supports a number of planar YUV colorspaces, including JPEG YUV
colorspaces (the ones with a 'J' in the names). The config_props() on an input pad is responsible for verifying that the properties of the input pad are supported by the filter, and to make any updates to the filter's context which are necessary for the link's properties. TODO: quick explanation of YUV colorspaces, chroma subsampling, difference in range of YUV and JPEG YUV. Let's take a look at the way in which this filter stores its context:
That's right. The priv_size member of the AVFilter structure tells the filter system how many bytes to reserve for this structure. The hsub and vsub members are used for chroma subsampling, and the offY and offUV members are used for handling the difference in range between YUV and JPEG YUV. Let's see how these are set in the input pad's config_props:
This simply calls avcodec_get_chroma_sub_sample() to get the chroma
subsampling shift factors, and stores those in the context. It then
stores a set of offsets for compensating for different luma/chroma value
ranges for JPEG YUV, and a different set of offsets for other YUV
colorspaces. It returns zero to indicate success, because there are no
possible input cases which this filter cannot handle. Finally, the function which actually does the processing for the filter, draw_slice():
The y parameter indicates the top of the current slice, and the h parameter the slice's height. Areas of the image outside this slice should not be assumed to be meaningful (though a method to allow this assumption in order to simplify boundary cases for some filters is coming in the future). This sets inrow to point to the beginning of the first row of the slice in the input, and outrow similarly for the output. Then, for each row, it loops through all the pixels, subtracting them from 255, and adding the offset which was determined in config_props() to account for different value ranges. It then does the same thing for the chroma planes. Note how the width and height are shifted right to account for the chroma subsampling. Once the drawing is completed, the slice is sent to the next filter by calling avfilter_draw_slice(). Chapter 2: 将滤镜添加进ffmpeg中编译和运行 1. configure中声明使用的协议
2. libavfilter/allfilter.c中注册自定义滤镜
3. libavfilter/Makfile中添加滤镜的链接
4. configure设置
5. 编译与运行
chapter 3: 反相滤镜源码
|
|
|
