USB摄像头SDK

======================================= How to get data from a DirectShow graph ======================================= A lot of people have asked, “how do I get data out of directshow and into my application?” Because DirectShow is an “open” streaming architecture, there are several ways to answer this question, all of which have a significant (but not impossible) learning curve. Here’s a list of possible solutions, in order of complexity: 1. Use the MultIMedia Streaming APIs. They’re simple, synchronous, and they do work. 2. Learn how to write a simple transform or rendering filter that can get the data for you, put as little effort as possible into that filter, and learn how to use that filter from your application. 3. Learn all about DirectShow, write a transform or render filter, and use that filter to process your data in a more tightly integrated way. The application becomes less important and the filter becomes more important. This document is going to explain how to do #2 above. #1 and #3 are beyond the scope of this paper. MultiMedia Streaming -------------------- You can find the documentation and sample code for how to use MultiMedia streaming in released versions of the DirectShow SDK. It is already well documented there. ======================================= The DirectShow Basics you need to know ======================================= Let’s start with a very basic summary of DirectShow and what it does. DirectShow is a streaming architecture runtime. It allows the user to create and connect objects together which stream data to and through each other, in a virtual space called a “graph”. A graph means “a collection of streaming objects, usually connected to each other” (a graph can contain unconnected objects, but those graphs aren’t very useful). The different kinds of DirectShow objects ----------------------------------------- In DirectShow there are three types of objects, collectively called “filters”, which are: “sources”, “transforms”, and “renderers”. Source objects create data and push it upon the next object it’s connected to. Transform objects both receive and transmit data, sometimes on more than one thread. Renderers receive data only. Pins on a DirectShow Filter --------------------------- Every DirectShow object (called a filter) has one or more “pins”, which represent a connection site. Each pin can connect to one and only one other pin. When two pins are connected to each other, it is assumed that data will flow from one pin to the other when the graph is in a state where data is supposed to flow. The Different States of a Graph ------------------------------- A graph (a collection of filters) has four different possible states it can be in: Stopped, Paused, Running, or “Transitioning”. Transitioning state means when the graph is going from one state to another, but it hasn’t quite finished yet because of the multithreaded nature of the sources. Paused state, with only several exceptions, is viewed as exactly the same to an individual filter as the running state. Only rendering filters and live capture source filters care about whether the graph is paused or running. Live capture filters will stop sending new frames of data when paused. Rendering filters will stop “displaying” data when paused and won’t allow any new information to come in, through the use of multithreading and event handle techniques. Other filters don’t really deal with the differences between states, but they do have to deal with the protocol which must be obeyed by filters when the graph transitions from one state to another. When the graph’s state transitions, messages will be sent both “upstream” and “downstream”, which must be handled correctly. You’ll need to read the documentation and probably a bit of the base class filters to understand this fully. Multithreading -------------- It is a must to partially understand multithreading when using any part of DirectShow besides MultiMedia Streaming. MultiMedia Streaming makes a great attempt to keep the user from having to deal with multithreading issues. But if you plan on writing any filter (source, transform, or renderer), be prepared to read the DirectShow docs, and be prepared to deal with multithreading, critical sections, autolocking, not using global data, and event handles. You can easily write a filter without dealing with these issues, but the filter will have a very high chance of locking up or performing incorrectly and creating a hassle for you and users of the filter. Don't be overwhelmed by this statement. Once you understand the issues, development will become much easier. The application You will need to understand multithreading when writing filters for DirectShow. Even if you write an application that just uses DirectShow in the simplest manner, you must understand (at the very least) that the data is travelling through the graph on different threads than the application thread. Source Filters In general, for each output pin that a source filter provides, a thread is created when the graph is in the paused or running state. That thread takes buffers which were created by it or another filter, fills them up, and delivers them to the next filter it’s connected to. Transform Filters Each transform filter has the option to create a new thread for any (or each) of its output pins to push data upon when the filter goes into paused or run mode. Most transform filters do not do this. In fact, unless necessary, it’s advised not to do this. Splitter filters, which parse interleaved data from an upstream source and separate the data into more basic (like audio or video) streams, normally do create threads for their output pins. This is because when one stream becomes two, you don’t want one stream which is blocked (for some unknown reason) to prevent the other stream from delivering data. Rendering Filters Nobody has ever heard of a render filter which creates it’s own thread. Thank goodness this is simple. Buffering Negotiations ---------------------- Each filter’s pin goes through a set of negotiations when it connects to another pin. The two pins must commonly decide on the type of media they will deliver and receive, the size of the buffers they will create or share, the number of buffers available for use, and the memory block size the memory must be aligned with. A pin when it’s dealing with data will be using buffers that either it created, or somebody else created. Put simply: there is a complex sequence of actions that are performed when two pins connect in order to get them to agree. This is described in the DirectShow docs. Why is all this important? Many potential users of DirectShow have been discouraged by the learning curve required just to get data out of a compressed file. MultiMedia Streaming can be used, but sometimes it may not be enough. Even if you know COM and C++, DirectShow still has a lot of rules that must be learned. ======================================= The Sample Grabber ======================================= The easiest way to get data out of a DirectShow graph, if you’re not going to use MultiMedia Streaming, is probably to write your own TransInPlace filter, a sub-variety of a Transform filter. Then connect this filter to the desired stream of data you wish to monitor, and then run, pause, seek, or otherwise control the graph. The data, as it passes through the transform filter, can be manipulated however you want. We call this kind of filter, a “sample grabber”. Microsoft released a limited-functionality sample grabber with DX8.0. This filter is limited because it doesn’t deal with DV Data or mediatypes with a format of VideoInfo2. It doesn’t allow the user to receive p