TraceEventProgrammersGuide.rar_ETW_tracing

The TraceEvent Library Programmers

Guide

Vance Morrison

Contents

Introduction: Strongly Typed (Semantic) Logging ........................................................................................3

Basic Logging Architecture ...........................................................................................................................3

Multi-player ..............................................................................................................................................4

Asynchronous ...........................................................................................................................................5

Quick Start Example .....................................................................................................................................5

Publication vs. Registration ....................................................................................................................13

Review of the Fundamental TraceEvent Architecture................................................................................14

What You Can Do With the TraceEvent Library..........................................................................................14

ETW Limitations..........................................................................................................................................16

Next Step: Code Samples............................................................................................................................17

Diagnostic Techniques: Using PerfView......................................................................................................17

Full dumps of ETW events in PerfVIew...................................................................................................17

PerfView’s /KeepAllEvents feature.........................................................................................................18

Debugging EventSource Authoring.........................................................................................................18

IObservable support in TraceEvent ............................................................................................................18

Source Code Support ..............................................................................................................................19

Best Practices for Versioning ETW Events ..................................................................................................19

On the fly filtering AND logging using ETWReloggerTraceEventSource .....................................................19

WPP logging Support in the TraceEvent Library .........................................................................................19

Introduction: Strongly Typed (Semantic) Logging

As long as there have been programs, developers have used logging system to diagnosis functional and

performance problems. Logging systems break down into two broad categories:

1. Weakly typed: The logging system only knows how to log strings and thus all data is converted

into strings before it is logged. Printf logging is an example of this. These are the simplest

logging systems and are a fine choice for ad hoc investigations by humans, however they suffer

c. Optional meta-data about the event (groups it belongs to, its verbosity etc.).

Because the events are strongly typed, the consumer of the logging data can assume that the events

conform to the schema which makes processing the events considerably easier and less fragile, as well

as improves the speed and compactness of logging as well as the post processing (no need to print to

strings and then parse back). The Event Tracing for Windows (ETW) system built into the Windows

Operating System is an example of a strongly typed logging system (see Introduction to ETW for more)

In general strongly types logging systems make sense in the same places where strongly typed

programming languages make sense, when

1. When the data is likely to be consumed by other automation rather than simply viewed by

humans.

Thus strongly typed eventing is a natural fit for the cloud.

The TraceEvent library is NuGET package that provides a set of .NET Runtime classes that make it easy to

control and consume (parse) the strongly typed Event Tracing for Windows (ETW) events. It is intended

to be used in conjunction with the System.Diagnostics.Tracing.EventSource class (which can generate

ETW events) to form a complete, end-to-end, strongly typed logging system.

Basic Logging Architecture

There are three basic parts to and ETW based logging system as shown in the figure below

1. The Event Session (represents the entity controlling the logging). The session has the ability

to tell the providers of events to start and stop logging and control how verbose the logging is.

directly written to a file (for maximum efficiency) or to send it to the session itself (for on the fly

1. The Event Session uses the Microsoft.Diagnostics.Tracing.TraceEventSession class.

2. The Event Provider uses the Microsoft.Diagnostics.Tracing.EventSource class.

3. The Event Consumer uses the Microsoft.Diagnostics.Tracing.TraceEventSource class.

It is not uncommon for each of these ‘roles’ to be implemented in a separate process, but it is also

common for some of the roles to be combined (most commonly the Session and the Consumer are in

the same process).

Because the logging system is strongly typed, each event conforms to a particular schema and the event

consumer needs to know this schema to decode the event data. The ETW system calls this schema the

manifest and one representation of this is an XML file, but it also is stored in a binary form for efficient

processing. One of the responsibilities of an event provider is to publish a manifest for the events that

each of which has a manifest for the events it might emit.

2. Multiple event sessions each of which are gathering different sets of events for different

purposes. Each of these sessions has the option of logging its events to a file or to the session

itself in real time.

3. Multiple event consumers that can process the events from a file or session. It is not

recommended however to have multiple consumers feeding from the same event session in real

time. Instead it is simpler and better to have multiple session each feeding a unique event

1. Logging is fast and scalable. Only the initial copy to the first buffer in the logging pipeline

actually delays the program. Note that this is independent of the number of sessions or

consumers. All the rest of the logging activity happens on other threads of execution and can

be parallelized (thus it scales well).

2. Logging has minimal and predictable impact on the program. There are no ‘global locks’ that

need to be taken by the provider when it logs events. Because of this it is much more likely

that behavior of threads (races) with logging on will be essentially the same (statistically

speaking) as with logging off.

3. There is the possibility of lost events. If the providers generate event faster than the file can

store them (or the real time processor can process them), then eventually events need to be