Inside Secrets

Not really new as it was previously called CtxBSOD v2.1 but was renamed to better show its purpose. In addition to renaming I added a command line option to dump a system remotely or from a command line locallywithout using its GUI interface. The main motivation for me to write this tool was the absence of similar tools for 64-bit Windows. SystemDumpcan dump a 64-bit server too!

You can download it form Citrix support web site (requires free registration):

http://support.citrix.com/article/CTX111072

Main features:

• The tool has both GUI and command line interfaces.
• Youcan type a message/text (or copy it from clipboard) before forcing a memory dump. This message is saved in a dump and a support engineer can read it after loading the dump in WinDbg.exe. This is implemented to encourage writing the symptoms and conditions explaining why the dump has to be forced.
• The tool can stay on top of any window (if you need this to quickly dump the server after a reproduction or during the process of an activity).
• It is supplied with Program Database (PDB) symbols for the driver (32-bit and 64-bit) which is useful when you want to have all symbols present on the bug check thread.
• The bug check clearly shows that the dump is manually generated.
• The tool can force a memory dump on both 32-bit and 64-bit platforms.
• Before forcing a fatal error on a server, the tool warns about potential damaging consequences: Users are disconnected and all the data which is not saved will be lost. It asks for a confirmation.
• You can specify a period of time (in minutes) when to force a memory dump.

The latter featureis implemented entirely in kernel. Additional commandthat not covered in the article is

>SystemDump.exe abort

allows you to abort the action if you ran the toolusing command line options.

I attached the UML component diagram showing the architecture of this tool.I recently developed apresentation about device drivers architectureand Citrix kernel drivers where I used this tool asone of examples.

- Dmitry Vostokov -

4 Responses to “Reverse Engineering Citrix ThinWire”

1. Toby Opferman Says:
   November 5th, 2006 at 11:00 am

   The diagram is slightly inaccurate. The video driver only ever directly links against WIN32K and would not call VideoPort.Sys directly without going through WIN32K. There is also another Citrix related path missing which I would not be able to mention in a public forum.

2. Dmitry Vostokov Says:
   November 5th, 2006 at 2:45 pm

   Thanks for pointing this out! This was adopted from the picture in OSR article:

   “From Andy’s Bookshelf: So you Wanna Write a Video Driver”
   Figure 2 - DDI and Miniport Drivers

   http://www.osronline.com/article.cfm?id=66

   I’ll add a label to that link: “via win32k!EngDeviceIoControl” similar I’ve done for vdtw30.dll and termdd.sys interaction

   Thanks,
   Dmitry

Bugchecks depicted: IRQL_NOT_LESS_OR_EQUAL

Duringkernel debugging training I’mprovidingI came up to the idea to use UML sequence diagrams to depict various Windows kernel behavior including bugchecks.Today I start with bugcheck A. To understand why this bugcheck is needed you need to understand the difference between thread scheduling and IRQL and I use the following diagram to illustrate it:

Then I explain interrupt masking:

Next I explain thread scheduling (thread dispatcher):

And finally here is the diagram showing whenbugcheck Ahappens and what would happenifit doesn’t exist:

This bugcheckhappens in the trap handler and IRQL checking before bugcheck happens in memory manager as you can see from the dump example below. There is no IRQL checking in disassembled handler so it must be in one of Mm functions:

BugCheck A, {3, 1c, 1, 8042d8f9}
0: kd> k
nt!KiTrap0E+0×210
driver!foo+0×209
0: kd> u nt!KiTrap0E nt!KiTrap0E+0×210
nt!KiTrap0E:
…
8046b05e call nt!MmAccessFault (8044bfba)
…
8046b189 call dword ptr [nt!_imp__KeGetCurrentIrql (8040063c)]
8046b18f lock inc dword ptr [nt!KiHardwareTrigger (80470cc0)]
8046b196 mov ecx,[ebp+0×64]
8046b199 and ecx,0×2
8046b19c shr ecx,1
8046b19e mov esi,[ebp+0×68]
8046b1a1 push esi
8046b1a2 push ecx
8046b1a3 push eax
8046b1a4 push edi
8046b1a5 push 0xa
8046b1a7 call nt!KeBugCheckEx (8042c1e2)

- Dmitry Vostokov -