Bluescreen Guide

[redhawk]

Hello Computer Hope
ive been reading the forums here for quite a while now. and though ive not made many posts ive found aloth of awnsers searching these forums over the last 2 years. onely posted and asked for help when needed.
felt like it might be my time to contribute so i hereby post a guide to bluescreen debugging ive put together.
i wrote this a while ago for reference to my collueges where i work. and i hope it will help ppl here aswell.

-------------------------------------------------------------------------------------------------------------

Guide - Bluescreen Problem Determination

introduction

a bluescreen or BSOD ( Bluescreen of death) can happen for a great number of reasons. faulty drivers, corrupt system files or hardware errors. to accuratly find out what realy happend you need to debugg the memory dumps created in c:\%SYSTEMROOT%\minidump (ex c:\windows\minidump\ ).

commonly the files are called minidumps. these files are used by microsoft devolpers to determin the error that occured but of course we want to be abel to debugg these files ourself instead of calling microsoft

The Minidumps

as i said erlier the minidumps are stored in c:\%SYSTEMROOT%\minidumps in the following format

miniMMDDYY-XX.dmp

so for exampel..

Mini121007-01.dmp should be read
Minidump 10/12 2007 - bluescreen number 1 (this day) .dmp

To Debugg the minidumps...



you need to install and configure windows debugging tools
this can be done from this link

http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx

when you have downloaded and installed this software you need to configure the symbol path in the debugger (windbg.exe).
to do this open windbg from the start menu.

in windbg open File -> Symbol File Path and copy the following into the window SRV*c:\symbols*http://msdl.microsoft.com/download/symbols

next we need to set the image file path

in windbg open FIle -> Image File Path either use "browse" or type in the searchpath(in CAPS!) to you i386 folder (eg C:\WINDOWS\I386 )

now the debugger is ready to use.


how does this work?
first of you need the minidump files, copy them to the computer you installed windbg on.

in windbg open a minidump from file -> Open crash dump

now the debugging process will start.

how do i read the output?

here follows an exampel from one debugg of a minidump where the GFX driver was corrupt. my comments are marked with a "//" in front of.


Microsoft (R) Windows Debugger Version 6.8.0004.0 X86
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\apps\debugg\grr\Mini121807-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

Symbol search path is: SRV*c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows XP Kernel Version 2600 (Service Pack 2) UP Free x86 compatible
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 2600.xpsp_sp2_gdr.070227-2254
Kernel base = 0x804d7000 PsLoadedModuleList = 0x8055a620
Debug session time: Tue Dec 18 07:57:23.834 2007 (GMT+0)
System Uptime: 0 days 0:02:36.414
Loading Kernel Symbols
...
Loading User Symbols
Loading unloaded module list
.....
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

Use !analyze -v to get detailed debugging information.
// the !analyze -v should apper in blue, you can click this text for more information

BugCheck 100000EA, {86cc2110, 86c824f0, f7998cb4, 1}

Probably caused by : ati2cqag.dll ( ati2cqag+8a31 )
Followup: MachineOwner

//here it becomes abvious that ati2cqag.dll is the file that couses the crash, ( this is a GFX driver, if
//you are unsure of what ati2cqag.dll does remember google is a verry nice friend =)

*** WARNING: Unable to verify timestamp for ati2cqag.dll
*** ERROR: Module load completed but symbols could not be loaded for ati2cqag.dll
Version too small


Probably caused by : ati2cqag.dll ( ati2cqag+8a31 )

Followup: MachineOwner
---------




kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

THREAD_STUCK_IN_DEVICE_DRIVER_M (100000ea)
The device driver is spinning in an infinite loop, most likely waiting for
hardware to become idle. This usually indicates problem with the hardware
itself or with the device driver programming the hardware incorrectly.
If the kernel debugger is connected and running when watchdog detects a
timeout condition then DbgBreakPoint() will be called instead of KeBugCheckEx()
and detailed message including bugcheck arguments will be printed to the
debugger. This way we can identify an offending thread, set breakpoints in it,
and hit go to return to the spinning code to debug it further. Because
KeBugCheckEx() is not called the .bugcheck directive will not return bugcheck
information in this case. The arguments are already printed out to the kernel
debugger. You can also retrieve them from a global variable via
"dd watchdog!g_WdBugCheckData l5" (use dq on NT64).
On MP machines it is possible to hit a timeout when the spinning thread is
interrupted by hardware interrupt and ISR or DPC routine is running at the time
of the bugcheck (this is because the timeout's work item can be delivered and
handled on the second CPU and the same time). If this is the case you will have
to look deeper at the offending thread's stack (e.g. using dds) to determine
spinning code which caused the timeout to occur.
Arguments:
Arg1: 86cc2110, Pointer to a stuck thread object. Do .thread then kb on it to find
the hung location.
Arg2: 86c824f0, Pointer to a DEFERRED_WATCHDOG object.
Arg3: f7998cb4, Pointer to offending driver name.
Arg4: 00000001, Number of times "intercepted" bugcheck 0xEA was hit (see notes).

Debugging Details:
------------------

//above is the real explenation of what happened, most of it is "mumbojumbo" to the common man, but (as been told to me) candy for devolpers

Version too small



FAULTING_THREAD: 86cc2110

DEFAULT_BUCKET_ID: GRAPHICS_DRIVER_FAULT // cannot be more clear can it?

CUSTOMER_CRASH_COUNT: 1

BUGCHECK_STR: 0xEA

PROCESS_NAME: csrss.exe // the program or process that crached becouse of ati2cqag.dll mentioned earlier

LAST_CONTROL_TRANSFER: from 806f2a5c to 804db8f3

STACK_TEXT:
f6a5677c 806f2a5c 00000000 00000002 bfa13a31 nt!KiDispatchInterrupt+0x7f
f6a5677c bfa13a31 00000000 00000002 bfa13a31 hal!HalpDispatchInterrupt2ndEntry+0x1b
WARNING: Stack unwind information not available. Following frames may be wrong.
f6a567f8 00000000 f6a568d8 e1cca598 00000401 ati2cqag+0x8a31


STACK_COMMAND: .thread 0xffffffff86cc2110 ; kb

FOLLOWUP_IP:
ati2cqag+8a31
bfa13a31 ??

SYMBOL_STACK_INDEX: 2

SYMBOL_NAME: ati2cqag+8a31

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: ati2cqag //this text should be marked blue aswell you can click on it for more information aswell

IMAGE_NAME: ati2cqag.dll

DEBUG_FLR_IMAGE_TIMESTAMP: 411096bd

FAILURE_BUCKET_ID: 0xEA_IMAGE_ati2cqag.dll_DATE_2004_08_04

BUCKET_ID: 0xEA_IMAGE_ati2cqag.dll_DATE_2004_08_04

Followup: MachineOwner
---------


now in this case update of the videodriver solved the problem, its a bit of detective work as you can see.


Common errors ive found using this method

first of, ANY and i mean ANY file witch ends with *.dmp or anything similiar is worth debugging. sometimes even "windows error reports" generates thouse files.
(usualy thouse kind of minisumps are stored in the temp folder, the searchpath should be in the "more details" section of the error message)
that being said. these are the most common errors ive encounted


FAULTY VIDEO DRIVERS

solution: no matter if its Nvidia/ATI or the intel based gfx drivers, thay all are usualy solved by updating the drivers.

FAULTY WLAN DRIVERS

usually the intel driver named wXXnXX.dll or .sys (the XX stands for diferent numbers) this is solved by updating the wlan drivers,
BUT! if you are using a laptop be sure to read the release log of the latest WLAN drivers. some WLAN drivers wont work properly if not other components arent updated aswelll!! this is always metioned in the wlan drivers release log witch usualy comes with the driver package as a readme.txt .

MEMORY CORRUPTION

if you get a whole bunch of minidumps with diffrent drivers and processes mentioned randomly it could becouse of faulty RAM memory (memory leak), a memory test is in order, the memory diagnostic tool i recommend is http://www.memtest86.com/ it should be run up to 7-12 cycels for most accuracy.

FAULTY WINDOWS SYSTEM FILES
this is verry verry dangerus stuff to play with if you dont know what you are doing. i suggest you read about the faulty system file on http://www.msdn.com before you try taking action at all. if decide you are brave and sure about what you are doing you can try to either replace the file or do a system file check ( start - run - type: sfc /scannow ) . Always bakup or rename the old file. if this does not help the onely solution is to do a reinstallation of windows on the system or close the service in computer manager that uses the file.

replacing corrupt system files through msconfig

i was a bit supprised when discovering some colluges of mine diddnt know about this feture in windows profesisonall, from run type msconfig -> click expand file. now you should have 3 fields.

File to restore: "here is where you type in the name of the windows system file you want to restore"
Restore from: "here is where you type in the path to a i386 dir. you can use windows installation cd. or the c:\windows\i386\ dir but i would not recommend that. "

save file in: "here is where you put in the search part to where the corrupted file is on your drive"

when all this is filled in correctly you click "expand" and then ok to close msconfig. now you restart and voila. you replaced a windows system file



although solving the problem without reinstalling windows has its benefits.

solving the problem leads to:
    1) an increase in knowledge
    2) a more stable environment
    3) prevention of future issues

I would encourage everyone to really examine issues (not just BSODs) and try to solve the issue as opposed the symptom.

----------------------------------------

[patio]

Nice contribution...

[davidoc]

What an Interesting and informative post Redhawk....Thank you.

[Broni]

Nice post

[redhawk]

bump! updated! - added a replacing windows system files section