Not sure if this is the right forum for this issue, but I need some assistance figuring out why one of our Exchange 2010 Database servers has crashed a couple of times in the last month.
Log Name: System
Source: Microsoft-Windows-WER-SystemErrorReporting
Date: 2/19/2013 8:00:46 AM
Event ID: 1001
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: FMT-MSEXCHDB-01
Description:
The computer has rebooted from a bugcheck. The bugcheck was: 0x000000d1 (0x0000000000000000, 0x0000000000000002, 0x0000000000000000, 0xfffff8800195dc12). A dump was saved in: C:\Windows\MEMORY.DMP. Report Id: 021913-48468-01.
Event Xml:
<Event xmlns="
<System>
<Provider Name="Microsoft-Windows-WER-SystemErrorReporting" Guid="{ABCE23E7-DE45-4366-8631-84FA6C525952}" EventSourceName="BugCheck" />
<EventID Qualifiers="16384">1001</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2013-02-19T14:00:46.000000000Z" />
<EventRecordID>40458</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>System</Channel>
<Computer>FMT-MSEXCHDB-01</Computer>
<Security />
</System>
<EventData>
<Data Name="param1">0x000000d1 (0x0000000000000000, 0x0000000000000002, 0x0000000000000000, 0xfffff8800195dc12)</Data>
<Data Name="param2">C:\Windows\MEMORY.DMP</Data>
<Data Name="param3">021913-48468-01</Data>
</EventData>
</Event>
I'm not fluent in analyzing memory.dmp files, but I tried my best to use WinDbg to open the memory.dmp file form this crash and the results are below:
Microsoft (R) Windows Debugger Version 6.12.0002.633 X86
Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [C:\Documents and Settings\Administrator\Desktop\MEMORY\MEMORY.DMP]
Kernel Summary Dump File: Only kernel address space is available
Symbol search path is: SRV*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows 7 Kernel Version 7601 (Service Pack 1) MP (2 procs) Free x64
Product: Server, suite: Enterprise TerminalServer SingleUserTS
Built by: 7601.17944.amd64fre.win7sp1_gdr.120830-0333
Machine Name:
Kernel base = 0xfffff800`01c62000 PsLoadedModuleList = 0xfffff800`01ea6670
Debug session time: Tue Feb 19 07:57:12.959 2013 (UTC - 6:00)
System Uptime: 13 days 9:18:20.250
Loading Kernel Symbols
...............................................................
................................................................
...........
Loading User Symbols
PEB is paged out (Peb.Ldr = 000007ff`fffde018). Type ".hh dbgerr001" for details
Loading unloaded module list
...................
*******************************************************************************
*
*
* Bugcheck Analysis
*
*
*
*******************************************************************************
Use !analyze -v to get detailed debugging information.
BugCheck D1, {0, 2, 0, fffff8800195dc12}
Probably caused by : NETIO.SYS ( NETIO!NetioDereferenceNetBufferListChain+132 )
Followup: MachineOwner
---------
0: kd> !analyze -v
*******************************************************************************
*
*
* Bugcheck Analysis
*
*
*
*******************************************************************************
DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000000000000, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
Arg4: fffff8800195dc12, address which referenced memory
Debugging Details:
------------------
READ_ADDRESS: 0000000000000000
CURRENT_IRQL: 2
FAULTING_IP:
tcpip! ?? ::FNODOBFM::`string'+5b24
fffff880`0195dc12 488b01 mov rax,qword ptr [rcx]
DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT
BUGCHECK_STR: 0xD1
PROCESS_NAME: store.exe
TRAP_FRAME: fffff800019e0330 -- (.trap 0xfffff800019e0330)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=fffffa800d799600 rbx=0000000000000000 rcx=0000000000000000
rdx=fffffa800d799601 rsi=0000000000000000 rdi=0000000000000000
rip=fffff8800195dc12 rsp=fffff800019e04c0 rbp=0000000000000000
r8=fffffa800d799600 r9=00000000000000d0 r10=fffff80001e54b80
r11=fffffa800a8f7940 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na po nc
tcpip! ?? ::FNODOBFM::`string'+0x5b24:
fffff880`0195dc12 488b01 mov rax,qword ptr [rcx] ds:df40:0000=????????????????
Resetting default scope
LAST_CONTROL_TRANSFER: from fffff80001ce0569 to fffff80001ce0fc0
STACK_TEXT:
fffff800`019e01e8 fffff800`01ce0569 : 00000000`0000000a 00000000`00000000 00000000`00000002 00000000`00000000 : nt!KeBugCheckEx
fffff800`019e01f0 fffff800`01cdf1e0 : 00000000`00000001 00000000`00000001 fffffa80`0a326e80 fffffa80`0ab25030 : nt!KiBugCheckDispatch+0x69
fffff800`019e0330 fffff880`0195dc12 : fffffa80`0ab245e0 00000000`00000000 00000000`206c644d 00000000`00000000 : nt!KiPageFault+0x260
fffff800`019e04c0 fffff880`010fc872 : fffffa80`0fecf968 00000000`00000002 00000000`00000002 00000000`00000000 : tcpip! ?? ::FNODOBFM::`string'+0x5b24
fffff800`019e0510 fffff880`01947551 : 00000000`000002dc fffff800`01d7a200 fffff800`019e05dc fffff800`01e0c1de : NETIO!NetioDereferenceNetBufferListChain+0x132
fffff800`019e05e0 fffff880`02c4f44d : fffffa80`0fecf968 fffffa80`0ab245e0 00000000`00000000 00000000`00000000 : tcpip!TcpTlProviderReleaseIndicationList+0x81
fffff800`019e0610 fffff880`02c8e0cd : fffffa80`0acd3660 fffff800`019e0758 00000000`00000000 fffffa80`0bd99cc0 : afd!AfdTLReleaseIndications+0x2d
fffff800`019e0660 fffff880`02c50c11 : fffffa80`000005b4 fffffa80`0fecf968 fffffa80`0fecf920 fffffa80`0ca62170 : afd!AfdReturnBuffer+0xbd
fffff800`019e06a0 fffff880`02c4e1e3 : 00000000`000003e8 00000000`000005b4 fffffa80`0fecfa08 00000000`44f1d85a : afd!AfdUpdateConnectionForTimerWheel+0x1d1
fffff800`019e0720 fffff800`01cebc3c : fffff800`019e0860 00000000`00000000 00000000`00000001 00000000`00000000 : afd!AfdTimerWheelHandler+0x1d7
fffff800`019e07a0 fffff800`01cebad6 : fffffa80`0c08b010 fffffa80`0c08b010 00000000`00000000 00000000`00000000 : nt!KiProcessTimerDpcTable+0x6c
fffff800`019e0810 fffff800`01ceb9be : 00000a85`27133ba0 fffff800`019e0e88 00000000`04699710 fffff800`01e56488 : nt!KiProcessExpiredTimerList+0xc6
fffff800`019e0e60 fffff800`01ceb7a7 : fffffa80`0ab68dc4 fffff800`04699710 00000000`00000000 00000000`00000010 : nt!KiTimerExpiration+0x1be
fffff800`019e0f00 fffff800`01ce4105 : 00000000`00000000 fffffa80`0dc0b060 00000000`00000000 fffff880`00eb3800 : nt!KiRetireDpcList+0x277
fffff800`019e0fb0 fffff800`01ce3f1c : 00000000`00000010 00000000`00000282 fffff880`04badc08 00000000`00000000 : nt!KxRetireDpcList+0x5
fffff880`04badbe0 fffff800`01d2be93 : fffff800`01cdd540 fffff800`01cdd5ac ffffffff`ffffffff fffff800`01c493c0 : nt!KiDispatchInterruptContinue
fffff880`04badc10 fffff800`01cdd5ac : ffffffff`ffffffff fffff800`01c493c0 00000000`00000000 fffffa80`0bbb9880 : nt!KiDpcInterruptBypass+0x13
fffff880`04badc20 00000000`77410028 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiInterruptDispatchNoLock+0x1fc
00000000`0c32f790 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x77410028
STACK_COMMAND: kb
FOLLOWUP_IP:
NETIO!NetioDereferenceNetBufferListChain+132
fffff880`010fc872 4c8bb42490000000 mov r14,qword ptr [rsp+90h]
SYMBOL_STACK_INDEX: 4
SYMBOL_NAME: NETIO!NetioDereferenceNetBufferListChain+132
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: NETIO
IMAGE_NAME: NETIO.SYS
DEBUG_FLR_IMAGE_TIMESTAMP: 5034f6a0
FAILURE_BUCKET_ID: X64_0xD1_NETIO!NetioDereferenceNetBufferListChain+132
BUCKET_ID: X64_0xD1_NETIO!NetioDereferenceNetBufferListChain+132
Followup: MachineOwner
---------
Any assistance would be appreciated.
Thank you,
Matt.