M Windows NT 4 0 Setup and Debugging

M Windows NT 4. 0 Setup and Debugging Joseph West Sr Technology Specialist

Agenda 4 Setup (build overview) – Three phases of Setup • Character-Based Setup • Boot from Character-Based to GUI-Based Setup • GUI-Based Setup 4 Troubleshooting – (Blue Screens & Stop Codes) 4 Latest information for NT 4. 0 – SP 4

Hardware Compatibility List 4 How important is it 4 Support parameters http: //www. microsoft. com/hwtest/ http: //support. microsoft. com/

Character-Based Setup Gathering of System Architecture Information 4 CPU Type 4 Motherboard Architecture 4 Hard Drive Controllers 4 File Systems 4 Disk Free Space 4 Memory

Info Gathered is Required for Basic System Initialization 4 ‘Failure to Detect’ will lead to failure of Setup 4 Unsupported components and enhancements – PCI 2. 1 – Special Bus Drivers – Caching Chips for Burst Mode

Boot from Character-Based to GUI-Based Setup 4 Windows NT Kernel is loaded completely for the first time – Finds a valid Hard Drive – Polls Adapters and tests Bus 4 Most likely point of failure – Drivers are loaded into Memory and Multi-threading is initialized

GUI-Based Setup 4 Install secondary Drivers 4 Create Accounts – Machine and Administrator 4 Configure Network Settings 4 Build final System Tree and Registry

Troubleshooting Character-Based Setup NTHQ Tool 4 Located in Support Directory 4 Purpose is to show all hardware peripheral settings 4 Works with PCI, Pn. P and Legacy peripherals

Troubleshooting Character-Based Setup NTHQ Demo

Troubleshooting Character-Based Setup Unsupported Controller and BIOS Enhancements 4 32 -bit I/O 4 Enhanced Drive Access 4 Multiple Block Access or Rapid IDE 4 Power Management Features

Troubleshooting Character-Based Setup Hangs During Initial Boot 4 Disable CD-Boot capability before installing – Needs to be done at both the Controller and BIOS levels

Troubleshooting Character-Based Setup Cannot Find Hard Drive 4 Scan System for Viruses 4 Make certain there is valid Boot Sector on the Hard Drive

Troubleshooting Character-Based Setup Cannot Find Hard Drive 4 If Hard Drive Controller is SCSI – Are devices properly terminated – Is SCSI BIOS enabled - first Controller (if at all) – On secondary Controllers, make certain BIOS is disabled – Partition and format using current Controller

Troubleshooting Character-Based Setup Cannot Find Hard Drive 4 If Hard Drive Controller is IDE or EIDE – Make certain drive is on primary Controller Channel – Make certain drive is jumpered correctly • (i. e. ) Master, Slave, Independent

Troubleshooting Character-Based Setup Does Not Detect Hard Drive Controller Correctly 4 Manually select Controller type 4 Make certain that an NT 4. 0 driver is being loaded 4 Use NTHQ Tool to check for correct IRQ and Memory addressing

Troubleshooting Character-Based Setup Cannot Find a Valid Partition 4 If Windows 95 is on the system, back-up and Fdisk Hard Drive (no support for Fat 32) – Recreate Partitions and Format with DOS 6. 22 4 Restore Windows 95 and proceed with Windows NT installation 4 Make certain that correct HAL is being loaded

Troubleshooting Failure to Reboot From Character-Based to GUI-Based Setup Stop Messages 4 Record Hex Value, 0 x 1 e, 0 x 7 b, etc. 4 Record Values in parentheses 4 Record component where failure occurred 4 Note where in Boot Process error occurred 4 Call PSS (installation support)

Troubleshooting Failure to Reboot Stop Messages Which can be Solved in the Field 4 0 x 7 b, (0 x 4, 0, 0, 0), or 0 x 8 b – Indicates problem with Master Boot Record – Scan for Viruses – Confirm correct Controller driver is loaded – Refresh Master Boot Record

Troubleshooting Failure to Reboot After Reboot, Video Remains “Black” 4 Check for devices using IRQ’s 2, 9 or 12 (PCI) 4 Scan Hard Drive for Viruses

Troubleshooting Failure to Reboot Stop Messages Which can be Solved in the Field 4 0 x 1 e or 0 xa – Disable any Third-party services or drivers which were loaded prior to Upgrade – Use NTHQ to confirm appropriate Memory and IRQ settings

Troubleshooting GUI-Based Setup Issues Setup Will Not Read From CD-ROM Drive 4 Make certain CD is on HCL 4 Copy I 386 directory to the Hard Drive and start again from the beginning 4 Make certain that the Controller and/or Hard Drive is correctly configured

Troubleshooting GUI-Based Setup Issues If Setup Fails During Copy of Files to Hard Drive 4 Disable all external Caches in BIOS 4 Make certain Hard Drives are terminated correctly; Active Preferred

Setup Enhancements in Windows NT 4. 0 Bootable CD-ROM 4 Supports only El Torrito Specification 4 Can only be used in ‘No Emulation Mode’ 4 Must be supported by both System and SCSI BIOS

Setup Enhancements in Windows NT 4. 0 Winnt Character-Based Setup Logging 4 Using Winnt or Winnt 32 /L: – Logs all actions during character-based setup to find last successful action – Helps to isolate where setup halted without requiring special DLL’s

Setup Enhancements in Windows NT 4. 0 Restartable GUI-Based Setup 4 If the machine fails during GUI-mode Setup; the problem can be fixed and setup will continue from reboot

Agenda 4 Setup (build overview) – Three phases of Setup • Character-Based Setup • Boot from Character-Based to GUI-Based Setup • GUI-Based Setup 4 Troubleshooting – (Blue Screens & Stop Codes) 4 Latest information for NT 4. 0 – SP 4

You’re Up and Running, But. . .

Debugging (the connection) 4 Connect – Modem, Null-modem cable, LAN 4 Boot. ini – / Debug /Debugport=com 1 / Baudrate=19200 4 Symbols – Retail NT CD (in the) supportdebug[platform]symbols sub-directory

Interpreting Blue Screens 4 The error code and parameters at the top of the screen 4 The list of modules that have successfully loaded and initialized in the middle of the screen 4 The list of modules that are currently on the stack at the bottom of the screen

Stop Codes Note: For a complete listing of stop codes, see Windows NTW 4. 0 Resource Kit, Chapter 39, “Windows NT Debugger”, or Q 142657 article on http: //support. microsoft. com

Common Stop Codes 4 0 x. A 4 0 x 1 E 4 0 x 24 4 0 x 3 F 4 0 x 50 4 0 x 7 B 4 0 x 7 F 4 0 x. C 000021 A

0 x. A 4 0 x 0000000 A IRQL_NOT_LESS_OR_EQUAL 4 Description – An attempt was made to touch paged out memory at a process interrupt request level (IRQL) that is too high. Code that runs at higher interrupt levels can’t touch paged-out memory because paging would be to expensive. If it happens that a pageable page is not committed, but it’s virtual address range is still in the translation buffer, high irql code can get away with touching it. But if the system is stressed – then the memory manager will have likely paged that page out and when an in page is attempted - the bugcheck will occur. So, this is why certain bugs tend to not show up on developers boxes which are less stressed than production. 4 Typical Scenarios – System configuration changes, virus scanners, other file I/O filters.

0 x 1 E 4 0 x 0000001 E KMODE_EXCEPTION_NOT_HANDLED 4 Description – Essentially, this bugcheck identifies an error that occurred in a section of code where no error detection routines were in place. Most exceptions are generated directly in the section of code that is executing. In this case, the error was not trapped in the middle of the code that was executing. Therefore, the error was allowed to fall through to this default error handler. This makes the error a very common exception. The actual instruction fault is usually similar to a STOP 0 x. A – that is a memory access violation. 4 Typical Scenarios – Invalid or obsolete third-party driver or system service, Microsoft driver or system service bug, file I/O filter drivers.

0 x 24 4 0 x 00000024 NTFS_FILE_SYSTEM 4 Description – A STOP 0 x 24 is the result of NTFS code that detects a problem with the structure of the NTFS file system. This is not a cut and dried exception code and debugging it is sometimes difficult. Disk corruption can generate a STOP 0 x 23 (FAT_FILE_SYSTEM) and 0 x 24. However any processes involved in reading or writing data from a FAT or NTFS file system could cause the disk data to appear corrupted. Therefore SCSI and IDE drivers as well as the disk structure itself (hard errors, i. e. bad blocks) can be suspect. The file system calls this bug check in multiple places and this will help us identify the actual source line that generated the bug check. Also, this bugcheck can be caused by I/O filter drivers (resource hangs, race conditions, etc. ). After the above is eliminated, more low-level constructs such as file system synchronization objects, scb attributes, etc. need to be examined by the debug engineer. 4 Typical Scenarios – This bugcheck is encountered when the NTFS file system has a corruption, or the hard drive has a bad block.

0 x 3 F 4 0 x 0000003 F NO_MORE_SYSTEM_PTES 4 Description – This stop isn’t as common as most of the others in this section, but a good explanation is warranted. A STOP 0 x 3 F is the result of a system doing lots of I/O, therefor fragmenting the system PTE’s. The bugcheck occurs not because the system is out of PTE's, but because a driver requests a huge chunk of memory that can’t be satisfied because a contiguous block that big isn’t available. 4 Typical Scenarios – Often video drivers will allocate large amounts of kernel memory that must succeed. Also, some backup programs do the same. – For these situations, consult a PSS engineer for the Registry hack that allows the increase of total system PTE’s.

0 x 50 4 0 x 00000050 PAGE_FAULT_IN_NONPAGED_AREA 4 Description – A STOP 0 x 50 is caused when a memory region that is not supposed to be paged out (usually for performance reasons) is paged out. This stop can be caused by a variety of problems including corrupt NTFS volumes, bad network packet data, and in general kernel mode drivers that corrupt memory. Also, drivers that free an MDL but don’t communicate it to all portions of the driver. Others include Disk, Controller, and Disk Driver problems. 4 Typical Scenarios – Usually third-party kernel mode drivers munging memory, or reading beyond allowable memory. Also, when the file system is pushed to the tested limits (large Mac volumes), bugs in NTFS are exposed that result in this STOP. This STOP can occur due to interaction problems between SCSI Controller firmware and Hard Drive firmware.

0 x 7 B 4 0 x 0000007 B INACCESSIBLE_BOOT_DEVICE 4 Description – During the initialization of the I/O system, the driver for the boot device may have failed to initialize the device that the system is attempting to boot from, or the file system that is supposed to read that device may have either failed its initialization or simply not recognized the data on the boot device as a file system structure. – If this is the initial setup of the system, this error may have occurred because the system was installed on an unsupported Hard Disk or SCSI Controller. – This error can also be caused by the installation of a new SCSI Adapter or Hard Disk Controller or by repartitioning the Hard Disk with the System Partition. 4 Typical Scenarios – VIRUS – LBA type problems, MBR type problems, SCSI Controller/Hard Drive geometry issues, etc.

0 x 7 F 4 0 x 0000007 F UNEXPECTED_KERNEL_MODE_TRAP 4 Description – This error means a trap occurred in kernel mode, either a kind of trap that the kernel is not allowed to have or catch (a bound trap), or a kind of trap that is always instant death (double fault). 4 Typical Scenarios – Hardware, kernel mode drivers that manipulate critical system data in an untimely fashion. – This STOP most often is the result of the processor taking a double 0 x 7 f (8, 0, 0, 0). Note that these parameters can also show up for a modern software issue involving Netmon (bhnt. sys).

0 x. C 000021 A 4 0 x. C 000021 A FATAL_SYSTEM_ERROR 4 Description – This is a typical description that accompanies this error: The Windows Subsystem System process terminated unexpectedly with a status of (0 x 6130 F 2 B 6 0 x 01 B 6 FBA 4). The system has been shutdown. – The failing process sometimes is listed in the blue screen itself. – This bugcheck occurs when a user-mode subsystem such as Winlogon or CSRSS is fatally compromised such that security can not be guaranteed. The Operating System makes a transition into kernel mode and throws this exception. 4 Typical Scenarios – A typical cause of this crash would be an extensible perfmon counter that overwrites it’s Winlogon shared data buffer (Q 171033), and in general any access violation that compromises a user-mode subsystem.

Break

Agenda 4 Setup (build overview) – Hardware Compatibility List – Three Phases of Setup – Character-Based Setup – Boot from Character-Based to GUI-Based Setup – GUI-Based Setup 4 Troubleshooting – (Blue Screens & Stop Codes) 4 Latest Information for NT 4. 0 – SP 4

A Day in the Life Video

NT 4 Service Pack 4 4 Contents – Hotfixes for important customer-reported problems – Resource and memory leak bugfixes from NT 5 – 30+ support, diagnostic and repair tools from the NT Resource Kit are included on the SP 4 CDROM – Event log entries for clean and dirty shutdown 4 Process Improvements – – Dedicated Service Pack test team Beta Program for Service Packs Improving the Knowledge Base, depth and ease of use Slipstreaming Service Packs into OEM releases

Resource / Memory Leaks 4 Problem – Leaks lead to hung systems and bluescreen crashes – Some customers do “preventive reboots” – Difficult to stop or kill the offending process 4 Solutions – Fix leaks: several hundred in NT 5, key fixes in NT 4 SP 4 – Job objects in NT 5, set memory limits on a collection of processes – Visual Studio adding leak checking to MFC and CRT 4 Next Work Items – Better leak detection – Logging in under low resource conditions – Stopping and killing processes

Bugchecks (Blue Screens) 4 Kernel mode code detected a serious error – Blue screens are still frequent and very hard to diagnose – Crash dumps take too long on large memory systems 4 Prevention – Find and fix bugs in our code – Review all calls to KEbugcheck by NT 5 RTM 4 Improve diagnosis – Reduced clutter on the blue screen, focus on key data, and add hints – Crash dumps are now dramatically faster in NT 5 – Developing comprehensive crashdump analysis tools for NT 4 and NT 5

Bugchecks (Blue Screens) Stop 0 x 0000001 E ( 0 x. C 0000005, 0 x. FDE 38 AF 9, 0 x 00000001, 0 x 7 E 8 B 0 EB 4 ) KMODE_EXCEPTION_NOT_HANDLED Address <x> has base at <x> - <filename> <manufacturer> <version> If this is the first time you've seen this Stop error screen, restart your computer. If this screen appears again, follow these steps: Check to make sure any new hardware or software is properly installed. If this is a new installation, ask your hardware or software manufacturer for any Windows NT updates you might need. If problems continue, disable or remove any newly installed hardware or software. Disable BIOS memory options such as caching or shadowing. If you need to use Safe Mode to remove or disable components, restart your computer, press F 8 to select Advanced Startup Options, and then select Safe Mode. Refer to your Getting Started manual for more information on troubleshooting Stop errors.

3 rd Party Drivers 4 Problem – One of the most common complaints from PSS – Source of pool corruption - difficult to diagnose 4 Solution – DDK driver samples and documentation is improved in NT 5 – Enhanced driver testing in NT 4 and NT 5, including pool corruption tests – NT 5 will have driver signing, “warning” level by default – WDM drivers will drive higher quality – We are testing major third-party anti-virus software regularly

Unnecessary Reboots in NT 5 4 Problem – Hardware and software configuration and maintenance 4 Solutions – Fixed 50 software configuration cases which required a reboot in NT 4. Key fixes include: • Adding, removing and configuring network protocols; changing IP addresses • Reconfiguring settings on PCI and other Pn. P hardware – Reboots still required for some rare cases • Machine name change, domain membership changes, system locale and system font changes, service pack installation – Hardware reconfiguration by clustering solutions in NTS/E – Where possible, hotfixes will avoid requiring a reboot

Diagnosis and Recovery 4 Recovery Involves – Detection (hard with a hung application or server) – Diagnosis (need good tools, need parallel installs, bad error messages) – System Recovery (chkdsk, crash dump biggest time hits) – Application recovery (SQL, Exchange Store, etc) 4 We are delivering – 30+ of the most critical support, diagnostic, and repair tools in SP 4 and NT 5 B 2 – Fixing 35 worst error messages by B 2+30, then next 200 as time allows – NT 5 Safe-mode Boot today and Floppy Boot by NT 5 RTM • Both support NTFS – Web-based trouble-shooter for most common bluescreens – Online chkdsk post NT 5

NT Test Initiatives 4 Long duration Server stress – 10 Servers running stress for a month+ starting at NT 5 Beta 2 – Mix of stress including Back. Office, IIS, Client/Server, etc – Specifically watching for memory and resource leaks 4 Improved driver testing for NT 4 and NT 5 – Catch pool corruption – Fault injection 4 Better integration testing of Server applications – – Back. Office applications: Exchange, SQL Server Using automated scripts from Back. Office teams Testing with Oracle, SAP R/3, Lotus Notes 100 Top Server Applications from Tier 1 RDP customers 4 Expanded tests for customer configurations – RDP Customer configurations, ISP

Resource Kit Tools 4 Network Diagnostic and Support Tools – nettest - quickly determine whether local uses network is configured properly (IDW) 4 Applications, Service Problems and Memory Leaks – memsnap - detection of memory and resource leaks over time (dump directory) 4 Disk Problems – fixacls - resets ACLs on system files to installation defaults, fixes users who hose their ACLs 4 Debugger Tools – debug wizard - easy setup of debuggers for customers 4 Other – windiff - file compare util, critical for many situations (reskit)

Event Log Analyst 4 Prototype tool for collecting and analyzing event log reliability data 4 Designed for collecting reliability trend data from an entire datacenter in few hours – Collected data from 800+ CDC servers in 5 hours – Analysis is manual with Excel, less than 3 hours 4 Provides trend analysis of reboots, bugchecks, and Dr Watsons

Event Log Analyst

Event Log Analyst Metrics 4 4 4 4 4 Mean time between reboots Mean time between bugchecks Mean time between Dr Watsons Trend analysis of reboots/server-year Trend analysis of bugchecks/server-year Trend analysis of Dr Watsons/server-year Bugcheck distribution Dr Watson distribution SP 4 Only: Availability percentage SP 4 Only: Mean time to repair

Tools for NT 4 SP 4 and NT 5 4 Network Diagnostic and Support Tools – browstat - only useful tool for diagnosing browser problems (reskit) – dhcpcmd - useful for fixing DHCP issues (reskit) – dnscmd - diagnose and repair DNS problems (reskit) – eseutil - used for WINS and DHCP database diagnosis and repair – nettest - quickly determine whether local uses network is configured properly (IDW) – winscl - diagnose and repair WINS (reskit) – winsadd - command line tool for batching static and dynamic entries in WINS – nltest - used for resetting secure channels, diagnosing and fixing trust problems (reskit)

Tools for NT 4 SP 4 and NT 5 4 Applications, Service Problems and Memory Leaks – depends - display and troubleshoot application dependency problems (IDW) – tlist - list running processes, used in conjunction with kill (reskit) – kill - forcibly terminate processes (reskit) – memsnap - detection of memory and resource leaks over time (dump directory) – pmon - detection of memory and resource leaks over time (reskit) – pviewer - gather extended information about running processes (reskit) – reg - registry utility, used for diagnosis and repair of many types of issues

Tools for NT 4 SP 4 and NT 5 4 Disk Problems – disksave - saves and restores the MBR (reskit) – fixacls - resets ACLs on system files to installation defaults, fixes users who hose their ACLs – ftedit - used daily to help customers repair fault tolerant volumes (reskit) 4 Debugger Tools – gflags - set global flags needed for various kinds of debugging (IDW) – remote - allow remote debugging by PSS (reskit) – debug wizard - easy setup of debuggers for customers – all standard debuggers - already ships in /support dir

Tools for NT 4 SP 4 and NT 5 4 Other – uptomp - update system from uniproc to multiproc (reskit) – robocopy - used daily by PSS during support calls, easiest way to move large amounts of data around very quickly. – shutdown - remote shutdown of systems (reskit) – ntevntlg. mdb & ntmsgs. hlp - better error message docs (reskit) – windiff - file compare utility; critical for many situations (reskit) – dumpel - dump event log messages from local or remote systems (reskit) – list - used daily by PSS for reviewing exceedingly large log files, etc.

Summary 4 Best Practices matter – Mature, disciplined planning & procedures – Design, Implement, Test – Configuration & Operational control 4 Technology matters – OS system services – UPS, RAID, ECC Memory, multi-homing – Cluster Services 4 We can deliver availability with Windows NT today 4 Microsoft is investing heavily in availability

References and Resources 4 http: //www. microsoft. com/ntserver/ 4 http: //www. microsoft. com/ntworkstation/ 4 http: //www. microsoft. com/windowsnt 5/ 4 http: //www. microsoft. com/hwtest/ 4 http: //support. microsoft. com/support/kb/articles/q 103/0/59. asp – Descriptions of Bug Codes for Windows NT

References and Resources 4 Inside Windows NT Second Edition, David A. Solomon MS Press 1998 4 Windows NTW 4. 0 Resource Kit – – Chapter 19: 21: 36: 39: “What Happens When You Start Your Computer” “Troubleshooting Startup and Disk Problems” “General Troubleshooting” “Windows NT Debugger”, or Q 142657 article 4 Supporting Windows NT Server in the Enterprise MS Press 1998 – Chapter 7: “Troubleshooting Tools and Methods”

Questions?

M