00-WindowsKernelOverview_University of Tokyo_July 2004.pdf

Windows Kernel Overview

1. Windows Kernel Internals II
Overview
University of Tokyo –
J
uly 2004*
Dave Probert, Ph.D.
Advanced Operating Systems Group
Windows Core Operating Systems Division
Microsoft Corporation
© Microsoft Corporation 2004 1

2. Contributors
Neill Clift Adrian Marinescu Eric Li Nar
G
anapathy
Jake Oshins Eric Traut Dragos
S
ambotin
Arun
Kishan
Brad Abrams
Brian Andrew Ben Leis Dan Lovinger
© Microsoft Corporation 2004 2

3. Course Overview
Four projects
–
W
riting kernel extensions for Windows
–
W
indows OS subsystems
–
N
TFS investigation using C#
–
M
onad (future)
Lecture topics
–
O
verview, kernel extensions, I/O drivers, WDF
–
O
bject manager, LPC, Processes & Threads
–
X
86, VirtualPC, Advanced File Systems
–
Longhorn, Monad, WinFX
© Microsoft Corporation 2004 3

4. Windows Overview
Current source base is Windows NT
–
F
oundation for NT4, Win2K, WinXP, WS03, Longhorn
–
API is still Win32 –
c
ompatible with Win9x
–
.
NET Framework pushing out Win32 for Longhorn
Most applications written in VB or VC++ today
–
F
uture is managed applications –
m
arrying VB
productivity with C++/Java richness => C#
Hot issues
–
T
rustworthy Computing
–
E
nable new computing experiences
–
C
reate new business opportunities
© Microsoft Corporation 2004 4

5. Security Issues
Lots of legacy code now hooked to the internet Most code written to work correctly under normal conditions Security design issues are subtle, particularly w.r.t. DoS Constantly evolving threats:
Stack-buffer overruns, Heap overruns, Format string overruns One byte overruns, Integer overflows
© Microsoft Corporation 2004 5

6. Reliability Issues
–
R
eboots required to do just about anything
–
H
uge base of third party code, esp. drivers
–
H
angs are hard to track and debug
–
P
atch management is tough
–
W
indows extension points poorly defined
•
A
pps break other apps
•
I
nstallation not idempotent
•
U
ninstall doesn’t return system to pre-install state
–
C
ompatibility issues everytime
O
S changes
•
A
pps test out the bugs in a particular OS environment
•
A
pps hardcode OS version information
–
W
indows management particularly hard
•
C
an’t ans
wer: what is the difference between 2 systems
•
R
egistry is too opaque and heavily abused
•
G
UI-based management doesn’t scale
© Microsoft Corporation 2004 6

7. Customer Experience
Establish tighter feedback loops
–
W
ATSON –
c
apture data on app crashes and hangs
–
O
CA –
c
apture data on BSODs
–
W
indows Update and SUS –
s
im
plify patching of systems
–
E
nterprise tools –
f
or deploymen
t, event log analysis
,
helpdesk
Use collected data to
–
p
rioritize fixes
–
w
ork with 3rd parties
–
analyz
e
common usage patterns
–
improve future products
Feedback loops pioneered by Office
© Microsoft Corporation 2004 7

8. Windows Architecture
© Microsoft Corporation 2004 8
User-m
ode
Kernel
-m
ode
Trap inter
f
ace / LPC
ntdll
/
run-
time library
Win32 GUI
Procs
&
threads
Kernel run-time / Hardware Adaptation Layer
Virtual memory
IO
Manager
Se
curity refmon
Cache mgr
File filter
s
Fi
le systems
Volume mgrs De
vice stacks
Scheduler
Kernel32
User32 / GDI
DLLs
Applications
System Services
Object Manager
/ Configuration Management
FS run-
time
exec synchr
Subsystem
servers
Login/GINA
Critical services

9. Windows Kernel Organization
Kernel-mode organized into
NTOS (kernel-mode services)
–
R
un-time Library, Scheduling, Executive services, object
manager, services for I/O, memory, processes, …
Hal (hardware-adaptation layer)
–
I
nsulates NTOS & drivers from hardware dependencies
–
P
roviders facilities, such as device access, timers, interrupt
servicing, clocks, spinlocks
Drivers
–
k
ernel extensions (primarily for device access)
© Microsoft Corporation 2004 9

10. Major Kernel Services
Process management
Process/thread creation
Security reference monitor
Access checks, token management
Memory manager
Pagefaults, virtual address, physical frame, and pagefile
management
Services for sharing, copy-on-write, mapped files, GC support, large apps
Lightwe
i
ght Procedure Call (LPC)
Native transport for RPC and user-mode system services.
I/O manager (& plug-and-play & power)
Maps user requests into IRP requests, configures/manages I/O devices,
implements services for drivers
Cache manager
Provides file-based caching for buffer file system I/O Built over the memory manager
Scheduler (aka
‘kernel’)
Schedules thread execution on each processor
© Microsoft Corporation 2004 10

11. CPU Scheduling & IRQLs
Thread scheduling occurs at
PASSIVE
or
APC
level
(IRQL < 2)
APCs
(
Asynchronous Procedure Calls) deliver I/O
completions, thread/process termination, etc (
IRQL ==
1
)
Not a general mechanis
m like unix
signals (user-mode code must
explicitly bloc
k pending APC deliv
e
ry)
Interrupt Service Routines run at
IRL >
2
ISRs
defer most processing to run at
IRQL==2
(
DISPATCH
level)
A pool of
worker threads
available for kernel components to
run in a normal thread context when user-mode thread is unavailable or inappropriate
Normal thread scheduling is round-robin among priority
levels, with priority adjustments (except for fixed priority real-time threads)
© Microsoft Corporation 2004 11

12. Process/Thread structure
Object
Manager
Any Handle
Table
Process
Object
Process’
Handle Table
Virtual
Address
Descriptors
Thread Thread Thread Thread Thread Thread
Files
Events
Devices Drivers
© Microsoft Corporation 2004 12

13. Process
Container for an address space and threads Associated User-mode Process Environment Block (PEB) Primary Access Token Quota, Debug port, Handle Table etc Unique process ID Queued to the Job, global process list and Session list MM structures like the WorkingSet, VAD tree, AWE etc
© Microsoft Corporation 2004 13

14. Thread
Fundamental schedulable entity in the system Represented by ETHREAD that includes a KTHREAD Queued to the process (both E and K thread) IRP list Impersonation Access Token Unique thread ID Associated User-mode Thread Environment Block (TEB) User-mode stack Kernel-mode stack Processor Control Block (in KTHREAD) for cpu
s
tate when
not running
© Microsoft Corporation 2004 14

15. Significant Windows Releases
•
Windows NT 3.1
•
Windows 95
•
Windows 98/98se/ME
•
Windows NT4
•
Windows 2000 (enterprise)
•
WindowsXP
(
consumer)
•
Windows Server 2003
•
Windows XP/SP2
•
“Longhorn”
© Microsoft Corporation 2004 15

16. Longhorn
Longhorn: codename for next major release
–
M
ost kernel improvements are clean-up,
scalability, URT support, fundamentals
–
B
ig bets: •
W
inFX
–
m
anaged replacement for Win32
•
W
inFS
–
n
ew unified information model
•
A
valon –
new GUI programming model
•
I
ndigo –
new messaging infrastructure for services
•
M
edia –
improve audio/video streaming
•
M
anagement, reliability, security
© Microsoft Corporation 2004 16