OS Fingerprint

Ping:
By default,
if TTL <= 128, the os is windows.
if TTL <= 64, the os is lunix.

Active detection
nmap:
nmap -O 192.168.1.1
T1: TCP SYN -> 21
T2: TCP NULL -> 21
T3: TCP SYN|FIN|URG|PSH -> 21
T4: TCP ACK -> 21
T5: TCP SYN -> 23
T6: TCP ACK -> 23
T7: TCP FIN|PSH|URG -> 23
PU: UDP -> 1
T1-T4: TCP OPEN T5-T7:TCP CLOSE PU: UDP CLOSE
According to nmap reply and use the result to compare with the nmap database, the os will be revealed.

Passive detection
p0f: running in a server to monitor incoming/outgoing (TCP/UDP/ICMP) packets, establishing the differential database. p0f will help to know what the source OS is.

Network sniffing
Plain text transfer protocol: TELENT, HTTP, FTP, POP3
Linux tools: Sniffit, Tcpdump, Ettercap, Ethereal.
Windows tools: Cain & Abel, Ethereal.