Subnets.ru

Всем привет.
На машине под управлением FreeBSD 7.2 поднят Network Access Server aka MPD5.
так же на машине собирает инфу ipcad и отсылает статистику на коллектор. Авторизация пользователей происходит при помощи radius'а.

Все бы ничего, но последнее время он начал подвисать. А подвисает так:
Сервер работает, но сетевые соединения не работают (ни на вход ни на выход).
Стал смотреть логи MPD, ничего интересного не нашел.
Стал ковырять /var/log/messages примерно от того времени, когда он завис. И наткнулся на это:

Код: Выделить всё

Jan 26 17:53:00 nas kernel:
Jan 26 17:53:00 nas kernel:
Jan 26 17:53:00 nas kernel: Fatal trap 12: page fault while in kernel mode
Jan 26 17:53:00 nas kernel: cpuid = 0; apic id = 00
Jan 26 17:53:00 nas kernel: fault virtual add
Jan 26 17:53:00 nas kernel: ress        = 0x188
Jan 26 17:53:00 nas kernel:
Jan 26 17:53:00 nas kernel: fault code
Jan 26 17:53:00 nas kernel: = su
Jan 26 17:53:00 nas kernel: perviso
Jan 26 17:53:00 nas kernel: r read,
Jan 26 17:53:00 nas kernel: page
Jan 26 17:53:00 nas kernel: not pr
Jan 26 17:53:00 nas kernel: esent
Jan 26 17:53:00 nas kernel:
Jan 26 17:53:00 nas kernel: instruction pointer = 0x20:0xc080bbc4
Jan 26 17:53:00 nas kernel: stack pointer               = 0x28:0xe7d75b58
Jan 26 17:53:00 nas kernel: frame pointer               = 0x28:0xe7d75b70
Jan 26 17:53:00 nas kernel: code segment                = base 0x0, limit 0xfffff
Jan 26 17:53:00 nas kernel: , typ
Jan 26 17:53:00 nas kernel: e 0x1
Jan 26 17:53:00 nas kernel: b

Подскажите что это за бяка и куда смотреть надо. Я правильно понимаю, что проблема в ядре?
Я не понимаю что могло случиться, т.к. сервер работает со 2.11.2009 и не было ни единого падения. Без падений он может работать сутки, двое, трое, неделю, а бывает за один день раза 2 - 3 падает.

Код: Выделить всё

nas# uname -a
FreeBSD nas.localhost 7.2-RELEASE FreeBSD 7.2-RELEASE #0: Fri Dec  4 06:03:51 YEKT 2009     nas@nas.localhost:/usr/obj/usr/src/sys/nas_  i386

Ядро собрано с этими, дописанными в конец, параметрами:

Код: Выделить всё

#PF
device pf
device pflog

options         ALTQ
options         ALTQ_CBQ        # Class Bases Queueing
options         ALTQ_RED        # Random Early Detection
options         ALTQ_RIO        # RED In/Out
options         ALTQ_HFSC       # Hierarchical Packet Scheduler
options         ALTQ_CDNR       # Traffic conditioner
options         ALTQ_PRIQ       # Priority Queueing
options         ALTQ_NOPCC      # Required for SMP build

options         NETGRAPH        #
options         NETGRAPH_PPP
options         NETGRAPH_PPTPGRE

Подскажите что в ней не так?

Заранее благодарен за ответ.

Добывай корку и смотри, что явилось причиной ребута.

посмотрел в /var/crash есть файлы с дампом ядра.
vmcore.6, info.6 и bound.
В bound лежит

Код: Выделить всё

7

в info.6:

Код: Выделить всё

# cat info.6
Dump header from device /dev/da0s1b
  Architecture: i386
  Architecture Version: 2
  Dump Length: 234610688B (223 MB)
  Blocksize: 512
  Dumptime: Tue Jan 26 17:53:00 2010
  Hostname: nas.localhost
  Magic: FreeBSD Kernel Dump
  Version String: FreeBSD 7.2-RELEASE #0: Fri Dec  4 06:03:51 YEKT 2009
    nas@nas.localhost:/usr/obj/usr/src/sys/nas_
  Panic String: page fault
  Dump Parity: 2758404424
  Bounds: 6
  Dump Status: good

vmcore.6 приводить не буду, т.к. он 234.6 Мб.
Хотя вот начало: в начале идет много строк с повторяющимися символами @^

Код: Выделить всё

@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.2-RELEASE #0: Fri Dec  4 06:03:51 YEKT 2009
    nas@nas.localhost:/usr/obj/usr/src/sys/nas_
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 3.00GHz (2992.70-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf41  Stepping = 1
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x641d<SSE3,DTES64,MON,DS_CPL,CNXT-ID,CX16,xTPR>
  AMD Features=0x20100000<NX,LM>
  Logical CPUs per core: 2
real memory  = 2147221504 (2047 MB)
avail memory = 2091536384 (1994 MB)
ACPI APIC Table: <DELL   PE BKC  >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP/HT): APIC ID:  1
 cpu2 (AP): APIC ID:  6
 cpu3 (AP/HT): APIC ID:  7
ioapic0: Changing APIC ID to 8
ioapic1: Changing APIC ID to 9
ioapic2: Changing APIC ID to 10
ioapic3: Changing APIC ID to 11
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 32-55 on motherboard
ioapic2 <Version 2.0> irqs 64-87 on motherboard
ioapic3 <Version 2.0> irqs 96-119 on motherboard
kbd1 at kbdmux0
acpi0: <DELL PE BKC> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> at device 0.0 on pci1
pci2: <ACPI PCI bus> on pcib2
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0xec00-0xecff mem 0xdfdf0000-0xdfdfffff,0xdfde0000-0xdfdeffff irq 34 at device 5.0 on pci2
mpt0: [ITHREAD]
mpt0: MPI Version=1.2.12.0
mpt1: <LSILogic 1030 Ultra4 Adapter> port 0xe800-0xe8ff mem 0xdfdd0000-0xdfddffff,0xdfdc0000-0xdfdcffff irq 33 at device 5.1 on pci2
mpt1: [ITHREAD]
mpt1: MPI Version=1.2.12.0
pcib3: <ACPI PCI-PCI bridge> at device 0.2 on pci1
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 5.0 on pci0
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 0.0 on pci5
pci6: <ACPI PCI bus> on pcib6
em0: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xdcc0-0xdcff mem 0xdfae0000-0xdfafffff irq 64 at device 7.0 on pci6
em0: [FILTER]
em0: Ethernet address: 00:11:43:e5:dd:d6
pcib7: <ACPI PCI-PCI bridge> at device 0.2 on pci5
pci7: <ACPI PCI bus> on pcib7
em1: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xccc0-0xccff mem 0xdf8e0000-0xdf8fffff irq 65 at device 8.0 on pci7
em1: [FILTER]
em1: Ethernet address: 00:11:43:e5:dd:d7
pcib8: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci8: <ACPI PCI bus> on pcib8
pcib9: <ACPI PCI-PCI bridge> at device 0.0 on pci8
pci9: <ACPI PCI bus> on pcib9
pcib10: <ACPI PCI-PCI bridge> at device 0.2 on pci8
pci10: <ACPI PCI bus> on pcib10
uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0xace0-0xacff irq 16 at device 29.0 on pci0
uhci0: [GIANT-LOCKED]
uhci0: [ITHREAD]
usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
usb0: USB revision 1.0
uhub0: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 2 ports with 2 removable, self powered
uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0xacc0-0xacdf irq 19 at device 29.1 on pci0
uhci1: [GIANT-LOCKED]
uhci1: [ITHREAD]
usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
usb1: USB revision 1.0
uhub1: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
uhub1: 2 ports with 2 removable, self powered
uhci2: <Intel 82801EB (ICH5) USB controller USB-C> port 0xaca0-0xacbf irq 18 at device 29.2 on pci0
uhci2: [GIANT-LOCKED]
uhci2: [ITHREAD]
usb2: <Intel 82801EB (ICH5) USB controller USB-C> on uhci2
usb2: USB revision 1.0
uhub2: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2
uhub2: 2 ports with 2 removable, self powered
ehci0: <Intel 82801EB/R (ICH5) USB 2.0 controller> mem 0xdff00000-0xdff003ff irq 23 at device 29.7 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb3: EHCI version 1.0
usb3: companion controllers, 2 ports each: usb0 usb1 usb2
usb3: <Intel 82801EB/R (ICH5) USB 2.0 controller> on ehci0
usb3: USB revision 2.0
uhub3: <Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb3
uhub3: 6 ports with 6 removable, self powered
uhub4: <vendor 0x413c product 0xa001, class 9/0, rev 2.00/0.00, addr 2> on uhub3
uhub4: multiple transaction translators
uhub4: 2 ports with 2 removable, self powered
pcib11: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci11: <ACPI PCI bus> on pcib11
pci11: <unknown> at device 5.0 (no driver attached)
uart0: <Non-standard ns8250 class UART with FIFOs> port 0xbc80-0xbcbf mem 0xdf4ff000-0xdf4fffff,0xd7f00000-0xd7f7ffff irq 21 at device 5.1 on pci11
uart0: [FILTER]
pci11: <unknown> at device 5.2 (no driver attached)
atapci0: <SiI SiI 0680 UDMA133 controller> port 0xbcf0-0xbcf7,0xbce4-0xbce7,0xbcd8-0xbcdf,0xbcd0-0xbcd3,0xbc70-0xbc7f mem 0xdf4fec00-0xdf4fecff irq 23 at device 6.0 on pci11
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
vgapci0: <VGA-compatible display> port 0xb800-0xb8ff mem 0xc8000000-0xcfffffff,0xdf4e0000-0xdf4effff irq 18 at device 13.0 on pci11
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci1: <Intel ICH5 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfc00-0xfc0f at device 31.1 on pci0
ata0: <ATA channel 0> on atapci1
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci1
ata1: [ITHREAD]
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FILTER]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A
sio0: [FILTER]
cpu0: <ACPI CPU> on acpi0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
p4tcc1: <CPU Frequency Thermal Control> on cpu1
cpu2: <ACPI CPU> on acpi0
p4tcc2: <CPU Frequency Thermal Control> on cpu2
cpu3: <ACPI CPU> on acpi0
p4tcc3: <CPU Frequency Thermal Control> on cpu3
pmtimer0 on isa0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xcafff,0xcb000-0xcefff,0xec000-0xeffff pnpid ORM0000 on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
ppc0: parallel port not found.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ukbd0: <Dell DRAC4, class 0/0, rev 1.10/0.00, addr 2> on uhub0
kbd2 at ukbd0
ums0: <Dell DRAC4, class 0/0, rev 1.10/0.00, addr 2> on uhub0
ums0: X report 0x0002 not supported
device_attach: ums0 attach returned 6
Timecounters tick every 1.000 msec
acd0: DVDROM <TEAC DVD-ROM DV-28E-C/D.4E> at ata0-master UDMA33
device_attach: afd0 attach returned 6
acd1: CDROM <VIRTUALCDROM DRIVE/> at ata2-slave PIO3
Waiting 5 seconds for SCSI devices to settle
(probe0:mpt0:0:6:0): TEST UNIT READY. CDB: 0 0 0 0 0 0
(probe0:mpt0:0:6:0): CAM Status: SCSI Status Error
(probe0:mpt0:0:6:0): SCSI Status: Check Condition
(probe0:mpt0:0:6:0): UNIT ATTENTION asc:29,0
(probe0:mpt0:0:6:0): Power on, reset, or bus device reset occurred
(probe0:mpt0:0:6:0): Retrying Command (per Sense Data)
da0 at mpt0 bus 0 target 0 lun 0
da0: <SEAGATE ST3300655LC 0003> Fixed Direct Access SCSI-3 device
da0: 320.000MB/s transfers (160.000MHz, offset 63, 16bit)
da0: Command Queueing Enabled
da0: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
ses0 at mpt0 bus 0 target 6 lun 0
ses0: <PE/PV 1x6 SCSI BP 1.0> Fixed Processor SCSI-2 device
ses0: 3.300MB/s transfers
ses0: SAF-TE Compliant Device
SMP: AP CPU #3 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
GEOM_LABEL: Label for provider da0s1a is ufsid/4ae7c8d413738ef1.
GEOM_LABEL: Label for provider da0s1d is ufsid/4ae7c8d4edf5d4f1.
GEOM_LABEL: Label for provider da0s1e is ufsid/4ae7c8d4aa844940.
GEOM_LABEL: Label for provider da0s1f is ufsid/4ae7c8d622d12e26.
Trying to mount root from ufs:/dev/da0s1a
WARNING: / was not properly dismounted
<118>Loading configuration files.
<118>kernel dumps on /dev/da0s1b
<118>Entropy harvesting:
<118> interrupts
<118> ethernet
<118> point_to_point
<118> kickstart
<118>.
<118>swapon: adding /dev/da0s1b as swap device
<118>Starting file system checks:
GEOM_LABEL: Label ufsid/4ae7c8d413738ef1 removed.
<118>/dev/da0s1a: 4067 files, 204214 used, 2334305 free (3801 frags, 291313 blocks, 0.1% fragmentation)
GEOM_LABEL: Label for provider da0s1a is ufsid/4ae7c8d413738ef1.
<118>/dev/da0s1d: DEFER FOR BACKGROUND CHECKING
<118>/dev/da0s1e: DEFER FOR BACKGROUND CHECKING
<118>/dev/da0s1f: DEFER FOR BACKGROUND CHECKING
GEOM_LABEL: Label ufsid/4ae7c8d413738ef1 removed.
<118>Setting hostuuid: 44454c4c-5100-1044-804b-b3c04f4d3731.
<118>Setting hostid: 0x59f3ca64.
<118>Mounting local file systems:
GEOM_LABEL: Label ufsid/4ae7c8d4edf5d4f1 removed.
WARNING: /tmp was not properly dismounted
GEOM_LABEL: Label ufsid/4ae7c8d4aa844940 removed.
WARNING: /usr was not properly dismounted
GEOM_LABEL: Label ufsid/4ae7c8d622d12e26 removed.
WARNING: /var was not properly dismounted
<118>.
<118>Setting hostname: nas.localhost.
<118>net.inet6.ip6.auto_linklocal:
<118>1
<118> ->
<118>0
<118>
<118>lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
<118>   inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
<118>   inet6 ::1 prefixlen 128
<118>   inet 127.0.0.1 netmask 0xff000000
<118>em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
<118>   options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
<118>   ether 00:11:43:e5:dd:d6
<118>   inet ***.***.***.251 netmask 0xffffff00 broadcast ***.***.***.255
<118>   media: Ethernet autoselect
<118>   status: no carrier
<118>em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
<118>   options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
<118>   ether 00:11:43:e5:dd:d7
<118>   inet 10.10.2.2 netmask 0xffff0000 broadcast 10.10.255.255
<118>   media: Ethernet autoselect
<118>   status: no carrier
<118>Enabling pf.
<118>pf enabled
<118>add net default: gateway ***.***.***.253
<118>add net 10.0.0.0: gateway 10.10.0.1
<118>add net 192.168.0.0: gateway 10.10.0.1
<118>Additional routing options:
<118> IP gateway=YES
<118>.
<118>Starting devd.
<118>Configuring keyboard:
<118> keymap
<118>.
<118>hw.acpi.cpu.cx_lowest:
<118>C1
<118> ->
<118>C1
<118>
<118>Additional IP options:
<118>.
<118>Mounting NFS file systems:
<118>.
<118>ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat
<118>a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout
<118>Creating and/or trimming log files:
<118>.
<118>Starting syslogd.
<118>Checking for core dump on /dev/da0s1b...
<118>savecore: reboot after panic: page fault
<118>Jan 23 15:23:00 nas savecore: reboot after panic: page fault
<118>savecore: writing core to vmcore.5
<5>em0: link state changed to UP
<5>em1: link state changed to UP
<118>Initial i386 initialization:
<118>.
<118>Additional ABI support:
<118>.
<118>Clearing /tmp (X related).
<118>Recovering vi editor sessions:
<118>.
<118>Starting mpd5.
<118>Starting local daemons:
<118>.
<118>Updating motd
WARNING: attempt to net_add_domain(netgraph) after domainfinalize()
<118>.
<118>Mounting late file systems:
<118>.
<118>Starting ipcad.
<118>Opening ng*...
<118>[LCap]
<118>[DYNAMIC]
<118>Initialized as 0
<118>Opening em0...
<118>[LCap]
<118>[4096]
<118>Initialized as 1
<118>Configured NetFlow destination at 10.10.0.2:9997
<118>Got 74899 elements from /usr/local/dump/flow/ipcadflow.dump.
<118>Storage: chained 8 in 36863 bucket out of 65537, mean 1.1 dev^2 1.1
<118>Daemonized.
<118>Starting ipcad.
<118>Opening ng*...
<118>[LCap]
<118>[DYNAMIC]
<118>Initialized as 0
<118>Opening em0...
<118>[LCap]
<118>[4096]
<118>Initialized as 1
<118>Configured NetFlow destination at 10.10.0.2:9997
<118>Can't start: another instance running, pid=759
<118>Can't initialize pid file /var/run/ipcadflow.pid: Operation not permitted
<118>Performing sanity check on apache22 configuration:
<118>Syntax OK
<118>Starting apache22.
<118>Configuring syscons:
<118> keymap
<118> scrnmap
<118> font8x16
<118> font8x14
<118> font8x8
<118> blanktime
<118>.
<118>Starting sshd.
<118>Starting cron.
<118>Local package initialization:
<118>.
<118>Starting utm5_rfw
<118>Starting background file system checks in 60 seconds.
<118>
<118>Sat Jan 23 15:23:11 YEKT 2010
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc623f100!
<118>Jan 23 15:51:18 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc623f100!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6a39d00!
<118>Jan 23 17:31:07 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6a39d00!
<118>Jan 23 17:36:09 nas sshd[13515]: error: PAM: authentication error for nas from 10.10.10.99
<118>Jan 23 17:36:28 nas su: nas to root on /dev/ttyp0
<6>pid 13710 (csh), uid 0: exited on signal 11 (core dumped)
<6>pid 13736 (csh), uid 0: exited on signal 11 (core dumped)
<6>pid 13779 (csh), uid 0: exited on signal 11 (core dumped)
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc62ea780!
<118>Jan 23 18:46:21 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc62ea780!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6a4bc00!
<118>Jan 23 20:31:04 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6a4bc00!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6778480!
<118>Jan 23 21:27:16 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6778480!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6166400!
<118>Jan 24 13:22:21 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6166400!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6cc0380!
<118>Jan 24 13:24:21 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6cc0380!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc5892200!
<118>Jan 24 13:33:53 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc5892200!
Limiting icmp ping response from 405 to 200 packets/sec
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xcaef7e00!
<118>Jan 24 17:36:30 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xcaef7e00!
<3>ng_mppc_decompress: too many (1194) packets dropped, disabling node 0xc6778180!
<118>Jan 24 18:05:14 nas kernel: ng_mppc_decompress: too many (1194) packets dropped, disabling node 0xc6778180!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc69c0280!
<118>Jan 24 18:45:33 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc69c0280!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc671a500!
<118>Jan 24 20:14:29 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc671a500!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6a3bb80!
<118>Jan 24 20:55:32 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6a3bb80!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6238a80!
<118>Jan 24 22:12:55 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6238a80!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc64a7d80!
<118>Jan 25 04:47:17 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc64a7d80!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6cc1400!
<118>Jan 25 13:14:12 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6cc1400!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc62ebe00!
<118>Jan 25 16:34:47 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc62ebe00!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6956580!
<118>Jan 25 17:10:44 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6956580!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6265480!
<118>Jan 25 17:14:41 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6265480!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6584b00!
<118>Jan 25 17:47:36 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6584b00!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc69bf800!
<118>Jan 25 18:15:31 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc69bf800!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6a39580!
<118>Jan 25 18:25:55 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6a39580!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6263980!
<118>Jan 25 20:11:17 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6263980!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc65b5800!
<118>Jan 25 20:19:31 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc65b5800!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6585400!
<118>Jan 25 20:45:58 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6585400!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6761a00!
<118>Jan 26 00:27:46 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6761a00!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc66e4a00!
<118>Jan 26 00:40:41 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc66e4a00!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc69bac00!
<118>Jan 26 01:13:55 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc69bac00!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6912b00!
<118>Jan 26 11:06:42 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6912b00!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc66e9b80!
<118>Jan 26 14:27:27 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc66e9b80!
<3>ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6a39400!
<118>Jan 26 16:56:45 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xc6a39400!


Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x188
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc080bbc4
stack pointer           = 0x28:0xe7d75b58
frame pointer           = 0x28:0xe7d75b70
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 759 (ipcad)
trap number             = 12
panic: page fault
cpuid = 2
Uptime: 3d2h30m14s
Physical memory: 2035 MB
Dumping 223 MB:

Может декомпрессию в mpd надо отключить?
Пробовал гуглить - натыкался только на упоминание, что линейка памяти может врать.

В dmesg нет ни малейшего у поминания о словах core (кроме проца), dump и fault.

Cделал так, как написано тут.
Вот что выдало:

Код: Выделить всё

# kgdb kernel.debug /var/crash/vmcore.6
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:

                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 759 (ipcad)
trap number             = 12
panic: page fault
cpuid = 2
Uptime: 3d2h30m14s
Physical memory: 2035 MB
Dumping 223 MB: 208 192 176 160 144 128 112 96 80 64 48 32 16

Reading symbols from /boot/kernel/ng_bpf.ko...Reading symbols from /boot/kernel/ng_bpf.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_bpf.ko
Reading symbols from /boot/kernel/ng_car.ko...Reading symbols from /boot/kernel/ng_car.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_car.ko
Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kernel/acpi.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/acpi.ko
Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols from /boot/kernel/ng_socket.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_socket.ko
Reading symbols from /boot/kernel/ng_mppc.ko...Reading symbols from /boot/kernel/ng_mppc.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_mppc.ko
Reading symbols from /boot/kernel/rc4.ko...Reading symbols from /boot/kernel/rc4.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/rc4.ko
Reading symbols from /boot/kernel/ng_tee.ko...Reading symbols from /boot/kernel/ng_tee.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_tee.ko
Reading symbols from /boot/kernel/ng_ksocket.ko...Reading symbols from /boot/kernel/ng_ksocket.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_ksocket.ko
Reading symbols from /boot/kernel/ng_iface.ko...Reading symbols from /boot/kernel/ng_iface.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_iface.ko
Reading symbols from /boot/kernel/ng_tcpmss.ko...Reading symbols from /boot/kernel/ng_tcpmss.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ng_tcpmss.ko
#0  doadump () at pcpu.h:196
196             __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) bt
#0  doadump () at pcpu.h:196
#1  0xc0819d17 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2  0xc0819fe9 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
#3  0xc0b26b3c in trap_fatal (frame=0xe7d75b18, eva=392)
    at /usr/src/sys/i386/i386/trap.c:939
#4  0xc0b26dc0 in trap_pfault (frame=0xe7d75b18, usermode=0, eva=392)
    at /usr/src/sys/i386/i386/trap.c:852
#5  0xc0b2776c in trap (frame=0xe7d75b18) at /usr/src/sys/i386/i386/trap.c:530
#6  0xc0b0be7b in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#7  0xc080bbc4 in _mtx_lock_sleep (m=0xc5db2458, tid=3319446816, opts=0,
    file=0x0, line=0) at /usr/src/sys/kern/kern_mutex.c:339
#8  0xc08ac31b in bpf_attachd (d=0xc6c6fa00, bp=0xc5db2440)
    at /usr/src/sys/net/bpf.c:337
#9  0xc08adcdd in bpfioctl (dev=0xc6255200, cmd=2149597804,
    addr=0xc6420ba0 "ng22", flags=5, td=0xc5dabd20)
    at /usr/src/sys/net/bpf.c:1179
#10 0xc07a70f7 in devfs_ioctl_f (fp=0xc58c1428, com=2149597804,
    data=0xc6420ba0, cred=0xc5472700, td=0xc5dabd20)
    at /usr/src/sys/fs/devfs/devfs_vnops.c:602
#11 0xc0852b35 in kern_ioctl (td=0xc5dabd20, fd=123, com=2149597804,
    data=0xc6420ba0 "ng22") at file.h:269
#12 0xc0852c94 in ioctl (td=0xc5dabd20, uap=0xe7d75cfc)
    at /usr/src/sys/kern/sys_generic.c:571
---Type <return> to continue, or q <return> to quit---
#13 0xc0b27115 in syscall (frame=0xe7d75d38)
    at /usr/src/sys/i386/i386/trap.c:1090
#14 0xc0b0bee0 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255
#15 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)


Больше ничего.

Судя по всему, факап случился в процессе сбора логов трафика ipcad'ом на интерфейсе ng22...

Значит буду сливать flow-статистику с помощью mpd5, а от ipcad'а уйду.
Жалко. такая хорошая программулина была.

есть ещё ipacctd

Код: Выделить всё

Port:   ipacctd-1.47
Path:   /usr/ports/net-mgmt/ipacctd
Info:   IP accounting using divert socket
Maint:  skv@FreeBSD.org

хорошая считалка, мы её пользуем

root писал(а):есть ещё ipacctd

Я думал за нее.
Но раз mpd пока нормально - буду пока mpd использовать.

потом видно будет.

Опять проблемка со сбором статистики с ифейсов ng.
Установил вновь запустил ipcad - статистика собиралась полтора дня, потмо все повисло. Только ребут помог.
ipacctd - не помог, т.к. он использует divert ipfw, которого у меня нет (что порадовало - ман на русском).
flow-funout не думаю что подойдет, т.к. он расчитан на разделение потоков и т.п.
Думал посмотреть ng_ipacct, но настройка у него странновата, хотя в конфигах приводятся и модели сетей.

Чем я еще могу собрать статистику и положить её на коллектор?

На биллинг сливается статистика при помощи mpd5, но сам биллинг не формирует файлы статистики. Может возможно ли отправить данные таким образом чтобы они попадали и на баллинг и на коллектор flow-capture из mpd5?
Можно ли использовать set netflow peer ip port в конфиге 2 раза и как себя поведет демон?
дело в том, что биллинг слушает на одном порту, а flow-capture на другом, а на один порт повесить биллинг и flow-capture нельзя .

Заранее благодарю за ответ.

ЗЫ. тут подсказали, что проблемы могут быть из-за

Код: Выделить всё

Limiting icmp ping response from 405 to 200 packets/sec

так ли это?
Кстати, еще периодически шлется в консоль это:

Код: Выделить всё

Feb 17 11:31:04 nas kernel: ng_mppc_decompress: too many (4094) packets dropped, disabling node 0xcc121180!

помогите разобраться.

Андрей писал(а):ipacctd - не помог, т.к. он использует divert ipfw, которого у меня нет (что порадовало - ман на русском).

а что мешает поднять ipfw с двумя правилами ?
лично я проблем не вижу
у меняесть точки где работает и PF и IPFW вместе и каждый занят своим делом

Андрей писал(а):Можно ли использовать set netflow peer ip port в конфиге 2 раза и как себя поведет демон?

мы не юзаем netflow, почему я уже как то говорил, а посему не можем ответить на этот вопрос

Андрей писал(а):ЗЫ. тут подсказали, что проблемы могут быть из-за

это можно регулировать с помошью переменной sysctl:

Код: Выделить всё

net.inet.icmp.icmplim

Андрей писал(а):Кстати, еще периодически шлется в консоль это:

mppc_decompress это скорее всего связано с шифрованием, смотри в эту сторону

root писал(а):мы не юзаем netflow, почему я уже как то говорил

А еще раз не напомните, почему не используете?
Насколько я знаю именно логи доступа по srcIP:port -> destIP:port интересуют органы и именно такие файлы надо хранить до 3х лет.
Или я не прав?
Как вы тогда обходитесь в жтом случае?

ЗЫ.
Мне тут еще посоветовали на коллектор поставить flow-fanout, который будет посредником между сниффером и коллектором, но возникает вопрос, нужен ли для его работы ng_netflow. Сейчас я работаю без него (хотя модуль загружен)