6,534
edits
Changes
rm translate tags
In case you are experiencing a kernel crash ([[oops]]) and have already [[Hardware testing|tested your hardware]], you should report what kernel says to the console (i.e. an [[oops]] text) to [httphttps://bugzillabugs.openvz.org/ Bugzillabug tracker]. Sometimes kernel crashes so badly that <tt>syslogd</tt> is not working and what kernel says is never written to a file. If this is the case, you have to catch what kernel says. There are several ways possible. == KDump == With RHEL6-based servers, kdump is pre-configured. See http://kb.odin.com/en/10044 to check the configuration. Dumps can be found under <code>/var/crash/</code> directory.
== Manual/Photo ==
=== Hardware setup ===
First of all you should make sure that your node has a [http[w://en.wikipedia.org/wiki/Serial_port Serial port|serial port]]. If there is no such port then
unfortunately this way is not for you.
Then you need to find a second machine with a serial port on it.
This machine will be used to collect logs from your primary machine. Further you need to acquire
so-called [http[w://en.wikipedia.org/wiki/Serial_cable Serial cable|null modem cable (a.k.a. serial cable)]] and it must be long enough to connect these two machines.
=== Software setup ===
</pre>
<!--T:10-->
{{Warning|make sure kernel command line does '''not''' contain the word '''<code>quiet</code>''', otherwise most of the kernel messages will not be printed to console.}}
<!--T:11-->
For example, in GRUB boot loader configuration file <tt>/boot/grub/grub.conf</tt> it looks like this:
<pre>
Kernel loaded with such parameters will send all kernel messages to /dev/ttyS0 (first serial port, a.k.a. COM1). If you have several ports, make sure that your null modem cable is connected to the appropriate port.
==== Receiving side ====<!--T:13-->
On the second node you should run any software that can log from /dev/ttyS0.
<!--T:14-->
It can be usual
<pre>
Save the file, then recompile the kernel:
Update your bootloader for the new updated kernel. In my case I use LILO so I just type lilo at the prompt.
Next you want your netconsole to send the request to somewhere. Load netconsole module, specifying the remote server parameters:
This will load the module with your settings. Replace your local IP address with where <tt>10.0.2.1</tt> is, <tt>eth0</tt> with your network interface card device, <tt>6666</tt> with the remote netconsole port (UDP), and <tt>10.0.2.2</tt> with your remote netconsole server IP. Also add in the mac address of your remote netconsole server, which in my case was 00:05:5D:34:11:AF. You can get the MAC address using arp utility:
Netconsole documentation is available from <tt>Documentation/networking/netconsole.txt</tt> file under your kernel source directory.
=== Setting from initrd === To log the boot process before root filesystem is mounted, network device driver and netconsole modules must be loaded from initd. RedHat 5/CentOS 5: echo 'MODULES+="<network-driver-module> netconsole "' > /etc/sysconfig/mkinitrd/netconsole chmod +x /etc/sysconfig/mkinitrd/netconsole echo 'options netconsole netconsole=<sport>@<saddr>/<dev>,<dport>@<daddr>/<dmac>' >> /etc/modprobe.conf Debian/Ubuntu: echo '<network-driver-module>' >> /etc/initramfs-tools/modules echo 'netconsole netconsole=<sport>@<saddr>/<dev>,<dport>@<daddr>/<dmac>' >> /etc/initramfs-tools/modules and rebuild initrd. === Setting up rsyslogd === /etc/rsyslog.d/netconsole.conf $template NetconsoleFile,"/var/log/netconsole/%FROMHOST%-%$NOW%.log" $template NetconsoleFormat,"%rawmsg%" $EscapeControlCharactersOnReceive off $DropTrailingLFOnReception off $RepeatedMsgReduction off $RuleSet NetconsoleRuleset *.* ?NetconsoleFile;NetconsoleFormat $RuleSet RSYSLOG_DefaultRuleset $ModLoad imudp $InputUDPServerBindRuleset NetconsoleRuleset $UDPServerRun 6666 === Setting up remote side === Set up '''netcat''' ('''nc''' on some Linux distributions) on your console server to listen on port 6666 UDP: netcat -u -l -p6666 or nc -lu 6666
When your kernel prints something on the console, the text will be also captured on this netconsole server.
==== Adding to inittab ====
For automatic care about capturing on console server you can use init respawn feature in this way:
For long term capturing you would like to do log rotating some way. With logrotate you can do it by creating config file /etc/logrotate.d/netconsole:
copytruncate
notifempty
# Need to restart logger after log file move
postrotate
# Below line assumes netcat will be restarted by init
killall -TERM netcat > /dev/null 2>&1 || true
}
</pre>
=== Testing netconsole ===First , check log level of console messages on OpenVZ side by:
First number should be 7 for testing. You can arrange it by:
== External links ==
* [http://kb.odin.com/en/10044 How to configure kdump (kernel crash dump)]
[[Category:QA]]
[[Category:HOWTO]]
[[Category:Kernel]]
[[Category:Troubleshooting]]