Editing Remote console setup
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
− | In case you are experiencing a kernel crash ([[oops]]) and have already [[Hardware testing|tested your hardware]], you should report what kernel says to the console (i.e. an [[oops]] text) to [ | + | In case you are experiencing a kernel crash ([[oops]]) and have already [[Hardware testing|tested your hardware]], you should report what kernel says to the console (i.e. an [[oops]] text) to [http://bugzilla.openvz.org/ Bugzilla]. Sometimes kernel crashes so badly that <tt>syslogd</tt> is not working and what kernel says is never written to a file. If this is the case, you have to catch what kernel says. There are several ways possible. |
− | |||
− | |||
− | |||
− | |||
== Manual/Photo == | == Manual/Photo == | ||
Line 12: | Line 8: | ||
=== Hardware setup === | === Hardware setup === | ||
− | First of all you should make sure that your node has a [ | + | First of all you should make sure that your node has a [http://en.wikipedia.org/wiki/Serial_port serial port]. If there is no such port then |
unfortunately this way is not for you. | unfortunately this way is not for you. | ||
Then you need to find a second machine with a serial port on it. | Then you need to find a second machine with a serial port on it. | ||
This machine will be used to collect logs from your primary machine. Further you need to acquire | This machine will be used to collect logs from your primary machine. Further you need to acquire | ||
− | so-called [ | + | so-called [http://en.wikipedia.org/wiki/Serial_cable null modem cable (a.k.a. serial cable)] and it must be long enough to connect these two machines. |
=== Software setup === | === Software setup === | ||
Line 27: | Line 23: | ||
</pre> | </pre> | ||
− | |||
− | |||
− | |||
− | |||
For example, in GRUB boot loader configuration file <tt>/boot/grub/grub.conf</tt> it looks like this: | For example, in GRUB boot loader configuration file <tt>/boot/grub/grub.conf</tt> it looks like this: | ||
<pre> | <pre> | ||
Line 41: | Line 33: | ||
Kernel loaded with such parameters will send all kernel messages to /dev/ttyS0 (first serial port, a.k.a. COM1). If you have several ports, make sure that your null modem cable is connected to the appropriate port. | Kernel loaded with such parameters will send all kernel messages to /dev/ttyS0 (first serial port, a.k.a. COM1). If you have several ports, make sure that your null modem cable is connected to the appropriate port. | ||
− | ==== Receiving side ==== | + | ==== Receiving side ==== |
On the second node you should run any software that can log from /dev/ttyS0. | On the second node you should run any software that can log from /dev/ttyS0. | ||
− | |||
It can be usual | It can be usual | ||
<pre> | <pre> | ||
Line 78: | Line 69: | ||
Save the file, then recompile the kernel: | Save the file, then recompile the kernel: | ||
− | + | <pre> | |
− | + | # make bzImage && make modules && make modules_install | |
+ | </pre> | ||
Update your bootloader for the new updated kernel. In my case I use LILO so I just type lilo at the prompt. | Update your bootloader for the new updated kernel. In my case I use LILO so I just type lilo at the prompt. | ||
Line 88: | Line 80: | ||
Next you want your netconsole to send the request to somewhere. Load netconsole module, specifying the remote server parameters: | Next you want your netconsole to send the request to somewhere. Load netconsole module, specifying the remote server parameters: | ||
− | + | <pre># modprobe netconsole netconsole=4444@10.0.2.1/eth0,6666@10.0.2.2/00:05:5D:34:11:AF</pre> | |
This will load the module with your settings. Replace your local IP address with where <tt>10.0.2.1</tt> is, <tt>eth0</tt> with your network interface card device, <tt>6666</tt> with the remote netconsole port (UDP), and <tt>10.0.2.2</tt> with your remote netconsole server IP. Also add in the mac address of your remote netconsole server, which in my case was 00:05:5D:34:11:AF. You can get the MAC address using arp utility: | This will load the module with your settings. Replace your local IP address with where <tt>10.0.2.1</tt> is, <tt>eth0</tt> with your network interface card device, <tt>6666</tt> with the remote netconsole port (UDP), and <tt>10.0.2.2</tt> with your remote netconsole server IP. Also add in the mac address of your remote netconsole server, which in my case was 00:05:5D:34:11:AF. You can get the MAC address using arp utility: | ||
Line 103: | Line 95: | ||
Netconsole documentation is available from <tt>Documentation/networking/netconsole.txt</tt> file under your kernel source directory. | Netconsole documentation is available from <tt>Documentation/networking/netconsole.txt</tt> file under your kernel source directory. | ||
− | + | === Setting up remote side === | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | === Setting up remote side === | ||
− | |||
− | |||
− | |||
− | |||
+ | Setup '''netcat''' ('''nc''' on some Linux distributions) on your console server to listen on port 6666 UDP: | ||
+ | <pre>netcat -u -l -p6666</pre> | ||
or | or | ||
− | + | <pre>nc -lu 6666</pre> | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
When your kernel prints something on the console, the text will be also captured on this netconsole server. | When your kernel prints something on the console, the text will be also captured on this netconsole server. | ||
Line 158: | Line 106: | ||
For automatic care about capturing on console server you can use init respawn feature in this way: | For automatic care about capturing on console server you can use init respawn feature in this way: | ||
− | + | <pre> | |
− | + | echo "n1:23:respawn:/bin/netcat -u -l -p 6666 >> /var/log/netconsole" >> /etc/inittab | |
+ | telinit q | ||
+ | </pre> | ||
==== Adding date/time to messages ==== | ==== Adding date/time to messages ==== | ||
Line 197: | Line 147: | ||
</pre> | </pre> | ||
− | |||
For more details, see man logrotate. | For more details, see man logrotate. | ||
− | === Testing netconsole === | + | === Testing netconsole === |
− | |||
− | + | First check log level of console messages on OpenVZ side by: | |
+ | |||
+ | <pre>cat /proc/sys/kernel/printk</pre> | ||
First number should be 7 for testing. You can arrange it by: | First number should be 7 for testing. You can arrange it by: | ||
− | + | <pre>sysctl -w kernel.printk="7 4 1 7"</pre> | |
After testing you can restore previous setting the same way. | After testing you can restore previous setting the same way. | ||
Line 217: | Line 167: | ||
If you see any console message on OpenVZ side, you should see message on console server too. If not, something is wrong. When debugging a problem, do not use tcpdump on OpenVZ side — it is not able to show netconsole packets. Instead, use tcpdump on console server. Quite a common source of problems with netconsole are firewalls. | If you see any console message on OpenVZ side, you should see message on console server too. If not, something is wrong. When debugging a problem, do not use tcpdump on OpenVZ side — it is not able to show netconsole packets. Instead, use tcpdump on console server. Quite a common source of problems with netconsole are firewalls. | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
[[Category:HOWTO]] | [[Category:HOWTO]] | ||
[[Category:Kernel]] | [[Category:Kernel]] | ||
[[Category:Troubleshooting]] | [[Category:Troubleshooting]] |