Shams

ssh authentication for non-root users in Mac OS X NetRestore images

After deployment of the image "SnowLeopard_Desktop-10-6-2" it is impossible to authorize to MAC node using password authentication method because it is disabled by default for this image (only publickey and keyboard-interactive are available).

As far as autotests could use only password authentication, it becomes impossible to run the tests at all. As a result, it blocks PVA4.6 release.

How to activate password authentication:

* login to Mac node (via direct console or SSH)
* open file /etc/sshd_config for edit
* in the file find the string: #PasswordAuthentication no
* replace it with the string: PasswordAuthentication yes
* save changes

Everything should be fine after that.

HowTo reproduce/check:

* open PuTTY and input Mac node address
* open SSH/Auth option, uncheck "Attempt "keyboard-interactive" auth (SSH-2)"
* click "Open" and try to login to node. If there is no error alert and login is fine, then problem is

fixed. Otherwise it still exists.

/etc/vztt/url.map

vim /etc/vztt/url.map for ez templates (not sure if it's even worth to remember):

# moved from /etc/vztt/vztt.conf for compatibility
#$SW_SERVER     http://vzdownload.swsoft.com
$SW_SERVER      http://repo/ez
$FC_SERVER      http://repo
#$FC_SERVER     http://fedora.redhat.com
#$CE_SERVER     http://mirror.centos.org
$CE_SERVER      http://repo/pub
#$SUSE_SERVER/pub/opensuse      http://download.opensuse.org 
$DEB_SERVER     http://ftp.ru.debian.org
$RH_SERVER      http://repo
$SLES_SERVER    http://repo
#$SUSE2_SERVER  ftp://ftp.suse.com
$UBU_SERVER     http://archive.ubuntu.com
$OR_SERVER      http://repo
$SUSE_SERVER    http://repo
$SUSE2_SERVER   http://repo

Assorted pieces of knowledge

Different Python version specified directly in jobs

At the moment PV SDK used in products is x64, so we fixed the NetRestore image and use the native Python, not x32 one. Back in good old PD6 days PV SDK was x32, so it's incompatible with x64 Python (maxint is '9' when it's x64 and '2' when it's x32):

>>> import prlsdkapi
dlopen(/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/prlsdkapi/prlsdk.so, 2): no suitable image found.  Did find:
	/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/prlsdkapi/prlsdk.so: mach-o, but wrong architecture
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/prlsdkapi/__init__.py", line 28, in <module>
    raise ImportError, err
ImportError: Cannot import module "prlsdk" !
>>> import sys
>>> sys.maxint
9223372036854775807

Here's a work-around (deployers will fix the issue in the future, Python x32 will be installed if a product is PD6):

from man python:

64-BIT SUPPORT
    % export VERSIONER_PYTHON_PREFER_32_BIT=yes # Bourne-like shells

so the idea is to replace every 'python' in jobs directory with this thing:

shams@shams pls-jobs$ grep python *
vzt_autocompact_lin.py:	cmdline = ("python vzt-autocompact/bin/vzt-autocompact "
shams@shams pls-jobs$ sed -i 's@python@VERSIONER_PYTHON_PREFER_32_BIT=yes &@' *.py
shams@shams pls-jobs$ grep python *
vzt_autocompact_lin.py:	cmdline = ("VERSIONER_PYTHON_PREFER_32_BIT=yes python vzt-autocompact/bin/vzt-autocompact "

there were several other service files which were damaged, so it's better to exclude them while fixing other stuff (it always appears that good old pipes never betray you):

shams@shams pls-jobs$ for i in `grep -l python * -R | grep -v configure.py | grep -v venet.py`; do echo sed -i 's@python@VERSIONER_PYTHON_PREFER_32_BIT=yes &@' $i; done

Installing products on Ubuntu (disk partitioning work-around)

Ubuntu disks are partitioned in a pretty weird way, so it's better to copy run files to /home, create '/tmp' there and slightly fix the executable file:

TMPROOT=${TMPDIR:=/home/tmp}

PMC for PWE Lin servers workaround

At the moment it's impossible to connect to PWE Lin servers via PMC because prl_disp_service doesn't listen to 64000 port by default. It's possible to fix it in two ways:

Change default value of <ListenAnyAddr>0</ListenAnyAddr> in /etc/parallels/dispatcher.workstation.xml to 1

Template:Note2

Check the box connect PMC directly to the host computer in PWE File -> Preferences --> Iphone

File:Pmc-hint-in-PWE.png Template:Note2

Missing libraries while test runs

DEBUG:root:local cmd: '/root/vzt-parallels-vm-install/lib/tools/BMPEntry4Lin /vzt/tmp/tmprFLsbI/current_screen.bmp /vzt/tmp/tmprFLsbI/01-300-220-190-40.bmp'
/root/vzt-parallels-vm-install/lib/tools/BMPEntry4Lin: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory

Library is missed, so:

# ldd /root/vzt-parallels-vm-install/lib/tools/BMPEntry4Lin
        linux-gate.so.1 =>  (0x006e7000)
        libstdc++.so.6 => not found
        libgcc_s.so.1 => not found
        libc.so.6 => /lib/libc.so.6 (0x00be7000)
        /lib/ld-linux.so.2 (0x00bc5000)
# rpm -q libstdc++
libstdc++-4.4.4-13.el6.x86_64
# rpm -ql libstdc++
/usr/lib64/libstdc++.so.6
/usr/lib64/libstdc++.so.6.0.13
# file /root/vzt-parallels-vm-install/lib/tools/BMPEntry4Lin
/root/vzt-parallels-vm-install/lib/tools/BMPEntry4Lin: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.2.5, not stripped

So it needs to be installed:

yum install libstdc++.i686

So now it's here and can be found, yay:

ldd /root/vzt-parallels-vm-install/lib/tools/BMPEntry4Lin
        linux-gate.so.1 =>  (0x002df000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x005d9000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00da3000)
        libc.so.6 => /lib/libc.so.6 (0x00be7000)
        libm.so.6 => /lib/libm.so.6 (0x0016a000)
        /lib/ld-linux.so.2 (0x00bc5000)

VMs' flags

force run in 32/64-bit mode	vm.app_mode=32/64
no vtx/svm (cne mode)	kernel.hvt_support=0
suspend/resume verbose	vm.sare_verbose=1

You don't need to stop VM to edit the config file (config.pvs) to add some flags. Simply suspend a VM, edit config.pvs file which is in VM's directory. SystemFlag is what you look for.

<SystemFlags>vm.default_answers=13014:16015; vm.assert=1</SystemFlags>

For hanging guests the best choice is vm.assert=1. Resume a VM and see the difference in parallels.log for the guest.

DNS dynamic dhcp clients

Note: There should be only nameserver 10.30.0.1 in /etc/resolv.conf, any other weird crap should be removed.

$GENERATE 1-254 $.255   PTR     dhcp27-${0,2,x}.qa.
---------------------------------------------------------
GENERATE 1-254 dhcp27-${0,2,x}         A       10.27.255.$

fedora/adobe sound troubles fix howto

sudo sed -i 's,\(^[[:space:]]*\)\(exec "$prog" ${1+"$@"}\),\1LD_PRELOAD=/usr/local/lib64/mymemcpy.so \2,' /usr/lib64/firefox-3.6/run-mozilla.sh

winsshd for win guests setting up

WinSSHD control panel -> Settings tab -> Edit/View settings button -> Virtual Accounts -> Add new virtual user root. Please make sure the following fields are filled properly:

Virtual account name	root
Virtual group	Virtual Admins
Windows account domain	hostname
Windows account name	Administrator
Public keys	import id_rsa.pub

Template:Warning2 Template:Note2

Don't forget to cache password for Administrator (as it shown on the picture below):

File:Winsshd-passwd-cache.png

If WinSSHD is configured properly, you're able to access your host/guest via ssh. Sometimes you've got the following SSH error:

Authentication partiall success.
Permission denied (publickey,gssapi-with-mic)

Try to reset Administrator password in Computer -> Manage -> System tools -> Local users and groups -> right-click Administrator -> Set password.

To get more info on dhcp conficts

tcpdump

qagw> sudo tcpdump -i vlan29 -s 1024 -vvvvv -e -n ether host M:A:C:Addr

mii-tool eth0 or ethtool eth0

To create a custom WinPE image

Note: You need to re-create WinPE image for the given server if you change its drivers in C:\Inetpub\wwwroot\inst\windows\drivers\SWID\!

Note: For new Intel servers only pnpbd-intel-15.6.1 is needed, skip all the Win* stuff from the good old days.

Login to rome
Create a folder with drivers in C:\Inetpub\wwwroot\inst\windows\drivers (folder name should be exactly the same as inv_no on rome)
Management console -> WinPE tab -> Manage WinPE

Choose WinPE version (1x for old Wins, 2x for new ones) and architecture type. Also you can create new image (empty one or including drivers you found) or copy one. For some Win distros (Win7) empty image would work (w/o drivers specified; some of them are already on board)

Press 'Start' and enjoy once it's over.

To set X for lin nodes

Custom packages install

yum install xclock xauth screen xorg-x11-fonts* xorg-x11-server-Xorg xorg-x11-apps xorg-x11-twm 
# if the browser is needed, then it's worth to remember that this is x64 filrefox:
firefox.x86_64

To install license key from command line:

prlsrvctl install-license -k RM3QZY-HDR34G-FN3FQR-QILKS8-54RGVB

Manual whole group install

for RHEL5:

# yum groupinstall "X Window System"
# yum groupinstall "GNOME Desktop Environment"

for RHEL6:

# yum groupinstall "X Window System"
# yum groupinstall "Desktop"

start X:xdpyinfo problems

Does work:

# export DISPLAY=:0; xdpyinfo
name of display:    :0.0
version number:    11.0
vendor string:    Red Hat, Inc.
vendor release number:    10707000
maximum request size:  16777212 bytes
motion buffer size:  256
bitmap unit, bit order, padding:    32, LSBFirst, 32
image byte order:    LSBFirst
number of supported pixmap formats:    7
supported pixmap formats:
    depth 1, bits_per_pixel 1, scanline_pad 32
    depth 4, bits_per_pixel 8, scanline_pad 32
    depth 8, bits_per_pixel 8, scanline_pad 32
    depth 15, bits_per_pixel 16, scanline_pad 32
    depth 16, bits_per_pixel 16, scanline_pad 32
    depth 24, bits_per_pixel 32, scanline_pad 32
    depth 32, bits_per_pixel 32, scanline_pad 32
keycode range:    minimum 8, maximum 255
focus:  window 0x1a0002c, revert to Parent
number of extensions:    26
  visual:
    visual id:    0x10c
    class:    DirectColor
    depth:    24 planes
    available colormap entries:    256 per subfield
    red, green, blue masks:    0xff0000, 0xff00, 0xff
    significant bits in color specification:    8 bits
  visual:
    visual id:    0x67
    class:    TrueColor
    depth:    32 planes
    available colormap entries:    256 per subfield
    red, green, blue masks:    0xff0000, 0xff00, 0xff
    significant bits in color specification:    8 bits

Doesn't work:

# export DISPLAY=:0; xdpyinfo
xdpyinfo: unable to open display ":0".

To view screens info

ps -o stime,pid,cmd -C screen

To resurrect deploy after power failure and when deploy is fucked up in general

1. Switch on both hosts, rome and carthage.
2. Log in to rome remotely and start services (dpsvc [parallels deployment service], prlwatchdog [parallels notification watchdog], ‘TFtpd32 svc’). DB on carthage starts automatically.
3. Run the cmd-string:
sc stop dpsvc&sc start dpsvc&sc stop prlwatchdog&sc start prlqatchdog&sc stop TFtpd32 svc&sc start TFtpd32 svc

To blow life into freshly deployed win server networking

If you deploy machine for the first time and it cannot find drivers, then do the following:

1. Insert  a flash with drivers
2. IE: ''\\10.29.0.1'' (or ''\\rome\c$'') Inetpub/wwwroot/inst/windows/drivers/
3. Create new folder = <deploy ID of this server> (for mccp8 this folder will be ''7192'')
4. Check the type of Ethernet card (''My PC -> Properties -> Hardware -> Device manager'') and copy to this new folder the proper folders with drivers from flash (for mccp8 these are ''winxp64'', ''win64'', ''win32'').
5. To check these drivers are really adopted by deploy, on the server you're setting go to ''Device manager'' right click ''Ethernet controller'' and choose ''Update drivers'' option from menu.

To reinstall mac via command line

python dpctl.py deploy_image -I 148 -i "Leopard_Desktop_10-5-7" -e user@domain.com --service=rome -o "Mac OS"
--attribute="dmgdrive=Macintosh HD"

PD5 mac hints: vnc enabling, knowing an arch of a given mac, installing a build manually

To start VNC Server on mac: ssh root@macx and execute:

# sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart 
-activate -configure -access -on -users kim -privs -all -restart -agent -menu -clientopts 
-setvnclegacy -vnclegacy yes -setvncpw -vncpw PASSWD

for a given mac to be available via ssh on port 22, you need to check the Remote Login check-box (System Preferences -> Sharing -> Remote Login service on). Deploy enables port 302 executing the postinstall script.
To know which ARCH is a given mac:

# sysctl -A | grep cpu64bit
hw.cpu64bit_capable:1 (this means it's x86_64)
hw.cpu64bit_capable:0 (this means it's i386)

or even like this:

mac16:~ root# sysctl -a | grep -e cpu64bit -e hw.model -e hw.ncpu
hw.model = MacBook2,1
hw.ncpu = 2
hw.ncpu: 2

If deploying an image with deploy fails, it's possible to install new PD build by

copying it from another mac (better store it in /), or simply vnc to a given mac, open http://s01/Sumer/ , sort by name, choose a new build NOT MARKED "in progress" it should also contain "mac" directory (avoid builds with "mac.r" and such)

If installing the tests via deploy also fails, it's possible to install them manually:

# curl http://s01/Sumer/3.0.3251.202085/mac/  (there's no wget, so use curl)
# curl -o pythontests-4.0.3251.202085.zip http://s01/Sumer/3.0.3251.202085/mac/
pythontests-4.0.3251.202085.zip
# curl -o desktop-unittests-4.0.3251.202085.zip http://s01/Sumer/3.0.3251.202085/
mac/desktop-unittests-4.0.3251.202085.zip

to force netboot on macs (if you press 'N' stubborn mac still ignores your wishes and wants to boot from hard disk):

# sudo bless --netboot --server bsdp://255.255.255.255

Template:Warning2

To reinstall lin server via command line

python dpctl.py deploy_image -e user@domain.com -i fedora-10-x86_64 -I 7043 --attribute='kargs="vzt debug "' --service='rome'

To deploy PSBM via tssetup-psbm

wget http://repo/pub/vzt-tools/tssetup-psbm
python tssetup-psbm --label ps/527-3.0.5368.407751 --kargs=vzt -r

Template:Note2

To start/stop VM and check the time

# while :; do time prlctl start "{7afcbfe5-c92d-4398-9c70-44ac35077c05}" && time prlctl stop VM_7afcbfe5 --fast || break; done
# while :; do prlctl set "{7afcbfe5-c92d-4398-9c70-44ac35077c05}" --mac "auto" --device-set "net1"; time prlctl start 
"{7afcbfe5-c92d-4398-9c70-44ac35077c05}" && time prlctl stop VM_7afcbfe5 --fast || break; done

To check if weekly report yaml is formatted properly

# PYTHONPATH=/home/shams/tests/vztlib/lib/
# python -c 'import yaml, pprint; pprint.pprint(yaml.load(open("shams-5Jun.yaml")))'
In case of mistakes there will be long parser output and something like:
# yaml.scanner.ScannerError: while scanning a simple key
  in "shams-5Jun.yaml", line 39, column 3
could not found expected ':'
  in "shams-5Jun.yaml", line 40, column 3

To execute test in cycle until it passes out (means manual test installing/unpacking/executing):

# a=0; while [ $? -eq 0 ]; do a=$(($a+1)); echo 'TEST RUN: ' $a; vzt-pxe/bin/vzt-pxe;done

launchctl: macosx processes

dispatcher restart

sudo launchctl stop com.parallels.server.launchdaemon
sudo launchctl start com.parallels.server.launchdaemon

to enable net boot for old school servers

Actually Legacy LAN in BIOS boot options is a fake. To enable true netboot, go to Integrated Peripherals -> Onboard Devices and set 'Enabled' for Onboard Lan device. Save and exit, on the next reboot go there again and specify the device correctly.

to create a stand-alone VM using setupvm

# curl http://testrepo/pub/vzt/setupvm.tar.bz2 | tar -xjvf -
# python setupvm/00main.py  execute -- --prl-template install:win_2k8_srv_dtc_r2_x86_64 --vt plsctl

Template:Note2

to remove VMs in cycle

Win hosts

C:\Users\Administrator>for /f %v in ('prlctl list -a -H -o name') do prlctl destroy  %v

Lin/Mac hosts

# for i in $(prlctl list -a -H -o name); do prlctl destroy $i; done

code coverage hints and tips

If scheduled test fails and doesn't export the results, before doing it manually please move all the coverage data from /var/tmp/--Build-Tmp--/pdfm-mac-something/ to test work directory.

Fixme: there's a script which does all the moving job, so place the link here

Fixme: also a couple of words about extracting coverage data and forming html won't hurt; it's a script also, dkomkov@ knows the pain

install license for Parallels products via cmd

mac16:~ root# prlsrvctl install-license --key RM3QZY-HDR34G-FN3FQR-QILKS8-54RGVB
The license has been successfully installed.

OS ISOs locations

1. Those which are used in automated tests: vztlib/lib/vztests/os_install/config.py 2. Those which are used in manual testing: on wiki

OS install on the fly using vzt-sample

Template:Note2

$ curl http://testrepo/pub/vzt/vzt-sample.tar.bz2 | tar -xjvf -
$ python vzt-sample/bin/vzt-sample --prl-template install:win_7_x86_64 --prl-tools --prl-cpus $cpu_num -D 4 -L --ts $test_server

mounting samba shares

on Linux

sudo mount -t cifs -o guest,file_mode=0644,dir_mode=0755,nocase,noacl,noperm,nosfu,noserverino,nosetuids,nouser_xattr 
//mir/incoming /home/shams/mnt/mir/

on Windows

This is pretty convenient for storing big dumps on racoon. Let me tell you this:

If you've got a VM in BSOD, then it's pretty hard to get its dump from C:\Windows\Memory.dmp. You can create another VM on the same server (create a dir, get a VM tarball, unpack it and prlctl register dir), add a HDD (existing image, that of broken VM) to this new VM in Configure.
IE -> Tools -> Map Network Drive (alternative: Computer -> Properties -> Map Network Drive
Any letter of drive is acceptable
Folder is in form \\racoon\Memory.Dmps\
Uncheck Reconnect as logon check-box
Different user name: swsoft/onovaselskaya: passwd
Now it's possible simply drag'n'drop the dump from E:\Windows\ to the bug folder on racoon

Template:Note2

scp -P 302 root@ts68.qa:Memory.dmp .
The authenticity of host '[ts68]:302 ([10.29.252.60]:302)' can't be established.
DSA key fingerprint is e9:6c:1a:cc:95:56:84:f8:04:7a:58:1f:e6:bd:19:5d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[ts68]:302,[10.29.252.60]:302' (DSA) to the list of known hosts.
root@ts68.qa's password: 
Memory.dmp                                                                                               100%  432MB  10.5MB/s   00:41

to burn something on CD

you can insert a CD and copy an image:

$ cat /dev/cdrom > file  #name the file

or it's possible to do it the other way:

$ dd if=/dev/cdrom of=file  #name the file

you can also mount the .iso file:

#if you're not root:
$ fuseiso file (what) dir (to where)
# if you're root:
# mount -o loop file (what) dir (to where)
# to unmount:
$ fusermount -

or you can copy .iso from mir to your local machine
burn a CD (perhaps you'll need to specify a lesser speed for some images):

$ cdrecord -v -eject -speed 12 file

Template:Warning2

If repo lacks some packages

Sometimes repo lacks some packages and that's why they cannot be installed into a VM. We can upload those packages to repo from some other repositories ( yandex mirror seems to be a good place to start) and then re-create repodata (note that createrepo command is executed inside directory where repodata/ is).

In this example we lack the package libusb-devel for fedora-10-x86_64.

Download the package from fedora mirror on yandex. Note2 that it's not in os but in Everything repo.

Upload it to proper Packages directory on repo:

$ scp libusb-devel-*.fc10.x86_64.rpm x@repo:/var/www/html/pub/fedora/linux/releases/10/Fedora/x86_64/os/Packages/

re-create repodata (once again, inside directory with repodata/:

@repo Packages]$ createrepo .
2814/2814 - libusb-devel-0.1.12-20.fc10.x86_64.rpm                              
Saving Primary metadata
Saving file lists metadata
Saving other metadata

if you tried yum install command before and failed because the package was missed, we need to clean yum metadata to be able to install package from renewed repo:
```
$ yum clean all
```

yum search (to make sure package is in the right place and problem is fixed):

$ yum search libusb-devel
========= Matched: libusb-devel ===============================================
libusb-devel.i386 : Development files for libusb.
libusb-devel.x86_64 : Development files for libusb.
$ yum install libusb-devel
...
Installed: libusb-devel.x86_64 0:0.1.12-5.1 libusb-devel.i386 0:0.1.12-5.1
Dependency Installed: pkgconfig.x86_64 1:0.21-2.el5
Complete!

robot.py hints and tips

create a job

python robot.py create_job --file="stm/mac24-vzt-pss-win-fixed-vm-num" --name="vzt-pss-win-fixed-vm-num" --url="http://testrepo/pub/vzt/vzt-pss.tar.bz2" --cmd="python vzt-pss/00main.py execute --shortcut fixednum -- --prl-template w2k3-ent-sp2-x86/default --min-mem=192 --max-mem=8192 --vm-num 4 -D4 --title vzt-pss-win-fixed-vm-num" --host="8140" Template:Note2

jobs

to replace something in the titles

$ for F in *snowleo*; do NF=$(echo $F | sed 's/snowleo/snow-leo/'); mv $F $NF; done
$ grep -F 'cpu.total>=2' * -l | xargs sed -i 's./cpu\.total>=2/cpu.total>1/'

Actually you can use any symbol to part instead of '/'. In case it's used in expression you want to edit, use '@' or something else.

to replace something in job lines

$ grep -i server_opts *
vzt-vm-install-stw-AMD64-hvt.py:			server_opts = "cpu.arch='x86_64'")
vzt-vm-install-stw-Intel32-hvt.py:			server_opts = "",
vzt-vm-install-stw-Intel32-sd.py:			server_opts = "",
vzt-vm-install-stw-Intel32-ska.py:			server_opts = "",
vzt-vm-install-stw-Intel64-hvt.py:			server_opts = "cpu.arch='x86_64'")
<pre>
# or it's possible to search exact strings in exact files and write a cycle
$ fgrep -l 'server_opts = ""' *
$ for f in $(fgrep -l 'server_opts = ""' *); do sed -i '/server_opts = ""/d' $f; done

one more example:

$ for f in $(fgrep -l "opts.get('server_opts')" *); do sed -i "s/opts.get('server_opts')/opts.get('server_opts', '')/g" $f; done

or it's possible to look for pattern and fix it in multiple lines:

$ sed -i "s/x86_64'\&/x86_64'/g" *-stw-AMD*

another sed intercourse:

# for f in $(fgrep -l "cpu.model?intel" *); do sed -i 's/&cpu.model?intel//g' $f; done

to check if search strings are valid:

python 
>>> import xmlrpclib
>>> s = xmlrpclib.ServerProxy('http://rome/xmlrpc/deploy.rem')
>>> s.servers.filter('')          # if there's one given server
>>> s.servers.filterEx(['server0', 'server1'], "server0.nodes.name=server1.nodes.name")        # if there's a massive of servers
# perhaps it's '''server0.name''' not '''server0.nodes.name'''

to find jobs inconsistencies

to find ones:

shams@shams pls-jobs$ for f in *.py; do python $f; done
shams@shams tps$ cat pd-regular-extended | wc
    148     148    4636

to test a new job on a given server(s) (in this case both client and master servers are specified):

$ python robot.py schedule --product='Parallels Desktop 5' --builds='5.0.9278.533221' --runlist-path=../jobs/pls-jobs/runlist 
--jobdir=../jobs/pls-jobs/ --owner='user@domain.com' --host-pool-search="nodes.inv_no=8137|nodes.inv_no=8216"

Template:Warning2 Template:Note2

when using runlist, do not forget to place it in pls-jobs directory:

python robot.py schedule --product='Parallels Desktop 5' --builds='5.0.9303.541720' \
--runlist-path=../jobs/pls-jobs/runlist --jobdir=../jobs/pls-jobs/ --owner='user@domain.com' \
 --host-pool-search="location_owner.department='PD at'" --enable-state-mgmt='1' --no-qainv='1'

Default build window is 20. You can specify another window (--window). If you want to specify build and then run tests on it and all the following builds, use --builds='x.x.xxxx.xxxxxx and build notification (--listen-addr='x.x.x.x with IP of server where robot is running):

 python robot.py schedule --product='Parallels Desktop 5' --builds='5.0.9077.527148' --listen-addr='10.30.1.27' 
--testplan-name='pd-regular-extended' --jobdir-path='../jobs/pls-jobs/' --owner='user@domain.com' 
--host-pool-search="location_owner.department='PD at'"

If there are some runs on old builds, you want to execute tests on new one, but consider the tests executed on old ones, too, then you need to specify all the builds in a manner --builds='x.x.old.build' --builds='x.x.another.old.build --builds='x.x.new.build
If there's a need to execute one or several jobs on a given man, it's possible to use --runlist-path option (it includes only test titles, not options like it was in run.py) and --host-pool-search="nodes.inv_no=ID'":

$ python robot.py schedule --product='Parallels Desktop 5' --builds='5.0.9142.528763' --runlist-path='../../logs/usbcv-sample'
 --jobdir-path='/home/shams/prltests/logs/usb-jobs/' --owner='user@domain.com' 
--host-pool-search="nodes.inv_no='8137'"

Template:Warning2

$ for f in $(fgrep -l 'w2k3-ent-sp2-x86' * | grep vzt_pss | grep i386); do sed -i "s/w2k3-ent-sp2-x86/winxp-sp3-i386/g" $f; done
$ for f in $(fgrep -l 'w2k3-ent-sp2-x86_64' * | grep vzt_pss | grep x86_64); do sed -i "s/w2k3-ent-sp2-x86_64/winxp-sp2-x86_64/g" $f; done

to add a line before pattern line

$ for f in vzt_usb*py; do sed -i '/nodes.type<8/i\\t# FIXME: this search string is to workaround bug #460648; \ 
replace with Mac type number' $f; done

scheduling for UTW7:

$ python robot.py schedule --product='Parallels Desktop UTW7' --builds='5.0.10714.546622' --testplan-name='utw7-Intel' 
--listen-to-addr='10.30.1.27' --enable-state-mgmt='1'
 --jobdir=../jobs/pls-jobs/ --owner='user@domain.com' --host-pool-search="location_owner.department='PDUtW7-Intel'"

Farm constraints fixes

to limit host search (farm constraints) in robot/jobs/pls-jobs/configure.py:

job.farm_constraints(
                        str(client) + '.link.other=' + \
                        str(server) + '.nodes.inv_no&' + \
                        str(client) + '.link.type=3&' + \
                        str(server) + ".cpu.arch?'%intel%'")

to limit host search (farm constraints) for usb jobs, right in parent jobs:

job.farm_constraints(
                        str(server) + ".link.other=" +
                        str(gadget) + ".nodes.inv_no&" +
                        str(server) + ".link.type=6&" + # 6 = USB server
                        str(server) + ".cpu.arch?'%intel%'"
                        )

Fake robot run

To emulate scheduler's work (fake robot run, with generating jobs and processing them, but no real server deployment) you need to edit robot/src/Scheduler.py. Comment a line id = self.deployJobWait(tp_current[i].title, job_doc) instead of #id = None. Once id is None, this is a fake run.

problem-reports

Template:Note2

There are two ways to send problem reports:
For a given VM (you need to specify VM ID in this case):

# prlctl problem-report VM_ID --send 
### or you can redirect this output to the file to send later: prlctl problem-report VM_ID > problem-report.tar.gz

For the whole server, this will catch all the crashes and all the spoiled things for last 7 days:

# prlsrvctl problem-report --send --stand-alone

If we've got a panic@, full path to detailed and mini dumps is specified in problem report in vm.log:

19:33:11.753 F /vm:2232:9c/ CMonitorDumpBuilder::Create(C:/vz/vm/win_vista_sp2_x32-15-Feb-2010-19.07.06.pvm, 0, #14)
19:33:11.753 F /vm:2232:9c/ CMonitorDumpBuilder::Create() full C:/vz/vm/win_vista_sp2_x32-15-Feb-2010-19.07.06.pvm/monitor.dmp, mini C:/vz/vm/win_vista_sp2_x32-15-Feb-2010-19.07.06.pvm/2010-02-15-193311.dmp

MonData has been changed a bit, thus we've got the error while using mem2cd.exe to generate a dump:

C:\Users\Administrator>mem2cd.exe vista-sp2-x32.mem MonData.txt vista-sp2-bsod.dmp
opening monitor data file "MonData.txt"
parse failure at "gdtr base 8167f000 limit 03ff
" against format "ES: sel %hx"

To avoid this error (until mem2cd is fixed) you need to remove the following lines from the MonData.txt file:

dr0 00000000                                                       
dr1 00000000                                                       
dr2 00000000                                                       
dr3 00000000                                                       
dr6 ffff0ff0
dr7 00000400

Generating dumps

Now it's possible to create memory dumps using product tools (everyone hug Max Nestratov!):

prlctl internal VMID dbgdump --file FILENAME --path DIRECTORY --wait TIMEOUT

where:

VMID is (surprise!) VM ID.
FILENAME is a name of generated dump (it's memory.dmp by default)
DIRECTORY is a full path to directory with a dump (it's VM home directory by default)
TIMEOUT lets you to specify (in seconds) how long the command will wait for a dump to be created (it's 60 sec by default)

Template:Warning2 Template:Note2 Some mind blowing insights about generating dumps are here: [1]

Performance counters

Fixme: Add meaningful details for perfcounters

If guest hangs after something happened inside him (like, guest OS hibernate), it's not bad idea to add perfcounters to the bug.

C:\Users\Administrator>prl_perf_ctl -a -n -l 5
# prl_perf_ctl -a -l 10 -c vcpu

[2]

Uploading the whole VM to the storage

If it's impossible to generate a report, it's worth to save a VM somewhere (for instance, on racoon). It's a pain to use scp/rdesktop, so (thanks to nadyak@) you can access racoon right from the test server (use your domain credentials):

C:\Users\Administrator>net use Y: \\racoon\Memory.Dmps /user:USER
Enter the password for 'USER' to connect to 'racoon'
The command completed successfully.

Then create a directory on Y: (there's a policy: every dir should be a bug number without #) and copy there whatever you want:

C:\Users\Administrator>copy win_vista_sp2_x32.pvm Y:\473518\                                                                               
win_vista_sp2_x32.pvm\2010-04-22-024551.dmp
win_vista_sp2_x32.pvm\2010-04-22-030830.dmp
win_vista_sp2_x32.pvm\config.pvs

bugzilla template on 'how to reproduce a bug' including automated test run:

On Win hosts:

1. Download test tarball (http://testrepo.qa/pub/vzt/vzt-pgov.tar.bz2) to C:\ and uncompress it.
2. python C:\vzt-pgov\vzt-pgov\bin\vzt-pgov  --plssdk --prl-cpus 1 --timeout 7000 --prl-vmm 32 --testset
stw-x32 -L -D4 --title=vzt-pgov-stw-up-Intel32-hvt
3. python C:\vzt-pgov\vztlib\lib\vztests\export_results.py -d C:\vzt-pgov\vzt-pgov\work\   (to export
results).

On Lin/Mac hosts:

Fixme: Add strings for Lin/Mac hosts: curl/tar/python + results

SDK related stuff:

mac33:~ root# python
Python 2.5.4 (r254:67916, Feb 11 2010, 00:50:55) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import prlsdkapi
>>> prlsdkapi.init_desktop_sdk()
>>> s = prlsdkapi.Server()
>>> s.login_local()
<prlsdkapi.Job object at 0x19aeb0>
>>> s.login_local().wait()
<prlsdkapi.Result object at 0x19f430>

git

If you've got series of committed patches, format and send them as a pack:

$ git format-patch -3 -n --thread
$ git-send-email --to bla@bla --cc bla@bla --cc bla@bla --dry-run 000*  
#to check if things are OK before sending then remove --dry-run

If you fed up to your ears with serial numbers on patches, you can omit numerating. By default, the subject of a single patch is

"[PATCH] First Line" and
       the subject when multiple patches are output is "[PATCH n/m] First
       Line". To force 1/1 to be added for a single patch, use -n. To omit
       patch numbers from the subject, use -N.

Colors in git-log:

$ GIT_PAGER="vim -R -" git log origin.. -p

to apply a patch sent but not committed yet:

vzt-patch -> save the patch as some.eml
go to git directory

apply the patching using git-am command (stands for git apply message):

$ git-am ~/prltests/logs/piglit/piglit-separator-patch.eml 
Applying: vzt_prep: added check for path separator at the end to name processing
b866e70557a29e7b4123ecaf801e419f7d7fd116

if you've got several different groups of files you want to commit independently, it's worth to use git-add for one group, then git-commit, then to do the same for another group.

To fix things in already committed patch (it's not on the top of the log):

~ git-log  ; to get the commit ID
~ git rebase -i commit-ID-long-series-of-letters-and-numbers^ ; 'edit' commit; wq
~ vim file-included-in-commit
~ git add file-included-in-commit
~ git commit --amend
~ git rebase --continue
~ git format-patch -N origin
~ git send-email --to --cc patch-fixed

To fix things in already committed patch (it's on top of the log):

~ git log origin..HEAD    ; to make sure it's really on top
~ vim file-included-in-commit
~ git commit -a --amend   ; to merge new changes with the old patch
~ git format-patch -N origin
~ git send-email --to --cc patch-fixed

vzt-results test runs queries

title (this was from the beginning):

title.title like 'vzt-pss-stress'
title.title like 'vzt-pss-fixed%num%rhel%'

hostname (just like the corresponding field on prlcov):

hardware.hostname='mac28'

other hardware details (like cpu, cores, cpu vendor):

hardware.cpus_vendor=''
hardware.cpus_family=''
hardware.cpus_count='4'
hardware.cpus_cores='8'
hardware.mem_size=''
hardware.cpus_flags=''

product (like the corresponding field on prlcov):

product_name.name='parallels-desktop' (or 'psbm' etc.)
Possible search strings for test title and product looks somewhat like:
title.title like 'vzt-parallels-snapshots-smp%' and product_name.name='parallels-desktop'

build (like in corresponding field on prlcov):

product_name.name='parallels-desktop' and 
product.version='5.0.9046.448229' (for psbm you need to specify it like 
'3.0.0-640')

invalid runs:

invalid=1

rome server search queries

New version of web client installed on rome together with updated client app and service ifself. It’s now possible to query(apply filter) on servers both from web client and stand-alone client. Query syntax is the same as in previous version except that new operator ‘?’ has been added. It’s used to search for substrings. Sample: “nodes.name?test” will find you all servers where name contain “test” substring. Sample: “cpu.arch=’i386’&net.driver=’e1000’” will find you all severs with i386 and Intel’s Gb Ethernet nics. Here is a list of what you can search right now:

nodes.inv_no,
nodes.name,
nodes.dirty,
nodes.online,
cpu.arch,
cpu.dev,
cpu.core,
cpu.ht,
cpu.total,
cpu.speed,
cpu.model,
cpu.vtd,
сpu.npt,    (rvi on amd/ept on intel)
net.mac,
net.driver,
net.speed,
net.ip,
net.mask,
net.mac,
net.ext,
net.basic,
net.port,
disks.drive,
disks.size,
mem.type,
mem.size,
mem.speed,
install_state.image_id,
install_state.os_id,
images.name,
images.os_id,
os_list.name,
os_list.type,
os_list.major,
os_list.minor,
location_owner.owner,
location_owner.department,
location_owner.comments,
location_owner.location,
serial_consoles.dev_num,
serial_consoles.speed,
serial_consoles.state

ToDO

script to check if nodes are plugged to conman and are writing to console logs

Fixme: Add meaningful details

(07:05:38 PM) shams: оно же покорраптит кусок кода а ещё можно запихать его в VIM и сделать там :TOhtml в новом окне появляется тот код, отформатированный в HTML начиная с тега следующего после body если не расцветил, надо сказать :set filetype=diff (где код), а не HTML кусок Поубирать [br], если стоит галочка autoformatting

OpenVZ Virtuozzo Containers Wiki β

Shams

ssh authentication for non-root users in Mac OS X NetRestore images

/etc/vztt/url.map

Assorted pieces of knowledge

Different Python version specified directly in jobs

Installing products on Ubuntu (disk partitioning work-around)

PMC for PWE Lin servers workaround

Missing libraries while test runs

VMs' flags

DNS dynamic dhcp clients

fedora/adobe sound troubles fix howto

winsshd for win guests setting up

To get more info on dhcp conficts

To create a custom WinPE image

To set X for lin nodes

Custom packages install

Manual whole group install

start X:xdpyinfo problems

To view screens info

To resurrect deploy after power failure and when deploy is fucked up in general

To blow life into freshly deployed win server networking

To reinstall mac via command line

PD5 mac hints: vnc enabling, knowing an arch of a given mac, installing a build manually

To reinstall lin server via command line

To deploy PSBM via tssetup-psbm

To start/stop VM and check the time

To check if weekly report yaml is formatted properly

launchctl: macosx processes

dispatcher restart

to enable net boot for old school servers

to create a stand-alone VM using setupvm

to remove VMs in cycle

Win hosts

Lin/Mac hosts

code coverage hints and tips

install license for Parallels products via cmd

OS ISOs locations

OS install on the fly using vzt-sample

mounting samba shares

on Linux

on Windows

to burn something on CD

If repo lacks some packages

robot.py hints and tips

create a job

jobs

to replace something in the titles

to replace something in job lines

to find jobs inconsistencies

to add a line before pattern line

Farm constraints fixes

Fake robot run

problem-reports

Generating dumps

Performance counters

Uploading the whole VM to the storage

git

vzt-results test runs queries

rome server search queries

ToDO

OpenVZ Virtuozzo Containers Wiki ^β