OpenVZ Virtuozzo Containers Wiki - User contributions [en]

Containers/Mini-summit 2008 notes

2008-07-23T19:32:47Z

Paulmenage:

[[Category: Containers]]

Intros (8:36am)

Dave Hansen
Eric Biederman
Jason Byron, Red Hat
Joe Ruscio, Evergrid
Joe McDonald
HP China
Sonny Rao
HP
HP
Matine Silberman HP
Sandy Harris
NEC Japan
John Schultz, AOL
Pavel Emelyanov, Parallels/OpenVZ
Denis Lunev, Parallels/OpenVZ
Andrey Mirkin, Parallels/OpenVZ
Constant Chan
Benjamin Thery, Bull
Daniel Lezcano, IBM
Serge Hallyn, IBM
Oren Laadan, Columbia University

On Phone:
Amy Griffis, HP
Dhaval Giani, IBM
Peter Zijlstra

(Later walk-ins):
Paul Menage, Google

== Namespaces and containers ==

Why do various companies want containers?
IBM, Google: workload management
EB: using containers as improved chroot
HP: wants similar to ibm, plus security
parallels: hosted providers

sysfs issues
EB gives status: should go into next merge window

mini-namespaces
NFS
clients should behave differently on diff. containers
currently uses single sunrpc transport for all containers
Dave: is there a list of all openvz mini-ns?
EB:
proposal:
create little filesystems
still store everything in nsproxy
currently:
some people want same process in different netns's
almost possible now, but can't open new sockets
namespace enter:
3 purposes
login
monitoring
configuring
may be worth prototyping the proposal
address mqns, or sunrpc, or fuse?
DH:
openvz addresses this using one big clone(), right?
(yes)

userid namespaces
EB summarizes his proposal
userid ns is unsharable without privilege
userids, capabilities, security labels become ns-local
hierarchical like pidns
openvz: just does chroot
DH:
observers that system vs. app containers have different requirements
EB:
so with userid namespaces, user has god-like powers over created namespaces
EB+SH will talk about hacking something this week during ols
Uses:
user unttrusted mounts
build systems

device namespaces
tty namespaces rejected
should be solved with generic device namespaces
virtualize the major:minor->device mapping
reserved device numbers (unnamed)
created with /proc?
get_unnamed_device()
tty ideas:
use selinux ptys
use user namespaces
use legacy ptys
leverage ptyfs
Suka is not on, so he gets volunteered to do pure /dev/pts fs approach

per-container LSMs:
SH: thinks LSMs should handle it
EB:
original purpose of chroot
set up policies from inside container
creating smack container inside selinux would be ideal

entering a container
netns: identified using pid of a ns
sh: can we solve this using EB's namespace filesystems proposal?
(EB goes to the board to demonstrate his proposal)
PM: Can we use control groups?
PE: Can we re-use /proc/pid/ ?
EB: could have a ns with no processes in it
Example of command using this:
ip set eth0 netns <pid>
becomes
ip set eth0 netns /proc/<pid>/
DL:
a real netns problem is knowing when a childns has died
the netnsfs mount could solve that
PE: EB, can you send POC patches for the namespace?
EB and EM will both send their own POC.

DL: people have complained about needing CAP_SYS_ADMIN to unshare ns
EB: example, setuid root sysvipc-using program could be fooled

PE: Entering a container:
reasons:
monitoring
enter an administrative command
DH: how do you do it now?
PE: numerical ID for each VE, use it to enter
EB:
one need for entering: /sbin/hotplug
(someone): does hijack suffice?
EB: two cases:
partial entering
full entering
sys_hijack does not address partial entering
DH:
why need partial entering?
fs stuff can be done without entering
PM: privileged process
PE:
will look at hijack patches
someone will re-send hijack to containers@
EB:
if we can do sys_hijack cleanly,
we can use it to solve kthread problem

== Control Groups and Resource Management ==

== Checkpoint/Restart [CR] ==

=== Uses of CR ===

* '''migration and live migration:''' e.g. for load balancing, maintenance, clusters and SSIs, etc. may or may not assume a shared file system between endpoints

* '''suspend/resume (aka hibernation):''' e.g. for hibernation, gang-scheduling and priority running, OS maintenance

* '''failure recovery / fault tolerance:''' periodic checkpoints, and restart from most recent (unlike the previous scenarios, here the applications continue to execute after the checkpoint, perhaps modify the file system)

* '''time-travel:''' periodic checkpoints and restart from any previous checkpoint (here, too, attention is required to capturing the state of the file system as well)

* [PE] '''fast-launch:''' reduce start-up time of heavy applications by restarting from a preset checkpoint instead of launching from scratch.

* [EB] '''remote fork:''' e.g. in a cluster

(the last two scenarios are likely to require adjustments during,
or after, the restart to tolerate changes in the file system or
otherwise in the environment)

* [EB,OL] '''distributed checkpoint:''' the ability to checkpoint and restart a distributed application
across multiple nodes as a whole.

EB reminded that at the last kernel summit nobody complained about the
wish to add CR capabilities to the kernel. The issue was and remains
related to technical choices.

=== General design ===

* '''Kernel-space vs user-space'''

OL: the issue of kernel-space vs. user-space is pivotal to design.
kernel support is mandatory to provide completeness and transparency.
Even the recent experience with "cryo" demonstrated that users-space
requires the kernel to expose a very fine-grained API.

Everyone agreed to aim at a monolithic interface, such that nearly
all of the CR will be done in the kernel. The kernel will return
(checkpoint) or receive (restart) a blob with the image of the state
of the container.

* '''Kernel-module ?'''

OL: can we implement mostly in a kernel module and then move CR into
the kernel later ?

EB: better to add CR functionality gradually directly to the kernel.

* '''Compatibility between kernels'''

DLu: there is an issue with compatibility between kernels - even same
kernel compiled with different options and/or compiler, and also if
the kernel ABI changes.

OL: suggest to use an intermediate representation for the checkpoint
image to avoid the issue as much as possible; conversion, if needed,
will take place with userland tools. No aim to bridge ABI changes in
case of migration: instead, fail the restart.

EB: format the blob such that userland tools it will be possible to
parse it and easily detect a version/configuration mismatch.

* '''Streaming checkpoint image ?'''

DLu: using sequential file (non seek-able) like a socket for the
checkpoint image is a challenge.

OL: with proper planning it is not complicated to achieve, and it has
advantage of possible to pass through a filter, e.g. for compression,
encryption, format conversion etc.

* '''Checkpoint operation'''

The procedure will entail five steps:
# Pre-dump
# Freeze the container
# Dump
# Thaw/Kill the container
# Post-dump

"pre-dump" works before freezing the container, e.g. the pre-copy for
live migration and minimize application downtime.

"post-dump" works after the container resumes execution, e.g. in the
case of a checkpoint (not migration) write-back the data to secondary
storage, again to minimize application downtime.

OL: we should be able to checkpoint from inside the container, keep
that in mind for later (also relates to the freezer).

* '''Restart operation'''

Restart is done by first creating a container, then creating the
process tree in it, and then each process restores its own state.
This allows to re-use existing kernel code (e.g., restoring a memory
region is a simple matter of calling mmap() and populating it).

OL: suggest that the process tree be created in userspace.

DLu: prefer to do everything, including process creation, in the
kernel, his experience shows that it isn't difficult.

* '''Error recovery'''

Should checkpoint fail, the container should continue execution
without noticing it. If either checkpoint or restart fail, there
should be a way to inform the caller/user of the reason (something
more informative than -EBUSY).

=== Road plan ===

A this point we want to create a proof of concept and CR a simple
application. We will add iteratively more and more kernel resources.

The first items to address:
# Create a container object (the context on which CR operates)
# Extend the container freezer cgroup ?)
# Interface via syscall or ioctl ?

First step - a simple application:
a single process, not using any files, no signal pending, no IPC etc.
Need to save state (registers, IDs), memory maps and contents (except
for read-only portions, e.g. text).
Assume that the file system state doesn't change between checkpoint
and restart.

Next steps:
# process hierarchy and relationships (multiple tasks and zombies)
# multiple threads (and shared memory)
# open files: regular file, fifo, pipe, socket-pair
# signals, timers
# TBD

=== Documentation ===

DH: proof of concept requires explicit documentation of what can be
checkpointed and what cannot be checkpointed, as well as what will
be the error returned in response to a failure.

Containers/Mini-summit 2008 notes

2008-07-23T19:31:38Z

Paulmenage:

[[Category: Containers]]

Intros (8:36am)

Dave Hansen
Eric Biederman
Jason Byron, Red Hat
Joe Ruscio, Evergrid
Joe McDonald
HP China
Sonny Rao
HP
HP
Matine Silberman HP
Sandy Harris
NEC Japan
John Schultz, AOL
Pavel Emelyanov, Parallels/OpenVZ
Denis Lunev, Parallels/OpenVZ
Andrey Mirkin, Parallels/OpenVZ
Constant Chan
Benjamin Thery, Bull
Daniel Lezcano, IBM
Serge Hallyn, IBM
Oren Laadan, Columbia University

On Phone:
Amy Griffis, HP
Dhaval Giani, IBM
Peter Zijlstra

(Later walk-ins):
Paul Menage, Google

== Namespaces and containers ==

Why do various companies want containers?
ibm: workload management
EB: using containers as improved chroot
HP: wants similar to ibm, plus security
parallels: hosted providers

sysfs issues
EB gives status: should go into next merge window

mini-namespaces
NFS
clients should behave differently on diff. containers
currently uses single sunrpc transport for all containers
Dave: is there a list of all openvz mini-ns?
EB:
proposal:
create little filesystems
still store everything in nsproxy
currently:
some people want same process in different netns's
almost possible now, but can't open new sockets
namespace enter:
3 purposes
login
monitoring
configuring
may be worth prototyping the proposal
address mqns, or sunrpc, or fuse?
DH:
openvz addresses this using one big clone(), right?
(yes)

userid namespaces
EB summarizes his proposal
userid ns is unsharable without privilege
userids, capabilities, security labels become ns-local
hierarchical like pidns
openvz: just does chroot
DH:
observers that system vs. app containers have different requirements
EB:
so with userid namespaces, user has god-like powers over created namespaces
EB+SH will talk about hacking something this week during ols
Uses:
user unttrusted mounts
build systems

device namespaces
tty namespaces rejected
should be solved with generic device namespaces
virtualize the major:minor->device mapping
reserved device numbers (unnamed)
created with /proc?
get_unnamed_device()
tty ideas:
use selinux ptys
use user namespaces
use legacy ptys
leverage ptyfs
Suka is not on, so he gets volunteered to do pure /dev/pts fs approach

per-container LSMs:
SH: thinks LSMs should handle it
EB:
original purpose of chroot
set up policies from inside container
creating smack container inside selinux would be ideal

entering a container
netns: identified using pid of a ns
sh: can we solve this using EB's namespace filesystems proposal?
(EB goes to the board to demonstrate his proposal)
PM: Can we use control groups?
PE: Can we re-use /proc/pid/ ?
EB: could have a ns with no processes in it
Example of command using this:
ip set eth0 netns <pid>
becomes
ip set eth0 netns /proc/<pid>/
DL:
a real netns problem is knowing when a childns has died
the netnsfs mount could solve that
PE: EB, can you send POC patches for the namespace?
EB and EM will both send their own POC.

DL: people have complained about needing CAP_SYS_ADMIN to unshare ns
EB: example, setuid root sysvipc-using program could be fooled

PE: Entering a container:
reasons:
monitoring
enter an administrative command
DH: how do you do it now?
PE: numerical ID for each VE, use it to enter
EB:
one need for entering: /sbin/hotplug
(someone): does hijack suffice?
EB: two cases:
partial entering
full entering
sys_hijack does not address partial entering
DH:
why need partial entering?
fs stuff can be done without entering
PM: privileged process
PE:
will look at hijack patches
someone will re-send hijack to containers@
EB:
if we can do sys_hijack cleanly,
we can use it to solve kthread problem

== Checkpoint/Restart [CR] ==

=== Uses of CR ===

* '''migration and live migration:''' e.g. for load balancing, maintenance, clusters and SSIs, etc. may or may not assume a shared file system between endpoints

* '''suspend/resume (aka hibernation):''' e.g. for hibernation, gang-scheduling and priority running, OS maintenance

* '''failure recovery / fault tolerance:''' periodic checkpoints, and restart from most recent (unlike the previous scenarios, here the applications continue to execute after the checkpoint, perhaps modify the file system)

* '''time-travel:''' periodic checkpoints and restart from any previous checkpoint (here, too, attention is required to capturing the state of the file system as well)

* [PE] '''fast-launch:''' reduce start-up time of heavy applications by restarting from a preset checkpoint instead of launching from scratch.

* [EB] '''remote fork:''' e.g. in a cluster

(the last two scenarios are likely to require adjustments during,
or after, the restart to tolerate changes in the file system or
otherwise in the environment)

* [EB,OL] '''distributed checkpoint:''' the ability to checkpoint and restart a distributed application
across multiple nodes as a whole.

EB reminded that at the last kernel summit nobody complained about the
wish to add CR capabilities to the kernel. The issue was and remains
related to technical choices.

=== General design ===

* '''Kernel-space vs user-space'''

OL: the issue of kernel-space vs. user-space is pivotal to design.
kernel support is mandatory to provide completeness and transparency.
Even the recent experience with "cryo" demonstrated that users-space
requires the kernel to expose a very fine-grained API.

Everyone agreed to aim at a monolithic interface, such that nearly
all of the CR will be done in the kernel. The kernel will return
(checkpoint) or receive (restart) a blob with the image of the state
of the container.

* '''Kernel-module ?'''

OL: can we implement mostly in a kernel module and then move CR into
the kernel later ?

EB: better to add CR functionality gradually directly to the kernel.

* '''Compatibility between kernels'''

DLu: there is an issue with compatibility between kernels - even same
kernel compiled with different options and/or compiler, and also if
the kernel ABI changes.

OL: suggest to use an intermediate representation for the checkpoint
image to avoid the issue as much as possible; conversion, if needed,
will take place with userland tools. No aim to bridge ABI changes in
case of migration: instead, fail the restart.

EB: format the blob such that userland tools it will be possible to
parse it and easily detect a version/configuration mismatch.

* '''Streaming checkpoint image ?'''

DLu: using sequential file (non seek-able) like a socket for the
checkpoint image is a challenge.

OL: with proper planning it is not complicated to achieve, and it has
advantage of possible to pass through a filter, e.g. for compression,
encryption, format conversion etc.

* '''Checkpoint operation'''

The procedure will entail five steps:
# Pre-dump
# Freeze the container
# Dump
# Thaw/Kill the container
# Post-dump

"pre-dump" works before freezing the container, e.g. the pre-copy for
live migration and minimize application downtime.

"post-dump" works after the container resumes execution, e.g. in the
case of a checkpoint (not migration) write-back the data to secondary
storage, again to minimize application downtime.

OL: we should be able to checkpoint from inside the container, keep
that in mind for later (also relates to the freezer).

* '''Restart operation'''

Restart is done by first creating a container, then creating the
process tree in it, and then each process restores its own state.
This allows to re-use existing kernel code (e.g., restoring a memory
region is a simple matter of calling mmap() and populating it).

OL: suggest that the process tree be created in userspace.

DLu: prefer to do everything, including process creation, in the
kernel, his experience shows that it isn't difficult.

* '''Error recovery'''

Should checkpoint fail, the container should continue execution
without noticing it. If either checkpoint or restart fail, there
should be a way to inform the caller/user of the reason (something
more informative than -EBUSY).

=== Road plan ===

A this point we want to create a proof of concept and CR a simple
application. We will add iteratively more and more kernel resources.

The first items to address:
# Create a container object (the context on which CR operates)
# Extend the container freezer cgroup ?)
# Interface via syscall or ioctl ?

First step - a simple application:
a single process, not using any files, no signal pending, no IPC etc.
Need to save state (registers, IDs), memory maps and contents (except
for read-only portions, e.g. text).
Assume that the file system state doesn't change between checkpoint
and restart.

Next steps:
# process hierarchy and relationships (multiple tasks and zombies)
# multiple threads (and shared memory)
# open files: regular file, fifo, pipe, socket-pair
# signals, timers
# TBD

=== Documentation ===

DH: proof of concept requires explicit documentation of what can be
checkpointed and what cannot be checkpointed, as well as what will
be the error returned in response to a failure.

Containers/Mini-summit 2008 notes

2008-07-23T19:04:54Z

Paulmenage:

[[Category: Containers]]

Intros (8:36am)

Dave Hansen
Eric Biederman
Jason Byron, Red Hat
Joe Ruscio, Evergrid
Joe McDonald
HP China
Sonny Rao
HP
HP
Matine Silberman HP
Sandy Harris
NEC Japan
John Schultz, AOL
Pavel Emelyanov, Parallels/OpenVZ
Denis Lunev, Parallels/OpenVZ
Andrey Mirkin, Parallels/OpenVZ
Constant Chan
Benjamin Thery, Bull
Daniel Lezcano, IBM
Serge Hallyn, IBM
Oren Laadan, Columbia University

On Phone:
Amy Griffis, HP
Dhaval Giani, IBM

(Later walk-ins):
Paul Menage, Google

== Namespaces and containers ==

Why do various companies want containers?
ibm: workload management
EB: using containers as improved chroot
HP: wants similar to ibm, plus security
parallels: hosted providers

sysfs issues
EB gives status: should go into next merge window

mini-namespaces
NFS
clients should behave differently on diff. containers
currently uses single sunrpc transport for all containers
Dave: is there a list of all openvz mini-ns?
EB:
proposal:
create little filesystems
still store everything in nsproxy
currently:
some people want same process in different netns's
almost possible now, but can't open new sockets
namespace enter:
3 purposes
login
monitoring
configuring
may be worth prototyping the proposal
address mqns, or sunrpc, or fuse?
DH:
openvz addresses this using one big clone(), right?
(yes)

userid namespaces
EB summarizes his proposal
userid ns is unsharable without privilege
userids, capabilities, security labels become ns-local
hierarchical like pidns
openvz: just does chroot
DH:
observers that system vs. app containers have different requirements
EB:
so with userid namespaces, user has god-like powers over created namespaces
EB+SH will talk about hacking something this week during ols
Uses:
user unttrusted mounts
build systems

device namespaces
tty namespaces rejected
should be solved with generic device namespaces
virtualize the major:minor->device mapping
reserved device numbers (unnamed)
created with /proc?
get_unnamed_device()
tty ideas:
use selinux ptys
use user namespaces
use legacy ptys
leverage ptyfs
Suka is not on, so he gets volunteered to do pure /dev/pts fs approach

per-container LSMs:
SH: thinks LSMs should handle it
EB:
original purpose of chroot
set up policies from inside container
creating smack container inside selinux would be ideal

entering a container
netns: identified using pid of a ns
sh: can we solve this using EB's namespace filesystems proposal?
(EB goes to the board to demonstrate his proposal)
PM: Can we use control groups?
PE: Can we re-use /proc/pid/ ?
EB: could have a ns with no processes in it
Example of command using this:
ip set eth0 netns <pid>
becomes
ip set eth0 netns /proc/<pid>/
DL:
a real netns problem is knowing when a childns has died
the netnsfs mount could solve that
PE: EB, can you send POC patches for the namespace?
EB and EM will both send their own POC.

DL: people have complained about needing CAP_SYS_ADMIN to unshare ns
EB: example, setuid root sysvipc-using program could be fooled

PE: Entering a container:
reasons:
monitoring
enter an administrative command
DH: how do you do it now?
PE: numerical ID for each VE, use it to enter
EB:
one need for entering: /sbin/hotplug
(someone): does hijack suffice?
EB: two cases:
partial entering
full entering
sys_hijack does not address partial entering
DH:
why need partial entering?
fs stuff can be done without entering
PM: privileged process
PE:
will look at hijack patches
someone will re-send hijack to containers@
EB:
if we can do sys_hijack cleanly,
we can use it to solve kthread problem

== Checkpoint/Restart [CR] ==

=== Uses of CR ===

* '''migration and live migration:''' e.g. for load balancing, maintenance, clusters and SSIs, etc. may or may not assume a shared file system between endpoints

* '''suspend/resume (aka hibernation):''' e.g. for hibernation, gang-scheduling and priority running, OS maintenance

* '''failure recovery / fault tolerance:''' periodic checkpoints, and restart from most recent (unlike the previous scenarios, here the applications continue to execute after the checkpoint, perhaps modify the file system)

* '''time-travel:''' periodic checkpoints and restart from any previous checkpoint (here, too, attention is required to capturing the state of the file system as well)

* [PE] '''fast-launch:''' reduce start-up time of heavy applications by restarting from a preset checkpoint instead of launching from scratch.

* [EB] '''remote fork:''' e.g. in a cluster

(the last two scenarios are likely to require adjustments during,
or after, the restart to tolerate changes in the file system or
otherwise in the environment)

* [EB,OL] '''distributed checkpoint:''' the ability to checkpoint and restart a distributed application
across multiple nodes as a whole.

EB reminded that at the last kernel summit nobody complained about the
wish to add CR capabilities to the kernel. The issue was and remains
related to technical choices.

=== General design ===

* '''Kernel-space vs user-space'''

OL: the issue of kernel-space vs. user-space is pivotal to design.
kernel support is mandatory to provide completeness and transparency.
Even the recent experience with "cryo" demonstrated that users-space
requires the kernel to expose a very fine-grained API.

Everyone agreed to aim at a monolithic interface, such that nearly
all of the CR will be done in the kernel. The kernel will return
(checkpoint) or receive (restart) a blob with the image of the state
of the container.

* '''Kernel-module ?'''

OL: can we implement mostly in a kernel module and then move CR into
the kernel later ?

EB: better to add CR functionality gradually directly to the kernel.

* '''Compatibility between kernels'''

DLu: there is an issue with compatibility between kernels - even same
kernel compiled with different options and/or compiler, and also if
the kernel ABI changes.

OL: suggest to use an intermediate representation for the checkpoint
image to avoid the issue as much as possible; conversion, if needed,
will take place with userland tools. No aim to bridge ABI changes in
case of migration: instead, fail the restart.

EB: format the blob such that userland tools it will be possible to
parse it and easily detect a version/configuration mismatch.

* '''Streaming checkpoint image ?'''

DLu: using sequential file (non seek-able) like a socket for the
checkpoint image is a challenge.

OL: with proper planning it is not complicated to achieve, and it has
advantage of possible to pass through a filter, e.g. for compression,
encryption, format conversion etc.

* '''Checkpoint operation'''

The procedure will entail five steps:
# Pre-dump
# Freeze the container
# Dump
# Thaw/Kill the container
# Post-dump

"pre-dump" works before freezing the container, e.g. the pre-copy for
live migration and minimize application downtime.

"post-dump" works after the container resumes execution, e.g. in the
case of a checkpoint (not migration) write-back the data to secondary
storage, again to minimize application downtime.

OL: we should be able to checkpoint from inside the container, keep
that in mind for later (also relates to the freezer).

* '''Restart operation'''

Restart is done by first creating a container, then creating the
process tree in it, and then each process restores its own state.
This allows to re-use existing kernel code (e.g., restoring a memory
region is a simple matter of calling mmap() and populating it).

OL: suggest that the process tree be created in userspace.

DLu: prefer to do everything, including process creation, in the
kernel, his experience shows that it isn't difficult.

* '''Error recovery'''

Should checkpoint fail, the container should continue execution
without noticing it. If either checkpoint or restart fail, there
should be a way to inform the caller/user of the reason (something
more informative than -EBUSY).

=== Road plan ===

A this point we want to create a proof of concept and CR a simple
application. We will add iteratively more and more kernel resources.

The first items to address:
# Create a container object (the context on which CR operates)
# Extend the container freezer cgroup ?)
# Interface via syscall or ioctl ?

First step - a simple application:
a single process, not using any files, no signal pending, no IPC etc.
Need to save state (registers, IDs), memory maps and contents (except
for read-only portions, e.g. text).
Assume that the file system state doesn't change between checkpoint
and restart.

Next steps:
# process hierarchy and relationships (multiple tasks and zombies)
# multiple threads (and shared memory)
# open files: regular file, fifo, pipe, socket-pair
# signals, timers
# TBD

=== Documentation ===

DH: proof of concept requires explicit documentation of what can be
checkpointed and what cannot be checkpointed, as well as what will
be the error returned in response to a failure.

Containers/Mini-summit 2008

2008-07-18T18:02:25Z

Paulmenage:

There will be a containers mini-summit at the [http://www.linuxsymposium.org/2008/ OLS'08]. This page is for organizing this mini-summit. Feel free to edit.

'''When''': 22nd of July 2008, 8:30-16:30<br/>
'''Where''': Ottawa, ON, Canada, Novotel Hotel (Albion A).

== Proposal ==

The mini-summit proposal sent to OLS organizers. '''See [[/Proposal|proposal]]'''.

== Topics to discuss ==

* Device accessibility cgroup (maybe with remap ability)
* TTYs
* Syslog
* Checkpoint/restart
* Memory controllers
* more?..

== List of attendees ==
Please fill in your name here if you are going to attend, or email kir at openvz dot org if you are too lazy. Surely the list is not final, so put your name even if you are not sure you can make it.

This list is in no particular order.


<div style="-moz-column-count:3; -webkit-column-count:3; column-count:3; text-align: left; background: #fefef0; border: 1px solid #ddddc0;">
# Pavel Emelyanov
# Denis Lunev
# Andrey Mirkin
# Serge Hallyn
# Dave Hansen
# Daniel Lezcano
# Srivatsa Vaddagiri
# Balbir Singh
# Sukadev Bhattiprolu
# Paul Menage
# Eric W. Biederman
# Oren Laadan
# Yamamoto Takashi
# Kamezawa Hiroyuki
# Benjamin Thery
# Herbert Pötzl
# Oleg Nesterov
# Dhaval Giani
# Bart Trojanowski
# Joseph Ruscio
# Constant Chan
# Linda Knippers
# Satoshi Uchida
# Masahiko Takahashi
# Martine Silbermann
# Benoit des Ligneris
# Patrick Naubert
# Daisuke Nishimura
# Sudhir Kumar
# Munehiro Ikeda
# Kamalesh Babulal
# John Schulz
# Poornima Nayak
# Gyuil Cha
# YoungHo Kim
</div>

== Agenda ==

* Namespaces/Containers (8:30am-11am)
** sysfs issues (and any /proc issues)
*** uevents/hotplug
** Network namespaces issues
*** multiple namespaces in one process
** Device namespace design?
** User namespace
** Additional needed namespaces
*** Small namespaces ''What to do with small subsystem that might need virtualization. E.g. in openvz we have FUSE, binfmt_misc and some other small stuff virtualized. But how to merge it in mainline? Create a separate namespace for each? Mere them into one? How to call this then?''
** Handling filesystem/namespace synchronization (not sure what the issue is)
** Container design
*** How to enter a container
*** Nature of a 'container' — kernel object or userspace fiction

* Cgroups+Resource management (11:30-2pm)
** Cgroup implementation
*** Locking (don't let cgroup_lock() become the BKL)
*** Transactional attachment
*** "procs" file
*** User-space notification API
**** Resource counter hit soft/hard limit
**** Task entered/left cgroup
**** OOM occurred
*** Binary statistics API
** Existing cgroups
*** Memory
**** Supporting over-commit and guarantees
**** Soft-limits
**** Hierarchical borrowing - in kernel or userspace?
**** Per-cgroup refault information?
*** Kernel memory
*** Device
*** Memrlimit
**** Some push-back over this - can we give real use cases?
*** CPU scheduler
** Additional cgroups and their design
*** Swap (separate subsystem or merge with memory?)
*** Disk I/O (several proposed designs)
*** Network traffic classification
*** Freezer
*** Signaller
*** OOM Handler
** libcg - userspace explotation of control groups/resource management
*** Overview so far
*** Is kernel-based reclassification needed?
*** Real use-cases
*** Future directions

* Checkpoint/Restart (2:30pm-5pm)
** Documentation : Look at "See Also" section below
** Goals and expectations of this summit
*** identify, discuss and (if possible) agree on the general design
*** identify, discuss and (if possible) agree on the technical points
*** decide on priorities for different components (eg. high, medium, low) such that the final outcome is a practical road-map that would keep us busy for (at least) until the next OLS (though the "O" may change ;)
** What are the problems that the linux community can solve with the checkpoint/restart ?
** Preparing the kernel internals
*** How we implement it without affecting long term maintainability ?
*** What are the kernel subsystems, process resources and framework for CR ?
*** Which pieces to target first ?

The following technical points can be discussed during the mini-summit if we have time or later at the OLS.

** Checkpointing / Restarting
*** Reaching a quescient point - network, processes, aio, avoiding side effects of quiesce/revive
*** Checkpoint - signal handler ? syscall ? crfs ? process hierarchy, resource dependencies, system and process resources
*** Restarting - New binary format handler ? converting between formats (from older kernel to newer)
*** Notification to processes which explicitly wish to be notified about quiesce, checkpoint and restart - container state ? new signals ?
** Determining the userspace API - Posix 1003.1m ?
** Passing the kernel internal state to/from userspace - coredump like file ? swap per container ? netlinks, CR filesystem ? army of different call for the CR (proc, existing syscalls, ...)
** Hopefully we can continue to discuss in the next days and get a bit of a hackfest going during OLS :)

== Moderators ==

* Namespaces/containers: Serge Hallyn, Dave Hansen
* Cgroups and resource management: Paul Menage, Balbir Singh, Dhaval Giani
* Checkpoint/restart: Daniel Lezcano, Oren Laadan

== See also ==
* http://www.linuxsymposium.org/2008/cfp.php — OLS call for papers
* https://lists.linux-foundation.org/pipermail/containers/2008-January/009688.html
* http://openvz.org/pipermail/devel/2008-July/012891.html
* Checkpoint/Restart
** Zap : http://www.ncl.cs.columbia.edu/publications/usenix2007_fordist.pdf
** Metacluster : http://lxc.sourceforge.net/doc/ols2006/lxc-ols2006.pdf
** OpenVZ : [[Checkpointing and live migration]]
** Checkpoint/Restart technology : http://en.wikipedia.org/wiki/Application_checkpointing
** Virtual Servers and Checkpoint/Restart in Mainstream Linux : Sigops document
** Remote fork: http://www.cse.nd.edu/~dthain/courses/classconf/wowsys2004/talks/rfork.pdf
** Vmadump : http://bproc.sourceforge.net/c268.html
** Posix CR : http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi/0650/bks/SGI_Admin/CPR_OG/sgi_html/ch03.html
** An OS services overview : http://sw-eng.falls-church.va.us/itsg/P08V31.htm

[[Category: Containers]]
[[Category: Events]]

Containers/Mini-summit 2008

2008-07-18T17:58:40Z

Paulmenage:

There will be a containers mini-summit at the [http://www.linuxsymposium.org/2008/ OLS'08]. This page is for organizing this mini-summit. Feel free to edit.

'''When''': 22nd of July 2008, 8:30-16:30<br/>
'''Where''': Ottawa, ON, Canada, Novotel Hotel (Albion A).

== Proposal ==

The mini-summit proposal sent to OLS organizers. '''See [[/Proposal|proposal]]'''.

== Topics to discuss ==

* Device accessibility cgroup (maybe with remap ability)
* TTYs
* Syslog
* Checkpoint/restart
* Memory controllers
* more?..

== List of attendees ==
Please fill in your name here if you are going to attend, or email kir at openvz dot org if you are too lazy. Surely the list is not final, so put your name even if you are not sure you can make it.

This list is in no particular order.


<div style="-moz-column-count:3; -webkit-column-count:3; column-count:3; text-align: left; background: #fefef0; border: 1px solid #ddddc0;">
# Pavel Emelyanov
# Denis Lunev
# Andrey Mirkin
# Serge Hallyn
# Dave Hansen
# Daniel Lezcano
# Srivatsa Vaddagiri
# Balbir Singh
# Sukadev Bhattiprolu
# Paul Menage
# Eric W. Biederman
# Oren Laadan
# Yamamoto Takashi
# Kamezawa Hiroyuki
# Benjamin Thery
# Herbert Pötzl
# Oleg Nesterov
# Dhaval Giani
# Bart Trojanowski
# Joseph Ruscio
# Constant Chan
# Linda Knippers
# Satoshi Uchida
# Masahiko Takahashi
# Martine Silbermann
# Benoit des Ligneris
# Patrick Naubert
# Daisuke Nishimura
# Sudhir Kumar
# Munehiro Ikeda
# Kamalesh Babulal
# John Schulz
# Poornima Nayak
# Gyuil Cha
# YoungHo Kim
</div>

== Agenda ==

* Namespaces/Containers (8:30am-11am)
** sysfs issues (and any /proc issues)
*** uevents/hotplug
** Network namespaces issues
*** multiple namespaces in one process
** Device namespace design?
** User namespace
** Additional needed namespaces
*** Small namespaces ''What to do with small subsystem that might need virtualization. E.g. in openvz we have FUSE, binfmt_misc and some other small stuff virtualized. But how to merge it in mainline? Create a separate namespace for each? Mere them into one? How to call this then?''
** Handling filesystem/namespace synchronization (not sure what the issue is)
** Container design
*** How to enter a container
*** Nature of a 'container' — kernel object or userspace fiction

* Cgroups+Resource management (11:30-2pm)
** Cgroup implementation
*** Locking (don't let cgroup_lock() become the BKL)
*** Transactional attachment
*** "procs" file
*** User-space notification API
*** Binary statistics API
** Existing cgroups
*** Memory
**** Supporting over-commit and guarantees
**** Soft-limits
**** Hierarchical borrowing - in kernel or userspace?
**** Per-cgroup refault information?
*** Kernel memory
*** Device
*** Memrlimit
**** Some push-back over this - can we give real use cases?
*** CPU scheduler
** Additional cgroups and their design
*** Swap (separate subsystem or merge with memory?)
*** Disk I/O (competing designs)
*** Network traffic classification
*** Freezer
*** Signaller
*** OOM Handler
** libcg - userspace explotation of control groups/resource management
*** Overview so far
*** Real use-cases
*** Future directions

* Checkpoint/Restart (2:30pm-5pm)
** Documentation : Look at "See Also" section below
** Goals and expectations of this summit
*** identify, discuss and (if possible) agree on the general design
*** identify, discuss and (if possible) agree on the technical points
*** decide on priorities for different components (eg. high, medium, low) such that the final outcome is a practical road-map that would keep us busy for (at least) until the next OLS (though the "O" may change ;)
** What are the problems that the linux community can solve with the checkpoint/restart ?
** Preparing the kernel internals
*** How we implement it without affecting long term maintainability ?
*** What are the kernel subsystems, process resources and framework for CR ?
*** Which pieces to target first ?

The following technical points can be discussed during the mini-summit if we have time or later at the OLS.

** Checkpointing / Restarting
*** Reaching a quescient point - network, processes, aio, avoiding side effects of quiesce/revive
*** Checkpoint - signal handler ? syscall ? crfs ? process hierarchy, resource dependencies, system and process resources
*** Restarting - New binary format handler ? converting between formats (from older kernel to newer)
*** Notification to processes which explicitly wish to be notified about quiesce, checkpoint and restart - container state ? new signals ?
** Determining the userspace API - Posix 1003.1m ?
** Passing the kernel internal state to/from userspace - coredump like file ? swap per container ? netlinks, CR filesystem ? army of different call for the CR (proc, existing syscalls, ...)
** Hopefully we can continue to discuss in the next days and get a bit of a hackfest going during OLS :)

== Moderators ==

* Namespaces/containers: Serge Hallyn, Dave Hansen
* Cgroups and resource management: Paul Menage, Balbir Singh, Dhaval Giani
* Checkpoint/restart: Daniel Lezcano, Oren Laadan

== See also ==
* http://www.linuxsymposium.org/2008/cfp.php — OLS call for papers
* https://lists.linux-foundation.org/pipermail/containers/2008-January/009688.html
* http://openvz.org/pipermail/devel/2008-July/012891.html
* Checkpoint/Restart
** Zap : http://www.ncl.cs.columbia.edu/publications/usenix2007_fordist.pdf
** Metacluster : http://lxc.sourceforge.net/doc/ols2006/lxc-ols2006.pdf
** OpenVZ : [[Checkpointing and live migration]]
** Checkpoint/Restart technology : http://en.wikipedia.org/wiki/Application_checkpointing
** Virtual Servers and Checkpoint/Restart in Mainstream Linux : Sigops document
** Remote fork: http://www.cse.nd.edu/~dthain/courses/classconf/wowsys2004/talks/rfork.pdf
** Vmadump : http://bproc.sourceforge.net/c268.html
** Posix CR : http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi/0650/bks/SGI_Admin/CPR_OG/sgi_html/ch03.html
** An OS services overview : http://sw-eng.falls-church.va.us/itsg/P08V31.htm

[[Category: Containers]]
[[Category: Events]]