See my recent posting in LKML "2.6.32 stalls as guest in virtualbox"
for more details that might be missing.
It seems a virtualbox guest running the Linux kernel 2.6.32 in certain
configurations like the one attached cannot deal with the code created
by the certain alternative_io for cmpxchg64 in
arch/x86/include/asm/cmpxchg_32.h. This causes the kernel to stall
rather early in the boot process as run as apply_alternatives is run.
Reverting commit 152f9d0710a62708710161bce1b29fa8292c8c11 works around
the problem by avoiding the code that calls cmpxchg64 and
'alternative_io("call cmpxchg8b_emu", "lock; cmpxchg8b (%%esi)" (...)'
inside of it.
Workarounds:
- Disable ACPI (i.e. acpi=off in the kernel command line)
- Enable VT-x/AMD-V (reportedly, couldn't check)
- Change the CPU to CONFIG_M686
Some of my findings: It appears apply_alternatives confuses the
virtualized kernel terribly.
The following patch
diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
index de7353c..48fbb20 100644
--- a/arch/x86/kernel/alternative.c
+++ b/arch/x86/kernel/alternative.c
@@ -210,6 +210,7 @@ void __init_or_module apply_alternatives(struct alt_instr *start,
DPRINTK("%s: alt table %p -> %p\n", __func__, start, end);
for (a = start; a < end; a++) {
u8 *instr = a->instr;
+printk("apply_alternatives at %p\n", instr);
BUG_ON(a->replacementlen > a->instrlen);
BUG_ON(a->instrlen > sizeof(insnbuf));
if (!boot_cpu_has(a->cpuid))
@@ -225,8 +226,11 @@ void __init_or_module apply_alternatives(struct alt_instr *start,
memcpy(insnbuf, a->replacement, a->replacementlen);
add_nops(insnbuf + a->replacementlen,
a->instrlen - a->replacementlen);
+printk("apply_alternatives: do it\n");
text_poke_early(instr, insnbuf, a->instrlen);
+printk("apply_alternatives: done this\n");
}
+printk("apply_alternatives: Here we go\n");
}
#ifdef CONFIG_SMP
yields as last messages
(...)
apply_alternatives at c1028dc1
apply_alternatives: do it
apply_alternatives: done this
apply_alternatives at c1028e5d
apply_alternatives: do it
apply_alternatives: done this
apply_alternatives at c10290c2
apply_alternatives: do it
I.e. there was no return from text_poke_early after patching
kernel/sched_clock.c
Checking vmlinux verifies the instruction at c10290c2 is indeed
cmpxchg8b_emu in sched_clock_local (kernel/sched_clock.c).
Another bit, technically disabling the concept of alternative_io for
cmpxchg8b by using the same code for what should be emulation and
alternative as an ugly workaround:
--- a/arch/x86/include/asm/cmpxchg_32.h
+++ b/arch/x86/include/asm/cmpxchg_32.h
@@ -317,7 +317,7 @@ extern unsigned long long cmpxchg_486_u64(volatile void *, u64, u64);
__typeof__(*(ptr)) __ret; \
__typeof__(*(ptr)) __old = (o); \
__typeof__(*(ptr)) __new = (n); \
- alternative_io("call cmpxchg8b_emu", \
+ alternative_io("lock; cmpxchg8b (%%esi)", \
"lock; cmpxchg8b (%%esi)" , \
X86_FEATURE_CX8, \
"=A" (__ret), \
fixes the problem and seems to show the problem is in modification of
the program, not writing it.
Now I'm stuck.
Version numbers and stuff:
- virtualbox-ose 3.0.8 (backport) running on Debian lenny, both 32bit and 64bit hosts
- Also verified on virtualbox-ose 3.1.2 running on Debian squeeze (32bit)
- guest kernel 2.6.32.6 (always 32bit), built using Debian squeeze. The config is attached
- Host CPU (32bit):
model name : Intel(R) Pentium(R) M processor 1600MHz
flags : fpu vme de pse tsc msr mce cx8 sep mtrr pge mca cmov clflush dts acpi mmx fxsr sse sse2 tm pbe bts est tm2
- Host CPU (64bit):
model name : Intel(R) Atom(TM) CPU 330 @ 1.60GHz
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl tm2 ssse3 cx16 xtpr lahf_lm
