What happens when
you resize a server?
You ask for a bigger flavor, or just a different host. Nova powers the server off, moves its disk and resource claim to a scheduler-selected destination compute, and parks it in VERIFY_RESIZE so you can confirm or undo the move. Throughout, Placement holds the old reservation under a migration UUID. This is the complete journey, traced from the real Nova source.
The cast of characters
A resize is a relay race between two compute hosts, refereed by the conductor and scheduler. Before we trace the messages, meet the players and the two ways they talk.
Two ways to send a message
Every interaction in this story travels over one of two channels. Knowing which is which is the single most useful idea in the whole walkthrough.
Nova's own services (API, conductor, scheduler, the two computes) talk through a message broker like RabbitMQ. A cast is fire-and-forget; a call waits for an answer.
When Nova needs another OpenStack project — Placement, Neutron, Cinder, Glance — it makes an ordinary HTTP REST request, just like a web app calling an API.
If both ends are Nova services, it's RPC on the queue. If the other end is a different OpenStack project, it's REST over HTTP. A resize bounces between the two computes entirely over RPC casts.
Meet the services (click each one)
These are the components your resize request will pass through. Notice there are two compute hosts: the one the server is leaving and the one it is arriving on. Click any box to learn its job.
The journey in one breath
Here is the whole thing compressed into a few beats. The rest of the course unpacks each one.
nova-api checks the flavor, guards quota for upsizes only, updates the RequestSpec, sets RESIZE_PREP, and casts to the conductor.
It creates the Migration record, moves the source allocation to the migration UUID, asks the scheduler for a destination, and casts prep_resize there.
The destination claims resources and casts back to the source, which powers off the guest and copies its disk over.
The destination retargets networking and volumes, recreates the guest at its new size, and lands in vm_state RESIZED (VERIFY_RESIZE).
Confirm deletes the old copy and drops the source allocation. Revert destroys the new copy and restores the original.
API validates and casts
Your request lands in nova/compute/api.py resize(). Nothing moves yet — this phase validates the flavor, guards quota, records intent as RESIZE_PREP, and hands off.
Migration is just resize with the same flavor
If no flavor_id is supplied, Nova assumes a migration and keeps the original flavor. Everything downstream is shared code — a cold migration is a resize that happens not to change size.
# If flavor_id is not provided, only migrate the instance.
volume_backed = None
if not flavor_id:
LOG.debug("flavor_id is None. Assuming migration.",
instance=instance)
new_flavor = current_flavor
else:
new_flavor = flavors.get_flavor_by_flavor_id(
flavor_id, read_deleted="no")
Quota is only checked for upsizes
Resizing up may exceed quota, so Nova checks headroom. Resizing down, or a pure migration, does not need a quota check here.
if flavor_id:
...
self._check_quota_for_upsize(context, instance,
current_flavor,
new_flavor, volume_backed,
is_revert=False)
if flavor_id: — a downsize or a flavor-less migration skips it entirely, because they cannot put you over your limit.
Stamp RESIZE_PREP and cast to conductor
Nova saves the task_state transition, records the action (RESIZE vs MIGRATE), then casts to the conductor with do_cast=True so it does not block on scheduling. The HTTP request returns immediately.
instance.task_state = task_states.RESIZE_PREP
instance.progress = 0
instance.auto_disk_config = auto_disk_config or False
instance.save(expected_task_state=[None])
...
self.compute_task_api.resize_instance(
context, instance,
scheduler_hint=scheduler_hint,
flavor=new_flavor,
...
do_cast=True)
expected_task_state=[None] is an optimistic guard: the save only succeeds if nothing else has already grabbed the instance. The do_cast=True means fire-and-forget, so your CLI returns at once.
The API accepts the request and walks away. Everything from here happens asynchronously while you poll the server status, waiting for it to reach VERIFY_RESIZE.
| RESIZE | resize_prep | qh2-rcc001 (still the source) |
# the migration ledger row is born in the CONDUCTOR — if it exists, the cast arrived
$ openstack server migration list --server $UUID
mysql> SELECT id, status FROM nova.migrations WHERE instance_uuid='$UUID' ORDER BY id DESC LIMIT 1; (cell DB)
# stuck at resize_prep with NO migration row → the cast to nova-conductor was lost;
# check the conductor service and the 'conductor' queue before anything else
Conductor schedules and swaps the allocation
The conductor picks up the cast, routes it to _cold_migrate(), and runs a MigrationTask that does the clever bookkeeping in Placement.
migrate_server routes cold migrate vs live migrate
A request that is not live, not a rebuild, and has a flavor is a cold migration or resize. The conductor decides this and starts the cold-migrate workflow.
elif not live and not rebuild and flavor:
instance_uuid = instance.uuid
with compute_utils.EventReporter(context, 'cold_migrate',
self.host, instance_uuid):
self._cold_migrate(context, instance, flavor,
scheduler_hint['filter_properties'],
clean_shutdown, request_spec,
host_list)
Move the source allocation to the migration UUID
Before scheduling, the instance's existing allocation on the source node is reassigned to the migration UUID. This frees the instance consumer to receive a fresh allocation on the destination, and gives a single object — the migration — that "owns" the source resources for confirm/revert.
success = reportclient.move_allocations(context,
instance.uuid,
migration.uuid)
if not success:
...
raise exception.NoValidHost(
reason=_('Unable to replace instance claim on source'))
NoValidHost, aborting the move cleanly.
Ask the scheduler for a destination
select_destinations returns a selected host plus alternates; as a side effect it claims the new-flavor resources in Placement against the instance UUID on the chosen node. This is the one place a blocking RPC call reaches into the scheduler.
selection_lists = self.query_client.select_destinations(
self.context, self.request_spec, [self.instance.uuid],
return_objects=True, return_alternates=True)
...
selection, self.host_list = selection_list[0], selection_list[1:]
scheduler_utils.fill_provider_mapping(self.request_spec, selection)
return_alternates=True brings back backup hosts so the destination can retry elsewhere if its claim fails. The new allocation lands on the instance UUID, while the migration UUID still holds the source.
Cast prep_resize to the destination compute
The task casts to the chosen destination host to begin the move. If the selected host is in another cell, it instead launches a cell-crossing CrossCellMigrationTask (a snapshot-based resize) instead.
# RPC cast to the destination host to start the migration process.
self.compute_rpcapi.prep_resize(
self.context, self.instance, self.request_spec.image,
self.flavor, host, migration,
request_spec=self.request_spec, filter_properties=legacy_props,
node=node, clean_shutdown=self.clean_shutdown,
host_list=self.host_list)
cast — fire-and-forget — carrying the migration record, the new flavor, and the list of alternate hosts so the destination can reschedule itself if its local claim fails.
$ openstack server migration list --server $UUID
| 42 | qh2-rcc001 | qh2-rcc002 | pre-migrating | migration_type: resize |
mysql> SELECT uuid, status, source_compute, dest_compute, migration_type FROM nova.migrations WHERE instance_uuid='$UUID' ORDER BY id DESC LIMIT 1;
# placement shows the swap: the MIGRATION uuid holds the source, the INSTANCE the dest claim
$ openstack resource provider allocation show $MIGRATION_UUID → old flavor on the source node
$ openstack resource provider allocation show $UUID → new flavor on the dest node
# migration row exists but never leaves pre-migrating → the prep_resize cast was lost,
# or nova-compute on the DEST is down; NoValidHost at this stage shows in conductor logs
Destination claims, source moves the disk
The destination reserves local resources and bounces the work back to the source. The source then powers off the guest, copies the disk over, and re-points the instance at the new host. States: RESIZE_MIGRATING → RESIZE_MIGRATED.
Destination resize_claim, then cast back to source
The destination claims local resources inside a Claim context manager — checking NUMA topology and PCI devices actually fit, and creating the MigrationContext. On success it casts resize_instance to the source. If the claim fails, _revert_allocation cleans up and a reschedule to an alternate may be attempted.
with self.rt.resize_claim(
context, instance, flavor, node, migration, allocs,
image_meta=image, limits=limits,
) as claim:
LOG.info('Migrating', instance=instance)
# RPC cast to the source host to start the actual resize/migration.
self.compute_rpcapi.resize_instance(
context, instance, claim.migration, image,
flavor, request_spec, clean_shutdown)
Source powers off and copies the disk
On the source, resize_instance flips task_state to RESIZE_MIGRATING, then the virt driver powers off the guest, unplugs VIFs, disconnects volumes, and copies the disk images to the destination (skipping swap, which finish_migration recreates).
self.power_off(instance, timeout, retry_interval)
self.unplug_vifs(instance, network_info)
block_device_mapping = driver.block_device_info_get_mapping(
block_device_info)
for vol in block_device_mapping:
connection_info = vol['connection_info']
self._disconnect_volume(context, connection_info, instance)
Source retargets the instance and casts finish_resize
After the disk is copied and networking is started, the source updates the instance's host/node/compute_id to the destination, stashes the old flavor, sets RESIZE_MIGRATED, and casts finish_resize to the destination.
instance.host = migration.dest_compute
instance.node = migration.dest_node
instance.compute_id = migration.get_dest_compute_id()
instance.old_flavor = instance.flavor
instance.task_state = task_states.RESIZE_MIGRATED
instance.save(expected_task_state=task_states.RESIZE_MIGRATING)
# RPC cast to the destination host to finish the resize/migration.
self.compute_rpcapi.finish_resize(context, instance,
migration, image, disk_info, migration.dest_compute,
request_spec)
instance.host = migration.dest_compute — is the moment ownership changes hands. Keeping old_flavor means a revert can restore the original size.
It is the source compute, in _resize_instance, that sets instance.host to the destination — before finish_resize even runs. This detail is a favourite exam question.
$ openstack server show $UUID -c OS-EXT-STS:task_state -c OS-EXT-SRV-ATTR:host
| resize_migrating | qh2-rcc001 | # host flips to the dest the moment resize_migrated is saved
mysql> SELECT task_state, host, node FROM nova.instances WHERE uuid='$UUID'; (cell DB — same truth)
mysql> SELECT status FROM nova.migrations WHERE uuid='$MIGRATION_UUID'; → migrating
# frozen at resize_migrating → the disk copy is still running (or died) on the SOURCE —
# watch the source's nova-compute log and the rsync/scp between the hosts
$ openstack server event list $UUID # the resize action journals every hop with its host
Destination finishes the move
finish_resize on the destination repoints networking and storage, recreates the guest at its new size, and lands the instance in vm_state=RESIZED — the state you see as VERIFY_RESIZE. States: RESIZE_FINISH → RESIZED.
Finish networking and volumes on the destination
Neutron port bindings are switched to this host with migrate_instance_finish, then volume attachments are updated with this host's connector before the driver recreates the guest.
self.network_api.migrate_instance_finish(
context, instance, migration, provider_mappings)
network_info = self.network_api.get_instance_nw_info(context, instance)
instance.task_state = task_states.RESIZE_FINISH
instance.save(expected_task_state=task_states.RESIZE_MIGRATED)
...
self._update_volume_attachments(context, instance, bdms)
migrate_instance_finish tells Neutron each port's binding:host_id is now the destination, and the volume attachments are rewritten with the destination's connector.
Recreate the guest and land in RESIZED
After finish_migration spawns the guest and the volume attachments are completed, the migration is marked finished and the instance enters vm_state=RESIZED — the state the user sees as VERIFY_RESIZE.
migration.status = 'finished'
migration.save()
instance.vm_state = vm_states.RESIZED
instance.task_state = None
instance.launched_at = timeutils.utcnow()
instance.save(expected_task_state=task_states.RESIZE_FINISH)
task_state=None means nothing is in flight; the ball is now in your court. Note the source copy is still on disk, and the migration UUID still holds the source allocation.
The move is not complete until you act. The old guest and its Placement allocation are deliberately kept so a revert is possible. Many deployments auto-confirm after a timeout via resize_confirm_window.
| VERIFY_RESIZE | resized | m3.large | qh2-rcc002 (the destination) |
$ openstack server migration list --server $UUID → status finished
# the fingerprint of a resize awaiting confirm: BOTH consumers hold allocations
$ openstack resource provider allocation show $UUID → new flavor on the dest node
$ openstack resource provider allocation show $MIGRATION_UUID → old flavor still held on the source
mysql> SELECT status FROM nova.migrations WHERE uuid='$MIGRATION_UUID'; → finished
# reached resize_finish but never RESIZED → the dest died mid-finish; check its nova-compute log
You confirm or revert
The server sits in RESIZED until you decide. Confirm commits the move and deletes the old copy. Revert undoes it and restores the original. The two paths run on opposite hosts.
Confirm runs on the source and deletes the old copy
confirm_resize is cast from the API to the source compute. The driver removes the old guest, the resource tracker drops the old-flavor usage, and the held allocation is removed from Placement.
self.driver.confirm_migration(context, migration, instance,
network_info)
# Free up the old_flavor usage from the resource tracker for this host.
self.rt.drop_move_claim_at_source(context, instance, migration)
drop_move_claim_at_source marks the migration confirmed
The migration record is marked 'confirmed' and the old-flavor usage on the source node is released, which deletes the migration-held Placement allocation. The instance returns to ACTIVE (or STOPPED).
def drop_move_claim_at_source(self, context, instance, migration):
"""Drop a move claim after confirming a resize or cold migration."""
migration.status = 'confirmed'
migration.save()
self._drop_move_claim(
context, instance, migration.source_node, instance.old_flavor,
prefix='old_')
'old_' prefix tells the tracker to release the previous flavor's bookkeeping. After this, the source host shows the freed CPU/RAM/disk again.
Revert destroys the dest copy and casts back to source
revert_resize runs on the destination first: it tears down dest networking, destroys the dest guest, terminates dest volume connections, drops the new-flavor claim (drop_move_claim_at_dest, migration → 'reverted'), then casts finish_revert_resize back to the source.
# Free up the new_flavor usage from the resource tracker for this
# host.
self.rt.drop_move_claim_at_dest(context, instance, migration)
# RPC cast back to the source host to finish the revert there.
self.compute_rpcapi.finish_revert_resize(context, instance,
migration, migration.source_compute, request_spec)
Source moves the allocation back to the instance
On the source, finish_revert_resize reverts the flavor/host/node and moves the allocation from the migration UUID back to the instance UUID, so the instance again owns its resources on the original host. The original guest is recreated and the instance ends ACTIVE/STOPPED.
def _revert_allocation(self, context, instance, migration):
"""Revert an allocation that is held by migration to our instance."""
orig_alloc = self.reportclient.get_allocations_for_consumer(
context, migration.uuid)
...
self.reportclient.move_allocations(context, migration.uuid,
instance.uuid)
move_allocations(migration.uuid, instance.uuid) instead of the other way round.
The migration UUID acts as a safe-deposit box for the source allocation. Confirm throws the box away; revert opens it and hands the contents back to the instance. Either way Placement stays consistent.
mysql> SELECT status FROM nova.migrations WHERE uuid='$MIGRATION_UUID'; → confirmed | reverted
# the migration's safe-deposit box must now be empty — one consumer, one host
$ openstack resource provider allocation show $MIGRATION_UUID → (empty ✓)
$ openstack resource provider allocation show $UUID → a single compute node
# stuck in VERIFY_RESIZE? confirm it yourself, or check resize_confirm_window auto-confirm
$ openstack server resize confirm $UUID
# leaked allocations after a botched confirm show up in the placement audit
$ nova-manage placement audit --verbose
The full journey, animated
Press Next step to watch the work travel between two compute hosts and back. Notice the CAST/CALL/REST tags, and how the two computes hand the instance to each other.
There is exactly one blocking RPC call (conductor → scheduler, step 7). Everything between Nova services is a non-blocking cast — including the round trip dest → source → dest that hands the instance across hosts. Steps 22–23 are the confirm path; 24–25 are the alternative revert path.
When it goes wrong
A move can fail at four distinct points, and the recovery differs at each. The migration UUID and the host_list of alternates are what make clean rollback possible.
NoValidHost at schedule time → roll the allocation back
If select_destinations finds no host (or move_allocations fails, mimicking "no space"), MigrationTask.rollback() sets migration.status='error' and calls revert_allocation_for_migration to move the source allocation back to the instance. _cold_migrate then resets vm_state and notifies.
Because nothing moved yet, recovery is simply un-swapping the allocation. The instance stays where it was, in its original state, with a fault recorded. Code: conductor/tasks/migrate.py rollback() · conductor/manager.py _cold_migrate()
prep_resize claim failure → reschedule to an alternate
If prep_resize raises on the destination, _revert_allocation drops the dest allocation and _reschedule_resize_or_reraise tries an alternate host from the host_list — casting resize_instance back to the conductor. A BuildAbortException (for example a PCI or placement mapping failure) skips reschedule and just reverts.
_revert_allocation removes the failed destination's reservation so its capacity is freed.
_reschedule_resize_or_reraise pulls the next host from host_list and casts back to the conductor to retry.
A BuildAbortException means another host would fail identically, so Nova reverts and errors instead of retrying.
resize_instance failure on the source → return the allocation
resize_instance wraps _resize_instance; on any exception it calls _revert_allocation(context, instance, migration) to return the source allocation to the instance. The server is still on the source and recoverable.
This handler runs while instance.host may still point at the source. Moving the allocation back to the instance UUID restores the pre-resize world. Code: compute/manager.py resize_instance()
finish_resize failure on the dest → recover on the destination
By the time finish_resize runs, instance.host already points at the destination. So on failure it deletes the source-node allocation held by the migration (_delete_allocation_after_move), leaving the dest allocation intact — so a hard reboot can recover the guest on the destination rather than the source.
Once the host has flipped to the destination, rolling back to the source would be wrong — the disk is already there. So this late failure keeps the destination allocation and lets a reboot finish the job. Code: compute/manager.py finish_resize()
Check yourself
Three questions and a matching exercise. If these click, you understand the resize flow.
Quiz
confirm_resize, and what does it do?instance.host to the destination, and when?Match each player to its job
Drag each service or method onto the task it performs during a resize.
From POST resize to vm_state=RESIZED and on to confirm or revert: validation in the API, an allocation swap to the migration UUID, a scheduler pick, a disk copy between two computes, and a clean rollback story at every failure point. That's the whole journey.