What really happens
when you click Create Desktop?
One button click. Behind the curtain, a Django app clones a volume, boots a VM with a secret password, waits for the machine to call back, and then hands your browser to Apache Guacamole, which streams the desktop as pixels. This is the complete journey, traced from the real Bumblebee source code.
The cast of characters
Bumblebee is the web service behind the Nectar Virtual Desktop Service. It gives researchers a full Linux or Windows desktop that runs in the browser, built on OpenStack. Before we trace a launch, meet the players.
Three ways the pieces talk
Every interaction in this story uses one of three channels. The third one is unusual, and it is the key to the whole Guacamole relationship.
The web app never does slow work itself. It drops a job onto RQ and returns at once. A separate rqworker process does the real work, and an rqscheduler re-runs polling jobs every few seconds.
For cloud resources, Bumblebee calls the OpenStack APIs (Cinder, Nova, Neutron, Keystone) over HTTP, using a single application credential.
Bumblebee and Guacamole read and write the same MariaDB database. Bumblebee writes rows straight into Guacamole's own tables. No API call between them, ever.
Nothing pushes state to your browser. Every workflow just updates one database row, a VMStatus, and your browser polls it. If you can read that row, you can see exactly where any launch is up to. That is what the ops checks in this course do.
Meet the services (click each one)
These are the components a desktop launch passes through. Click any box to learn its job.
The journey in one breath
Here is the whole launch compressed into five phases. The rest of the course unpacks each one.
The web app writes a VMStatus row (progress 0) and enqueues launch_vm_worker. Your browser starts polling.
The worker finds the newest pre-built desktop volume in your chosen availability zone and asks Cinder to clone it.
Nova boots a VM from the clone. Its cloud-init user-data carries a generated RDP username and password.
When the machine finishes booting, it calls Bumblebee back. Only then does the status flip to VM_Okay (progress 100).
Open Desktop writes the RDP settings into Guacamole's tables and sends your browser to Guacamole, which streams the desktop.
The click: recording intent
You press Create Desktop and pick an availability zone. The request lands in vm_manager/views.py. Nothing is built yet. This phase is about guard checks and writing down the plan.
Two guards at the door
Before anything is created, launch_vm() asks two questions. Both are answered from the database alone, without touching OpenStack.
The policy is one desktop per user. If a live instance exists, the launch is refused.
desktop_limit_check() · vm_manager/views.pyAn old volume with an error flag, or an instance record in an inconsistent state, blocks the launch. These need manual cleanup, so the user gets an error status instead of a half-broken build.
_check_launch_blocked() · vm_manager/views.pyWrite the plan, then hand it off
Both guards passed. Now the view writes a VMStatus row and puts the real work on the queue. The web request then returns straight away.
vm_status = VMStatus(
user=user, requesting_feature=desktop_type.feature,
operating_system=desktop_type.id, status=VM_CREATING,
wait_time=after_time(launch_time),
status_progress=0,
status_message="Starting desktop creation",
status_done="has been created"
)
vm_status.save()
queue = django_rq.get_queue('default')
queue.enqueue(launch_vm_worker, user=user,
desktop_type=desktop_type, zone=zone)
wait_time is a deadline: if the whole launch has not finished by then, the status will be treated as an error. The enqueue call drops launch_vm_worker onto the Redis queue and does not wait for it.
The click just wrote one row and one queue message. Everything slow happens later, inside rqworker. From this moment on, your browser simply polls the VMStatus row and draws its progress bar.
mysql> SELECT status, status_progress, status_message FROM vm_manager_vmstatus WHERE user_id=$UID ORDER BY created DESC LIMIT 1;
| VM_Waiting | 0 | Starting desktop creation |
# the job should be on (or already taken off) the 'default' queue
$ rq info --url redis://$REDIS_HOST:6379
default | 1 queued, 1 workers
# stuck at 0% forever? no rqworker is consuming the queue;
# check the worker container/pod is up and watching 'default'
$ docker compose ps rqworker rqscheduler # or: kubectl get pods | grep rq
Cloning the golden volume
The worker picks up the job in vm_manager/vm_functions/create_vm.py. A desktop is never installed from scratch. It is photocopied from a master.
The master copy
Every desktop type (Ubuntu, Fedora, TERN, RStudio ...) has a pre-built golden volume sitting in each availability zone. The worker looks for the newest one whose name starts with the desktop's image name, ranked by its nectar_build number.
matches = sorted(
[v for v in candidates
if v.name.startswith(desktop_type.image_name)],
key=lambda v: int(v.metadata.get('nectar_build', 0)),
reverse=True)
available. Sorting by the nectar_build metadata means a freshly published image is used automatically, with no code change.
Photocopy, do not install
With the master found, one Cinder call makes your personal copy, and a second marks it bootable. The clone becomes your disk: your files live on it from now on.
volume_result = n.cinder.volumes.create(
source_volid=source_volume_id,
size=desktop_type.volume_size,
name=name,
metadata={'readonly': 'False'},
availability_zone=zone.name)
n.cinder.volumes.set_bootable(
volume=volume_result, flag=True)
source_volid: this is a clone, not an empty disk. A matching Volume row is then saved in Bumblebee's database, and the volume gets metadata tags (hostname, user email, desktop type, environment) so operators can tell whose volume it is from the OpenStack side.
Because the disk then outlives the VM. When a desktop is shelved, the instance is deleted but the volume stays, files and all. Unshelving just boots a fresh instance from the same volume. Fast launches, durable data.
The 5-second heartbeat
Cloning takes time, and the worker refuses to sit blocked. Instead, wait_to_create_instance checks the clone once, and if it is not ready, schedules itself to run again in 5 seconds. This self-rescheduling poll is the signature move of the whole codebase.
else:
scheduler = django_rq.get_scheduler('default')
scheduler.enqueue_in(timedelta(seconds=5),
wait_to_create_instance,
user, desktop_type, volume, start_time)
available means move on and boot the instance. error means fail fast. Anything else, and it re-books itself with rqscheduler, carrying the original start_time so it can give up after VOLUME_CREATION_WAIT (180 seconds by default).
mysql> SELECT status_progress, status_message FROM vm_manager_vmstatus WHERE user_id=$UID ORDER BY created DESC LIMIT 1;
| 15 | Creating volume |
# the clone on the OpenStack side: creating -> available, cloned from the golden volume
$ openstack volume show $VOL_UUID -c status -c bootable -c source_volid -c properties
| available | true | $GOLDEN_UUID | hostname='vdu-abc123', user='you@uni.edu.au', ... |
# and Bumblebee's own record of it (ready flips to 1 only after phone home)
mysql> SELECT id, zone, hostname_id, ready, checked_in FROM vm_manager_volume WHERE user_id=$UID AND deleted IS NULL;
| $VOL_UUID | melbourne-qh2 | abc123 | 0 | 0 |
# stuck at 15% for ~3 minutes then VM_Error 'Volume took too long to create'?
# that is the VOLUME_CREATION_WAIT timeout; check Cinder capacity/health in that AZ
Boot, then wait for the call
The clone is ready. Now Nova boots your VM from it, carrying a sealed envelope of instructions. Bumblebee then waits for the new machine to ring back.
Boot from the clone, with a secret inside
_create_instance() builds the Nova request. Note image='': there is no image, the boot source is your cloned volume. The interesting part is userdata, a rendered cloud-init config that carries a freshly generated RDP username and password.
launch_result = n.nova.servers.create(
name=name,
image='',
flavor=desktop_type.default_flavor.id,
userdata=user_data,
security_groups=desktop_type.security_groups,
block_device_mapping_v2=block_device_mapping,
nics=nics,
availability_zone=zone,
meta=metadata_server,
key_name=settings.OS_KEYNAME,
)
nics come from the availability zone's private network, which allows no incoming connections from the internet. The password inside user_data is hashed with sha512 for Linux desktops, and plain text for Windows, because that is what each OS expects.
The username (vdiuser) and generated password are saved on the Instance row in Bumblebee's database, and injected into the VM at first boot. You never see or type them. Module 04 shows who does use them: Guacamole.
The sealed envelope: cloud-init
The user-data template does two jobs: create the desktop user, and install a tiny systemd service whose only purpose is to call Bumblebee when boot has finished.
#!/bin/bash
METADATA=$(curl -s http://169.254.169.254
/openstack/latest/meta_data.json)
INSTANCE_ID=$(echo "$METADATA" | grep -oP
'"uuid":\s*"\K[^"]+')
curl -s --retry 10 \
-d "instance_id=${INSTANCE_ID}" \
{{ phone_home_url }}
phone_home endpoint, retrying up to 10 times. A systemd unit runs it once, after the network is up.
Meanwhile, Bumblebee watches and waits
The same 5-second heartbeat pattern now polls Nova. When the server goes ACTIVE, progress reaches 45 and the message becomes honest: Instance launched; waiting for boot. ACTIVE only means the VM started. It says nothing about whether the desktop inside is ready. The real finish line is the phone-home call:
volume = instance.boot_volume
volume.ready = True
volume.save()
vm_status.status_progress = 100
vm_status.status_message = 'Instance ready'
vm_status.status = status
vm_status.save()
return HttpResponse("OK")
ready, and the VMStatus jumps to 100 with state VM_Okay. On the next poll, your browser swaps the progress bar for the desktop card with the Open Desktop button.
The full progress ladder, all read from the one VMStatus row:
Nova saying ACTIVE proves the hypervisor started the VM. The phone-home call proves the operating system booted, the network works, DHCP gave an address, and the machine can reach Bumblebee. That is a far stronger health check, which is why the status only turns green here.
$ openstack server show $UUID -c status -c addresses -c properties
| ACTIVE | private-net=172.16.x.y | allow_user='you', environment='prod', ... |
# poll the ladder in the DB: 30 -> 45 -> (phone home) -> 100
mysql> SELECT status, status_progress, status_message FROM vm_manager_vmstatus WHERE user_id=$UID ORDER BY created DESC LIMIT 1;
| VM_Okay | 100 | Instance ready | # stuck at 45 = the VM never phoned home
# did the callback arrive? volume.ready flips to 1 on phone home
mysql> SELECT ready, checked_in FROM vm_manager_volume WHERE id='$VOL_UUID'; → 1 | 1
# replay it by hand from inside the VM (console) to test the path:
vm$ curl -d "instance_id=$UUID" $SITE_URL/researcher_desktop/phone_home/ → OK
The Guacamole connection
Your desktop is running, but Bumblebee never streams a single pixel. That job belongs to Apache Guacamole, and the two services cooperate in an unusual way: through the database, not an API.
Guacamole in one minute
Apache Guacamole has two halves. The web app is the face: it draws the remote desktop on an HTML5 canvas in your browser. guacd is the interpreter: a daemon that speaks real RDP to the desktop VM on the private network. Bumblebee's job is simply to tell Guacamole where to connect and with which credentials.
Normally Guacamole owns its configuration tables. Here, Bumblebee's Django models are mapped straight onto Guacamole's table names, and Bumblebee writes them directly. The README is blunt about it: "Guacamole must use Bumblebee's database as Bumblebee manages the Guacamole's tables." No REST call, no sync job. One write, instantly visible to the other side.
class Meta:
db_table = 'guacamole_connection'
class Meta:
db_table = 'guacamole_connection_parameter'
class Meta:
db_table = 'guacamole_connection_permission'
GuacamoleConnection object, a row appears in the exact table Guacamole reads its connections from. The two applications share one source of truth.
Three writes at three moments
Bumblebee populates Guacamole's tables in stages, each at the moment the information first exists.
The OIDC auth backend creates a guacamole_entity and guacamole_user row named after your email. Both apps log in against the same Keycloak, so the names line up automatically.
get_or_create_guac_objects() · researcher_workspace/auth.pyWhile booting your VM, the worker creates a GuacamoleConnection row named, for example, "Grace Hopper's Ubuntu desktop". No address or password yet.
_create_instance() · create_vm.pyWhen the desktop card renders its URL, Bumblebee fills in the RDP parameters and grants your entity READ permission on the connection.
create_guac_connection() · vm_manager/models.pyparams = [
('hostname', self.get_ip_addr()),
('username', self.username),
('password', self.password),
('security', 'any'),
('enable-drive', 'true'),
('drive-path',
f'/var/lib/guacd/shared-drive/{self.id}'),
...
]
gentity = GuacamoleEntity.objects.get(
name=self.user.email)
GuacamoleConnectionPermission.objects.get_or_create(
entity=gentity,
connection=self.guac_connection,
permission='READ')
vdiuser username and password from Module 03, plus quality-of-life settings: drag-and-drop file transfer through a shared drive folder, display resizing, font smoothing. The final line is the authorisation: only your entity gets READ on this connection, so no other Guacamole user can even see it.
The RDP password never travels to your browser and you never type it. The desktop VM accepts no incoming traffic from the internet; only guacd reaches it, on port 3389, from inside. Your browser only ever talks HTTPS to Guacamole, authenticated by the same Keycloak login as Bumblebee.
The strange little URL
The Open Desktop button needs a link that lands directly on your connection inside Guacamole. Guacamole encodes connection identity in the URL fragment, so Bumblebee reproduces the format:
components = [str(conn.connection_id), 'c', 'mysql']
joined_components = '\x00'.join(components).encode('utf-8')
hash_str = base64.b64encode(joined_components)\
.decode('utf-8')
fixed_hash = hash_str.replace('=', '')
return f'#/client/{quote_hash_str}'
#/client/NDIAYwBteXNxbA. The full URL comes from GUACAMOLE_URL_TEMPLATE, which slots in the environment and the availability zone, because each zone runs its own Guacamole cluster close to its desktops.
The whole handoff, as a conversation
Press Next message to watch what happens when you click Open Desktop, told as a group chat between the five actors involved.
mysql> SELECT connection_id, connection_name, protocol FROM guacamole_connection ORDER BY connection_id DESC LIMIT 1;
| 42 | Grace Hopper's Ubuntu desktop | rdp |
# the RDP parameters (filled in when Open Desktop was rendered)
mysql> SELECT parameter_name, parameter_value FROM guacamole_connection_parameter WHERE connection_id=42;
| hostname | 172.16.1.23 | | username | vdiuser | | password | ... | | enable-drive | true |
# the permission: exactly one entity, your email, with READ
mysql> SELECT e.name, p.permission FROM guacamole_connection_permission p JOIN guacamole_entity e ON p.entity_id=e.entity_id WHERE p.connection_id=42;
| grace@uni.edu.au | READ |
# desktop shows 'connecting' forever? guacd probably cannot reach RDP;
# test the path from a guacd host on that zone's private network
guacd$ nc -zv 172.16.1.23 3389 → succeeded (else: routing / security groups)
The full journey, animated
Press Next step to watch one desktop launch travel the whole path, from the Create Desktop click to pixels in the browser. Watch how often the answer is "a row changed in the database".
Bumblebee and Guacamole never exchange a message directly, in the whole 17 steps. Every handoff between them (steps 12 to 14) goes through the shared database. And every status your browser ever sees is one VMStatus row, polled.
When it goes wrong
Each waiting loop has two exits besides success: fail fast when OpenStack reports a terminal error, and time out when nothing happens for too long. Both end in the same place: a VMStatus in VM_Error with the reason attached.
Fail fast, and keep the evidence
If Cinder puts the clone into error, or Nova puts the server into ERROR, there is no point waiting out the timer. The worker fails immediately, and it goes looking for the reason: Nova attaches a fault message to a failed server, and Cinder records user messages for failed volumes. That detail is copied into the status the user and the operator see.
fault = instance.get_fault()
detail = f"{msg}: {fault}" if fault else msg
...
vm_status.status = VM_ERROR
vm_status.status_message = detail
vm_status.save()
instance.error(detail)
status_message, and the instance row gets an error_flag plus the same message. Nothing is silently retried, and nothing vanishes.
The classic: stuck at 45, no phone call
The most common real-world failure is a desktop that reaches Instance launched; waiting for boot and never finishes. Nova says ACTIVE, but the phone-home call never arrives. When the VMStatus wait_time deadline passes, the next poll flips it to an error.
The VM cannot reach Bumblebee's URL: DHCP failed on the private network, or routing / security groups block the way out. The phone-home curl retries 10 times, then gives up.
The golden volume's boot is broken: cloud-init errors out, or the phone-home service never runs. Check the VM console log.
If the call turns up after the timeout, phone_home notices the status is VM_Error but the instance is ACTIVE, and handles the late event instead of rejecting it.
phone_home() · vm_manager/views.pyA volume with an error_flag makes _check_launch_blocked() refuse new launches for that desktop with "Needs manual cleanup". This prevents a pile-up of half-built resources. An operator must repair or delete the broken records (the admin panel has actions for this) before the user can try again.
mysql> SELECT status, status_message FROM vm_manager_vmstatus WHERE user_id=$UID ORDER BY created DESC LIMIT 1;
| VM_Error | Instance launch failed: No valid host was found. ... |
# the same fault from the OpenStack side
$ openstack server show $UUID -c status -c fault
$ openstack volume message list | grep $VOL_UUID # cinder's reasons for volume failures
# which records are now error-flagged (these block the next launch)
mysql> SELECT id, error_flag, error_message FROM vm_manager_volume WHERE user_id=$UID AND deleted IS NULL;
# the worker logs the failing job with a stack trace
$ docker compose logs rqworker | grep -A5 "$UUID" # or the pod logs in k8s
# drifted? compare every DB record against real OpenStack resources
$ python3 manage.py audit_openstack
Check yourself
Three questions and a matching exercise. If these click, you understand the Virtual Desktop Service.
Quiz
VM_Okay / 100?Match the actor to its job
Drag each actor onto the task it performs during a launch.
From the Create Desktop click to pixels in a browser tab: a VMStatus row for intent, a Cinder clone of a golden volume, a Nova boot carrying secrets via cloud-init, a phone-home callback as the real health check, and a handoff to Apache Guacamole through a shared database. That is the whole Virtual Desktop Service.