[2026-11-13T08:37:33.054Z] Started by upstream project "anvil/anvil-build-all-kmods" build number 724
[2026-11-13T08:37:33.054Z] originally caused by:
[2026-11-13T08:37:33.054Z]  Started by upstream project "anvil/anvil-build-cron" build number 224
[2026-11-13T08:37:33.054Z]  originally caused by:
[2026-11-13T08:37:33.054Z]   Started by timer
[2026-11-13T08:37:33.430Z] Obtained pipelines/projects/anvil/build-kmods/Jenkinsfile from git https://github.com/kronosnet/ci-tools
[2026-11-13T08:37:33.436Z] Loading library ProjectLib@main:pipelines/projects/anvil/
[2026-11-13T08:37:33.704Z] Examining kronosnet/ci-tools
[2026-11-13T08:37:33.704Z] Attempting to resolve main as a branch
[2026-11-13T08:37:33.876Z] Resolved main as branch main at revision fd5667069c7b69778ce97df718cf25626f5beefc
[2026-11-13T08:37:33.877Z] Selected Git installation does not exist. Using Default
[2026-11-13T08:37:33.877Z] The recommended git tool is: NONE
[2026-11-13T08:37:33.877Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-11-13T08:37:33.881Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-11-13T08:37:33.881Z] Wiping out workspace first.
[2026-11-13T08:37:33.908Z] Cloning the remote Git repository
[2026-11-13T08:37:33.908Z] Using shallow clone with depth 1
[2026-11-13T08:37:33.908Z] Avoid fetching tags
[2026-11-13T08:37:33.908Z] Honoring refspec on initial clone
[2026-11-13T08:37:33.909Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-11-13T08:37:33.909Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-kmods@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b@2 # timeout=10
[2026-11-13T08:37:33.923Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-11-13T08:37:33.923Z]  > git --version # timeout=10
[2026-11-13T08:37:33.931Z]  > git --version # 'git version 2.52.0'
[2026-11-13T08:37:33.931Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-11-13T08:37:33.939Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-11-13T08:37:34.200Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-11-13T08:37:34.207Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-11-13T08:37:34.215Z] Avoid second fetch
[2026-11-13T08:37:34.215Z] Checking out Revision fd5667069c7b69778ce97df718cf25626f5beefc (main)
[2026-11-13T08:37:34.215Z]  > git config core.sparsecheckout # timeout=10
[2026-11-13T08:37:34.222Z]  > git checkout -f fd5667069c7b69778ce97df718cf25626f5beefc # timeout=10
[2026-11-13T08:37:34.259Z] Commit message: "pipelines: fix ansible managed update pipelines to process all nodes in parallel"
[2026-11-13T08:37:34.269Z] Loading library GlobalLib@main:./pipelines/libs/global/
[2026-11-13T08:37:34.558Z] Examining kronosnet/ci-tools
[2026-11-13T08:37:34.558Z] Attempting to resolve main as a branch
[2026-11-13T08:37:34.823Z] Resolved main as branch main at revision fd5667069c7b69778ce97df718cf25626f5beefc
[2026-11-13T08:37:34.824Z] Selected Git installation does not exist. Using Default
[2026-11-13T08:37:34.824Z] The recommended git tool is: NONE
[2026-11-13T08:37:34.825Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-11-13T08:37:34.828Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-11-13T08:37:34.828Z] Wiping out workspace first.
[2026-11-13T08:37:34.861Z] Cloning the remote Git repository
[2026-11-13T08:37:34.861Z] Using shallow clone with depth 1
[2026-11-13T08:37:34.861Z] Avoid fetching tags
[2026-11-13T08:37:34.861Z] Honoring refspec on initial clone
[2026-11-13T08:37:34.862Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-11-13T08:37:34.862Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-kmods@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b@5 # timeout=10
[2026-11-13T08:37:34.875Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-11-13T08:37:34.875Z]  > git --version # timeout=10
[2026-11-13T08:37:34.883Z]  > git --version # 'git version 2.52.0'
[2026-11-13T08:37:34.883Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-11-13T08:37:34.891Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-11-13T08:37:35.112Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-11-13T08:37:35.121Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-11-13T08:37:35.131Z] Avoid second fetch
[2026-11-13T08:37:35.131Z] Checking out Revision fd5667069c7b69778ce97df718cf25626f5beefc (main)
[2026-11-13T08:37:35.131Z]  > git config core.sparsecheckout # timeout=10
[2026-11-13T08:37:35.141Z]  > git checkout -f fd5667069c7b69778ce97df718cf25626f5beefc # timeout=10
[2026-11-13T08:37:35.179Z] Commit message: "pipelines: fix ansible managed update pipelines to process all nodes in parallel"
[2026-11-13T08:37:35.233Z] [Pipeline] Start of Pipeline
[2026-11-13T08:37:35.398Z] [Pipeline] node
[2026-11-13T08:37:35.403Z] Running on anvil-ci-bm-phy07 in /srv/workspace/anvil/anvil-build-kmods
[2026-11-13T08:37:35.404Z] [Pipeline] {
[2026-11-13T08:37:35.415Z] [Pipeline] stage
[2026-11-13T08:37:35.415Z] [Pipeline] { (Prep build env)
[2026-11-13T08:37:35.435Z] [Pipeline] cleanWs
[2026-11-13T08:37:35.484Z] [WS-CLEANUP] Deleting project workspace...
[2026-11-13T08:37:35.484Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-11-13T08:37:35.508Z] [WS-CLEANUP] done
[2026-11-13T08:37:35.534Z] [Pipeline] lock
[2026-11-13T08:37:35.534Z] Trying to acquire lock on [Resource: MAINTENANCE LOCKOUT]
[2026-11-13T08:37:35.534Z] Resource [MAINTENANCE LOCKOUT] did not exist. Created.
[2026-11-13T08:37:35.535Z] Lock acquired on [Resource: MAINTENANCE LOCKOUT], resources: [MAINTENANCE LOCKOUT]
[2026-11-13T08:37:35.535Z] [Pipeline] {
[2026-11-13T08:37:35.537Z] [Pipeline] }
[2026-11-13T08:37:35.539Z] Lock released on resource [Resource: MAINTENANCE LOCKOUT], resources: [MAINTENANCE LOCKOUT]
[2026-11-13T08:37:35.541Z] [Pipeline] // lock
[2026-11-13T08:37:35.544Z] [Pipeline] sh
[2026-11-13T08:37:36.015Z] + /root/ci-tools/ci-destroy-anvil-bm-vm
[2026-11-13T08:37:36.015Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-11-13T08:37:36.360Z] Genering ssh keys and ssh config
[2026-11-13T08:37:37.099Z] Cleaning up simengine configuration
[2026-11-13T08:37:40.157Z] Destroying VMs (if any)
[2026-11-13T08:37:40.572Z] [Pipeline] }
[2026-11-13T08:37:40.574Z] [Pipeline] // stage
[2026-11-13T08:37:40.577Z] [Pipeline] stage
[2026-11-13T08:37:40.577Z] [Pipeline] { (Build kmod)
[2026-11-13T08:37:40.586Z] [Pipeline] sh
[2026-11-13T08:37:41.059Z] + /root/ci-tools/ci-setup-anvil-bm-vm-kmods almalinux-9 ci ci
[2026-11-13T08:37:41.059Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-11-13T08:37:41.402Z] Genering ssh keys and ssh config
[2026-11-13T08:37:41.746Z] Destroying VMs (if any)
[2026-11-13T08:37:41.746Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-11-13T08:37:42.485Z] Genering ssh keys and ssh config
[2026-11-13T08:37:42.829Z] Cache already updated
[2026-11-13T08:37:42.829Z] Building drbd kmod for almalinux-9 repo: ci anvil: ci
[2026-11-13T08:37:42.829Z] Formatting 'an-cache-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/almalinux-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-11-13T08:37:42.829Z] wrote an-cache-seed.img with filesystem=iso9660 and diskformat=raw
[2026-11-13T08:37:44.023Z] 
[2026-11-13T08:37:44.023Z] Starting install...
[2026-11-13T08:37:44.023Z] 
[2026-11-13T08:37:44.023Z] Creating domain...                                          |         00:00     
[2026-11-13T08:37:44.023Z] Domain creation completed.
[2026-11-13T08:37:44.023Z] Waiting for nodes and cloud-init to complete full init
[2026-11-13T08:37:47.908Z] cloud-init status node an-cache: 
[2026-11-13T08:37:47.908Z] Fri Nov 13 03:37:47 EST 2026 actual: 0 expected: 1
[2026-11-13T08:38:01.227Z] cloud-init status node an-cache: 
[2026-11-13T08:38:01.227Z] Fri Nov 13 03:37:59 EST 2026 actual: 0 expected: 1
[2026-11-13T08:38:10.299Z] cloud-init status node an-cache: running
[2026-11-13T08:38:10.299Z] Fri Nov 13 03:38:10 EST 2026 actual: 0 expected: 1
[2026-11-13T08:38:21.299Z] cloud-init status node an-cache: running
[2026-11-13T08:38:21.299Z] Fri Nov 13 03:38:21 EST 2026 actual: 0 expected: 1
[2026-11-13T08:38:32.429Z] cloud-init status node an-cache: running
[2026-11-13T08:38:32.429Z] Fri Nov 13 03:38:31 EST 2026 actual: 0 expected: 1
[2026-11-13T08:38:43.437Z] cloud-init status node an-cache: running
[2026-11-13T08:38:43.437Z] Fri Nov 13 03:38:42 EST 2026 actual: 0 expected: 1
[2026-11-13T08:38:54.449Z] cloud-init status node an-cache: running
[2026-11-13T08:38:54.449Z] Fri Nov 13 03:38:53 EST 2026 actual: 0 expected: 1
[2026-11-13T08:39:05.463Z] cloud-init status node an-cache: running
[2026-11-13T08:39:05.463Z] Fri Nov 13 03:39:04 EST 2026 actual: 0 expected: 1
[2026-11-13T08:39:16.583Z] cloud-init status node an-cache: running
[2026-11-13T08:39:16.583Z] Fri Nov 13 03:39:14 EST 2026 actual: 0 expected: 1
[2026-11-13T08:39:25.646Z] cloud-init status node an-cache: running
[2026-11-13T08:39:25.646Z] Fri Nov 13 03:39:25 EST 2026 actual: 0 expected: 1
[2026-11-13T08:39:36.653Z] cloud-init status node an-cache: running
[2026-11-13T08:39:36.654Z] Fri Nov 13 03:39:36 EST 2026 actual: 0 expected: 1
[2026-11-13T08:39:47.771Z] cloud-init status node an-cache: done
[2026-11-13T08:39:47.771Z] Fri Nov 13 03:39:47 EST 2026 actual: 1 expected: 1
[2026-11-13T08:39:47.771Z] Waiting for akmods to complete build
[2026-11-13T08:39:47.771Z] build not found, sleeping 10 seconds
[2026-11-13T08:39:58.782Z] build not found, sleeping 10 seconds
[2026-11-13T08:40:09.788Z] build not found, sleeping 10 seconds
[2026-11-13T08:40:18.978Z] build not found, sleeping 10 seconds
[2026-11-13T08:40:29.996Z] build not found, sleeping 10 seconds
[2026-11-13T08:40:39.059Z] build not found, sleeping 10 seconds
[2026-11-13T08:40:50.164Z] build not found, sleeping 10 seconds
[2026-11-13T08:41:01.168Z] build not found, sleeping 10 seconds
[2026-11-13T08:41:10.235Z] build not found, sleeping 10 seconds
[2026-11-13T08:41:21.338Z] /var/cache/akmods/drbd/kmod-drbd-5.14.0-687.el9_8-9.2.16-1.el9.x86_64.rpm
[2026-11-13T08:41:21.338Z] build found
[2026-11-13T08:41:21.338Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-11-13T08:41:21.684Z] Directory walk started
[2026-11-13T08:41:21.684Z] Directory walk done - 10 packages
[2026-11-13T08:41:21.684Z] Temporary output repo path: ./.repodata/
[2026-11-13T08:41:21.684Z] Preparing sqlite DBs
[2026-11-13T08:41:21.684Z] Pool started (with 5 workers)
[2026-11-13T08:41:21.684Z] Pool finished
[2026-11-13T08:41:21.684Z] /srv/anvil/cache
[2026-11-13T08:41:22.028Z] bash: line 1: rhc: command not found
[2026-11-13T08:41:22.372Z] bash: line 1: insights-client: command not found
[2026-11-13T08:41:22.372Z] bash: line 1: subscription-manager: command not found
[2026-11-13T08:41:24.705Z] Domain 'an-cache' destroyed
[2026-11-13T08:41:24.705Z] 
[2026-11-13T08:41:24.705Z] Domain 'an-cache' has been undefined
[2026-11-13T08:41:24.705Z] 
[2026-11-13T08:41:25.048Z] Done building drbd kmods
[2026-11-13T08:41:25.120Z] [Pipeline] }
[2026-11-13T08:41:25.122Z] [Pipeline] // stage
[2026-11-13T08:41:25.126Z] [Pipeline] stage
[2026-11-13T08:41:25.126Z] [Pipeline] { (Declarative: Post Actions)
[2026-11-13T08:41:25.132Z] [Pipeline] sh
[2026-11-13T08:41:25.602Z] + /root/ci-tools/ci-destroy-anvil-bm-vm
[2026-11-13T08:41:25.947Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-11-13T08:41:26.291Z] Genering ssh keys and ssh config
[2026-11-13T08:41:26.635Z] Cleaning up simengine configuration
[2026-11-13T08:41:30.521Z] Destroying VMs (if any)
[2026-11-13T08:41:30.689Z] [Pipeline] cleanWs
[2026-11-13T08:41:30.738Z] [WS-CLEANUP] Deleting project workspace...
[2026-11-13T08:41:30.738Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-11-13T08:41:30.761Z] [WS-CLEANUP] done
[2026-11-13T08:41:30.768Z] [Pipeline] script
[2026-11-13T08:41:30.770Z] [Pipeline] {
[2026-11-13T08:41:30.785Z] [Pipeline] stage
[2026-11-13T08:41:30.786Z] [Pipeline] { (Copying logs)
[2026-11-13T08:41:30.820Z] [Pipeline] node
[2026-11-13T08:41:30.920Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-kmods@2
[2026-11-13T08:41:30.921Z] [Pipeline] {
[2026-11-13T08:41:30.958Z] [Pipeline] node
[2026-11-13T08:41:31.063Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-kmods@3
[2026-11-13T08:41:31.064Z] [Pipeline] {
[2026-11-13T08:41:31.067Z] [Pipeline] sh
[2026-11-13T08:41:31.323Z] + mkdir -p /var/lib/jenkins/locks
[2026-11-13T08:41:31.332Z] [Pipeline] echo
[2026-11-13T08:41:31.332Z] RWLock: FD for lock log_archive in stage projectFinishUp is 639
[2026-11-13T08:41:31.335Z] [Pipeline] sleep
[2026-11-13T08:41:31.335Z] No need to sleep any longer
[2026-11-13T08:41:31.338Z] [Pipeline] echo
[2026-11-13T08:41:31.338Z] Waiting for lock log_archive
[2026-11-13T08:41:31.341Z] [Pipeline] sleep
[2026-11-13T08:41:31.341Z] Sleeping for 1 min 0 sec
[2026-11-13T08:42:31.342Z] [Pipeline] echo
[2026-11-13T08:42:31.342Z] RWLock: log_archive in stage projectFinishUp locked for WRITE
[2026-11-13T08:42:31.346Z] [Pipeline] }
[2026-11-13T08:42:31.348Z] [Pipeline] // node
[2026-11-13T08:42:31.352Z] [Pipeline] echo
[2026-11-13T08:42:31.352Z] Copying logs to /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-kmods/4835/
[2026-11-13T08:42:31.353Z] [Pipeline] sh
[2026-11-13T08:42:31.608Z] + mkdir -p /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-kmods/4835/
[2026-11-13T08:42:31.610Z] [Pipeline] sh