[2026-04-30T08:40:15.338Z] Started by upstream project "anvil/anvil-build-all-kmods" build number 543
[2026-04-30T08:40:15.338Z] originally caused by:
[2026-04-30T08:40:15.338Z]  Started by upstream project "anvil/anvil-build-cron" build number 40
[2026-04-30T08:40:15.338Z]  originally caused by:
[2026-04-30T08:40:15.338Z]   Started by timer
[2026-04-30T08:40:16.286Z] Obtained pipelines/projects/anvil/build-kmods/Jenkinsfile from git https://github.com/kronosnet/ci-tools
[2026-04-30T08:40:16.294Z] Loading library ProjectLib@main:pipelines/projects/anvil/
[2026-04-30T08:40:16.606Z] Examining kronosnet/ci-tools
[2026-04-30T08:40:16.607Z] Attempting to resolve main as a branch
[2026-04-30T08:40:16.809Z] Resolved main as branch main at revision 9ad09a984fa36f4673663554aea24fb7e9688b8f
[2026-04-30T08:40:16.811Z] Selected Git installation does not exist. Using Default
[2026-04-30T08:40:16.811Z] The recommended git tool is: NONE
[2026-04-30T08:40:16.811Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-04-30T08:40:16.814Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-04-30T08:40:16.814Z] Wiping out workspace first.
[2026-04-30T08:40:16.837Z] Cloning the remote Git repository
[2026-04-30T08:40:16.837Z] Using shallow clone with depth 1
[2026-04-30T08:40:16.837Z] Avoid fetching tags
[2026-04-30T08:40:16.837Z] Honoring refspec on initial clone
[2026-04-30T08:40:16.838Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-04-30T08:40:16.838Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-kmods@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b@4 # timeout=10
[2026-04-30T08:40:16.850Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-04-30T08:40:16.850Z]  > git --version # timeout=10
[2026-04-30T08:40:16.859Z]  > git --version # 'git version 2.47.3'
[2026-04-30T08:40:16.859Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-04-30T08:40:16.865Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-30T08:40:17.086Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-04-30T08:40:17.095Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-30T08:40:17.104Z] Avoid second fetch
[2026-04-30T08:40:17.105Z] Checking out Revision 9ad09a984fa36f4673663554aea24fb7e9688b8f (main)
[2026-04-30T08:40:17.105Z]  > git config core.sparsecheckout # timeout=10
[2026-04-30T08:40:17.113Z]  > git checkout -f 9ad09a984fa36f4673663554aea24fb7e9688b8f # timeout=10
[2026-04-30T08:40:17.148Z] Commit message: "Merge pull request #44 from nrwahl2/nrwahl2-pacemaker"
[2026-04-30T08:40:17.157Z] Loading library GlobalLib@main:./pipelines/libs/global/
[2026-04-30T08:40:17.375Z] Examining kronosnet/ci-tools
[2026-04-30T08:40:17.375Z] Attempting to resolve main as a branch
[2026-04-30T08:40:17.571Z] Resolved main as branch main at revision 9ad09a984fa36f4673663554aea24fb7e9688b8f
[2026-04-30T08:40:17.572Z] Selected Git installation does not exist. Using Default
[2026-04-30T08:40:17.572Z] The recommended git tool is: NONE
[2026-04-30T08:40:17.572Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-04-30T08:40:17.576Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-04-30T08:40:17.576Z] Wiping out workspace first.
[2026-04-30T08:40:17.594Z] Cloning the remote Git repository
[2026-04-30T08:40:17.594Z] Using shallow clone with depth 1
[2026-04-30T08:40:17.594Z] Avoid fetching tags
[2026-04-30T08:40:17.594Z] Honoring refspec on initial clone
[2026-04-30T08:40:17.595Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-04-30T08:40:17.595Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-kmods@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b # timeout=10
[2026-04-30T08:40:17.605Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-04-30T08:40:17.605Z]  > git --version # timeout=10
[2026-04-30T08:40:17.612Z]  > git --version # 'git version 2.47.3'
[2026-04-30T08:40:17.613Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-04-30T08:40:17.620Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-30T08:40:17.831Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-04-30T08:40:17.838Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-30T08:40:17.846Z] Avoid second fetch
[2026-04-30T08:40:17.847Z] Checking out Revision 9ad09a984fa36f4673663554aea24fb7e9688b8f (main)
[2026-04-30T08:40:17.847Z]  > git config core.sparsecheckout # timeout=10
[2026-04-30T08:40:17.855Z]  > git checkout -f 9ad09a984fa36f4673663554aea24fb7e9688b8f # timeout=10
[2026-04-30T08:40:17.890Z] Commit message: "Merge pull request #44 from nrwahl2/nrwahl2-pacemaker"
[2026-04-30T08:40:17.936Z] [Pipeline] Start of Pipeline
[2026-04-30T08:40:18.072Z] [Pipeline] node
[2026-04-30T08:40:18.087Z] Running on anvil-ci-bm-phy05 in /srv/workspace/anvil/anvil-build-kmods
[2026-04-30T08:40:18.088Z] [Pipeline] {
[2026-04-30T08:40:18.096Z] [Pipeline] stage
[2026-04-30T08:40:18.096Z] [Pipeline] { (Prep build env)
[2026-04-30T08:40:18.109Z] [Pipeline] cleanWs
[2026-04-30T08:40:18.174Z] [WS-CLEANUP] Deleting project workspace...
[2026-04-30T08:40:18.174Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-04-30T08:40:18.206Z] [WS-CLEANUP] done
[2026-04-30T08:40:18.227Z] [Pipeline] lock
[2026-04-30T08:40:18.228Z] Trying to acquire lock on [Resource: MAINTENANCE LOCKOUT]
[2026-04-30T08:40:18.228Z] Resource [MAINTENANCE LOCKOUT] did not exist. Created.
[2026-04-30T08:40:18.228Z] Lock acquired on [Resource: MAINTENANCE LOCKOUT]
[2026-04-30T08:40:18.228Z] [Pipeline] {
[2026-04-30T08:40:18.231Z] [Pipeline] }
[2026-04-30T08:40:18.231Z] Lock released on resource [Resource: MAINTENANCE LOCKOUT]
[2026-04-30T08:40:18.233Z] [Pipeline] // lock
[2026-04-30T08:40:18.236Z] [Pipeline] sh
[2026-04-30T08:40:18.778Z] + /root/ci-tools/ci-destroy-anvil-bm-vm
[2026-04-30T08:40:18.778Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-30T08:40:19.152Z] Genering ssh keys and ssh config
[2026-04-30T08:40:19.152Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:19.152Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:19.152Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:19.527Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:19.527Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:19.527Z] Cleaning up simengine configuration
[2026-04-30T08:40:23.627Z] Destroying VMs (if any)
[2026-04-30T08:40:23.723Z] [Pipeline] }
[2026-04-30T08:40:23.725Z] [Pipeline] // stage
[2026-04-30T08:40:23.728Z] [Pipeline] stage
[2026-04-30T08:40:23.728Z] [Pipeline] { (Build kmod)
[2026-04-30T08:40:23.738Z] [Pipeline] sh
[2026-04-30T08:40:24.290Z] + /root/ci-tools/ci-setup-anvil-bm-vm-kmods almalinux-9 enterprise none
[2026-04-30T08:40:24.290Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-30T08:40:24.666Z] Genering ssh keys and ssh config
[2026-04-30T08:40:24.666Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:24.666Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:25.042Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:25.042Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:25.042Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:25.042Z] Destroying VMs (if any)
[2026-04-30T08:40:25.421Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-30T08:40:25.806Z] Genering ssh keys and ssh config
[2026-04-30T08:40:25.806Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:25.806Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:26.181Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:26.181Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:26.181Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:40:26.181Z] Cache already updated
[2026-04-30T08:40:26.181Z] Building drbd kmod for almalinux-9 repo: enterprise anvil: none
[2026-04-30T08:40:26.181Z] Formatting 'an-cache-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/almalinux-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-30T08:40:26.559Z] wrote an-cache-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-30T08:40:29.065Z] 
[2026-04-30T08:40:29.065Z] Starting install...
[2026-04-30T08:40:29.065Z] 
[2026-04-30T08:40:29.065Z] Creating domain...                                          |         00:00     
[2026-04-30T08:40:29.065Z] Domain creation completed.
[2026-04-30T08:40:29.065Z] Waiting for nodes and cloud-init to complete full init
[2026-04-30T08:40:32.306Z] cloud-init status node an-cache: 
[2026-04-30T08:40:32.306Z] Thu Apr 30 04:40:32 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:40:46.046Z] cloud-init status node an-cache: 
[2026-04-30T08:40:46.046Z] Thu Apr 30 04:40:45 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:40:55.488Z] cloud-init status node an-cache: 
[2026-04-30T08:40:55.488Z] Thu Apr 30 04:40:55 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:41:06.861Z] cloud-init status node an-cache: running
[2026-04-30T08:41:06.861Z] Thu Apr 30 04:41:06 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:41:18.239Z] cloud-init status node an-cache: running
[2026-04-30T08:41:18.239Z] Thu Apr 30 04:41:17 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:41:29.571Z] cloud-init status node an-cache: running
[2026-04-30T08:41:29.571Z] Thu Apr 30 04:41:28 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:41:41.353Z] cloud-init status node an-cache: running
[2026-04-30T08:41:41.353Z] Thu Apr 30 04:41:40 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:41:52.681Z] cloud-init status node an-cache: running
[2026-04-30T08:41:52.681Z] Thu Apr 30 04:41:51 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:42:04.041Z] cloud-init status node an-cache: running
[2026-04-30T08:42:04.041Z] Thu Apr 30 04:42:03 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:42:17.746Z] cloud-init status node an-cache: running
[2026-04-30T08:42:17.746Z] Thu Apr 30 04:42:15 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:42:29.113Z] cloud-init status node an-cache: running
[2026-04-30T08:42:29.113Z] Thu Apr 30 04:42:27 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:42:40.472Z] cloud-init status node an-cache: running
[2026-04-30T08:42:40.472Z] Thu Apr 30 04:42:39 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:42:51.823Z] cloud-init status node an-cache: running
[2026-04-30T08:42:51.823Z] Thu Apr 30 04:42:51 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:43:05.492Z] cloud-init status node an-cache: running
[2026-04-30T08:43:05.492Z] Thu Apr 30 04:43:03 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:43:14.846Z] cloud-init status node an-cache: running
[2026-04-30T08:43:14.846Z] Thu Apr 30 04:43:14 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:43:26.186Z] cloud-init status node an-cache: running
[2026-04-30T08:43:26.186Z] Thu Apr 30 04:43:25 EDT 2026 actual: 0 expected: 1
[2026-04-30T08:43:37.532Z] cloud-init status node an-cache: done
[2026-04-30T08:43:37.532Z] Thu Apr 30 04:43:36 EDT 2026 actual: 1 expected: 1
[2026-04-30T08:43:37.532Z] Waiting for akmods to complete build
[2026-04-30T08:43:37.532Z] build not found, sleeping 10 seconds
[2026-04-30T08:43:46.886Z] build not found, sleeping 10 seconds
[2026-04-30T08:43:58.222Z] build not found, sleeping 10 seconds
[2026-04-30T08:44:09.554Z] build not found, sleeping 10 seconds
[2026-04-30T08:44:20.877Z] build not found, sleeping 10 seconds
[2026-04-30T08:44:30.249Z] build not found, sleeping 10 seconds
[2026-04-30T08:44:41.597Z] build not found, sleeping 10 seconds
[2026-04-30T08:44:52.933Z] build not found, sleeping 10 seconds
[2026-04-30T08:45:04.274Z] build not found, sleeping 10 seconds
[2026-04-30T08:45:13.647Z] build not found, sleeping 10 seconds
[2026-04-30T08:45:25.118Z] build not found, sleeping 10 seconds
[2026-04-30T08:45:36.563Z] build not found, sleeping 10 seconds
[2026-04-30T08:45:45.964Z] build not found, sleeping 10 seconds
[2026-04-30T08:45:57.311Z] build not found, sleeping 10 seconds
[2026-04-30T08:46:08.663Z] build not found, sleeping 10 seconds
[2026-04-30T08:46:18.043Z] build not found, sleeping 10 seconds
[2026-04-30T08:46:29.615Z] build not found, sleeping 10 seconds
[2026-04-30T08:46:41.166Z] build not found, sleeping 10 seconds
[2026-04-30T08:46:50.540Z] build not found, sleeping 10 seconds
[2026-04-30T08:47:01.876Z] build not found, sleeping 10 seconds
[2026-04-30T08:47:13.231Z] build not found, sleeping 10 seconds
[2026-04-30T08:47:22.609Z] build not found, sleeping 10 seconds
[2026-04-30T08:47:34.042Z] /var/cache/akmods/drbd/kmod-drbd-5.14.0-611.el9_7-9.2.16-1.el9.x86_64.rpm
[2026-04-30T08:47:34.042Z] build found
[2026-04-30T08:47:34.042Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-30T08:47:35.321Z] Directory walk started
[2026-04-30T08:47:35.321Z] Directory walk done - 7 packages
[2026-04-30T08:47:35.321Z] Temporary output repo path: ./.repodata/
[2026-04-30T08:47:35.321Z] Preparing sqlite DBs
[2026-04-30T08:47:35.321Z] Pool started (with 5 workers)
[2026-04-30T08:47:35.321Z] Pool finished
[2026-04-30T08:47:35.321Z] /srv/anvil/cache
[2026-04-30T08:47:35.693Z] bash: line 1: rhc: command not found
[2026-04-30T08:47:36.488Z] bash: line 1: insights-client: command not found
[2026-04-30T08:47:36.861Z] bash: line 1: subscription-manager: command not found
[2026-04-30T08:47:39.337Z] Domain 'an-cache' destroyed
[2026-04-30T08:47:39.337Z] 
[2026-04-30T08:47:39.337Z] Domain 'an-cache' has been undefined
[2026-04-30T08:47:39.337Z] 
[2026-04-30T08:47:39.337Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:47:39.337Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:47:39.337Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:47:39.709Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:47:39.709Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:47:39.709Z] Done building drbd kmods
[2026-04-30T08:47:39.801Z] [Pipeline] }
[2026-04-30T08:47:39.804Z] [Pipeline] // stage
[2026-04-30T08:47:39.808Z] [Pipeline] stage
[2026-04-30T08:47:39.808Z] [Pipeline] { (Declarative: Post Actions)
[2026-04-30T08:47:39.815Z] [Pipeline] sh
[2026-04-30T08:47:40.437Z] + /root/ci-tools/ci-destroy-anvil-bm-vm
[2026-04-30T08:47:40.437Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-30T08:47:40.809Z] Genering ssh keys and ssh config
[2026-04-30T08:47:40.809Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:47:40.809Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:47:41.182Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:47:41.182Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:47:41.182Z] # github.com:22 SSH-2.0-a59182e
[2026-04-30T08:47:41.182Z] Cleaning up simengine configuration
[2026-04-30T08:47:45.270Z] Destroying VMs (if any)
[2026-04-30T08:47:45.366Z] [Pipeline] cleanWs
[2026-04-30T08:47:45.430Z] [WS-CLEANUP] Deleting project workspace...
[2026-04-30T08:47:45.430Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-04-30T08:47:45.461Z] [WS-CLEANUP] done
[2026-04-30T08:47:45.466Z] [Pipeline] script
[2026-04-30T08:47:45.471Z] [Pipeline] {
[2026-04-30T08:47:45.494Z] [Pipeline] stage
[2026-04-30T08:47:45.494Z] [Pipeline] { (Copying logs)
[2026-04-30T08:47:45.533Z] [Pipeline] node
[2026-04-30T08:47:45.668Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-kmods@3
[2026-04-30T08:47:45.669Z] [Pipeline] {
[2026-04-30T08:47:45.722Z] [Pipeline] node
[2026-04-30T08:47:45.826Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-kmods@5
[2026-04-30T08:47:45.827Z] [Pipeline] {
[2026-04-30T08:47:45.831Z] [Pipeline] sh
[2026-04-30T08:47:46.088Z] + mkdir -p /var/lib/jenkins/locks
[2026-04-30T08:47:46.098Z] [Pipeline] echo
[2026-04-30T08:47:46.098Z] RWLock: FD for lock log_archive in stage projectFinishUp is 376
[2026-04-30T08:47:46.101Z] [Pipeline] sleep
[2026-04-30T08:47:46.101Z] No need to sleep any longer
[2026-04-30T08:47:46.104Z] [Pipeline] echo
[2026-04-30T08:47:46.104Z] RWLock: log_archive in stage projectFinishUp locked for WRITE
[2026-04-30T08:47:46.107Z] [Pipeline] }
[2026-04-30T08:47:46.109Z] [Pipeline] // node
[2026-04-30T08:47:46.112Z] [Pipeline] echo
[2026-04-30T08:47:46.112Z] Copying logs to /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-kmods/3751/
[2026-04-30T08:47:46.113Z] [Pipeline] sh
[2026-04-30T08:47:46.369Z] + mkdir -p /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-kmods/3751/
[2026-04-30T08:47:46.371Z] [Pipeline] sh