[2026-04-14T09:07:00.340Z] Started by upstream project "anvil/anvil-build-all-kmods" build number 528
[2026-04-14T09:07:00.340Z] originally caused by:
[2026-04-14T09:07:00.340Z]  Started by upstream project "anvil/anvil-build-cron" build number 25
[2026-04-14T09:07:00.340Z]  originally caused by:
[2026-04-14T09:07:00.340Z]   Started by timer
[2026-04-14T09:07:01.283Z] Obtained pipelines/projects/anvil/build-kmods/Jenkinsfile from git https://github.com/kronosnet/ci-tools
[2026-04-14T09:07:01.291Z] Loading library ProjectLib@main:pipelines/projects/anvil/
[2026-04-14T09:07:01.515Z] Examining kronosnet/ci-tools
[2026-04-14T09:07:01.515Z] Attempting to resolve main as a branch
[2026-04-14T09:07:01.653Z] Resolved main as branch main at revision 122331b302b30271242a259844a3b4a95de3e23f
[2026-04-14T09:07:01.654Z] Selected Git installation does not exist. Using Default
[2026-04-14T09:07:01.654Z] The recommended git tool is: NONE
[2026-04-14T09:07:01.654Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-04-14T09:07:01.657Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-04-14T09:07:01.657Z] Wiping out workspace first.
[2026-04-14T09:07:01.685Z] Cloning the remote Git repository
[2026-04-14T09:07:01.685Z] Using shallow clone with depth 1
[2026-04-14T09:07:01.685Z] Avoid fetching tags
[2026-04-14T09:07:01.685Z] Honoring refspec on initial clone
[2026-04-14T09:07:01.685Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-04-14T09:07:01.685Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-kmods@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b@5 # timeout=10
[2026-04-14T09:07:01.697Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-04-14T09:07:01.697Z]  > git --version # timeout=10
[2026-04-14T09:07:01.704Z]  > git --version # 'git version 2.47.3'
[2026-04-14T09:07:01.704Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-04-14T09:07:01.710Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-14T09:07:01.896Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-04-14T09:07:01.904Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-14T09:07:01.913Z] Avoid second fetch
[2026-04-14T09:07:01.913Z] Checking out Revision 122331b302b30271242a259844a3b4a95de3e23f (main)
[2026-04-14T09:07:01.914Z]  > git config core.sparsecheckout # timeout=10
[2026-04-14T09:07:01.921Z]  > git checkout -f 122331b302b30271242a259844a3b4a95de3e23f # timeout=10
[2026-04-14T09:07:01.962Z] Commit message: "fn-testing: move aws to us-west-2"
[2026-04-14T09:07:01.967Z] Loading library GlobalLib@main:./pipelines/libs/global/
[2026-04-14T09:07:02.194Z] Examining kronosnet/ci-tools
[2026-04-14T09:07:02.194Z] Attempting to resolve main as a branch
[2026-04-14T09:07:02.338Z] Resolved main as branch main at revision 122331b302b30271242a259844a3b4a95de3e23f
[2026-04-14T09:07:02.339Z] Selected Git installation does not exist. Using Default
[2026-04-14T09:07:02.339Z] The recommended git tool is: NONE
[2026-04-14T09:07:02.339Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-04-14T09:07:02.341Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-04-14T09:07:02.341Z] Wiping out workspace first.
[2026-04-14T09:07:02.361Z] Cloning the remote Git repository
[2026-04-14T09:07:02.361Z] Using shallow clone with depth 1
[2026-04-14T09:07:02.361Z] Avoid fetching tags
[2026-04-14T09:07:02.361Z] Honoring refspec on initial clone
[2026-04-14T09:07:02.361Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-04-14T09:07:02.361Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-kmods@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b@4 # timeout=10
[2026-04-14T09:07:02.370Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-04-14T09:07:02.370Z]  > git --version # timeout=10
[2026-04-14T09:07:02.377Z]  > git --version # 'git version 2.47.3'
[2026-04-14T09:07:02.377Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-04-14T09:07:02.383Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-14T09:07:02.592Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-04-14T09:07:02.599Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-14T09:07:02.608Z] Avoid second fetch
[2026-04-14T09:07:02.608Z] Checking out Revision 122331b302b30271242a259844a3b4a95de3e23f (main)
[2026-04-14T09:07:02.608Z]  > git config core.sparsecheckout # timeout=10
[2026-04-14T09:07:02.614Z]  > git checkout -f 122331b302b30271242a259844a3b4a95de3e23f # timeout=10
[2026-04-14T09:07:02.648Z] Commit message: "fn-testing: move aws to us-west-2"
[2026-04-14T09:07:02.686Z] [Pipeline] Start of Pipeline
[2026-04-14T09:07:02.774Z] [Pipeline] node
[2026-04-14T09:07:02.775Z] Running on anvil-ci-bm-phy05 in /srv/workspace/anvil/anvil-build-kmods
[2026-04-14T09:07:02.776Z] [Pipeline] {
[2026-04-14T09:07:02.784Z] [Pipeline] stage
[2026-04-14T09:07:02.784Z] [Pipeline] { (Prep build env)
[2026-04-14T09:07:02.793Z] [Pipeline] cleanWs
[2026-04-14T09:07:02.856Z] [WS-CLEANUP] Deleting project workspace...
[2026-04-14T09:07:02.856Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-04-14T09:07:02.888Z] [WS-CLEANUP] done
[2026-04-14T09:07:02.904Z] [Pipeline] lock
[2026-04-14T09:07:02.904Z] Trying to acquire lock on [Resource: MAINTENANCE LOCKOUT]
[2026-04-14T09:07:02.904Z] Resource [MAINTENANCE LOCKOUT] did not exist. Created.
[2026-04-14T09:07:02.904Z] Lock acquired on [Resource: MAINTENANCE LOCKOUT]
[2026-04-14T09:07:02.904Z] [Pipeline] {
[2026-04-14T09:07:02.906Z] [Pipeline] }
[2026-04-14T09:07:02.907Z] Lock released on resource [Resource: MAINTENANCE LOCKOUT]
[2026-04-14T09:07:02.909Z] [Pipeline] // lock
[2026-04-14T09:07:02.912Z] [Pipeline] sh
[2026-04-14T09:07:03.475Z] + /root/ci-tools/ci-destroy-anvil-bm-vm
[2026-04-14T09:07:03.475Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-14T09:07:03.847Z] Genering ssh keys and ssh config
[2026-04-14T09:07:03.847Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:03.847Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:04.219Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:04.219Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:04.219Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:04.219Z] Cleaning up simengine configuration
[2026-04-14T09:07:08.294Z] Destroying VMs (if any)
[2026-04-14T09:07:08.385Z] [Pipeline] }
[2026-04-14T09:07:08.388Z] [Pipeline] // stage
[2026-04-14T09:07:08.391Z] [Pipeline] stage
[2026-04-14T09:07:08.392Z] [Pipeline] { (Build kmod)
[2026-04-14T09:07:08.401Z] [Pipeline] sh
[2026-04-14T09:07:08.962Z] + /root/ci-tools/ci-setup-anvil-bm-vm-kmods almalinux-9 enterprise none
[2026-04-14T09:07:08.962Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-14T09:07:09.333Z] Genering ssh keys and ssh config
[2026-04-14T09:07:09.333Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:09.333Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:09.333Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:09.704Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:09.704Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:09.704Z] Destroying VMs (if any)
[2026-04-14T09:07:10.075Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-14T09:07:10.075Z] Genering ssh keys and ssh config
[2026-04-14T09:07:10.447Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:10.447Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:10.447Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:10.447Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:10.447Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:07:10.818Z] Cache already updated
[2026-04-14T09:07:10.818Z] Building drbd kmod for almalinux-9 repo: enterprise anvil: none
[2026-04-14T09:07:10.818Z] Formatting 'an-cache-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/almalinux-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-14T09:07:10.818Z] wrote an-cache-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-14T09:07:14.029Z] 
[2026-04-14T09:07:14.029Z] Starting install...
[2026-04-14T09:07:14.029Z] 
[2026-04-14T09:07:14.029Z] Creating domain...                                          |         00:00     
[2026-04-14T09:07:14.029Z] Domain creation completed.
[2026-04-14T09:07:14.400Z] Waiting for nodes and cloud-init to complete full init
[2026-04-14T09:07:17.608Z] cloud-init status node an-cache: 
[2026-04-14T09:07:17.609Z] Tue Apr 14 05:07:17 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:07:31.707Z] cloud-init status node an-cache: 
[2026-04-14T09:07:31.707Z] Tue Apr 14 05:07:30 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:07:41.179Z] cloud-init status node an-cache: 
[2026-04-14T09:07:41.179Z] Tue Apr 14 05:07:40 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:07:52.553Z] cloud-init status node an-cache: running
[2026-04-14T09:07:52.553Z] Tue Apr 14 05:07:51 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:08:04.573Z] cloud-init status node an-cache: running
[2026-04-14T09:08:04.573Z] Tue Apr 14 05:08:03 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:08:15.979Z] cloud-init status node an-cache: running
[2026-04-14T09:08:15.979Z] Tue Apr 14 05:08:14 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:08:25.617Z] cloud-init status node an-cache: running
[2026-04-14T09:08:25.617Z] Tue Apr 14 05:08:25 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:08:37.930Z] cloud-init status node an-cache: running
[2026-04-14T09:08:37.930Z] Tue Apr 14 05:08:37 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:08:49.256Z] cloud-init status node an-cache: running
[2026-04-14T09:08:49.256Z] Tue Apr 14 05:08:48 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:09:02.926Z] cloud-init status node an-cache: running
[2026-04-14T09:09:02.926Z] Tue Apr 14 05:09:00 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:09:12.861Z] cloud-init status node an-cache: running
[2026-04-14T09:09:12.861Z] Tue Apr 14 05:09:12 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:09:26.544Z] cloud-init status node an-cache: running
[2026-04-14T09:09:26.544Z] Tue Apr 14 05:09:24 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:09:37.865Z] cloud-init status node an-cache: running
[2026-04-14T09:09:37.865Z] Tue Apr 14 05:09:35 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:09:49.191Z] cloud-init status node an-cache: running
[2026-04-14T09:09:49.191Z] Tue Apr 14 05:09:47 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:09:58.548Z] cloud-init status node an-cache: running
[2026-04-14T09:09:58.548Z] Tue Apr 14 05:09:58 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:10:09.878Z] cloud-init status node an-cache: running
[2026-04-14T09:10:09.878Z] Tue Apr 14 05:10:09 EDT 2026 actual: 0 expected: 1
[2026-04-14T09:10:21.219Z] cloud-init status node an-cache: done
[2026-04-14T09:10:21.219Z] Tue Apr 14 05:10:19 EDT 2026 actual: 1 expected: 1
[2026-04-14T09:10:21.219Z] Waiting for akmods to complete build
[2026-04-14T09:10:21.219Z] build not found, sleeping 10 seconds
[2026-04-14T09:10:32.532Z] build not found, sleeping 10 seconds
[2026-04-14T09:10:41.876Z] build not found, sleeping 10 seconds
[2026-04-14T09:10:53.429Z] build not found, sleeping 10 seconds
[2026-04-14T09:11:04.760Z] build not found, sleeping 10 seconds
[2026-04-14T09:11:14.128Z] build not found, sleeping 10 seconds
[2026-04-14T09:11:25.735Z] build not found, sleeping 10 seconds
[2026-04-14T09:11:37.066Z] build not found, sleeping 10 seconds
[2026-04-14T09:11:46.427Z] build not found, sleeping 10 seconds
[2026-04-14T09:11:57.757Z] build not found, sleeping 10 seconds
[2026-04-14T09:12:07.147Z] build not found, sleeping 10 seconds
[2026-04-14T09:12:18.507Z] build not found, sleeping 10 seconds
[2026-04-14T09:12:29.837Z] build not found, sleeping 10 seconds
[2026-04-14T09:12:41.161Z] build not found, sleeping 10 seconds
[2026-04-14T09:12:50.517Z] build not found, sleeping 10 seconds
[2026-04-14T09:13:02.331Z] build not found, sleeping 10 seconds
[2026-04-14T09:13:13.664Z] build not found, sleeping 10 seconds
[2026-04-14T09:13:23.014Z] build not found, sleeping 10 seconds
[2026-04-14T09:13:34.350Z] build not found, sleeping 10 seconds
[2026-04-14T09:13:45.678Z] build not found, sleeping 10 seconds
[2026-04-14T09:13:55.293Z] /var/cache/akmods/drbd/kmod-drbd-5.14.0-611.el9_7-9.2.16-1.el9.x86_64.rpm
[2026-04-14T09:13:55.293Z] build found
[2026-04-14T09:13:55.664Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-14T09:13:56.456Z] Directory walk started
[2026-04-14T09:13:56.456Z] Directory walk done - 7 packages
[2026-04-14T09:13:56.456Z] Temporary output repo path: ./.repodata/
[2026-04-14T09:13:56.456Z] Preparing sqlite DBs
[2026-04-14T09:13:56.456Z] Pool started (with 5 workers)
[2026-04-14T09:13:56.456Z] Pool finished
[2026-04-14T09:13:56.456Z] /srv/anvil/cache
[2026-04-14T09:13:57.247Z] bash: line 1: rhc: command not found
[2026-04-14T09:13:57.625Z] bash: line 1: insights-client: command not found
[2026-04-14T09:13:58.000Z] bash: line 1: subscription-manager: command not found
[2026-04-14T09:14:00.469Z] Domain 'an-cache' destroyed
[2026-04-14T09:14:00.469Z] 
[2026-04-14T09:14:00.469Z] Domain 'an-cache' has been undefined
[2026-04-14T09:14:00.469Z] 
[2026-04-14T09:14:00.469Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:14:00.469Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:14:00.469Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:14:00.842Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:14:00.842Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:14:00.842Z] Done building drbd kmods
[2026-04-14T09:14:00.935Z] [Pipeline] }
[2026-04-14T09:14:00.937Z] [Pipeline] // stage
[2026-04-14T09:14:00.940Z] [Pipeline] stage
[2026-04-14T09:14:00.940Z] [Pipeline] { (Declarative: Post Actions)
[2026-04-14T09:14:00.945Z] [Pipeline] sh
[2026-04-14T09:14:01.583Z] + /root/ci-tools/ci-destroy-anvil-bm-vm
[2026-04-14T09:14:01.583Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-14T09:14:01.954Z] Genering ssh keys and ssh config
[2026-04-14T09:14:01.954Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:14:01.954Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:14:01.954Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:14:02.414Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:14:02.414Z] # github.com:22 SSH-2.0-3992d52
[2026-04-14T09:14:02.414Z] Cleaning up simengine configuration
[2026-04-14T09:14:06.491Z] Destroying VMs (if any)
[2026-04-14T09:14:06.585Z] [Pipeline] cleanWs
[2026-04-14T09:14:06.646Z] [WS-CLEANUP] Deleting project workspace...
[2026-04-14T09:14:06.647Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-04-14T09:14:06.678Z] [WS-CLEANUP] done
[2026-04-14T09:14:06.682Z] [Pipeline] script
[2026-04-14T09:14:06.683Z] [Pipeline] {
[2026-04-14T09:14:06.696Z] [Pipeline] stage
[2026-04-14T09:14:06.696Z] [Pipeline] { (Copying logs)
[2026-04-14T09:14:06.726Z] [Pipeline] node
[2026-04-14T09:14:06.733Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-kmods
[2026-04-14T09:14:06.734Z] [Pipeline] {
[2026-04-14T09:14:06.768Z] [Pipeline] node
[2026-04-14T09:14:06.774Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-kmods@2
[2026-04-14T09:14:06.774Z] [Pipeline] {
[2026-04-14T09:14:06.777Z] [Pipeline] sh
[2026-04-14T09:14:07.032Z] + mkdir -p /var/lib/jenkins/locks
[2026-04-14T09:14:07.040Z] [Pipeline] echo
[2026-04-14T09:14:07.040Z] RWLock: FD for lock log_archive in stage projectFinishUp is 416
[2026-04-14T09:14:07.042Z] [Pipeline] sleep
[2026-04-14T09:14:07.043Z] No need to sleep any longer
[2026-04-14T09:14:07.045Z] [Pipeline] echo
[2026-04-14T09:14:07.045Z] RWLock: log_archive in stage projectFinishUp locked for WRITE
[2026-04-14T09:14:07.047Z] [Pipeline] }
[2026-04-14T09:14:07.049Z] [Pipeline] // node
[2026-04-14T09:14:07.051Z] [Pipeline] echo
[2026-04-14T09:14:07.052Z] Copying logs to /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-kmods/3661/
[2026-04-14T09:14:07.052Z] [Pipeline] sh
[2026-04-14T09:14:07.307Z] + mkdir -p /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-kmods/3661/
[2026-04-14T09:14:07.308Z] [Pipeline] sh