[2026-11-03T08:16:00.393Z] Started by timer
[2026-11-03T08:16:00.613Z] Obtained pipelines/projects/anvil/build-cron/Jenkinsfile from git https://github.com/kronosnet/ci-tools
[2026-11-03T08:16:00.632Z] Loading library ProjectLib@main:pipelines/projects/anvil/
[2026-11-03T08:16:01.242Z] Examining kronosnet/ci-tools
[2026-11-03T08:16:01.242Z] Attempting to resolve main as a branch
[2026-11-03T08:16:01.449Z] Resolved main as branch main at revision fd5667069c7b69778ce97df718cf25626f5beefc
[2026-11-03T08:16:01.451Z] Selected Git installation does not exist. Using Default
[2026-11-03T08:16:01.451Z] The recommended git tool is: NONE
[2026-11-03T08:16:01.452Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-11-03T08:16:01.454Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-11-03T08:16:01.454Z] Wiping out workspace first.
[2026-11-03T08:16:01.480Z] Cloning the remote Git repository
[2026-11-03T08:16:01.480Z] Using shallow clone with depth 1
[2026-11-03T08:16:01.480Z] Avoid fetching tags
[2026-11-03T08:16:01.480Z] Honoring refspec on initial clone
[2026-11-03T08:16:01.480Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-11-03T08:16:01.480Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-cron@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b # timeout=10
[2026-11-03T08:16:01.490Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-11-03T08:16:01.490Z]  > git --version # timeout=10
[2026-11-03T08:16:01.498Z]  > git --version # 'git version 2.52.0'
[2026-11-03T08:16:01.498Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-11-03T08:16:01.505Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-11-03T08:16:01.775Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-11-03T08:16:01.782Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-11-03T08:16:01.790Z] Avoid second fetch
[2026-11-03T08:16:01.790Z] Checking out Revision fd5667069c7b69778ce97df718cf25626f5beefc (main)
[2026-11-03T08:16:01.790Z]  > git config core.sparsecheckout # timeout=10
[2026-11-03T08:16:01.797Z]  > git checkout -f fd5667069c7b69778ce97df718cf25626f5beefc # timeout=10
[2026-11-03T08:16:01.832Z] Commit message: "pipelines: fix ansible managed update pipelines to process all nodes in parallel"
[2026-11-03T08:16:01.841Z] Loading library GlobalLib@main:./pipelines/libs/global/
[2026-11-03T08:16:02.108Z] Examining kronosnet/ci-tools
[2026-11-03T08:16:02.108Z] Attempting to resolve main as a branch
[2026-11-03T08:16:02.303Z] Resolved main as branch main at revision fd5667069c7b69778ce97df718cf25626f5beefc
[2026-11-03T08:16:02.304Z] Selected Git installation does not exist. Using Default
[2026-11-03T08:16:02.304Z] The recommended git tool is: NONE
[2026-11-03T08:16:02.304Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-11-03T08:16:02.307Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-11-03T08:16:02.307Z] Wiping out workspace first.
[2026-11-03T08:16:02.326Z] Cloning the remote Git repository
[2026-11-03T08:16:02.326Z] Using shallow clone with depth 1
[2026-11-03T08:16:02.326Z] Avoid fetching tags
[2026-11-03T08:16:02.326Z] Honoring refspec on initial clone
[2026-11-03T08:16:02.326Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-11-03T08:16:02.326Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-cron@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b # timeout=10
[2026-11-03T08:16:02.335Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-11-03T08:16:02.335Z]  > git --version # timeout=10
[2026-11-03T08:16:02.342Z]  > git --version # 'git version 2.52.0'
[2026-11-03T08:16:02.342Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-11-03T08:16:02.347Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-11-03T08:16:02.616Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-11-03T08:16:02.623Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-11-03T08:16:02.630Z] Avoid second fetch
[2026-11-03T08:16:02.631Z] Checking out Revision fd5667069c7b69778ce97df718cf25626f5beefc (main)
[2026-11-03T08:16:02.631Z]  > git config core.sparsecheckout # timeout=10
[2026-11-03T08:16:02.637Z]  > git checkout -f fd5667069c7b69778ce97df718cf25626f5beefc # timeout=10
[2026-11-03T08:16:02.672Z] Commit message: "pipelines: fix ansible managed update pipelines to process all nodes in parallel"
[2026-11-03T08:16:02.947Z] [Pipeline] Start of Pipeline
[2026-11-03T08:16:03.092Z] [Pipeline] node
[2026-11-03T08:16:03.196Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron
[2026-11-03T08:16:03.197Z] [Pipeline] {
[2026-11-03T08:16:03.209Z] [Pipeline] stage
[2026-11-03T08:16:03.209Z] [Pipeline] { (Setup build env)
[2026-11-03T08:16:03.219Z] [Pipeline] script
[2026-11-03T08:16:03.222Z] [Pipeline] {
[2026-11-03T08:16:03.226Z] [Pipeline] cleanWs
[2026-11-03T08:16:03.227Z] [WS-CLEANUP] Deleting project workspace...
[2026-11-03T08:16:03.227Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-11-03T08:16:03.227Z] [WS-CLEANUP] done
[2026-11-03T08:16:03.250Z] [Pipeline] lock
[2026-11-03T08:16:03.251Z] Trying to acquire lock on [Resource: MAINTENANCE LOCKOUT]
[2026-11-03T08:16:03.251Z] Resource [MAINTENANCE LOCKOUT] did not exist. Created.
[2026-11-03T08:16:03.251Z] Lock acquired on [Resource: MAINTENANCE LOCKOUT]
[2026-11-03T08:16:03.251Z] [Pipeline] {
[2026-11-03T08:16:03.254Z] [Pipeline] }
[2026-11-03T08:16:03.254Z] Lock released on resource [Resource: MAINTENANCE LOCKOUT]
[2026-11-03T08:16:03.256Z] [Pipeline] // lock
[2026-11-03T08:16:03.259Z] [Pipeline] }
[2026-11-03T08:16:03.262Z] [Pipeline] // script
[2026-11-03T08:16:03.264Z] [Pipeline] }
[2026-11-03T08:16:03.266Z] [Pipeline] // stage
[2026-11-03T08:16:03.269Z] [Pipeline] stage
[2026-11-03T08:16:03.269Z] [Pipeline] { (Update repos and cache)
[2026-11-03T08:16:03.275Z] [Pipeline] parallel
[2026-11-03T08:16:03.275Z] [Pipeline] { (Branch: anvil build cache)
[2026-11-03T08:16:03.275Z] [Pipeline] { (Branch: anvil build dep repos)
[2026-11-03T08:16:03.279Z] [Pipeline] stage
[2026-11-03T08:16:03.279Z] [Pipeline] { (anvil build cache)
[2026-11-03T08:16:03.280Z] [Pipeline] stage
[2026-11-03T08:16:03.280Z] [Pipeline] { (anvil build dep repos)
[2026-11-03T08:16:03.287Z] [Pipeline] script
[2026-11-03T08:16:03.288Z] [Pipeline] {
[2026-11-03T08:16:03.289Z] [Pipeline] script
[2026-11-03T08:16:03.290Z] [Pipeline] {
[2026-11-03T08:16:03.303Z] [Pipeline] build
[2026-11-03T08:16:03.304Z] Scheduling project: anvil » anvil-build-cache
[2026-11-03T08:16:03.305Z] [Pipeline] build
[2026-11-03T08:16:03.305Z] Scheduling project: anvil » anvil-build-dep-repos » main
[2026-11-03T08:16:10.600Z] Starting building: anvil » anvil-build-cache #784
[2026-11-03T08:16:10.601Z] Starting building: anvil » anvil-build-dep-repos » main #219
[2026-11-03T08:16:10.602Z] [Pipeline] waitForBuild
[2026-11-03T08:16:10.602Z] Waiting for anvil » anvil-build-cache #784 to complete
[2026-11-03T08:16:10.609Z] [Pipeline] waitForBuild
[2026-11-03T08:16:10.609Z] Waiting for anvil » anvil-build-dep-repos » main #219 to complete
[2026-11-03T08:38:08.846Z] Build anvil » anvil-build-cache #784 completed: SUCCESS
[2026-11-03T08:38:08.848Z] [Pipeline] }
[2026-11-03T08:38:08.851Z] [Pipeline] // script
[2026-11-03T08:38:08.853Z] [Pipeline] }
[2026-11-03T08:38:08.855Z] [Pipeline] // stage
[2026-11-03T08:38:08.857Z] [Pipeline] }
[2026-11-03T08:39:11.756Z] Build anvil » anvil-build-dep-repos » main #219 completed: SUCCESS
[2026-11-03T08:39:11.759Z] [Pipeline] }
[2026-11-03T08:39:11.762Z] [Pipeline] // script
[2026-11-03T08:39:11.764Z] [Pipeline] }
[2026-11-03T08:39:11.766Z] [Pipeline] // stage
[2026-11-03T08:39:11.769Z] [Pipeline] }
[2026-11-03T08:39:11.771Z] [Pipeline] // parallel
[2026-11-03T08:39:11.773Z] [Pipeline] }
[2026-11-03T08:39:11.775Z] [Pipeline] // stage
[2026-11-03T08:39:11.779Z] [Pipeline] stage
[2026-11-03T08:39:11.779Z] [Pipeline] { (Update kernel modules)
[2026-11-03T08:39:11.784Z] [Pipeline] parallel
[2026-11-03T08:39:11.784Z] [Pipeline] { (Branch: anvil build all kmods)
[2026-11-03T08:39:11.788Z] [Pipeline] stage
[2026-11-03T08:39:11.788Z] [Pipeline] { (anvil build all kmods)
[2026-11-03T08:39:11.795Z] [Pipeline] script
[2026-11-03T08:39:11.797Z] [Pipeline] {
[2026-11-03T08:39:11.800Z] [Pipeline] build
[2026-11-03T08:39:11.800Z] Scheduling project: anvil » anvil-build-all-kmods
[2026-11-03T08:39:21.113Z] Starting building: anvil » anvil-build-all-kmods #715
[2026-11-03T08:39:21.114Z] [Pipeline] waitForBuild
[2026-11-03T08:39:21.115Z] Waiting for anvil » anvil-build-all-kmods #715 to complete
[2026-11-03T08:48:49.955Z] Build anvil » anvil-build-all-kmods #715 completed: SUCCESS
[2026-11-03T08:48:49.957Z] [Pipeline] }
[2026-11-03T08:48:49.960Z] [Pipeline] // script
[2026-11-03T08:48:49.962Z] [Pipeline] }
[2026-11-03T08:48:49.964Z] [Pipeline] // stage
[2026-11-03T08:48:49.966Z] [Pipeline] }
[2026-11-03T08:48:49.968Z] [Pipeline] // parallel
[2026-11-03T08:48:49.971Z] [Pipeline] }
[2026-11-03T08:48:49.973Z] [Pipeline] // stage
[2026-11-03T08:48:49.976Z] [Pipeline] stage
[2026-11-03T08:48:49.976Z] [Pipeline] { (Run daily functional testing)
[2026-11-03T08:48:49.982Z] [Pipeline] parallel
[2026-11-03T08:48:49.982Z] [Pipeline] { (Branch: almalinux-9 ci ci)
[2026-11-03T08:48:49.982Z] [Pipeline] { (Branch: rhel-9 ci ci)
[2026-11-03T08:48:49.983Z] [Pipeline] { (Branch: almalinux-9 enterprise none)
[2026-11-03T08:48:49.983Z] [Pipeline] { (Branch: rhel-9 enterprise none)
[2026-11-03T08:48:49.983Z] [Pipeline] { (Branch: almalinux-9 enterprise-staging none)
[2026-11-03T08:48:49.983Z] [Pipeline] { (Branch: rhel-9 enterprise-staging none)
[2026-11-03T08:48:49.990Z] [Pipeline] stage
[2026-11-03T08:48:49.991Z] [Pipeline] { (almalinux-9 ci ci)
[2026-11-03T08:48:49.991Z] [Pipeline] stage
[2026-11-03T08:48:49.992Z] [Pipeline] { (rhel-9 ci ci)
[2026-11-03T08:48:49.992Z] [Pipeline] stage
[2026-11-03T08:48:49.992Z] [Pipeline] { (almalinux-9 enterprise none)
[2026-11-03T08:48:49.993Z] [Pipeline] stage
[2026-11-03T08:48:49.993Z] [Pipeline] { (rhel-9 enterprise none)
[2026-11-03T08:48:49.994Z] [Pipeline] stage
[2026-11-03T08:48:49.994Z] [Pipeline] { (almalinux-9 enterprise-staging none)
[2026-11-03T08:48:49.995Z] [Pipeline] stage
[2026-11-03T08:48:49.995Z] [Pipeline] { (rhel-9 enterprise-staging none)
[2026-11-03T08:48:50.011Z] [Pipeline] script
[2026-11-03T08:48:50.012Z] [Pipeline] {
[2026-11-03T08:48:50.013Z] [Pipeline] script
[2026-11-03T08:48:50.014Z] [Pipeline] {
[2026-11-03T08:48:50.014Z] [Pipeline] script
[2026-11-03T08:48:50.015Z] [Pipeline] {
[2026-11-03T08:48:50.016Z] [Pipeline] script
[2026-11-03T08:48:50.016Z] [Pipeline] {
[2026-11-03T08:48:50.017Z] [Pipeline] script
[2026-11-03T08:48:50.018Z] [Pipeline] {
[2026-11-03T08:48:50.019Z] [Pipeline] script
[2026-11-03T08:48:50.019Z] [Pipeline] {
[2026-11-03T08:48:50.024Z] [Pipeline] build
[2026-11-03T08:48:50.024Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-11-03T08:48:50.026Z] [Pipeline] build
[2026-11-03T08:48:50.026Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-11-03T08:48:50.028Z] [Pipeline] build
[2026-11-03T08:48:50.028Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-11-03T08:48:50.029Z] [Pipeline] build
[2026-11-03T08:48:50.029Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-11-03T08:48:50.031Z] [Pipeline] build
[2026-11-03T08:48:50.031Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-11-03T08:48:50.032Z] [Pipeline] build
[2026-11-03T08:48:50.033Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-11-03T08:48:56.283Z] Starting building: anvil » anvil-build-functional-testing #1493
[2026-11-03T08:48:56.285Z] [Pipeline] waitForBuild
[2026-11-03T08:48:56.285Z] Waiting for anvil » anvil-build-functional-testing #1493 to complete
[2026-11-03T08:48:56.285Z] Starting building: anvil » anvil-build-functional-testing #1492
[2026-11-03T08:48:56.286Z] Starting building: anvil » anvil-build-functional-testing #1489
[2026-11-03T08:48:56.286Z] Starting building: anvil » anvil-build-functional-testing #1491
[2026-11-03T08:48:56.287Z] Starting building: anvil » anvil-build-functional-testing #1490
[2026-11-03T08:48:56.287Z] Starting building: anvil » anvil-build-functional-testing #1494
[2026-11-03T08:48:56.301Z] [Pipeline] waitForBuild
[2026-11-03T08:48:56.301Z] Waiting for anvil » anvil-build-functional-testing #1489 to complete
[2026-11-03T08:48:56.302Z] [Pipeline] waitForBuild
[2026-11-03T08:48:56.302Z] Waiting for anvil » anvil-build-functional-testing #1490 to complete
[2026-11-03T08:48:56.303Z] [Pipeline] waitForBuild
[2026-11-03T08:48:56.303Z] Waiting for anvil » anvil-build-functional-testing #1491 to complete
[2026-11-03T08:48:56.304Z] [Pipeline] waitForBuild
[2026-11-03T08:48:56.304Z] Waiting for anvil » anvil-build-functional-testing #1492 to complete
[2026-11-03T08:48:56.305Z] [Pipeline] waitForBuild
[2026-11-03T08:48:56.305Z] Waiting for anvil » anvil-build-functional-testing #1494 to complete
[2026-11-03T11:11:54.322Z] Build anvil » anvil-build-functional-testing #1494 completed: SUCCESS
[2026-11-03T11:11:54.324Z] [Pipeline] }
[2026-11-03T11:11:54.327Z] [Pipeline] // script
[2026-11-03T11:11:54.329Z] [Pipeline] }
[2026-11-03T11:11:54.332Z] [Pipeline] // stage
[2026-11-03T11:11:54.334Z] [Pipeline] }
[2026-11-03T11:26:29.472Z] Build anvil » anvil-build-functional-testing #1492 completed: SUCCESS
[2026-11-03T11:26:29.474Z] [Pipeline] }
[2026-11-03T11:26:29.476Z] [Pipeline] // script
[2026-11-03T11:26:29.479Z] [Pipeline] }
[2026-11-03T11:26:29.481Z] [Pipeline] // stage
[2026-11-03T11:26:29.483Z] [Pipeline] }
[2026-11-03T11:40:55.819Z] Build anvil » anvil-build-functional-testing #1493 completed: SUCCESS
[2026-11-03T11:40:55.821Z] [Pipeline] }
[2026-11-03T11:40:55.824Z] [Pipeline] // script
[2026-11-03T11:40:55.827Z] [Pipeline] }
[2026-11-03T11:40:55.830Z] [Pipeline] // stage
[2026-11-03T11:40:55.832Z] [Pipeline] }
[2026-11-03T11:41:44.863Z] Build anvil » anvil-build-functional-testing #1489 completed: SUCCESS
[2026-11-03T11:41:44.865Z] [Pipeline] }
[2026-11-03T11:41:44.867Z] [Pipeline] // script
[2026-11-03T11:41:44.869Z] [Pipeline] }
[2026-11-03T11:41:44.872Z] [Pipeline] // stage
[2026-11-03T11:41:44.874Z] [Pipeline] }
[2026-11-03T12:26:53.344Z] Build anvil » anvil-build-functional-testing #1490 completed: SUCCESS
[2026-11-03T12:26:53.346Z] [Pipeline] }
[2026-11-03T12:26:53.349Z] [Pipeline] // script
[2026-11-03T12:26:53.352Z] [Pipeline] }
[2026-11-03T12:26:53.354Z] [Pipeline] // stage
[2026-11-03T12:26:53.357Z] [Pipeline] }
[2026-11-03T13:15:51.066Z] Build anvil » anvil-build-functional-testing #1491 completed: FAILURE
[2026-11-03T13:15:51.068Z] [Pipeline] catchError
[2026-11-03T13:15:51.069Z] [Pipeline] {
[2026-11-03T13:15:51.080Z] [Pipeline] sh
[2026-11-03T13:15:51.336Z] [Pipeline] }
[2026-11-03T13:15:51.336Z] ERROR: script returned exit code 1
[2026-11-03T13:15:51.336Z] Setting overall build result to FAILURE
[2026-11-03T13:15:51.341Z] [Pipeline] // catchError
[2026-11-03T13:15:51.344Z] [Pipeline] }
[2026-11-03T13:15:51.347Z] [Pipeline] // script
[2026-11-03T13:15:51.349Z] [Pipeline] }
[2026-11-03T13:15:51.351Z] [Pipeline] // stage
[2026-11-03T13:15:51.354Z] [Pipeline] }
[2026-11-03T13:15:51.356Z] [Pipeline] // parallel
[2026-11-03T13:15:51.358Z] [Pipeline] }
[2026-11-03T13:15:51.360Z] [Pipeline] // stage
[2026-11-03T13:15:51.364Z] [Pipeline] stage
[2026-11-03T13:15:51.364Z] [Pipeline] { (Declarative: Post Actions)
[2026-11-03T13:15:51.380Z] [Pipeline] stage
[2026-11-03T13:15:51.380Z] [Pipeline] { (Copying logs)
[2026-11-03T13:15:51.413Z] [Pipeline] node
[2026-11-03T13:15:51.520Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron@2
[2026-11-03T13:15:51.520Z] [Pipeline] {
[2026-11-03T13:15:51.554Z] [Pipeline] node
[2026-11-03T13:15:51.656Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron@3
[2026-11-03T13:15:51.657Z] [Pipeline] {
[2026-11-03T13:15:51.660Z] [Pipeline] sh
[2026-11-03T13:15:51.915Z] + mkdir -p /var/lib/jenkins/locks
[2026-11-03T13:15:51.923Z] [Pipeline] echo
[2026-11-03T13:15:51.923Z] RWLock: FD for lock log_archive in stage projectFinishUp is 385
[2026-11-03T13:15:51.927Z] [Pipeline] sleep
[2026-11-03T13:15:51.927Z] No need to sleep any longer
[2026-11-03T13:15:51.930Z] [Pipeline] echo
[2026-11-03T13:15:51.930Z] RWLock: log_archive in stage projectFinishUp locked for WRITE
[2026-11-03T13:15:51.933Z] [Pipeline] }
[2026-11-03T13:15:51.936Z] [Pipeline] // node
[2026-11-03T13:15:51.939Z] [Pipeline] echo
[2026-11-03T13:15:51.939Z] Copying logs to /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-cron/215/
[2026-11-03T13:15:51.940Z] [Pipeline] sh
[2026-11-03T13:15:52.195Z] + mkdir -p /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-cron/215/
[2026-11-03T13:15:52.197Z] [Pipeline] sh