[2026-10-26T08:16:00.393Z] Started by timer
[2026-10-26T08:16:00.633Z] Obtained pipelines/projects/anvil/build-cron/Jenkinsfile from git https://github.com/kronosnet/ci-tools
[2026-10-26T08:16:00.652Z] Loading library ProjectLib@main:pipelines/projects/anvil/
[2026-10-26T08:16:01.319Z] Examining kronosnet/ci-tools
[2026-10-26T08:16:01.319Z] Attempting to resolve main as a branch
[2026-10-26T08:16:01.513Z] Resolved main as branch main at revision fd5667069c7b69778ce97df718cf25626f5beefc
[2026-10-26T08:16:01.515Z] Selected Git installation does not exist. Using Default
[2026-10-26T08:16:01.515Z] The recommended git tool is: NONE
[2026-10-26T08:16:01.515Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-10-26T08:16:01.517Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-10-26T08:16:01.517Z] Wiping out workspace first.
[2026-10-26T08:16:01.543Z] Cloning the remote Git repository
[2026-10-26T08:16:01.543Z] Using shallow clone with depth 1
[2026-10-26T08:16:01.543Z] Avoid fetching tags
[2026-10-26T08:16:01.543Z] Honoring refspec on initial clone
[2026-10-26T08:16:01.544Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-10-26T08:16:01.544Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-cron@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b # timeout=10
[2026-10-26T08:16:01.553Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-10-26T08:16:01.553Z]  > git --version # timeout=10
[2026-10-26T08:16:01.559Z]  > git --version # 'git version 2.52.0'
[2026-10-26T08:16:01.559Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-10-26T08:16:01.565Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-10-26T08:16:01.852Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-10-26T08:16:01.860Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-10-26T08:16:01.867Z] Avoid second fetch
[2026-10-26T08:16:01.868Z] Checking out Revision fd5667069c7b69778ce97df718cf25626f5beefc (main)
[2026-10-26T08:16:01.868Z]  > git config core.sparsecheckout # timeout=10
[2026-10-26T08:16:01.875Z]  > git checkout -f fd5667069c7b69778ce97df718cf25626f5beefc # timeout=10
[2026-10-26T08:16:01.911Z] Commit message: "pipelines: fix ansible managed update pipelines to process all nodes in parallel"
[2026-10-26T08:16:01.920Z] Loading library GlobalLib@main:./pipelines/libs/global/
[2026-10-26T08:16:02.223Z] Examining kronosnet/ci-tools
[2026-10-26T08:16:02.223Z] Attempting to resolve main as a branch
[2026-10-26T08:16:02.470Z] Resolved main as branch main at revision fd5667069c7b69778ce97df718cf25626f5beefc
[2026-10-26T08:16:02.470Z] Selected Git installation does not exist. Using Default
[2026-10-26T08:16:02.470Z] The recommended git tool is: NONE
[2026-10-26T08:16:02.471Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-10-26T08:16:02.473Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-10-26T08:16:02.473Z] Wiping out workspace first.
[2026-10-26T08:16:02.493Z] Cloning the remote Git repository
[2026-10-26T08:16:02.493Z] Using shallow clone with depth 1
[2026-10-26T08:16:02.493Z] Avoid fetching tags
[2026-10-26T08:16:02.493Z] Honoring refspec on initial clone
[2026-10-26T08:16:02.493Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-10-26T08:16:02.493Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-cron@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b # timeout=10
[2026-10-26T08:16:02.502Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-10-26T08:16:02.502Z]  > git --version # timeout=10
[2026-10-26T08:16:02.508Z]  > git --version # 'git version 2.52.0'
[2026-10-26T08:16:02.508Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-10-26T08:16:02.514Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-10-26T08:16:02.790Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-10-26T08:16:02.797Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-10-26T08:16:02.805Z] Avoid second fetch
[2026-10-26T08:16:02.807Z] Checking out Revision fd5667069c7b69778ce97df718cf25626f5beefc (main)
[2026-10-26T08:16:02.807Z]  > git config core.sparsecheckout # timeout=10
[2026-10-26T08:16:02.814Z]  > git checkout -f fd5667069c7b69778ce97df718cf25626f5beefc # timeout=10
[2026-10-26T08:16:02.851Z] Commit message: "pipelines: fix ansible managed update pipelines to process all nodes in parallel"
[2026-10-26T08:16:03.236Z] [Pipeline] Start of Pipeline
[2026-10-26T08:16:03.405Z] [Pipeline] node
[2026-10-26T08:16:03.526Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron
[2026-10-26T08:16:03.527Z] [Pipeline] {
[2026-10-26T08:16:03.541Z] [Pipeline] stage
[2026-10-26T08:16:03.541Z] [Pipeline] { (Setup build env)
[2026-10-26T08:16:03.554Z] [Pipeline] script
[2026-10-26T08:16:03.557Z] [Pipeline] {
[2026-10-26T08:16:03.562Z] [Pipeline] cleanWs
[2026-10-26T08:16:03.563Z] [WS-CLEANUP] Deleting project workspace...
[2026-10-26T08:16:03.563Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-10-26T08:16:03.563Z] [WS-CLEANUP] done
[2026-10-26T08:16:03.594Z] [Pipeline] lock
[2026-10-26T08:16:03.594Z] Trying to acquire lock on [Resource: MAINTENANCE LOCKOUT]
[2026-10-26T08:16:03.595Z] Resource [MAINTENANCE LOCKOUT] did not exist. Created.
[2026-10-26T08:16:03.595Z] Lock acquired on [Resource: MAINTENANCE LOCKOUT]
[2026-10-26T08:16:03.596Z] [Pipeline] {
[2026-10-26T08:16:03.598Z] [Pipeline] }
[2026-10-26T08:16:03.599Z] Lock released on resource [Resource: MAINTENANCE LOCKOUT]
[2026-10-26T08:16:03.601Z] [Pipeline] // lock
[2026-10-26T08:16:03.604Z] [Pipeline] }
[2026-10-26T08:16:03.607Z] [Pipeline] // script
[2026-10-26T08:16:03.609Z] [Pipeline] }
[2026-10-26T08:16:03.611Z] [Pipeline] // stage
[2026-10-26T08:16:03.614Z] [Pipeline] stage
[2026-10-26T08:16:03.615Z] [Pipeline] { (Update repos and cache)
[2026-10-26T08:16:03.621Z] [Pipeline] parallel
[2026-10-26T08:16:03.621Z] [Pipeline] { (Branch: anvil build cache)
[2026-10-26T08:16:03.621Z] [Pipeline] { (Branch: anvil build dep repos)
[2026-10-26T08:16:03.626Z] [Pipeline] stage
[2026-10-26T08:16:03.626Z] [Pipeline] { (anvil build cache)
[2026-10-26T08:16:03.627Z] [Pipeline] stage
[2026-10-26T08:16:03.627Z] [Pipeline] { (anvil build dep repos)
[2026-10-26T08:16:03.635Z] [Pipeline] script
[2026-10-26T08:16:03.636Z] [Pipeline] {
[2026-10-26T08:16:03.637Z] [Pipeline] script
[2026-10-26T08:16:03.638Z] [Pipeline] {
[2026-10-26T08:16:03.653Z] [Pipeline] build
[2026-10-26T08:16:03.653Z] Scheduling project: anvil » anvil-build-cache
[2026-10-26T08:16:03.654Z] [Pipeline] build
[2026-10-26T08:16:03.655Z] Scheduling project: anvil » anvil-build-dep-repos » main
[2026-10-26T08:16:10.740Z] Starting building: anvil » anvil-build-cache #776
[2026-10-26T08:16:10.740Z] Starting building: anvil » anvil-build-dep-repos » main #211
[2026-10-26T08:16:10.743Z] [Pipeline] waitForBuild
[2026-10-26T08:16:10.743Z] Waiting for anvil » anvil-build-cache #776 to complete
[2026-10-26T08:16:10.744Z] [Pipeline] waitForBuild
[2026-10-26T08:16:10.744Z] Waiting for anvil » anvil-build-dep-repos » main #211 to complete
[2026-10-26T08:38:05.480Z] Build anvil » anvil-build-cache #776 completed: SUCCESS
[2026-10-26T08:38:05.482Z] [Pipeline] }
[2026-10-26T08:38:05.485Z] [Pipeline] // script
[2026-10-26T08:38:05.487Z] [Pipeline] }
[2026-10-26T08:38:05.489Z] [Pipeline] // stage
[2026-10-26T08:38:05.491Z] [Pipeline] }
[2026-10-26T08:38:17.150Z] Build anvil » anvil-build-dep-repos » main #211 completed: SUCCESS
[2026-10-26T08:38:17.152Z] [Pipeline] }
[2026-10-26T08:38:17.154Z] [Pipeline] // script
[2026-10-26T08:38:17.157Z] [Pipeline] }
[2026-10-26T08:38:17.159Z] [Pipeline] // stage
[2026-10-26T08:38:17.161Z] [Pipeline] }
[2026-10-26T08:38:17.163Z] [Pipeline] // parallel
[2026-10-26T08:38:17.166Z] [Pipeline] }
[2026-10-26T08:38:17.168Z] [Pipeline] // stage
[2026-10-26T08:38:17.171Z] [Pipeline] stage
[2026-10-26T08:38:17.171Z] [Pipeline] { (Update kernel modules)
[2026-10-26T08:38:17.176Z] [Pipeline] parallel
[2026-10-26T08:38:17.176Z] [Pipeline] { (Branch: anvil build all kmods)
[2026-10-26T08:38:17.179Z] [Pipeline] stage
[2026-10-26T08:38:17.180Z] [Pipeline] { (anvil build all kmods)
[2026-10-26T08:38:17.186Z] [Pipeline] script
[2026-10-26T08:38:17.188Z] [Pipeline] {
[2026-10-26T08:38:17.191Z] [Pipeline] build
[2026-10-26T08:38:17.191Z] Scheduling project: anvil » anvil-build-all-kmods
[2026-10-26T08:38:26.236Z] Starting building: anvil » anvil-build-all-kmods #707
[2026-10-26T08:38:26.238Z] [Pipeline] waitForBuild
[2026-10-26T08:38:26.238Z] Waiting for anvil » anvil-build-all-kmods #707 to complete
[2026-10-26T08:48:43.284Z] Build anvil » anvil-build-all-kmods #707 completed: SUCCESS
[2026-10-26T08:48:43.286Z] [Pipeline] }
[2026-10-26T08:48:43.289Z] [Pipeline] // script
[2026-10-26T08:48:43.291Z] [Pipeline] }
[2026-10-26T08:48:43.293Z] [Pipeline] // stage
[2026-10-26T08:48:43.295Z] [Pipeline] }
[2026-10-26T08:48:43.298Z] [Pipeline] // parallel
[2026-10-26T08:48:43.300Z] [Pipeline] }
[2026-10-26T08:48:43.302Z] [Pipeline] // stage
[2026-10-26T08:48:43.305Z] [Pipeline] stage
[2026-10-26T08:48:43.305Z] [Pipeline] { (Run daily functional testing)
[2026-10-26T08:48:43.311Z] [Pipeline] parallel
[2026-10-26T08:48:43.311Z] [Pipeline] { (Branch: almalinux-9 ci ci)
[2026-10-26T08:48:43.311Z] [Pipeline] { (Branch: rhel-9 ci ci)
[2026-10-26T08:48:43.311Z] [Pipeline] { (Branch: almalinux-9 enterprise none)
[2026-10-26T08:48:43.312Z] [Pipeline] { (Branch: rhel-9 enterprise none)
[2026-10-26T08:48:43.312Z] [Pipeline] { (Branch: almalinux-9 enterprise-staging none)
[2026-10-26T08:48:43.312Z] [Pipeline] { (Branch: rhel-9 enterprise-staging none)
[2026-10-26T08:48:43.319Z] [Pipeline] stage
[2026-10-26T08:48:43.319Z] [Pipeline] { (almalinux-9 ci ci)
[2026-10-26T08:48:43.320Z] [Pipeline] stage
[2026-10-26T08:48:43.320Z] [Pipeline] { (rhel-9 ci ci)
[2026-10-26T08:48:43.321Z] [Pipeline] stage
[2026-10-26T08:48:43.321Z] [Pipeline] { (almalinux-9 enterprise none)
[2026-10-26T08:48:43.322Z] [Pipeline] stage
[2026-10-26T08:48:43.322Z] [Pipeline] { (rhel-9 enterprise none)
[2026-10-26T08:48:43.323Z] [Pipeline] stage
[2026-10-26T08:48:43.323Z] [Pipeline] { (almalinux-9 enterprise-staging none)
[2026-10-26T08:48:43.323Z] [Pipeline] stage
[2026-10-26T08:48:43.324Z] [Pipeline] { (rhel-9 enterprise-staging none)
[2026-10-26T08:48:43.340Z] [Pipeline] script
[2026-10-26T08:48:43.341Z] [Pipeline] {
[2026-10-26T08:48:43.342Z] [Pipeline] script
[2026-10-26T08:48:43.342Z] [Pipeline] {
[2026-10-26T08:48:43.343Z] [Pipeline] script
[2026-10-26T08:48:43.344Z] [Pipeline] {
[2026-10-26T08:48:43.344Z] [Pipeline] script
[2026-10-26T08:48:43.345Z] [Pipeline] {
[2026-10-26T08:48:43.346Z] [Pipeline] script
[2026-10-26T08:48:43.347Z] [Pipeline] {
[2026-10-26T08:48:43.347Z] [Pipeline] script
[2026-10-26T08:48:43.348Z] [Pipeline] {
[2026-10-26T08:48:43.352Z] [Pipeline] build
[2026-10-26T08:48:43.353Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-26T08:48:43.354Z] [Pipeline] build
[2026-10-26T08:48:43.354Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-26T08:48:43.356Z] [Pipeline] build
[2026-10-26T08:48:43.356Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-26T08:48:43.357Z] [Pipeline] build
[2026-10-26T08:48:43.357Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-26T08:48:43.359Z] [Pipeline] build
[2026-10-26T08:48:43.359Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-26T08:48:43.360Z] [Pipeline] build
[2026-10-26T08:48:43.360Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-26T08:48:51.352Z] Starting building: anvil » anvil-build-functional-testing #1444
[2026-10-26T08:48:51.352Z] Starting building: anvil » anvil-build-functional-testing #1442
[2026-10-26T08:48:51.353Z] Starting building: anvil » anvil-build-functional-testing #1441
[2026-10-26T08:48:51.353Z] Starting building: anvil » anvil-build-functional-testing #1446
[2026-10-26T08:48:51.353Z] [Pipeline] waitForBuild
[2026-10-26T08:48:51.353Z] Starting building: anvil » anvil-build-functional-testing #1445
[2026-10-26T08:48:51.354Z] Waiting for anvil » anvil-build-functional-testing #1444 to complete
[2026-10-26T08:48:51.354Z] Starting building: anvil » anvil-build-functional-testing #1443
[2026-10-26T08:48:51.370Z] [Pipeline] waitForBuild
[2026-10-26T08:48:51.370Z] Waiting for anvil » anvil-build-functional-testing #1441 to complete
[2026-10-26T08:48:51.372Z] [Pipeline] waitForBuild
[2026-10-26T08:48:51.372Z] Waiting for anvil » anvil-build-functional-testing #1442 to complete
[2026-10-26T08:48:51.374Z] [Pipeline] waitForBuild
[2026-10-26T08:48:51.374Z] Waiting for anvil » anvil-build-functional-testing #1445 to complete
[2026-10-26T08:48:51.375Z] [Pipeline] waitForBuild
[2026-10-26T08:48:51.375Z] Waiting for anvil » anvil-build-functional-testing #1446 to complete
[2026-10-26T08:48:51.376Z] [Pipeline] waitForBuild
[2026-10-26T08:48:51.377Z] Waiting for anvil » anvil-build-functional-testing #1443 to complete
[2026-10-26T11:18:15.389Z] Build anvil » anvil-build-functional-testing #1446 completed: SUCCESS
[2026-10-26T11:18:15.391Z] [Pipeline] }
[2026-10-26T11:18:15.394Z] [Pipeline] // script
[2026-10-26T11:18:15.397Z] [Pipeline] }
[2026-10-26T11:18:15.399Z] [Pipeline] // stage
[2026-10-26T11:18:15.401Z] [Pipeline] }
[2026-10-26T11:34:04.128Z] Build anvil » anvil-build-functional-testing #1443 completed: SUCCESS
[2026-10-26T11:34:04.130Z] [Pipeline] }
[2026-10-26T11:34:04.132Z] [Pipeline] // script
[2026-10-26T11:34:04.135Z] [Pipeline] }
[2026-10-26T11:34:04.137Z] [Pipeline] // stage
[2026-10-26T11:34:04.139Z] [Pipeline] }
[2026-10-26T11:37:16.385Z] Build anvil » anvil-build-functional-testing #1445 completed: SUCCESS
[2026-10-26T11:37:16.386Z] [Pipeline] }
[2026-10-26T11:37:16.389Z] [Pipeline] // script
[2026-10-26T11:37:16.392Z] [Pipeline] }
[2026-10-26T11:37:16.394Z] [Pipeline] // stage
[2026-10-26T11:37:16.396Z] [Pipeline] }
[2026-10-26T11:44:51.477Z] Build anvil » anvil-build-functional-testing #1444 completed: FAILURE
[2026-10-26T11:44:51.479Z] [Pipeline] catchError
[2026-10-26T11:44:51.479Z] [Pipeline] {
[2026-10-26T11:44:51.490Z] [Pipeline] sh
[2026-10-26T11:44:51.745Z] [Pipeline] }
[2026-10-26T11:44:51.745Z] ERROR: script returned exit code 1
[2026-10-26T11:44:51.745Z] Setting overall build result to FAILURE
[2026-10-26T11:44:51.748Z] [Pipeline] // catchError
[2026-10-26T11:44:51.751Z] [Pipeline] }
[2026-10-26T11:44:51.753Z] [Pipeline] // script
[2026-10-26T11:44:51.756Z] [Pipeline] }
[2026-10-26T11:44:51.758Z] [Pipeline] // stage
[2026-10-26T11:44:51.760Z] [Pipeline] }
[2026-10-26T11:46:51.320Z] Build anvil » anvil-build-functional-testing #1442 completed: SUCCESS
[2026-10-26T11:46:51.322Z] [Pipeline] }
[2026-10-26T11:46:51.325Z] [Pipeline] // script
[2026-10-26T11:46:51.328Z] [Pipeline] }
[2026-10-26T11:46:51.330Z] [Pipeline] // stage
[2026-10-26T11:46:51.333Z] [Pipeline] }
[2026-10-26T12:09:54.537Z] Build anvil » anvil-build-functional-testing #1441 completed: FAILURE
[2026-10-26T12:09:54.538Z] [Pipeline] catchError
[2026-10-26T12:09:54.539Z] [Pipeline] {
[2026-10-26T12:09:54.542Z] [Pipeline] sh
[2026-10-26T12:09:54.797Z] [Pipeline] }
[2026-10-26T12:09:54.798Z] ERROR: script returned exit code 1
[2026-10-26T12:09:54.801Z] [Pipeline] // catchError
[2026-10-26T12:09:54.804Z] [Pipeline] }
[2026-10-26T12:09:54.806Z] [Pipeline] // script
[2026-10-26T12:09:54.808Z] [Pipeline] }
[2026-10-26T12:09:54.811Z] [Pipeline] // stage
[2026-10-26T12:09:54.813Z] [Pipeline] }
[2026-10-26T12:09:54.815Z] [Pipeline] // parallel
[2026-10-26T12:09:54.818Z] [Pipeline] }
[2026-10-26T12:09:54.820Z] [Pipeline] // stage
[2026-10-26T12:09:54.823Z] [Pipeline] stage
[2026-10-26T12:09:54.824Z] [Pipeline] { (Declarative: Post Actions)
[2026-10-26T12:09:54.840Z] [Pipeline] stage
[2026-10-26T12:09:54.840Z] [Pipeline] { (Copying logs)
[2026-10-26T12:09:54.874Z] [Pipeline] node
[2026-10-26T12:09:54.980Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron@2
[2026-10-26T12:09:54.980Z] [Pipeline] {
[2026-10-26T12:09:55.014Z] [Pipeline] node
[2026-10-26T12:09:55.112Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron@3
[2026-10-26T12:09:55.112Z] [Pipeline] {
[2026-10-26T12:09:55.116Z] [Pipeline] sh
[2026-10-26T12:09:55.371Z] + mkdir -p /var/lib/jenkins/locks
[2026-10-26T12:09:55.378Z] [Pipeline] echo
[2026-10-26T12:09:55.378Z] RWLock: FD for lock log_archive in stage projectFinishUp is 387
[2026-10-26T12:09:55.381Z] [Pipeline] sleep
[2026-10-26T12:09:55.382Z] No need to sleep any longer
[2026-10-26T12:09:55.384Z] [Pipeline] echo
[2026-10-26T12:09:55.385Z] RWLock: log_archive in stage projectFinishUp locked for WRITE
[2026-10-26T12:09:55.388Z] [Pipeline] }
[2026-10-26T12:09:55.390Z] [Pipeline] // node
[2026-10-26T12:09:55.394Z] [Pipeline] echo
[2026-10-26T12:09:55.394Z] Copying logs to /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-cron/207/
[2026-10-26T12:09:55.394Z] [Pipeline] sh
[2026-10-26T12:09:55.649Z] + mkdir -p /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-cron/207/
[2026-10-26T12:09:55.651Z] [Pipeline] sh