[2026-10-28T08:16:00.392Z] Started by timer
[2026-10-28T08:16:00.612Z] Obtained pipelines/projects/anvil/build-cron/Jenkinsfile from git https://github.com/kronosnet/ci-tools
[2026-10-28T08:16:00.629Z] Loading library ProjectLib@main:pipelines/projects/anvil/
[2026-10-28T08:16:01.270Z] Examining kronosnet/ci-tools
[2026-10-28T08:16:01.270Z] Attempting to resolve main as a branch
[2026-10-28T08:16:01.464Z] Resolved main as branch main at revision fd5667069c7b69778ce97df718cf25626f5beefc
[2026-10-28T08:16:01.465Z] Selected Git installation does not exist. Using Default
[2026-10-28T08:16:01.465Z] The recommended git tool is: NONE
[2026-10-28T08:16:01.466Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-10-28T08:16:01.468Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-10-28T08:16:01.468Z] Wiping out workspace first.
[2026-10-28T08:16:01.493Z] Cloning the remote Git repository
[2026-10-28T08:16:01.493Z] Using shallow clone with depth 1
[2026-10-28T08:16:01.493Z] Avoid fetching tags
[2026-10-28T08:16:01.493Z] Honoring refspec on initial clone
[2026-10-28T08:16:01.493Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-10-28T08:16:01.493Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-cron@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b # timeout=10
[2026-10-28T08:16:01.503Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-10-28T08:16:01.503Z]  > git --version # timeout=10
[2026-10-28T08:16:01.509Z]  > git --version # 'git version 2.52.0'
[2026-10-28T08:16:01.509Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-10-28T08:16:01.515Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-10-28T08:16:01.950Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-10-28T08:16:01.957Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-10-28T08:16:01.964Z] Avoid second fetch
[2026-10-28T08:16:01.965Z] Checking out Revision fd5667069c7b69778ce97df718cf25626f5beefc (main)
[2026-10-28T08:16:01.965Z]  > git config core.sparsecheckout # timeout=10
[2026-10-28T08:16:01.971Z]  > git checkout -f fd5667069c7b69778ce97df718cf25626f5beefc # timeout=10
[2026-10-28T08:16:02.004Z] Commit message: "pipelines: fix ansible managed update pipelines to process all nodes in parallel"
[2026-10-28T08:16:02.012Z] Loading library GlobalLib@main:./pipelines/libs/global/
[2026-10-28T08:16:02.252Z] Examining kronosnet/ci-tools
[2026-10-28T08:16:02.252Z] Attempting to resolve main as a branch
[2026-10-28T08:16:02.456Z] Resolved main as branch main at revision fd5667069c7b69778ce97df718cf25626f5beefc
[2026-10-28T08:16:02.456Z] Selected Git installation does not exist. Using Default
[2026-10-28T08:16:02.456Z] The recommended git tool is: NONE
[2026-10-28T08:16:02.456Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-10-28T08:16:02.460Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-10-28T08:16:02.460Z] Wiping out workspace first.
[2026-10-28T08:16:02.479Z] Cloning the remote Git repository
[2026-10-28T08:16:02.479Z] Using shallow clone with depth 1
[2026-10-28T08:16:02.479Z] Avoid fetching tags
[2026-10-28T08:16:02.479Z] Honoring refspec on initial clone
[2026-10-28T08:16:02.479Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-10-28T08:16:02.479Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-cron@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b # timeout=10
[2026-10-28T08:16:02.488Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-10-28T08:16:02.488Z]  > git --version # timeout=10
[2026-10-28T08:16:02.495Z]  > git --version # 'git version 2.52.0'
[2026-10-28T08:16:02.495Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-10-28T08:16:02.500Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-10-28T08:16:02.796Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-10-28T08:16:02.803Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-10-28T08:16:02.810Z] Avoid second fetch
[2026-10-28T08:16:02.810Z] Checking out Revision fd5667069c7b69778ce97df718cf25626f5beefc (main)
[2026-10-28T08:16:02.810Z]  > git config core.sparsecheckout # timeout=10
[2026-10-28T08:16:02.816Z]  > git checkout -f fd5667069c7b69778ce97df718cf25626f5beefc # timeout=10
[2026-10-28T08:16:02.849Z] Commit message: "pipelines: fix ansible managed update pipelines to process all nodes in parallel"
[2026-10-28T08:16:03.041Z] [Pipeline] Start of Pipeline
[2026-10-28T08:16:03.160Z] [Pipeline] node
[2026-10-28T08:16:03.273Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron
[2026-10-28T08:16:03.274Z] [Pipeline] {
[2026-10-28T08:16:03.284Z] [Pipeline] stage
[2026-10-28T08:16:03.284Z] [Pipeline] { (Setup build env)
[2026-10-28T08:16:03.295Z] [Pipeline] script
[2026-10-28T08:16:03.297Z] [Pipeline] {
[2026-10-28T08:16:03.301Z] [Pipeline] cleanWs
[2026-10-28T08:16:03.302Z] [WS-CLEANUP] Deleting project workspace...
[2026-10-28T08:16:03.302Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-10-28T08:16:03.302Z] [WS-CLEANUP] done
[2026-10-28T08:16:03.326Z] [Pipeline] lock
[2026-10-28T08:16:03.326Z] Trying to acquire lock on [Resource: MAINTENANCE LOCKOUT]
[2026-10-28T08:16:03.326Z] Resource [MAINTENANCE LOCKOUT] did not exist. Created.
[2026-10-28T08:16:03.326Z] Lock acquired on [Resource: MAINTENANCE LOCKOUT]
[2026-10-28T08:16:03.327Z] [Pipeline] {
[2026-10-28T08:16:03.329Z] [Pipeline] }
[2026-10-28T08:16:03.330Z] Lock released on resource [Resource: MAINTENANCE LOCKOUT]
[2026-10-28T08:16:03.332Z] [Pipeline] // lock
[2026-10-28T08:16:03.335Z] [Pipeline] }
[2026-10-28T08:16:03.337Z] [Pipeline] // script
[2026-10-28T08:16:03.339Z] [Pipeline] }
[2026-10-28T08:16:03.341Z] [Pipeline] // stage
[2026-10-28T08:16:03.344Z] [Pipeline] stage
[2026-10-28T08:16:03.344Z] [Pipeline] { (Update repos and cache)
[2026-10-28T08:16:03.350Z] [Pipeline] parallel
[2026-10-28T08:16:03.350Z] [Pipeline] { (Branch: anvil build cache)
[2026-10-28T08:16:03.350Z] [Pipeline] { (Branch: anvil build dep repos)
[2026-10-28T08:16:03.354Z] [Pipeline] stage
[2026-10-28T08:16:03.354Z] [Pipeline] { (anvil build cache)
[2026-10-28T08:16:03.355Z] [Pipeline] stage
[2026-10-28T08:16:03.355Z] [Pipeline] { (anvil build dep repos)
[2026-10-28T08:16:03.362Z] [Pipeline] script
[2026-10-28T08:16:03.363Z] [Pipeline] {
[2026-10-28T08:16:03.364Z] [Pipeline] script
[2026-10-28T08:16:03.365Z] [Pipeline] {
[2026-10-28T08:16:03.379Z] [Pipeline] build
[2026-10-28T08:16:03.379Z] Scheduling project: anvil » anvil-build-cache
[2026-10-28T08:16:03.381Z] [Pipeline] build
[2026-10-28T08:16:03.381Z] Scheduling project: anvil » anvil-build-dep-repos » main
[2026-10-28T08:16:08.404Z] Starting building: anvil » anvil-build-dep-repos » main #213
[2026-10-28T08:16:08.404Z] Starting building: anvil » anvil-build-cache #778
[2026-10-28T08:16:08.405Z] [Pipeline] waitForBuild
[2026-10-28T08:16:08.405Z] Waiting for anvil » anvil-build-dep-repos » main #213 to complete
[2026-10-28T08:16:08.411Z] [Pipeline] waitForBuild
[2026-10-28T08:16:08.411Z] Waiting for anvil » anvil-build-cache #778 to complete
[2026-10-28T08:38:08.123Z] Build anvil » anvil-build-cache #778 completed: SUCCESS
[2026-10-28T08:38:08.125Z] [Pipeline] }
[2026-10-28T08:38:08.128Z] [Pipeline] // script
[2026-10-28T08:38:08.131Z] [Pipeline] }
[2026-10-28T08:38:08.133Z] [Pipeline] // stage
[2026-10-28T08:38:08.136Z] [Pipeline] }
[2026-10-28T08:39:13.239Z] Build anvil » anvil-build-dep-repos » main #213 completed: SUCCESS
[2026-10-28T08:39:13.242Z] [Pipeline] }
[2026-10-28T08:39:13.246Z] [Pipeline] // script
[2026-10-28T08:39:13.249Z] [Pipeline] }
[2026-10-28T08:39:13.252Z] [Pipeline] // stage
[2026-10-28T08:39:13.255Z] [Pipeline] }
[2026-10-28T08:39:13.257Z] [Pipeline] // parallel
[2026-10-28T08:39:13.260Z] [Pipeline] }
[2026-10-28T08:39:13.262Z] [Pipeline] // stage
[2026-10-28T08:39:13.265Z] [Pipeline] stage
[2026-10-28T08:39:13.266Z] [Pipeline] { (Update kernel modules)
[2026-10-28T08:39:13.272Z] [Pipeline] parallel
[2026-10-28T08:39:13.272Z] [Pipeline] { (Branch: anvil build all kmods)
[2026-10-28T08:39:13.276Z] [Pipeline] stage
[2026-10-28T08:39:13.276Z] [Pipeline] { (anvil build all kmods)
[2026-10-28T08:39:13.284Z] [Pipeline] script
[2026-10-28T08:39:13.287Z] [Pipeline] {
[2026-10-28T08:39:13.291Z] [Pipeline] build
[2026-10-28T08:39:13.291Z] Scheduling project: anvil » anvil-build-all-kmods
[2026-10-28T08:39:20.778Z] Starting building: anvil » anvil-build-all-kmods #709
[2026-10-28T08:39:20.781Z] [Pipeline] waitForBuild
[2026-10-28T08:39:20.781Z] Waiting for anvil » anvil-build-all-kmods #709 to complete
[2026-10-28T08:48:41.339Z] Build anvil » anvil-build-all-kmods #709 completed: SUCCESS
[2026-10-28T08:48:41.341Z] [Pipeline] }
[2026-10-28T08:48:41.343Z] [Pipeline] // script
[2026-10-28T08:48:41.346Z] [Pipeline] }
[2026-10-28T08:48:41.348Z] [Pipeline] // stage
[2026-10-28T08:48:41.351Z] [Pipeline] }
[2026-10-28T08:48:41.353Z] [Pipeline] // parallel
[2026-10-28T08:48:41.355Z] [Pipeline] }
[2026-10-28T08:48:41.357Z] [Pipeline] // stage
[2026-10-28T08:48:41.360Z] [Pipeline] stage
[2026-10-28T08:48:41.361Z] [Pipeline] { (Run daily functional testing)
[2026-10-28T08:48:41.367Z] [Pipeline] parallel
[2026-10-28T08:48:41.367Z] [Pipeline] { (Branch: almalinux-9 ci ci)
[2026-10-28T08:48:41.368Z] [Pipeline] { (Branch: rhel-9 ci ci)
[2026-10-28T08:48:41.368Z] [Pipeline] { (Branch: almalinux-9 enterprise none)
[2026-10-28T08:48:41.368Z] [Pipeline] { (Branch: rhel-9 enterprise none)
[2026-10-28T08:48:41.368Z] [Pipeline] { (Branch: almalinux-9 enterprise-staging none)
[2026-10-28T08:48:41.368Z] [Pipeline] { (Branch: rhel-9 enterprise-staging none)
[2026-10-28T08:48:41.376Z] [Pipeline] stage
[2026-10-28T08:48:41.377Z] [Pipeline] { (almalinux-9 ci ci)
[2026-10-28T08:48:41.377Z] [Pipeline] stage
[2026-10-28T08:48:41.378Z] [Pipeline] { (rhel-9 ci ci)
[2026-10-28T08:48:41.378Z] [Pipeline] stage
[2026-10-28T08:48:41.378Z] [Pipeline] { (almalinux-9 enterprise none)
[2026-10-28T08:48:41.379Z] [Pipeline] stage
[2026-10-28T08:48:41.379Z] [Pipeline] { (rhel-9 enterprise none)
[2026-10-28T08:48:41.380Z] [Pipeline] stage
[2026-10-28T08:48:41.380Z] [Pipeline] { (almalinux-9 enterprise-staging none)
[2026-10-28T08:48:41.381Z] [Pipeline] stage
[2026-10-28T08:48:41.381Z] [Pipeline] { (rhel-9 enterprise-staging none)
[2026-10-28T08:48:41.400Z] [Pipeline] script
[2026-10-28T08:48:41.401Z] [Pipeline] {
[2026-10-28T08:48:41.402Z] [Pipeline] script
[2026-10-28T08:48:41.403Z] [Pipeline] {
[2026-10-28T08:48:41.404Z] [Pipeline] script
[2026-10-28T08:48:41.405Z] [Pipeline] {
[2026-10-28T08:48:41.405Z] [Pipeline] script
[2026-10-28T08:48:41.406Z] [Pipeline] {
[2026-10-28T08:48:41.407Z] [Pipeline] script
[2026-10-28T08:48:41.407Z] [Pipeline] {
[2026-10-28T08:48:41.408Z] [Pipeline] script
[2026-10-28T08:48:41.409Z] [Pipeline] {
[2026-10-28T08:48:41.414Z] [Pipeline] build
[2026-10-28T08:48:41.414Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-28T08:48:41.416Z] [Pipeline] build
[2026-10-28T08:48:41.416Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-28T08:48:41.455Z] [Pipeline] build
[2026-10-28T08:48:41.456Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-28T08:48:41.495Z] [Pipeline] build
[2026-10-28T08:48:41.495Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-28T08:48:41.533Z] [Pipeline] build
[2026-10-28T08:48:41.533Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-28T08:48:41.575Z] [Pipeline] build
[2026-10-28T08:48:41.575Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-10-28T08:48:50.244Z] Starting building: anvil » anvil-build-functional-testing #1454
[2026-10-28T08:48:50.245Z] Starting building: anvil » anvil-build-functional-testing #1456
[2026-10-28T08:48:50.245Z] Starting building: anvil » anvil-build-functional-testing #1457
[2026-10-28T08:48:50.245Z] Starting building: anvil » anvil-build-functional-testing #1455
[2026-10-28T08:48:50.246Z] [Pipeline] waitForBuild
[2026-10-28T08:48:50.246Z] Waiting for anvil » anvil-build-functional-testing #1454 to complete
[2026-10-28T08:48:50.246Z] Starting building: anvil » anvil-build-functional-testing #1453
[2026-10-28T08:48:50.246Z] Starting building: anvil » anvil-build-functional-testing #1458
[2026-10-28T08:48:50.260Z] [Pipeline] waitForBuild
[2026-10-28T08:48:50.261Z] Waiting for anvil » anvil-build-functional-testing #1455 to complete
[2026-10-28T08:48:50.262Z] [Pipeline] waitForBuild
[2026-10-28T08:48:50.262Z] Waiting for anvil » anvil-build-functional-testing #1456 to complete
[2026-10-28T08:48:50.263Z] [Pipeline] waitForBuild
[2026-10-28T08:48:50.263Z] Waiting for anvil » anvil-build-functional-testing #1457 to complete
[2026-10-28T08:48:50.269Z] [Pipeline] waitForBuild
[2026-10-28T08:48:50.269Z] Waiting for anvil » anvil-build-functional-testing #1453 to complete
[2026-10-28T08:48:50.270Z] [Pipeline] waitForBuild
[2026-10-28T08:48:50.270Z] Waiting for anvil » anvil-build-functional-testing #1458 to complete
[2026-10-28T11:14:11.837Z] Build anvil » anvil-build-functional-testing #1456 completed: SUCCESS
[2026-10-28T11:14:11.840Z] [Pipeline] }
[2026-10-28T11:14:11.843Z] [Pipeline] // script
[2026-10-28T11:14:11.845Z] [Pipeline] }
[2026-10-28T11:14:11.848Z] [Pipeline] // stage
[2026-10-28T11:14:11.850Z] [Pipeline] }
[2026-10-28T11:19:52.378Z] Build anvil » anvil-build-functional-testing #1457 completed: SUCCESS
[2026-10-28T11:19:52.379Z] [Pipeline] }
[2026-10-28T11:19:52.382Z] [Pipeline] // script
[2026-10-28T11:19:52.385Z] [Pipeline] }
[2026-10-28T11:19:52.387Z] [Pipeline] // stage
[2026-10-28T11:19:52.389Z] [Pipeline] }
[2026-10-28T11:45:28.270Z] Build anvil » anvil-build-functional-testing #1454 completed: SUCCESS
[2026-10-28T11:45:28.272Z] [Pipeline] }
[2026-10-28T11:45:28.276Z] [Pipeline] // script
[2026-10-28T11:45:28.279Z] [Pipeline] }
[2026-10-28T11:45:28.281Z] [Pipeline] // stage
[2026-10-28T11:45:28.284Z] [Pipeline] }
[2026-10-28T11:45:41.824Z] Build anvil » anvil-build-functional-testing #1458 completed: FAILURE
[2026-10-28T11:45:41.826Z] [Pipeline] catchError
[2026-10-28T11:45:41.826Z] [Pipeline] {
[2026-10-28T11:45:41.838Z] [Pipeline] sh
[2026-10-28T11:45:42.095Z] [Pipeline] }
[2026-10-28T11:45:42.095Z] ERROR: script returned exit code 1
[2026-10-28T11:45:42.095Z] Setting overall build result to FAILURE
[2026-10-28T11:45:42.098Z] [Pipeline] // catchError
[2026-10-28T11:45:42.102Z] [Pipeline] }
[2026-10-28T11:45:42.104Z] [Pipeline] // script
[2026-10-28T11:45:42.107Z] [Pipeline] }
[2026-10-28T11:45:42.109Z] [Pipeline] // stage
[2026-10-28T11:45:42.111Z] [Pipeline] }
[2026-10-28T11:58:05.255Z] Build anvil » anvil-build-functional-testing #1455 completed: SUCCESS
[2026-10-28T11:58:05.258Z] [Pipeline] }
[2026-10-28T11:58:05.261Z] [Pipeline] // script
[2026-10-28T11:58:05.265Z] [Pipeline] }
[2026-10-28T11:58:05.267Z] [Pipeline] // stage
[2026-10-28T11:58:05.270Z] [Pipeline] }
[2026-10-28T13:05:51.828Z] Build anvil » anvil-build-functional-testing #1453 completed: FAILURE
[2026-10-28T13:05:51.830Z] [Pipeline] catchError
[2026-10-28T13:05:51.830Z] [Pipeline] {
[2026-10-28T13:05:51.835Z] [Pipeline] sh
[2026-10-28T13:05:52.092Z] [Pipeline] }
[2026-10-28T13:05:52.092Z] ERROR: script returned exit code 1
[2026-10-28T13:05:52.095Z] [Pipeline] // catchError
[2026-10-28T13:05:52.098Z] [Pipeline] }
[2026-10-28T13:05:52.100Z] [Pipeline] // script
[2026-10-28T13:05:52.103Z] [Pipeline] }
[2026-10-28T13:05:52.105Z] [Pipeline] // stage
[2026-10-28T13:05:52.107Z] [Pipeline] }
[2026-10-28T13:05:52.109Z] [Pipeline] // parallel
[2026-10-28T13:05:52.112Z] [Pipeline] }
[2026-10-28T13:05:52.114Z] [Pipeline] // stage
[2026-10-28T13:05:52.117Z] [Pipeline] stage
[2026-10-28T13:05:52.117Z] [Pipeline] { (Declarative: Post Actions)
[2026-10-28T13:05:52.134Z] [Pipeline] stage
[2026-10-28T13:05:52.135Z] [Pipeline] { (Copying logs)
[2026-10-28T13:05:52.168Z] [Pipeline] node
[2026-10-28T13:05:52.270Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron@2
[2026-10-28T13:05:52.271Z] [Pipeline] {
[2026-10-28T13:05:52.305Z] [Pipeline] node
[2026-10-28T13:05:52.409Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron@3
[2026-10-28T13:05:52.410Z] [Pipeline] {
[2026-10-28T13:05:52.413Z] [Pipeline] sh
[2026-10-28T13:05:52.668Z] + mkdir -p /var/lib/jenkins/locks
[2026-10-28T13:05:52.677Z] [Pipeline] echo
[2026-10-28T13:05:52.677Z] RWLock: FD for lock log_archive in stage projectFinishUp is 98
[2026-10-28T13:05:52.680Z] [Pipeline] sleep
[2026-10-28T13:05:52.681Z] No need to sleep any longer
[2026-10-28T13:05:52.684Z] [Pipeline] echo
[2026-10-28T13:05:52.684Z] RWLock: log_archive in stage projectFinishUp locked for WRITE
[2026-10-28T13:05:52.687Z] [Pipeline] }
[2026-10-28T13:05:52.690Z] [Pipeline] // node
[2026-10-28T13:05:52.694Z] [Pipeline] echo
[2026-10-28T13:05:52.694Z] Copying logs to /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-cron/209/
[2026-10-28T13:05:52.695Z] [Pipeline] sh
[2026-10-28T13:05:52.950Z] + mkdir -p /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-cron/209/
[2026-10-28T13:05:52.951Z] [Pipeline] sh