[2026-06-29T08:16:00.685Z] Started by timer
[2026-06-29T08:16:00.939Z] Obtained pipelines/projects/anvil/build-cron/Jenkinsfile from git https://github.com/kronosnet/ci-tools
[2026-06-29T08:16:00.958Z] Loading library ProjectLib@main:pipelines/projects/anvil/
[2026-06-29T08:16:02.343Z] Examining kronosnet/ci-tools
[2026-06-29T08:16:02.343Z] Attempting to resolve main as a branch
[2026-06-29T08:16:02.489Z] Resolved main as branch main at revision d6e2b83422a0946fedbf297fe2b01b56a947c833
[2026-06-29T08:16:02.491Z] Selected Git installation does not exist. Using Default
[2026-06-29T08:16:02.491Z] The recommended git tool is: NONE
[2026-06-29T08:16:02.491Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-06-29T08:16:02.493Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-06-29T08:16:02.493Z] Wiping out workspace first.
[2026-06-29T08:16:02.515Z] Cloning the remote Git repository
[2026-06-29T08:16:02.515Z] Using shallow clone with depth 1
[2026-06-29T08:16:02.515Z] Avoid fetching tags
[2026-06-29T08:16:02.515Z] Honoring refspec on initial clone
[2026-06-29T08:16:02.515Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-06-29T08:16:02.515Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-cron@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b # timeout=10
[2026-06-29T08:16:02.525Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-06-29T08:16:02.525Z]  > git --version # timeout=10
[2026-06-29T08:16:02.532Z]  > git --version # 'git version 2.52.0'
[2026-06-29T08:16:02.532Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-06-29T08:16:02.538Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-06-29T08:16:03.028Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-06-29T08:16:03.035Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-06-29T08:16:03.043Z] Avoid second fetch
[2026-06-29T08:16:03.043Z] Checking out Revision d6e2b83422a0946fedbf297fe2b01b56a947c833 (main)
[2026-06-29T08:16:03.043Z]  > git config core.sparsecheckout # timeout=10
[2026-06-29T08:16:03.049Z]  > git checkout -f d6e2b83422a0946fedbf297fe2b01b56a947c833 # timeout=10
[2026-06-29T08:16:03.083Z] Commit message: "fn-testing-weekly: Send interim emails in their own stage() for clarity"
[2026-06-29T08:16:03.092Z] Loading library GlobalLib@main:./pipelines/libs/global/
[2026-06-29T08:16:03.285Z] Examining kronosnet/ci-tools
[2026-06-29T08:16:03.285Z] Attempting to resolve main as a branch
[2026-06-29T08:16:03.452Z] Resolved main as branch main at revision d6e2b83422a0946fedbf297fe2b01b56a947c833
[2026-06-29T08:16:03.453Z] Selected Git installation does not exist. Using Default
[2026-06-29T08:16:03.453Z] The recommended git tool is: NONE
[2026-06-29T08:16:03.453Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-06-29T08:16:03.456Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-06-29T08:16:03.456Z] Wiping out workspace first.
[2026-06-29T08:16:03.474Z] Cloning the remote Git repository
[2026-06-29T08:16:03.474Z] Using shallow clone with depth 1
[2026-06-29T08:16:03.474Z] Avoid fetching tags
[2026-06-29T08:16:03.474Z] Honoring refspec on initial clone
[2026-06-29T08:16:03.474Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-06-29T08:16:03.474Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-cron@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b # timeout=10
[2026-06-29T08:16:03.483Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-06-29T08:16:03.483Z]  > git --version # timeout=10
[2026-06-29T08:16:03.490Z]  > git --version # 'git version 2.52.0'
[2026-06-29T08:16:03.490Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-06-29T08:16:03.496Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-06-29T08:16:03.690Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-06-29T08:16:03.697Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-06-29T08:16:03.706Z] Avoid second fetch
[2026-06-29T08:16:03.706Z] Checking out Revision d6e2b83422a0946fedbf297fe2b01b56a947c833 (main)
[2026-06-29T08:16:03.706Z]  > git config core.sparsecheckout # timeout=10
[2026-06-29T08:16:03.713Z]  > git checkout -f d6e2b83422a0946fedbf297fe2b01b56a947c833 # timeout=10
[2026-06-29T08:16:03.744Z] Commit message: "fn-testing-weekly: Send interim emails in their own stage() for clarity"
[2026-06-29T08:16:03.809Z] [Pipeline] Start of Pipeline
[2026-06-29T08:16:03.935Z] [Pipeline] node
[2026-06-29T08:16:04.035Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron
[2026-06-29T08:16:04.035Z] [Pipeline] {
[2026-06-29T08:16:04.047Z] [Pipeline] stage
[2026-06-29T08:16:04.047Z] [Pipeline] { (Setup build env)
[2026-06-29T08:16:04.058Z] [Pipeline] script
[2026-06-29T08:16:04.062Z] [Pipeline] {
[2026-06-29T08:16:04.066Z] [Pipeline] cleanWs
[2026-06-29T08:16:04.067Z] [WS-CLEANUP] Deleting project workspace...
[2026-06-29T08:16:04.067Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-06-29T08:16:04.067Z] [WS-CLEANUP] done
[2026-06-29T08:16:04.091Z] [Pipeline] lock
[2026-06-29T08:16:04.091Z] Trying to acquire lock on [Resource: MAINTENANCE LOCKOUT]
[2026-06-29T08:16:04.092Z] Resource [MAINTENANCE LOCKOUT] did not exist. Created.
[2026-06-29T08:16:04.092Z] Lock acquired on [Resource: MAINTENANCE LOCKOUT]
[2026-06-29T08:16:04.092Z] [Pipeline] {
[2026-06-29T08:16:04.095Z] [Pipeline] }
[2026-06-29T08:16:04.095Z] Lock released on resource [Resource: MAINTENANCE LOCKOUT]
[2026-06-29T08:16:04.098Z] [Pipeline] // lock
[2026-06-29T08:16:04.101Z] [Pipeline] }
[2026-06-29T08:16:04.103Z] [Pipeline] // script
[2026-06-29T08:16:04.105Z] [Pipeline] }
[2026-06-29T08:16:04.107Z] [Pipeline] // stage
[2026-06-29T08:16:04.110Z] [Pipeline] stage
[2026-06-29T08:16:04.111Z] [Pipeline] { (Update repos and cache)
[2026-06-29T08:16:04.117Z] [Pipeline] parallel
[2026-06-29T08:16:04.117Z] [Pipeline] { (Branch: anvil build cache)
[2026-06-29T08:16:04.117Z] [Pipeline] { (Branch: anvil build dep repos)
[2026-06-29T08:16:04.121Z] [Pipeline] stage
[2026-06-29T08:16:04.121Z] [Pipeline] { (anvil build cache)
[2026-06-29T08:16:04.122Z] [Pipeline] stage
[2026-06-29T08:16:04.122Z] [Pipeline] { (anvil build dep repos)
[2026-06-29T08:16:04.130Z] [Pipeline] script
[2026-06-29T08:16:04.133Z] [Pipeline] {
[2026-06-29T08:16:04.133Z] [Pipeline] script
[2026-06-29T08:16:04.135Z] [Pipeline] {
[2026-06-29T08:16:04.148Z] [Pipeline] build
[2026-06-29T08:16:04.149Z] Scheduling project: anvil » anvil-build-cache
[2026-06-29T08:16:04.150Z] [Pipeline] build
[2026-06-29T08:16:04.150Z] Scheduling project: anvil » anvil-build-dep-repos » main
[2026-06-29T08:16:11.606Z] Starting building: anvil » anvil-build-cache #669
[2026-06-29T08:16:11.607Z] Starting building: anvil » anvil-build-dep-repos » main #104
[2026-06-29T08:16:11.608Z] [Pipeline] waitForBuild
[2026-06-29T08:16:11.608Z] Waiting for anvil » anvil-build-cache #669 to complete
[2026-06-29T08:16:11.614Z] [Pipeline] waitForBuild
[2026-06-29T08:16:11.615Z] Waiting for anvil » anvil-build-dep-repos » main #104 to complete
[2026-06-29T08:33:55.315Z] Build anvil » anvil-build-cache #669 completed: SUCCESS
[2026-06-29T08:33:55.317Z] [Pipeline] }
[2026-06-29T08:33:55.321Z] [Pipeline] // script
[2026-06-29T08:33:55.324Z] [Pipeline] }
[2026-06-29T08:33:55.326Z] [Pipeline] // stage
[2026-06-29T08:33:55.329Z] [Pipeline] }
[2026-06-29T08:34:59.328Z] Build anvil » anvil-build-dep-repos » main #104 completed: SUCCESS
[2026-06-29T08:34:59.331Z] [Pipeline] }
[2026-06-29T08:34:59.333Z] [Pipeline] // script
[2026-06-29T08:34:59.337Z] [Pipeline] }
[2026-06-29T08:34:59.339Z] [Pipeline] // stage
[2026-06-29T08:34:59.341Z] [Pipeline] }
[2026-06-29T08:34:59.344Z] [Pipeline] // parallel
[2026-06-29T08:34:59.347Z] [Pipeline] }
[2026-06-29T08:34:59.349Z] [Pipeline] // stage
[2026-06-29T08:34:59.353Z] [Pipeline] stage
[2026-06-29T08:34:59.353Z] [Pipeline] { (Update kernel modules)
[2026-06-29T08:34:59.360Z] [Pipeline] parallel
[2026-06-29T08:34:59.360Z] [Pipeline] { (Branch: anvil build all kmods)
[2026-06-29T08:34:59.364Z] [Pipeline] stage
[2026-06-29T08:34:59.365Z] [Pipeline] { (anvil build all kmods)
[2026-06-29T08:34:59.374Z] [Pipeline] script
[2026-06-29T08:34:59.378Z] [Pipeline] {
[2026-06-29T08:34:59.382Z] [Pipeline] build
[2026-06-29T08:34:59.383Z] Scheduling project: anvil » anvil-build-all-kmods
[2026-06-29T08:35:06.990Z] Starting building: anvil » anvil-build-all-kmods #602
[2026-06-29T08:35:06.992Z] [Pipeline] waitForBuild
[2026-06-29T08:35:06.992Z] Waiting for anvil » anvil-build-all-kmods #602 to complete
[2026-06-29T08:43:28.931Z] Build anvil » anvil-build-all-kmods #602 completed: SUCCESS
[2026-06-29T08:43:28.934Z] [Pipeline] }
[2026-06-29T08:43:28.936Z] [Pipeline] // script
[2026-06-29T08:43:28.939Z] [Pipeline] }
[2026-06-29T08:43:28.941Z] [Pipeline] // stage
[2026-06-29T08:43:28.943Z] [Pipeline] }
[2026-06-29T08:43:28.946Z] [Pipeline] // parallel
[2026-06-29T08:43:28.949Z] [Pipeline] }
[2026-06-29T08:43:28.951Z] [Pipeline] // stage
[2026-06-29T08:43:28.954Z] [Pipeline] stage
[2026-06-29T08:43:28.954Z] [Pipeline] { (Run daily functional testing)
[2026-06-29T08:43:28.960Z] [Pipeline] parallel
[2026-06-29T08:43:28.960Z] [Pipeline] { (Branch: almalinux-9 ci ci)
[2026-06-29T08:43:28.960Z] [Pipeline] { (Branch: rhel-9 ci ci)
[2026-06-29T08:43:28.961Z] [Pipeline] { (Branch: almalinux-9 enterprise none)
[2026-06-29T08:43:28.961Z] [Pipeline] { (Branch: rhel-9 enterprise none)
[2026-06-29T08:43:28.961Z] [Pipeline] { (Branch: almalinux-9 enterprise-staging none)
[2026-06-29T08:43:28.961Z] [Pipeline] { (Branch: rhel-9 enterprise-staging none)
[2026-06-29T08:43:28.968Z] [Pipeline] stage
[2026-06-29T08:43:28.968Z] [Pipeline] { (almalinux-9 ci ci)
[2026-06-29T08:43:28.969Z] [Pipeline] stage
[2026-06-29T08:43:28.969Z] [Pipeline] { (rhel-9 ci ci)
[2026-06-29T08:43:28.970Z] [Pipeline] stage
[2026-06-29T08:43:28.970Z] [Pipeline] { (almalinux-9 enterprise none)
[2026-06-29T08:43:28.971Z] [Pipeline] stage
[2026-06-29T08:43:28.971Z] [Pipeline] { (rhel-9 enterprise none)
[2026-06-29T08:43:28.972Z] [Pipeline] stage
[2026-06-29T08:43:28.972Z] [Pipeline] { (almalinux-9 enterprise-staging none)
[2026-06-29T08:43:28.972Z] [Pipeline] stage
[2026-06-29T08:43:28.973Z] [Pipeline] { (rhel-9 enterprise-staging none)
[2026-06-29T08:43:28.991Z] [Pipeline] script
[2026-06-29T08:43:28.993Z] [Pipeline] {
[2026-06-29T08:43:28.994Z] [Pipeline] script
[2026-06-29T08:43:28.996Z] [Pipeline] {
[2026-06-29T08:43:28.996Z] [Pipeline] script
[2026-06-29T08:43:28.998Z] [Pipeline] {
[2026-06-29T08:43:28.998Z] [Pipeline] script
[2026-06-29T08:43:29.000Z] [Pipeline] {
[2026-06-29T08:43:29.000Z] [Pipeline] script
[2026-06-29T08:43:29.002Z] [Pipeline] {
[2026-06-29T08:43:29.002Z] [Pipeline] script
[2026-06-29T08:43:29.004Z] [Pipeline] {
[2026-06-29T08:43:29.009Z] [Pipeline] build
[2026-06-29T08:43:29.009Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-06-29T08:43:29.011Z] [Pipeline] build
[2026-06-29T08:43:29.012Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-06-29T08:43:29.013Z] [Pipeline] build
[2026-06-29T08:43:29.013Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-06-29T08:43:29.014Z] [Pipeline] build
[2026-06-29T08:43:29.015Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-06-29T08:43:29.016Z] [Pipeline] build
[2026-06-29T08:43:29.016Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-06-29T08:43:29.017Z] [Pipeline] build
[2026-06-29T08:43:29.018Z] Scheduling project: anvil » anvil-build-functional-testing
[2026-06-29T08:43:37.089Z] Starting building: anvil » anvil-build-functional-testing #804
[2026-06-29T08:43:37.089Z] Starting building: anvil » anvil-build-functional-testing #799
[2026-06-29T08:43:37.090Z] Starting building: anvil » anvil-build-functional-testing #803
[2026-06-29T08:43:37.090Z] Starting building: anvil » anvil-build-functional-testing #801
[2026-06-29T08:43:37.090Z] Starting building: anvil » anvil-build-functional-testing #802
[2026-06-29T08:43:37.090Z] [Pipeline] waitForBuild
[2026-06-29T08:43:37.091Z] Starting building: anvil » anvil-build-functional-testing #800
[2026-06-29T08:43:37.091Z] Waiting for anvil » anvil-build-functional-testing #804 to complete
[2026-06-29T08:43:37.106Z] [Pipeline] waitForBuild
[2026-06-29T08:43:37.106Z] Waiting for anvil » anvil-build-functional-testing #799 to complete
[2026-06-29T08:43:37.107Z] [Pipeline] waitForBuild
[2026-06-29T08:43:37.107Z] Waiting for anvil » anvil-build-functional-testing #800 to complete
[2026-06-29T08:43:37.108Z] [Pipeline] waitForBuild
[2026-06-29T08:43:37.108Z] Waiting for anvil » anvil-build-functional-testing #801 to complete
[2026-06-29T08:43:37.109Z] [Pipeline] waitForBuild
[2026-06-29T08:43:37.109Z] Waiting for anvil » anvil-build-functional-testing #802 to complete
[2026-06-29T08:43:37.110Z] [Pipeline] waitForBuild
[2026-06-29T08:43:37.110Z] Waiting for anvil » anvil-build-functional-testing #803 to complete
[2026-06-29T08:59:04.386Z] Build anvil » anvil-build-functional-testing #804 completed: FAILURE
[2026-06-29T08:59:04.388Z] [Pipeline] catchError
[2026-06-29T08:59:04.388Z] [Pipeline] {
[2026-06-29T08:59:04.399Z] [Pipeline] sh
[2026-06-29T08:59:04.655Z] [Pipeline] }
[2026-06-29T08:59:04.655Z] ERROR: script returned exit code 1
[2026-06-29T08:59:04.655Z] Setting overall build result to FAILURE
[2026-06-29T08:59:04.658Z] [Pipeline] // catchError
[2026-06-29T08:59:04.661Z] [Pipeline] }
[2026-06-29T08:59:04.664Z] [Pipeline] // script
[2026-06-29T08:59:04.666Z] [Pipeline] }
[2026-06-29T08:59:04.668Z] [Pipeline] // stage
[2026-06-29T08:59:04.671Z] [Pipeline] }
[2026-06-29T11:02:51.270Z] Build anvil » anvil-build-functional-testing #803 completed: SUCCESS
[2026-06-29T11:02:51.272Z] [Pipeline] }
[2026-06-29T11:02:51.275Z] [Pipeline] // script
[2026-06-29T11:02:51.277Z] [Pipeline] }
[2026-06-29T11:02:51.279Z] [Pipeline] // stage
[2026-06-29T11:02:51.282Z] [Pipeline] }
[2026-06-29T11:34:49.806Z] Build anvil » anvil-build-functional-testing #801 completed: SUCCESS
[2026-06-29T11:34:49.808Z] [Pipeline] }
[2026-06-29T11:34:49.811Z] [Pipeline] // script
[2026-06-29T11:34:49.813Z] [Pipeline] }
[2026-06-29T11:34:49.815Z] [Pipeline] // stage
[2026-06-29T11:34:49.817Z] [Pipeline] }
[2026-06-29T11:38:39.612Z] Build anvil » anvil-build-functional-testing #799 completed: SUCCESS
[2026-06-29T11:38:39.614Z] [Pipeline] }
[2026-06-29T11:38:39.617Z] [Pipeline] // script
[2026-06-29T11:38:39.620Z] [Pipeline] }
[2026-06-29T11:38:39.622Z] [Pipeline] // stage
[2026-06-29T11:38:39.624Z] [Pipeline] }
[2026-06-29T11:43:19.497Z] Build anvil » anvil-build-functional-testing #800 completed: SUCCESS
[2026-06-29T11:43:19.499Z] [Pipeline] }
[2026-06-29T11:43:19.501Z] [Pipeline] // script
[2026-06-29T11:43:19.504Z] [Pipeline] }
[2026-06-29T11:43:19.506Z] [Pipeline] // stage
[2026-06-29T11:43:19.508Z] [Pipeline] }
[2026-06-29T11:45:38.302Z] Build anvil » anvil-build-functional-testing #802 completed: SUCCESS
[2026-06-29T11:45:38.304Z] [Pipeline] }
[2026-06-29T11:45:38.307Z] [Pipeline] // script
[2026-06-29T11:45:38.309Z] [Pipeline] }
[2026-06-29T11:45:38.311Z] [Pipeline] // stage
[2026-06-29T11:45:38.313Z] [Pipeline] }
[2026-06-29T11:45:38.316Z] [Pipeline] // parallel
[2026-06-29T11:45:38.318Z] [Pipeline] }
[2026-06-29T11:45:38.320Z] [Pipeline] // stage
[2026-06-29T11:45:38.324Z] [Pipeline] stage
[2026-06-29T11:45:38.324Z] [Pipeline] { (Declarative: Post Actions)
[2026-06-29T11:45:38.341Z] [Pipeline] stage
[2026-06-29T11:45:38.341Z] [Pipeline] { (Copying logs)
[2026-06-29T11:45:38.373Z] [Pipeline] node
[2026-06-29T11:45:38.469Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron@2
[2026-06-29T11:45:38.470Z] [Pipeline] {
[2026-06-29T11:45:38.501Z] [Pipeline] node
[2026-06-29T11:45:38.596Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-cron@3
[2026-06-29T11:45:38.597Z] [Pipeline] {
[2026-06-29T11:45:38.600Z] [Pipeline] sh
[2026-06-29T11:45:38.855Z] + mkdir -p /var/lib/jenkins/locks
[2026-06-29T11:45:38.863Z] [Pipeline] echo
[2026-06-29T11:45:38.863Z] RWLock: FD for lock log_archive in stage projectFinishUp is 433
[2026-06-29T11:45:38.866Z] [Pipeline] sleep
[2026-06-29T11:45:38.867Z] No need to sleep any longer
[2026-06-29T11:45:38.869Z] [Pipeline] echo
[2026-06-29T11:45:38.870Z] RWLock: log_archive in stage projectFinishUp locked for WRITE
[2026-06-29T11:45:38.872Z] [Pipeline] }
[2026-06-29T11:45:38.875Z] [Pipeline] // node
[2026-06-29T11:45:38.879Z] [Pipeline] echo
[2026-06-29T11:45:38.879Z] Copying logs to /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-cron/99/
[2026-06-29T11:45:38.880Z] [Pipeline] sh
[2026-06-29T11:45:39.135Z] + mkdir -p /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-cron/99/
[2026-06-29T11:45:39.136Z] [Pipeline] sh