[2026-04-05T09:22:58.147Z] Started by upstream project "anvil/anvil-build-cron" build number 16
[2026-04-05T09:22:58.147Z] originally caused by:
[2026-04-05T09:22:58.147Z]  Started by timer
[2026-04-05T09:22:59.054Z] Obtained pipelines/projects/anvil/build-functional-testing/Jenkinsfile from git https://github.com/kronosnet/ci-tools
[2026-04-05T09:22:59.063Z] Loading library ProjectLib@main:pipelines/projects/anvil/
[2026-04-05T09:22:59.279Z] Examining kronosnet/ci-tools
[2026-04-05T09:22:59.279Z] Attempting to resolve main as a branch
[2026-04-05T09:22:59.391Z] Resolved main as branch main at revision ec5fa371f9bb750f83c25da62a8ed5e8ce97a911
[2026-04-05T09:22:59.392Z] Selected Git installation does not exist. Using Default
[2026-04-05T09:22:59.392Z] The recommended git tool is: NONE
[2026-04-05T09:22:59.392Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-04-05T09:22:59.396Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-04-05T09:22:59.396Z] Wiping out workspace first.
[2026-04-05T09:22:59.414Z] Cloning the remote Git repository
[2026-04-05T09:22:59.414Z] Using shallow clone with depth 1
[2026-04-05T09:22:59.414Z] Avoid fetching tags
[2026-04-05T09:22:59.414Z] Honoring refspec on initial clone
[2026-04-05T09:22:59.414Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-04-05T09:22:59.415Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-functional-testing@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b # timeout=10
[2026-04-05T09:22:59.426Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-04-05T09:22:59.426Z]  > git --version # timeout=10
[2026-04-05T09:22:59.434Z]  > git --version # 'git version 2.47.3'
[2026-04-05T09:22:59.435Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-04-05T09:22:59.442Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-05T09:22:59.636Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-04-05T09:22:59.645Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-05T09:22:59.654Z] Avoid second fetch
[2026-04-05T09:22:59.654Z] Checking out Revision ec5fa371f9bb750f83c25da62a8ed5e8ce97a911 (main)
[2026-04-05T09:22:59.654Z]  > git config core.sparsecheckout # timeout=10
[2026-04-05T09:22:59.665Z]  > git checkout -f ec5fa371f9bb750f83c25da62a8ed5e8ce97a911 # timeout=10
[2026-04-05T09:22:59.700Z] Commit message: "Get rid of [[$class: 'LabelParameterValue', as it's just a string and needs an unmaintained plugin to work"
[2026-04-05T09:22:59.706Z] Loading library GlobalLib@main:./pipelines/libs/global/
[2026-04-05T09:22:59.869Z] Examining kronosnet/ci-tools
[2026-04-05T09:22:59.869Z] Attempting to resolve main as a branch
[2026-04-05T09:22:59.997Z] Resolved main as branch main at revision ec5fa371f9bb750f83c25da62a8ed5e8ce97a911
[2026-04-05T09:22:59.998Z] Selected Git installation does not exist. Using Default
[2026-04-05T09:22:59.998Z] The recommended git tool is: NONE
[2026-04-05T09:22:59.999Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-04-05T09:23:00.003Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-04-05T09:23:00.003Z] Wiping out workspace first.
[2026-04-05T09:23:00.028Z] Cloning the remote Git repository
[2026-04-05T09:23:00.028Z] Using shallow clone with depth 1
[2026-04-05T09:23:00.028Z] Avoid fetching tags
[2026-04-05T09:23:00.028Z] Honoring refspec on initial clone
[2026-04-05T09:23:00.028Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-04-05T09:23:00.028Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-functional-testing@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b@2 # timeout=10
[2026-04-05T09:23:00.039Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-04-05T09:23:00.039Z]  > git --version # timeout=10
[2026-04-05T09:23:00.053Z]  > git --version # 'git version 2.47.3'
[2026-04-05T09:23:00.053Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-04-05T09:23:00.060Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-05T09:23:00.239Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-04-05T09:23:00.246Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-05T09:23:00.256Z] Avoid second fetch
[2026-04-05T09:23:00.256Z] Checking out Revision ec5fa371f9bb750f83c25da62a8ed5e8ce97a911 (main)
[2026-04-05T09:23:00.256Z]  > git config core.sparsecheckout # timeout=10
[2026-04-05T09:23:00.263Z]  > git checkout -f ec5fa371f9bb750f83c25da62a8ed5e8ce97a911 # timeout=10
[2026-04-05T09:23:00.295Z] Commit message: "Get rid of [[$class: 'LabelParameterValue', as it's just a string and needs an unmaintained plugin to work"
[2026-04-05T09:23:00.357Z] [Pipeline] Start of Pipeline
[2026-04-05T09:23:00.458Z] [Pipeline] node
[2026-04-05T09:23:00.562Z] Running on anvil-ci-bm-phy04 in /srv/workspace/anvil/anvil-build-functional-testing
[2026-04-05T09:23:00.563Z] [Pipeline] {
[2026-04-05T09:23:00.572Z] [Pipeline] stage
[2026-04-05T09:23:00.572Z] [Pipeline] { (Prep build env)
[2026-04-05T09:23:00.583Z] [Pipeline] script
[2026-04-05T09:23:00.585Z] [Pipeline] {
[2026-04-05T09:23:00.589Z] [Pipeline] cleanWs
[2026-04-05T09:23:00.650Z] [WS-CLEANUP] Deleting project workspace...
[2026-04-05T09:23:00.650Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-04-05T09:23:00.680Z] [WS-CLEANUP] done
[2026-04-05T09:23:00.700Z] [Pipeline] lock
[2026-04-05T09:23:00.700Z] Trying to acquire lock on [Resource: MAINTENANCE LOCKOUT]
[2026-04-05T09:23:00.703Z] Resource [MAINTENANCE LOCKOUT] did not exist. Created.
[2026-04-05T09:23:00.707Z] Lock acquired on [Resource: MAINTENANCE LOCKOUT]
[2026-04-05T09:23:00.708Z] [Pipeline] {
[2026-04-05T09:23:00.712Z] [Pipeline] }
[2026-04-05T09:23:00.715Z] Lock released on resource [Resource: MAINTENANCE LOCKOUT]
[2026-04-05T09:23:00.717Z] [Pipeline] // lock
[2026-04-05T09:23:00.722Z] [Pipeline] }
[2026-04-05T09:23:00.724Z] [Pipeline] // script
[2026-04-05T09:23:00.726Z] [Pipeline] }
[2026-04-05T09:23:00.727Z] [Pipeline] // stage
[2026-04-05T09:23:00.730Z] [Pipeline] stage
[2026-04-05T09:23:00.730Z] [Pipeline] { (Run functional testing)
[2026-04-05T09:23:00.736Z] [Pipeline] script
[2026-04-05T09:23:00.737Z] [Pipeline] {
[2026-04-05T09:23:00.759Z] [Pipeline] echo
[2026-04-05T09:23:00.759Z] runWithTimeout 360
[2026-04-05T09:23:00.760Z] [Pipeline] timeout
[2026-04-05T09:23:00.760Z] Timeout set to expire in 6 hr 0 min
[2026-04-05T09:23:00.760Z] [Pipeline] {
[2026-04-05T09:23:00.765Z] [Pipeline] sh
[2026-04-05T09:23:01.315Z] + /root/ci-tools/ci-wrap ci-setup-anvil-bm-vm rhel-9 enterprise-staging none all
[2026-04-05T09:23:01.315Z] CITBRANCH:  main
[2026-04-05T09:23:01.315Z] CITHOME:    /root/ci-tools
[2026-04-05T09:23:01.315Z] CITCMD:     ci-setup-anvil-bm-vm
[2026-04-05T09:23:01.315Z] CITCMDARGS: rhel-9 enterprise-staging none all
[2026-04-05T09:23:01.315Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-05T09:23:01.315Z] Genering ssh keys and ssh config
[2026-04-05T09:23:01.684Z] # github.com:22 SSH-2.0-ab54611
[2026-04-05T09:23:01.684Z] # github.com:22 SSH-2.0-ab54611
[2026-04-05T09:23:01.684Z] # github.com:22 SSH-2.0-ab54611
[2026-04-05T09:23:01.684Z] # github.com:22 SSH-2.0-ab54611
[2026-04-05T09:23:02.051Z] # github.com:22 SSH-2.0-ab54611
[2026-04-05T09:23:02.051Z] Delay build by 180 seconds to allow Jenkins to cancel the job
[2026-04-05T09:26:11.844Z] Destroying VMs (if any)
[2026-04-05T09:26:11.844Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-05T09:26:11.844Z] Genering ssh keys and ssh config
[2026-04-05T09:26:11.844Z] # github.com:22 SSH-2.0-ab54611
[2026-04-05T09:26:11.844Z] # github.com:22 SSH-2.0-ab54611
[2026-04-05T09:26:11.844Z] # github.com:22 SSH-2.0-ab54611
[2026-04-05T09:26:11.844Z] # github.com:22 SSH-2.0-ab54611
[2026-04-05T09:26:11.844Z] # github.com:22 SSH-2.0-ab54611
[2026-04-05T09:26:11.845Z] Cache already updated
[2026-04-05T09:26:11.845Z] Creating nodes disks
[2026-04-05T09:26:11.845Z] Formatting 'an-striker01-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/rhel-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-05T09:26:11.845Z] Formatting 'an-striker01-vol2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 lazy_refcounts=off refcount_bits=16
[2026-04-05T09:26:11.845Z] Formatting 'an-striker02-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/rhel-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-05T09:26:11.845Z] Formatting 'an-striker02-vol2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 lazy_refcounts=off refcount_bits=16
[2026-04-05T09:26:11.845Z] Formatting 'an-a01n01-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/rhel-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-05T09:26:11.845Z] Formatting 'an-a01n01-vol2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 lazy_refcounts=off refcount_bits=16
[2026-04-05T09:26:11.845Z] Formatting 'an-a01n02-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/rhel-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-05T09:26:11.845Z] Formatting 'an-a01n02-vol2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 lazy_refcounts=off refcount_bits=16
[2026-04-05T09:26:11.845Z] Formatting 'an-a01dr01-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/rhel-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-05T09:26:11.845Z] Formatting 'an-a01dr01-vol2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 lazy_refcounts=off refcount_bits=16
[2026-04-05T09:26:11.845Z] Generating an-test-deployX kickstart files/isos
[2026-04-05T09:26:11.845Z] xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.
[2026-04-05T09:26:11.845Z] 
[2026-04-05T09:26:11.845Z] Drive current: -outdev 'stdio:/srv/anvil/snapshots/deploy1.iso'
[2026-04-05T09:26:11.845Z] Media current: stdio file, overwriteable
[2026-04-05T09:26:11.845Z] Media status : is blank
[2026-04-05T09:26:11.845Z] Media summary: 0 sessions, 0 data blocks, 0 data, 1726g free
[2026-04-05T09:26:11.845Z] Added to ISO image: directory '/'='/srv/anvil/configs/ks1'
[2026-04-05T09:26:11.845Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-05T09:26:11.845Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-05T09:26:11.845Z] ISO image produced: 184 sectors
[2026-04-05T09:26:11.845Z] Written to medium : 184 sectors at LBA 0
[2026-04-05T09:26:11.845Z] Writing to 'stdio:/srv/anvil/snapshots/deploy1.iso' completed successfully.
[2026-04-05T09:26:11.845Z] 
[2026-04-05T09:26:11.845Z] xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.
[2026-04-05T09:26:11.845Z] 
[2026-04-05T09:26:11.845Z] Drive current: -outdev 'stdio:/srv/anvil/snapshots/deploy2.iso'
[2026-04-05T09:26:11.845Z] Media current: stdio file, overwriteable
[2026-04-05T09:26:11.845Z] Media status : is blank
[2026-04-05T09:26:11.845Z] Media summary: 0 sessions, 0 data blocks, 0 data, 1730g free
[2026-04-05T09:26:11.845Z] Added to ISO image: directory '/'='/srv/anvil/configs/ks2'
[2026-04-05T09:26:11.845Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-05T09:26:11.845Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-05T09:26:11.845Z] ISO image produced: 184 sectors
[2026-04-05T09:26:11.845Z] Written to medium : 184 sectors at LBA 0
[2026-04-05T09:26:11.845Z] Writing to 'stdio:/srv/anvil/snapshots/deploy2.iso' completed successfully.
[2026-04-05T09:26:11.845Z] 
[2026-04-05T09:26:11.845Z] xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.
[2026-04-05T09:26:11.845Z] 
[2026-04-05T09:26:11.845Z] Drive current: -outdev 'stdio:/srv/anvil/snapshots/deploy3.iso'
[2026-04-05T09:26:11.845Z] Media current: stdio file, overwriteable
[2026-04-05T09:26:11.845Z] Media status : is blank
[2026-04-05T09:26:11.845Z] Media summary: 0 sessions, 0 data blocks, 0 data, 1730g free
[2026-04-05T09:26:11.845Z] Added to ISO image: directory '/'='/srv/anvil/configs/ks3'
[2026-04-05T09:26:11.845Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-05T09:26:11.845Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-05T09:26:11.845Z] ISO image produced: 184 sectors
[2026-04-05T09:26:11.845Z] Written to medium : 184 sectors at LBA 0
[2026-04-05T09:26:11.845Z] Writing to 'stdio:/srv/anvil/snapshots/deploy3.iso' completed successfully.
[2026-04-05T09:26:11.845Z] 
[2026-04-05T09:26:11.845Z] xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.
[2026-04-05T09:26:11.845Z] 
[2026-04-05T09:26:11.845Z] Drive current: -outdev 'stdio:/srv/anvil/snapshots/deploy4.iso'
[2026-04-05T09:26:11.845Z] Media current: stdio file, overwriteable
[2026-04-05T09:26:11.845Z] Media status : is blank
[2026-04-05T09:26:11.845Z] Media summary: 0 sessions, 0 data blocks, 0 data, 1730g free
[2026-04-05T09:26:11.845Z] Added to ISO image: directory '/'='/srv/anvil/configs/ks4'
[2026-04-05T09:26:11.845Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-05T09:26:11.845Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-05T09:26:11.845Z] ISO image produced: 184 sectors
[2026-04-05T09:26:11.845Z] Written to medium : 184 sectors at LBA 0
[2026-04-05T09:26:11.845Z] Writing to 'stdio:/srv/anvil/snapshots/deploy4.iso' completed successfully.
[2026-04-05T09:26:11.845Z] 
[2026-04-05T09:26:11.845Z] xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.
[2026-04-05T09:26:11.845Z] 
[2026-04-05T09:26:11.845Z] Drive current: -outdev 'stdio:/srv/anvil/snapshots/deploy5.iso'
[2026-04-05T09:26:11.845Z] Media current: stdio file, overwriteable
[2026-04-05T09:26:11.845Z] Media status : is blank
[2026-04-05T09:26:11.845Z] Media summary: 0 sessions, 0 data blocks, 0 data, 1730g free
[2026-04-05T09:26:11.845Z] Added to ISO image: directory '/'='/srv/anvil/configs/ks5'
[2026-04-05T09:26:11.845Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-05T09:26:11.845Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-05T09:26:11.845Z] ISO image produced: 184 sectors
[2026-04-05T09:26:11.845Z] Written to medium : 184 sectors at LBA 0
[2026-04-05T09:26:11.845Z] Writing to 'stdio:/srv/anvil/snapshots/deploy5.iso' completed successfully.
[2026-04-05T09:26:11.845Z] 
[2026-04-05T09:26:11.845Z] Generating cloud-init configs
[2026-04-05T09:26:11.845Z] Generating cloud-init metadata and seed images
[2026-04-05T09:26:11.845Z] wrote /srv/anvil/snapshots/an-striker01-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-05T09:26:11.845Z] wrote /srv/anvil/snapshots/an-striker02-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-05T09:26:11.845Z] wrote /srv/anvil/snapshots/an-a01n01-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-05T09:26:11.845Z] wrote /srv/anvil/snapshots/an-a01n02-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-05T09:26:11.845Z] wrote /srv/anvil/snapshots/an-a01dr01-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-05T09:26:11.845Z] Creating VMs
[2026-04-05T09:26:13.112Z] 
[2026-04-05T09:26:13.112Z] Starting install...
[2026-04-05T09:26:13.112Z] 
[2026-04-05T09:26:13.112Z] Creating domain...                                          |         00:00     
[2026-04-05T09:26:13.112Z] Domain creation completed.
[2026-04-05T09:26:14.378Z] 
[2026-04-05T09:26:14.378Z] Starting install...
[2026-04-05T09:26:14.378Z] 
[2026-04-05T09:26:14.378Z] Creating domain...                                          |         00:00     
[2026-04-05T09:26:14.378Z] Domain creation completed.
[2026-04-05T09:26:16.194Z] 
[2026-04-05T09:26:16.194Z] Starting install...
[2026-04-05T09:26:16.194Z] 
[2026-04-05T09:26:16.194Z] Creating domain...                                          |         00:00     
[2026-04-05T09:26:16.194Z] Domain creation completed.
[2026-04-05T09:26:18.014Z] 
[2026-04-05T09:26:18.014Z] Starting install...
[2026-04-05T09:26:18.014Z] 
[2026-04-05T09:26:18.014Z] Creating domain...                                          |         00:00     
[2026-04-05T09:26:18.014Z] Domain creation completed.
[2026-04-05T09:26:19.292Z] 
[2026-04-05T09:26:19.292Z] Starting install...
[2026-04-05T09:26:19.292Z] 
[2026-04-05T09:26:19.292Z] Creating domain...                                          |         00:00     
[2026-04-05T09:26:19.292Z] Domain creation completed.
[2026-04-05T09:26:19.661Z] Waiting for nodes and cloud-init to complete full init
[2026-04-05T09:26:22.856Z] cloud-init status node an-striker01: 
[2026-04-05T09:26:26.057Z] cloud-init status node an-striker02: 
[2026-04-05T09:26:29.259Z] cloud-init status node an-a01n01: 
[2026-04-05T09:26:32.454Z] cloud-init status node an-a01n02: 
[2026-04-05T09:26:35.650Z] cloud-init status node an-a01dr01: 
[2026-04-05T09:26:35.651Z] Sun Apr  5 05:26:34 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:26:46.989Z] cloud-init status node an-striker01: running
[2026-04-05T09:26:47.788Z] cloud-init status node an-striker02: running
[2026-04-05T09:26:48.586Z] cloud-init status node an-a01n01: running
[2026-04-05T09:26:49.868Z] cloud-init status node an-a01n02: running
[2026-04-05T09:26:50.665Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:26:50.665Z] Sun Apr  5 05:26:50 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:27:02.057Z] cloud-init status node an-striker01: running
[2026-04-05T09:27:02.445Z] cloud-init status node an-striker02: running
[2026-04-05T09:27:03.736Z] cloud-init status node an-a01n01: running
[2026-04-05T09:27:10.120Z] cloud-init status node an-a01n02: running
[2026-04-05T09:27:10.120Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:27:10.120Z] Sun Apr  5 05:27:09 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:27:21.523Z] cloud-init status node an-striker01: running
[2026-04-05T09:27:21.900Z] cloud-init status node an-striker02: running
[2026-04-05T09:27:22.700Z] cloud-init status node an-a01n01: running
[2026-04-05T09:27:23.988Z] cloud-init status node an-a01n02: running
[2026-04-05T09:27:24.790Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:27:24.790Z] Sun Apr  5 05:27:24 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:27:36.185Z] cloud-init status node an-striker01: running
[2026-04-05T09:27:36.560Z] cloud-init status node an-striker02: running
[2026-04-05T09:27:37.846Z] cloud-init status node an-a01n01: running
[2026-04-05T09:27:38.660Z] cloud-init status node an-a01n02: running
[2026-04-05T09:27:39.460Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:27:39.460Z] Sun Apr  5 05:27:39 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:27:50.844Z] cloud-init status node an-striker01: running
[2026-04-05T09:27:51.640Z] cloud-init status node an-striker02: running
[2026-04-05T09:27:52.442Z] cloud-init status node an-a01n01: running
[2026-04-05T09:27:58.776Z] cloud-init status node an-a01n02: running
[2026-04-05T09:27:59.150Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:27:59.150Z] Sun Apr  5 05:27:59 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:28:10.675Z] cloud-init status node an-striker01: running
[2026-04-05T09:28:11.484Z] cloud-init status node an-striker02: running
[2026-04-05T09:28:12.455Z] cloud-init status node an-a01n01: running
[2026-04-05T09:28:18.946Z] cloud-init status node an-a01n02: running
[2026-04-05T09:28:19.322Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:28:19.322Z] Sun Apr  5 05:28:19 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:28:31.005Z] cloud-init status node an-striker01: running
[2026-04-05T09:28:31.378Z] cloud-init status node an-striker02: running
[2026-04-05T09:28:32.189Z] cloud-init status node an-a01n01: running
[2026-04-05T09:28:32.994Z] cloud-init status node an-a01n02: running
[2026-04-05T09:28:33.800Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:28:33.800Z] Sun Apr  5 05:28:33 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:28:45.569Z] cloud-init status node an-striker01: running
[2026-04-05T09:28:45.946Z] cloud-init status node an-striker02: running
[2026-04-05T09:28:46.775Z] cloud-init status node an-a01n01: running
[2026-04-05T09:28:47.572Z] cloud-init status node an-a01n02: running
[2026-04-05T09:28:48.373Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:28:48.373Z] Sun Apr  5 05:28:48 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:28:59.760Z] cloud-init status node an-striker01: running
[2026-04-05T09:29:00.590Z] cloud-init status node an-striker02: running
[2026-04-05T09:29:01.403Z] cloud-init status node an-a01n01: running
[2026-04-05T09:29:02.202Z] cloud-init status node an-a01n02: running
[2026-04-05T09:29:03.006Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:29:03.006Z] Sun Apr  5 05:29:02 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:29:14.377Z] cloud-init status node an-striker01: running
[2026-04-05T09:29:15.176Z] cloud-init status node an-striker02: running
[2026-04-05T09:29:15.978Z] cloud-init status node an-a01n01: running
[2026-04-05T09:29:17.261Z] cloud-init status node an-a01n02: running
[2026-04-05T09:29:18.061Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:29:18.061Z] Sun Apr  5 05:29:17 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:29:29.437Z] cloud-init status node an-striker01: running
[2026-04-05T09:29:29.810Z] cloud-init status node an-striker02: running
[2026-04-05T09:29:31.095Z] cloud-init status node an-a01n01: running
[2026-04-05T09:29:31.894Z] cloud-init status node an-a01n02: running
[2026-04-05T09:29:32.695Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:29:32.695Z] Sun Apr  5 05:29:32 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:29:44.070Z] cloud-init status node an-striker01: running
[2026-04-05T09:29:44.871Z] cloud-init status node an-striker02: running
[2026-04-05T09:29:45.669Z] cloud-init status node an-a01n01: running
[2026-04-05T09:29:46.466Z] cloud-init status node an-a01n02: running
[2026-04-05T09:29:47.750Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:29:47.750Z] Sun Apr  5 05:29:47 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:29:59.115Z] cloud-init status node an-striker01: running
[2026-04-05T09:29:59.489Z] cloud-init status node an-striker02: running
[2026-04-05T09:30:00.285Z] cloud-init status node an-a01n01: running
[2026-04-05T09:30:01.082Z] cloud-init status node an-a01n02: running
[2026-04-05T09:30:02.366Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:30:02.366Z] Sun Apr  5 05:30:01 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:30:13.736Z] cloud-init status node an-striker01: running
[2026-04-05T09:30:13.736Z] cloud-init status node an-striker02: running
[2026-04-05T09:30:15.015Z] cloud-init status node an-a01n01: running
[2026-04-05T09:30:21.319Z] cloud-init status node an-a01n02: running
[2026-04-05T09:30:21.693Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:30:21.693Z] Sun Apr  5 05:30:21 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:30:33.042Z] cloud-init status node an-striker01: running
[2026-04-05T09:30:33.835Z] cloud-init status node an-striker02: running
[2026-04-05T09:30:40.110Z] cloud-init status node an-a01n01: running
[2026-04-05T09:30:40.478Z] cloud-init status node an-a01n02: running
[2026-04-05T09:30:41.266Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:30:41.266Z] Sun Apr  5 05:30:41 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:30:52.565Z] cloud-init status node an-striker01: running
[2026-04-05T09:30:53.355Z] cloud-init status node an-striker02: running
[2026-04-05T09:30:54.144Z] cloud-init status node an-a01n01: running
[2026-04-05T09:30:55.413Z] cloud-init status node an-a01n02: running
[2026-04-05T09:30:56.201Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:30:56.201Z] Sun Apr  5 05:30:55 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:31:07.501Z] cloud-init status node an-striker01: running
[2026-04-05T09:31:07.870Z] cloud-init status node an-striker02: running
[2026-04-05T09:31:08.657Z] cloud-init status node an-a01n01: running
[2026-04-05T09:31:09.923Z] cloud-init status node an-a01n02: running
[2026-04-05T09:31:10.711Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:31:10.711Z] Sun Apr  5 05:31:10 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:31:22.006Z] cloud-init status node an-striker01: running
[2026-04-05T09:31:22.794Z] cloud-init status node an-striker02: running
[2026-04-05T09:31:23.582Z] cloud-init status node an-a01n01: running
[2026-04-05T09:31:24.369Z] cloud-init status node an-a01n02: running
[2026-04-05T09:31:25.637Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:31:25.637Z] Sun Apr  5 05:31:25 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:31:36.939Z] cloud-init status node an-striker01: running
[2026-04-05T09:31:37.308Z] cloud-init status node an-striker02: running
[2026-04-05T09:31:38.096Z] cloud-init status node an-a01n01: running
[2026-04-05T09:31:38.885Z] cloud-init status node an-a01n02: running
[2026-04-05T09:31:40.153Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:31:40.153Z] Sun Apr  5 05:31:39 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:31:51.460Z] cloud-init status node an-striker01: running
[2026-04-05T09:31:51.829Z] cloud-init status node an-striker02: running
[2026-04-05T09:31:52.618Z] cloud-init status node an-a01n01: running
[2026-04-05T09:31:53.885Z] cloud-init status node an-a01n02: running
[2026-04-05T09:31:54.673Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:31:54.673Z] Sun Apr  5 05:31:54 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:32:05.972Z] cloud-init status node an-striker01: running
[2026-04-05T09:32:06.342Z] cloud-init status node an-striker02: running
[2026-04-05T09:32:07.610Z] cloud-init status node an-a01n01: running
[2026-04-05T09:32:08.400Z] cloud-init status node an-a01n02: running
[2026-04-05T09:32:09.188Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:32:09.188Z] Sun Apr  5 05:32:09 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:32:20.492Z] cloud-init status node an-striker01: running
[2026-04-05T09:32:21.280Z] cloud-init status node an-striker02: running
[2026-04-05T09:32:22.069Z] cloud-init status node an-a01n01: running
[2026-04-05T09:32:22.867Z] cloud-init status node an-a01n02: running
[2026-04-05T09:32:23.673Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:32:23.673Z] Sun Apr  5 05:32:23 EDT 2026 actual: 0 expected: 5
[2026-04-05T09:32:34.979Z] cloud-init status node an-striker01: running
[2026-04-05T09:32:35.769Z] cloud-init status node an-striker02: running
[2026-04-05T09:32:37.040Z] cloud-init status node an-a01n01: running
[2026-04-05T09:32:38.308Z] cloud-init status node an-a01n02: done
[2026-04-05T09:32:39.097Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:32:39.097Z] Sun Apr  5 05:32:38 EDT 2026 actual: 1 expected: 5
[2026-04-05T09:32:50.408Z] cloud-init status node an-striker01: running
[2026-04-05T09:32:51.197Z] cloud-init status node an-striker02: running
[2026-04-05T09:32:51.987Z] cloud-init status node an-a01n01: running
[2026-04-05T09:32:52.775Z] cloud-init status node an-a01n02: done
[2026-04-05T09:32:54.042Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:32:54.042Z] Sun Apr  5 05:32:53 EDT 2026 actual: 1 expected: 5
[2026-04-05T09:33:05.375Z] cloud-init status node an-striker01: running
[2026-04-05T09:33:06.165Z] cloud-init status node an-striker02: running
[2026-04-05T09:33:06.958Z] cloud-init status node an-a01n01: done
[2026-04-05T09:33:08.228Z] cloud-init status node an-a01n02: done
[2026-04-05T09:33:09.017Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:33:09.017Z] Sun Apr  5 05:33:08 EDT 2026 actual: 2 expected: 5
[2026-04-05T09:33:20.335Z] cloud-init status node an-striker01: running
[2026-04-05T09:33:21.124Z] cloud-init status node an-striker02: running
[2026-04-05T09:33:22.394Z] cloud-init status node an-a01n01: done
[2026-04-05T09:33:23.197Z] cloud-init status node an-a01n02: done
[2026-04-05T09:33:23.987Z] cloud-init status node an-a01dr01: running
[2026-04-05T09:33:23.987Z] Sun Apr  5 05:33:23 EDT 2026 actual: 2 expected: 5
[2026-04-05T09:33:35.289Z] cloud-init status node an-striker01: running
[2026-04-05T09:33:36.078Z] cloud-init status node an-striker02: running
[2026-04-05T09:33:37.348Z] cloud-init status node an-a01n01: done
[2026-04-05T09:33:38.137Z] cloud-init status node an-a01n02: done
[2026-04-05T09:33:38.927Z] cloud-init status node an-a01dr01: done
[2026-04-05T09:33:38.927Z] Sun Apr  5 05:33:38 EDT 2026 actual: 3 expected: 5
[2026-04-05T09:33:50.235Z] cloud-init status node an-striker01: running
[2026-04-05T09:33:52.692Z] cloud-init status node an-striker02: running
[2026-04-05T09:33:53.482Z] cloud-init status node an-a01n01: done
[2026-04-05T09:33:54.271Z] cloud-init status node an-a01n02: done
[2026-04-05T09:33:55.540Z] cloud-init status node an-a01dr01: done
[2026-04-05T09:33:55.540Z] Sun Apr  5 05:33:55 EDT 2026 actual: 3 expected: 5
[2026-04-05T09:34:06.850Z] cloud-init status node an-striker01: running
[2026-04-05T09:34:07.641Z] cloud-init status node an-striker02: running
[2026-04-05T09:34:08.436Z] cloud-init status node an-a01n01: done
[2026-04-05T09:34:09.702Z] cloud-init status node an-a01n02: done
[2026-04-05T09:34:10.491Z] cloud-init status node an-a01dr01: done
[2026-04-05T09:34:10.491Z] Sun Apr  5 05:34:10 EDT 2026 actual: 3 expected: 5
[2026-04-05T09:34:21.792Z] cloud-init status node an-striker01: running
[2026-04-05T09:34:22.580Z] cloud-init status node an-striker02: running
[2026-04-05T09:34:23.859Z] cloud-init status node an-a01n01: done
[2026-04-05T09:34:24.649Z] cloud-init status node an-a01n02: done
[2026-04-05T09:34:25.438Z] cloud-init status node an-a01dr01: done
[2026-04-05T09:34:25.438Z] Sun Apr  5 05:34:25 EDT 2026 actual: 3 expected: 5
[2026-04-05T09:34:36.741Z] cloud-init status node an-striker01: done
[2026-04-05T09:34:38.008Z] cloud-init status node an-striker02: done
[2026-04-05T09:34:38.796Z] cloud-init status node an-a01n01: done
[2026-04-05T09:34:39.583Z] cloud-init status node an-a01n02: done
[2026-04-05T09:34:40.372Z] cloud-init status node an-a01dr01: done
[2026-04-05T09:34:40.372Z] Sun Apr  5 05:34:40 EDT 2026 actual: 5 expected: 5
[2026-04-05T09:34:40.372Z] Building simengine model
[2026-04-05T09:34:56.797Z] Generating anvil striker auto config
[2026-04-05T09:35:06.127Z] Cloning into 'anvil-test-suite'...
[2026-04-05T09:35:06.127Z] generated ansible config:
[2026-04-05T09:35:06.127Z] =========================
[2026-04-05T09:35:06.127Z] [defaults]
[2026-04-05T09:35:06.127Z] inventory = ansible-inventory
[2026-04-05T09:35:06.127Z] forks = 500
[2026-04-05T09:35:06.127Z] any_errors_fatal = true
[2026-04-05T09:35:06.127Z] 
[2026-04-05T09:35:06.127Z] [ssh_connection]
[2026-04-05T09:35:06.127Z] retries = 20
[2026-04-05T09:35:06.127Z] =========================
[2026-04-05T09:35:06.127Z] 
[2026-04-05T09:35:06.127Z] generated ansible inventory:
[2026-04-05T09:35:06.127Z] ============================
[2026-04-05T09:35:06.127Z] [strikers]
[2026-04-05T09:35:06.127Z] an-striker01
[2026-04-05T09:35:06.127Z] an-striker02
[2026-04-05T09:35:06.127Z] 
[2026-04-05T09:35:06.127Z] [nodes]
[2026-04-05T09:35:06.127Z] an-a01n01
[2026-04-05T09:35:06.127Z] an-a01n02
[2026-04-05T09:35:06.127Z] 
[2026-04-05T09:35:06.127Z] [drnodes]
[2026-04-05T09:35:06.127Z] ============================
[2026-04-05T09:35:06.127Z] 
[2026-04-05T09:35:06.127Z] Checking connectivity to nodes
[2026-04-05T09:35:06.127Z] 
[2026-04-05T09:35:07.947Z] an-a01n01 | SUCCESS => {
[2026-04-05T09:35:07.947Z]     "ansible_facts": {
[2026-04-05T09:35:07.947Z]         "discovered_interpreter_python": "/usr/bin/python3"
[2026-04-05T09:35:07.947Z]     },
[2026-04-05T09:35:07.947Z]     "changed": false,
[2026-04-05T09:35:07.947Z]     "ping": "pong"
[2026-04-05T09:35:07.947Z] }
[2026-04-05T09:35:07.947Z] an-a01n02 | SUCCESS => {
[2026-04-05T09:35:07.947Z]     "ansible_facts": {
[2026-04-05T09:35:07.947Z]         "discovered_interpreter_python": "/usr/bin/python3"
[2026-04-05T09:35:07.947Z]     },
[2026-04-05T09:35:07.947Z]     "changed": false,
[2026-04-05T09:35:07.947Z]     "ping": "pong"
[2026-04-05T09:35:07.947Z] }
[2026-04-05T09:35:07.947Z] an-striker01 | SUCCESS => {
[2026-04-05T09:35:07.947Z]     "ansible_facts": {
[2026-04-05T09:35:07.947Z]         "discovered_interpreter_python": "/usr/bin/python3"
[2026-04-05T09:35:07.947Z]     },
[2026-04-05T09:35:07.947Z]     "changed": false,
[2026-04-05T09:35:07.947Z]     "ping": "pong"
[2026-04-05T09:35:07.947Z] }
[2026-04-05T09:35:07.947Z] an-striker02 | SUCCESS => {
[2026-04-05T09:35:07.947Z]     "ansible_facts": {
[2026-04-05T09:35:07.947Z]         "discovered_interpreter_python": "/usr/bin/python3"
[2026-04-05T09:35:07.947Z]     },
[2026-04-05T09:35:07.947Z]     "changed": false,
[2026-04-05T09:35:07.947Z]     "ping": "pong"
[2026-04-05T09:35:07.947Z] }
[2026-04-05T09:35:08.327Z] 
[2026-04-05T09:35:09.115Z] 
[2026-04-05T09:35:09.115Z] PLAY [Setup all nodes] *********************************************************
[2026-04-05T09:35:09.115Z] 
[2026-04-05T09:35:09.115Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:35:11.571Z] ok: [an-a01n02]
[2026-04-05T09:35:11.571Z] ok: [an-a01n01]
[2026-04-05T09:35:11.571Z] ok: [an-striker02]
[2026-04-05T09:35:11.571Z] ok: [an-striker01]
[2026-04-05T09:35:11.571Z] 
[2026-04-05T09:35:11.571Z] TASK [create /opt/MegaRAID/storcli/] *******************************************
[2026-04-05T09:35:11.940Z] changed: [an-a01n01]
[2026-04-05T09:35:11.940Z] changed: [an-a01n02]
[2026-04-05T09:35:11.940Z] changed: [an-striker01]
[2026-04-05T09:35:11.940Z] changed: [an-striker02]
[2026-04-05T09:35:11.940Z] 
[2026-04-05T09:35:11.940Z] TASK [wipe old storcli64] ******************************************************
[2026-04-05T09:35:12.728Z] ok: [an-a01n01]
[2026-04-05T09:35:12.728Z] ok: [an-a01n02]
[2026-04-05T09:35:12.728Z] ok: [an-striker01]
[2026-04-05T09:35:12.728Z] ok: [an-striker02]
[2026-04-05T09:35:12.728Z] 
[2026-04-05T09:35:12.728Z] TASK [get latest Simengine storcli64] ******************************************
[2026-04-05T09:35:13.995Z] changed: [an-a01n02]
[2026-04-05T09:35:13.995Z] changed: [an-a01n01]
[2026-04-05T09:35:13.995Z] changed: [an-striker02]
[2026-04-05T09:35:13.995Z] changed: [an-striker01]
[2026-04-05T09:35:13.995Z] 
[2026-04-05T09:35:13.995Z] PLAY [setup strikers] **********************************************************
[2026-04-05T09:35:13.995Z] 
[2026-04-05T09:35:13.995Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:35:15.814Z] ok: [an-striker02]
[2026-04-05T09:35:15.814Z] ok: [an-striker01]
[2026-04-05T09:35:15.814Z] 
[2026-04-05T09:35:15.814Z] TASK [copy striker autogenerated config] ***************************************
[2026-04-05T09:35:17.632Z] changed: [an-striker02]
[2026-04-05T09:35:17.632Z] changed: [an-striker01]
[2026-04-05T09:35:17.632Z] 
[2026-04-05T09:35:17.632Z] PLAY [setup lvm storage] *******************************************************
[2026-04-05T09:35:17.632Z] 
[2026-04-05T09:35:17.632Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:35:19.451Z] ok: [an-a01n02]
[2026-04-05T09:35:19.451Z] ok: [an-a01n01]
[2026-04-05T09:35:19.451Z] 
[2026-04-05T09:35:19.451Z] TASK [check if vg already exists] **********************************************
[2026-04-05T09:35:20.239Z] ok: [an-a01n02]
[2026-04-05T09:35:20.239Z] ok: [an-a01n01]
[2026-04-05T09:35:20.239Z] 
[2026-04-05T09:35:20.239Z] TASK [vgcheck] *****************************************************************
[2026-04-05T09:35:20.239Z] skipping: [an-a01n01]
[2026-04-05T09:35:20.239Z] 
[2026-04-05T09:35:20.239Z] TASK [create anvil-test-vg] ****************************************************
[2026-04-05T09:35:21.507Z] changed: [an-a01n02]
[2026-04-05T09:35:21.507Z] changed: [an-a01n01]
[2026-04-05T09:35:21.507Z] 
[2026-04-05T09:35:21.507Z] PLAY [deploy anvil] ************************************************************
[2026-04-05T09:35:21.507Z] 
[2026-04-05T09:35:21.507Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:35:23.326Z] ok: [an-striker01]
[2026-04-05T09:35:23.326Z] ok: [an-striker02]
[2026-04-05T09:35:23.326Z] 
[2026-04-05T09:35:23.326Z] TASK [check if installation is in progress] ************************************
[2026-04-05T09:35:24.118Z] ok: [an-striker01]
[2026-04-05T09:35:24.118Z] ok: [an-striker02]
[2026-04-05T09:35:24.118Z] 
[2026-04-05T09:35:24.118Z] TASK [register installation is in progress] ************************************
[2026-04-05T09:35:25.387Z] changed: [an-striker02]
[2026-04-05T09:35:25.387Z] changed: [an-striker01]
[2026-04-05T09:35:25.387Z] 
[2026-04-05T09:35:25.387Z] TASK [start anvil deployment and configuration] ********************************
[2026-04-05T09:35:28.585Z] changed: [an-striker01]
[2026-04-05T09:35:28.585Z] changed: [an-striker02]
[2026-04-05T09:35:28.585Z] 
[2026-04-05T09:35:28.585Z] TASK [Wait up to 20 minutes for DB to be running on both strikers] *************
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (120 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (120 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (119 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (119 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (118 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (118 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (117 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (117 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (116 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (116 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (115 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (115 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (114 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (114 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (113 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (113 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (112 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (112 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (111 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (111 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (110 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (110 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (109 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (109 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (108 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (108 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (107 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (107 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (106 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (106 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (105 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (105 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (104 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (104 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (103 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (103 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (102 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (102 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (101 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (101 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (100 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (100 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (99 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (99 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (98 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (98 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (97 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (97 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (96 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (95 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (94 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (93 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (96 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (92 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (95 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (91 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (94 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (90 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (93 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (89 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (92 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (88 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (91 retries left).
[2026-04-05T09:41:24.737Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (87 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (90 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (89 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (88 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (87 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (86 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (86 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (85 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (85 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (84 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (84 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (83 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (83 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (82 retries left).
[2026-04-05T09:43:03.846Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (82 retries left).
[2026-04-05T09:43:03.846Z] ok: [an-striker01 -> localhost]
[2026-04-05T09:43:03.846Z] ok: [an-striker02 -> localhost]
[2026-04-05T09:43:03.846Z] 
[2026-04-05T09:43:03.846Z] TASK [Wait up to 20 minutes for both striker-db-status to sync] ****************
[2026-04-05T09:43:23.624Z] FAILED - RETRYING: [an-striker02]: Wait up to 20 minutes for both striker-db-status to sync (120 retries left).
[2026-04-05T09:43:23.624Z] ok: [an-striker01]
[2026-04-05T09:43:23.624Z] FAILED - RETRYING: [an-striker02]: Wait up to 20 minutes for both striker-db-status to sync (119 retries left).
[2026-04-05T09:43:23.624Z] ok: [an-striker02]
[2026-04-05T09:43:23.624Z] 
[2026-04-05T09:43:23.624Z] TASK [wait up to 20 minutes for nodes to join the database] ********************
[2026-04-05T09:49:04.711Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (120 retries left).
[2026-04-05T09:49:04.711Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (120 retries left).
[2026-04-05T09:49:04.711Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (119 retries left).
[2026-04-05T09:49:04.711Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (119 retries left).
[2026-04-05T09:49:04.711Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (118 retries left).
[2026-04-05T09:49:04.711Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (118 retries left).
[2026-04-05T09:49:04.711Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (117 retries left).
[2026-04-05T09:49:04.711Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (117 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (116 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (116 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (115 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (115 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (114 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (114 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (113 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (113 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (112 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (112 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (111 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (111 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (110 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (110 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (109 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (109 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (108 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (108 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (107 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (107 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (106 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (106 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (105 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (105 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (104 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (104 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (103 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (103 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (102 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (102 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (101 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (101 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (100 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (100 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (99 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (99 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (98 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (98 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (97 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (97 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (96 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (96 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (95 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (95 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (94 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (94 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (93 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (93 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (92 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (92 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (91 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (91 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (90 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (90 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (89 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (89 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (88 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (88 retries left).
[2026-04-05T09:49:04.712Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (87 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (87 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (86 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (86 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (85 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (85 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (84 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (84 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (83 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (83 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (82 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (82 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (81 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (81 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (80 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (80 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (79 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (79 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (78 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (78 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (77 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (77 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (76 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (76 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (75 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (75 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (74 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (74 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (73 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (73 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (72 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (72 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (71 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (71 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (70 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (70 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (69 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (69 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (68 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (68 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (67 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (67 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (66 retries left).
[2026-04-05T09:52:59.889Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (66 retries left).
[2026-04-05T09:52:59.889Z] ok: [an-striker02 -> localhost]
[2026-04-05T09:52:59.889Z] ok: [an-striker01 -> localhost]
[2026-04-05T09:52:59.889Z] 
[2026-04-05T09:52:59.889Z] TASK [wait up to 20 minutes for anvil nodes to form] ***************************
[2026-04-05T09:52:59.889Z] ok: [an-striker01 -> localhost]
[2026-04-05T09:52:59.889Z] ok: [an-striker02 -> localhost]
[2026-04-05T09:52:59.889Z] 
[2026-04-05T09:52:59.889Z] TASK [wait up to 60 minutes for pacemaker to be configured] ********************
[2026-04-05T09:54:23.911Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (360 retries left).
[2026-04-05T09:54:23.911Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (360 retries left).
[2026-04-05T09:54:23.911Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (359 retries left).
[2026-04-05T09:54:23.911Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (359 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (358 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (358 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (357 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (357 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (356 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (356 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (355 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (355 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (354 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (354 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (353 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (353 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (352 retries left).
[2026-04-05T09:54:23.912Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (352 retries left).
[2026-04-05T09:54:23.912Z] ok: [an-striker01 -> localhost]
[2026-04-05T09:54:23.912Z] ok: [an-striker02 -> localhost]
[2026-04-05T09:54:23.912Z] 
[2026-04-05T09:54:23.912Z] TASK [wait up to 20 minutes for drbd to be configured] *************************
[2026-04-05T09:54:43.663Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for drbd to be configured (120 retries left).
[2026-04-05T09:54:43.663Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for drbd to be configured (120 retries left).
[2026-04-05T09:54:43.663Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for drbd to be configured (119 retries left).
[2026-04-05T09:54:43.663Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for drbd to be configured (119 retries left).
[2026-04-05T09:54:43.663Z] ok: [an-striker02 -> localhost]
[2026-04-05T09:54:43.663Z] ok: [an-striker01 -> localhost]
[2026-04-05T09:54:43.663Z] 
[2026-04-05T09:54:43.663Z] TASK [wait up to 20 minutes for any other configuration job to complete] *******
[2026-04-05T09:54:43.663Z] ok: [an-striker01 -> localhost]
[2026-04-05T09:54:43.663Z] ok: [an-striker02 -> localhost]
[2026-04-05T09:54:43.663Z] 
[2026-04-05T09:54:43.663Z] PLAY [check network status after deployment] ***********************************
[2026-04-05T09:54:43.663Z] 
[2026-04-05T09:54:43.663Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:54:46.123Z] ok: [an-a01n02]
[2026-04-05T09:54:46.123Z] ok: [an-a01n01]
[2026-04-05T09:54:46.123Z] ok: [an-striker02]
[2026-04-05T09:54:46.123Z] ok: [an-striker01]
[2026-04-05T09:54:46.123Z] 
[2026-04-05T09:54:46.123Z] TASK [check network manager autoconnect is set to yes] *************************
[2026-04-05T09:54:46.123Z] skipping: [an-striker01] => (item=lo) 
[2026-04-05T09:54:53.796Z] changed: [an-a01n01] => (item=sn1_bond1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n02] => (item=ifn1_bridge1)
[2026-04-05T09:54:53.796Z] changed: [an-striker02] => (item=bcn1_link2)
[2026-04-05T09:54:53.796Z] changed: [an-striker01] => (item=ifn1_link1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n01] => (item=bcn1_link2)
[2026-04-05T09:54:53.796Z] changed: [an-a01n02] => (item=ifn1_bond1)
[2026-04-05T09:54:53.796Z] changed: [an-striker01] => (item=ifn1_bond1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n01] => (item=sn1_link2)
[2026-04-05T09:54:53.796Z] changed: [an-a01n02] => (item=sn1_link2)
[2026-04-05T09:54:53.796Z] changed: [an-striker02] => (item=bcn1_bond1)
[2026-04-05T09:54:53.796Z] skipping: [an-a01n01] => (item=lo) 
[2026-04-05T09:54:53.796Z] changed: [an-a01n01] => (item=ifn1_link1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n02] => (item=ifn1_link1)
[2026-04-05T09:54:53.796Z] changed: [an-striker02] => (item=bcn1_link1)
[2026-04-05T09:54:53.796Z] changed: [an-striker01] => (item=bcn1_link2)
[2026-04-05T09:54:53.796Z] changed: [an-a01n01] => (item=bcn1_link1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n02] => (item=bcn1_link1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n01] => (item=sn1_link1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n02] => (item=bcn1_bridge1)
[2026-04-05T09:54:53.796Z] skipping: [an-a01n02] => (item=lo) 
[2026-04-05T09:54:53.796Z] changed: [an-striker02] => (item=ifn1_link2)
[2026-04-05T09:54:53.796Z] changed: [an-striker01] => (item=ifn1_link2)
[2026-04-05T09:54:53.796Z] changed: [an-a01n01] => (item=bcn1_bridge1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n02] => (item=bcn1_bond1)
[2026-04-05T09:54:53.796Z] changed: [an-striker02] => (item=ifn1_link1)
[2026-04-05T09:54:53.796Z] skipping: [an-striker02] => (item=lo) 
[2026-04-05T09:54:53.796Z] changed: [an-striker01] => (item=bcn1_link1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n01] => (item=ifn1_bridge1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n02] => (item=sn1_bond1)
[2026-04-05T09:54:53.796Z] changed: [an-striker02] => (item=ifn1_bond1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n01] => (item=ifn1_link2)
[2026-04-05T09:54:53.796Z] changed: [an-a01n02] => (item=sn1_link1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n01] => (item=ifn1_bond1)
[2026-04-05T09:54:53.796Z] changed: [an-striker01] => (item=bcn1_bond1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n02] => (item=bcn1_link2)
[2026-04-05T09:54:53.796Z] changed: [an-a01n01] => (item=bcn1_bond1)
[2026-04-05T09:54:53.796Z] changed: [an-a01n02] => (item=ifn1_link2)
[2026-04-05T09:54:53.796Z] 
[2026-04-05T09:54:53.796Z] PLAY [check pacemaker status after deployment] *********************************
[2026-04-05T09:54:53.796Z] 
[2026-04-05T09:54:53.796Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:54:55.064Z] ok: [an-a01n02]
[2026-04-05T09:54:55.064Z] ok: [an-a01n01]
[2026-04-05T09:54:55.064Z] 
[2026-04-05T09:54:55.064Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T09:54:55.434Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T09:54:55.434Z] 
[2026-04-05T09:54:55.434Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T09:54:59.498Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T09:54:59.498Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T09:54:59.498Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T09:54:59.498Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T09:54:59.498Z] 
[2026-04-05T09:54:59.498Z] PLAY [check drbd kmod] *********************************************************
[2026-04-05T09:54:59.498Z] 
[2026-04-05T09:54:59.498Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:55:00.768Z] ok: [an-a01n01]
[2026-04-05T09:55:00.768Z] ok: [an-a01n02]
[2026-04-05T09:55:00.768Z] 
[2026-04-05T09:55:00.768Z] TASK [check if drbd kernel module is already loaded] ***************************
[2026-04-05T09:55:01.137Z] ok: [an-a01n01]
[2026-04-05T09:55:01.137Z] ok: [an-a01n02]
[2026-04-05T09:55:01.137Z] 
[2026-04-05T09:55:01.137Z] TASK [load drbd kernel module] *************************************************
[2026-04-05T09:55:01.137Z] skipping: [an-a01n01]
[2026-04-05T09:55:01.137Z] skipping: [an-a01n02]
[2026-04-05T09:55:01.137Z] 
[2026-04-05T09:55:01.137Z] TASK [check if drbd kernel module is loaded] ***********************************
[2026-04-05T09:55:01.506Z] skipping: [an-a01n01]
[2026-04-05T09:55:01.506Z] skipping: [an-a01n02]
[2026-04-05T09:55:01.506Z] 
[2026-04-05T09:55:01.506Z] PLAY [Copy iso images to striker01] ********************************************
[2026-04-05T09:55:01.506Z] 
[2026-04-05T09:55:01.506Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:55:03.332Z] ok: [an-striker01]
[2026-04-05T09:55:03.332Z] 
[2026-04-05T09:55:03.332Z] TASK [Wipe old snapshots dir] **************************************************
[2026-04-05T09:55:04.599Z] ok: [an-striker01]
[2026-04-05T09:55:04.599Z] 
[2026-04-05T09:55:04.599Z] TASK [Create snapshots dir] ****************************************************
[2026-04-05T09:55:05.389Z] changed: [an-striker01]
[2026-04-05T09:55:05.389Z] 
[2026-04-05T09:55:05.389Z] TASK [Copy C9S kickstart images] ***********************************************
[2026-04-05T09:55:16.700Z] changed: [an-striker01] => (item=deploy1.iso)
[2026-04-05T09:55:16.700Z] changed: [an-striker01] => (item=deploy2.iso)
[2026-04-05T09:55:16.700Z] changed: [an-striker01] => (item=deploy3.iso)
[2026-04-05T09:55:16.700Z] changed: [an-striker01] => (item=deploy4.iso)
[2026-04-05T09:55:16.700Z] changed: [an-striker01] => (item=deploy5.iso)
[2026-04-05T09:55:16.700Z] 
[2026-04-05T09:55:16.700Z] TASK [Move install isos to anvil upload dir] ***********************************
[2026-04-05T09:55:17.969Z] changed: [an-striker01]
[2026-04-05T09:55:17.969Z] 
[2026-04-05T09:55:17.969Z] TASK [Wipe old snapshots dir] **************************************************
[2026-04-05T09:55:18.758Z] changed: [an-striker01]
[2026-04-05T09:55:18.758Z] 
[2026-04-05T09:55:18.758Z] PLAY [Check iso images propagation] ********************************************
[2026-04-05T09:55:18.758Z] 
[2026-04-05T09:55:18.758Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:55:21.216Z] ok: [an-a01n01]
[2026-04-05T09:55:21.216Z] ok: [an-a01n02]
[2026-04-05T09:55:21.216Z] ok: [an-striker02]
[2026-04-05T09:55:21.216Z] ok: [an-striker01]
[2026-04-05T09:55:21.216Z] 
[2026-04-05T09:55:21.216Z] TASK [wait up to 30 minutes for all images to be propagated to all nodes] ******
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (180 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (180 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (180 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (180 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (179 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (179 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (179 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (179 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (178 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (178 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (178 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (178 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (177 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (177 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (177 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (177 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (176 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (176 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (176 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (176 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (175 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (175 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (175 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (175 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (174 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (174 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (174 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (174 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (173 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (173 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (173 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (173 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (172 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (172 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (172 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (172 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (171 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (171 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (171 retries left).
[2026-04-05T09:57:15.441Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (171 retries left).
[2026-04-05T09:57:15.441Z] ok: [an-a01n02]
[2026-04-05T09:57:15.441Z] ok: [an-a01n01]
[2026-04-05T09:57:15.441Z] ok: [an-striker02]
[2026-04-05T09:57:15.441Z] ok: [an-striker01]
[2026-04-05T09:57:15.441Z] 
[2026-04-05T09:57:15.441Z] PLAY [example tests] ***********************************************************
[2026-04-05T09:57:15.441Z] 
[2026-04-05T09:57:15.441Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:57:15.441Z] ok: [an-a01n01]
[2026-04-05T09:57:15.441Z] ok: [an-a01n02]
[2026-04-05T09:57:15.441Z] ok: [an-striker02]
[2026-04-05T09:57:15.441Z] ok: [an-striker01]
[2026-04-05T09:57:15.441Z] 
[2026-04-05T09:57:15.441Z] TASK [this test should be executed only on strikers] ***************************
[2026-04-05T09:57:15.441Z] skipping: [an-a01n01]
[2026-04-05T09:57:15.441Z] skipping: [an-a01n02]
[2026-04-05T09:57:15.441Z] ok: [an-striker02]
[2026-04-05T09:57:15.441Z] ok: [an-striker01]
[2026-04-05T09:57:15.441Z] 
[2026-04-05T09:57:15.441Z] TASK [this test should be executed only on non strikers] ***********************
[2026-04-05T09:57:15.441Z] skipping: [an-striker01]
[2026-04-05T09:57:15.441Z] skipping: [an-striker02]
[2026-04-05T09:57:15.441Z] ok: [an-a01n02]
[2026-04-05T09:57:15.441Z] ok: [an-a01n01]
[2026-04-05T09:57:15.441Z] 
[2026-04-05T09:57:15.441Z] TASK [inject failure] **********************************************************
[2026-04-05T09:57:15.441Z] skipping: [an-striker01]
[2026-04-05T09:57:15.441Z] skipping: [an-striker02]
[2026-04-05T09:57:15.441Z] skipping: [an-a01n01]
[2026-04-05T09:57:15.441Z] skipping: [an-a01n02]
[2026-04-05T09:57:15.441Z] 
[2026-04-05T09:57:15.441Z] PLAY [Record anvil available resources from striker01] *************************
[2026-04-05T09:57:15.441Z] 
[2026-04-05T09:57:15.441Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:57:16.714Z] ok: [an-striker01]
[2026-04-05T09:57:16.714Z] 
[2026-04-05T09:57:16.714Z] TASK [Record anvil available resources] ****************************************
[2026-04-05T09:57:20.776Z] ok: [an-striker01]
[2026-04-05T09:57:20.776Z] 
[2026-04-05T09:57:20.776Z] PLAY [check selinux status] ****************************************************
[2026-04-05T09:57:20.776Z] 
[2026-04-05T09:57:20.776Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:57:23.233Z] ok: [an-a01n02]
[2026-04-05T09:57:23.233Z] ok: [an-a01n01]
[2026-04-05T09:57:23.233Z] ok: [an-striker02]
[2026-04-05T09:57:23.233Z] ok: [an-striker01]
[2026-04-05T09:57:23.233Z] 
[2026-04-05T09:57:23.233Z] TASK [check selinux status permissive (8)] *************************************
[2026-04-05T09:57:23.603Z] skipping: [an-striker01]
[2026-04-05T09:57:23.603Z] skipping: [an-striker02]
[2026-04-05T09:57:23.603Z] skipping: [an-a01n01]
[2026-04-05T09:57:23.603Z] skipping: [an-a01n02]
[2026-04-05T09:57:23.603Z] 
[2026-04-05T09:57:23.603Z] TASK [check selinux status permissive (9)] *************************************
[2026-04-05T09:57:23.973Z] ok: [an-striker01] => {
[2026-04-05T09:57:23.973Z]     "ansible_selinux": {
[2026-04-05T09:57:23.973Z]         "config_mode": "enforcing",
[2026-04-05T09:57:23.973Z]         "mode": "enforcing",
[2026-04-05T09:57:23.973Z]         "policyvers": 33,
[2026-04-05T09:57:23.973Z]         "status": "enabled",
[2026-04-05T09:57:23.973Z]         "type": "targeted"
[2026-04-05T09:57:23.973Z]     },
[2026-04-05T09:57:23.973Z]     "failed_when_result": false
[2026-04-05T09:57:23.973Z] }
[2026-04-05T09:57:23.973Z] ok: [an-striker02] => {
[2026-04-05T09:57:23.973Z]     "ansible_selinux": {
[2026-04-05T09:57:23.973Z]         "config_mode": "enforcing",
[2026-04-05T09:57:23.973Z]         "mode": "enforcing",
[2026-04-05T09:57:23.973Z]         "policyvers": 33,
[2026-04-05T09:57:23.973Z]         "status": "enabled",
[2026-04-05T09:57:23.973Z]         "type": "targeted"
[2026-04-05T09:57:23.973Z]     },
[2026-04-05T09:57:23.973Z]     "failed_when_result": false
[2026-04-05T09:57:23.973Z] }
[2026-04-05T09:57:23.973Z] ok: [an-a01n01] => {
[2026-04-05T09:57:23.973Z]     "ansible_selinux": {
[2026-04-05T09:57:23.973Z]         "config_mode": "enforcing",
[2026-04-05T09:57:23.973Z]         "mode": "enforcing",
[2026-04-05T09:57:23.973Z]         "policyvers": 33,
[2026-04-05T09:57:23.973Z]         "status": "enabled",
[2026-04-05T09:57:23.973Z]         "type": "targeted"
[2026-04-05T09:57:23.973Z]     },
[2026-04-05T09:57:23.973Z]     "failed_when_result": false
[2026-04-05T09:57:23.973Z] }
[2026-04-05T09:57:23.973Z] ok: [an-a01n02] => {
[2026-04-05T09:57:23.973Z]     "ansible_selinux": {
[2026-04-05T09:57:23.973Z]         "config_mode": "enforcing",
[2026-04-05T09:57:23.973Z]         "mode": "enforcing",
[2026-04-05T09:57:23.973Z]         "policyvers": 33,
[2026-04-05T09:57:23.973Z]         "status": "enabled",
[2026-04-05T09:57:23.973Z]         "type": "targeted"
[2026-04-05T09:57:23.973Z]     },
[2026-04-05T09:57:23.973Z]     "failed_when_result": false
[2026-04-05T09:57:23.973Z] }
[2026-04-05T09:57:23.973Z] 
[2026-04-05T09:57:23.973Z] PLAY [Create random file on striker01] *****************************************
[2026-04-05T09:57:23.973Z] 
[2026-04-05T09:57:23.973Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:57:25.793Z] ok: [an-striker01]
[2026-04-05T09:57:25.793Z] 
[2026-04-05T09:57:25.793Z] TASK [Create archive file] *****************************************************
[2026-04-05T09:57:27.611Z] changed: [an-striker01]
[2026-04-05T09:57:27.611Z] 
[2026-04-05T09:57:27.611Z] PLAY [Check archive file propagation] ******************************************
[2026-04-05T09:57:27.611Z] 
[2026-04-05T09:57:27.611Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:57:29.433Z] ok: [an-a01n01]
[2026-04-05T09:57:29.433Z] ok: [an-a01n02]
[2026-04-05T09:57:29.433Z] ok: [an-striker02]
[2026-04-05T09:57:29.433Z] ok: [an-striker01]
[2026-04-05T09:57:29.433Z] 
[2026-04-05T09:57:29.433Z] TASK [wait up to 30 minutes for archive to be propagated to all nodes] *********
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for archive to be propagated to all nodes (180 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for archive to be propagated to all nodes (180 retries left).
[2026-04-05T09:58:53.444Z] ok: [an-striker01]
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for archive to be propagated to all nodes (180 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for archive to be propagated to all nodes (179 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for archive to be propagated to all nodes (179 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for archive to be propagated to all nodes (179 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for archive to be propagated to all nodes (178 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for archive to be propagated to all nodes (178 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for archive to be propagated to all nodes (178 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for archive to be propagated to all nodes (177 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for archive to be propagated to all nodes (177 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for archive to be propagated to all nodes (177 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for archive to be propagated to all nodes (176 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for archive to be propagated to all nodes (176 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for archive to be propagated to all nodes (176 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for archive to be propagated to all nodes (175 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for archive to be propagated to all nodes (175 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for archive to be propagated to all nodes (175 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for archive to be propagated to all nodes (174 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for archive to be propagated to all nodes (174 retries left).
[2026-04-05T09:58:53.444Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for archive to be propagated to all nodes (174 retries left).
[2026-04-05T09:58:53.444Z] ok: [an-a01n02]
[2026-04-05T09:58:53.444Z] ok: [an-a01n01]
[2026-04-05T09:58:53.444Z] ok: [an-striker02]
[2026-04-05T09:58:53.444Z] 
[2026-04-05T09:58:53.444Z] PLAY [test fencing] ************************************************************
[2026-04-05T09:58:53.444Z] 
[2026-04-05T09:58:53.444Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T09:58:53.444Z] ok: [an-a01n02]
[2026-04-05T09:58:53.444Z] ok: [an-a01n01]
[2026-04-05T09:58:53.444Z] 
[2026-04-05T09:58:53.444Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T09:58:53.444Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T09:58:53.444Z] 
[2026-04-05T09:58:53.444Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T09:58:53.444Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T09:58:53.444Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T09:58:53.444Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T09:58:53.444Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T09:58:53.444Z] 
[2026-04-05T09:58:53.444Z] TASK [kill node "an-a01n01"] ***************************************************
[2026-04-05T09:58:53.444Z] included: /srv/anvil/anvil-test-suite/playbooks/common/kill_node.yml for an-a01n01, an-a01n02
[2026-04-05T09:58:53.444Z] 
[2026-04-05T09:58:53.444Z] TASK [kill node "an-a01n01"] ***************************************************
[2026-04-05T09:58:53.444Z] skipping: [an-a01n02]
[2026-04-05T09:58:53.444Z] changed: [an-a01n01]
[2026-04-05T09:58:53.444Z] 
[2026-04-05T09:58:53.444Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-05T09:58:53.444Z] skipping: [an-a01n02]
[2026-04-05T09:58:53.444Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01
[2026-04-05T09:58:53.444Z] 
[2026-04-05T09:58:53.444Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-05T10:02:07.604Z] ok: [an-a01n01 -> localhost]
[2026-04-05T10:02:07.604Z] 
[2026-04-05T10:02:07.604Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T10:02:07.604Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T10:02:07.604Z] 
[2026-04-05T10:02:07.604Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T10:02:07.604Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T10:02:07.604Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T10:02:07.604Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T10:02:07.604Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T10:02:07.604Z] 
[2026-04-05T10:02:07.604Z] TASK [kill node "an-a01n02"] ***************************************************
[2026-04-05T10:02:07.604Z] included: /srv/anvil/anvil-test-suite/playbooks/common/kill_node.yml for an-a01n01, an-a01n02
[2026-04-05T10:02:07.604Z] 
[2026-04-05T10:02:07.604Z] TASK [kill node "an-a01n02"] ***************************************************
[2026-04-05T10:02:08.121Z] skipping: [an-a01n01]
[2026-04-05T10:02:08.121Z] changed: [an-a01n02]
[2026-04-05T10:02:08.121Z] 
[2026-04-05T10:02:08.121Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-05T10:02:08.121Z] skipping: [an-a01n01]
[2026-04-05T10:02:08.121Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n02
[2026-04-05T10:02:08.121Z] 
[2026-04-05T10:02:08.121Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-05T10:05:18.728Z] ok: [an-a01n02 -> localhost]
[2026-04-05T10:05:18.728Z] 
[2026-04-05T10:05:18.728Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T10:05:18.728Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T10:05:18.728Z] 
[2026-04-05T10:05:18.728Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T10:05:22.791Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T10:05:22.791Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T10:05:22.791Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T10:05:22.791Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T10:05:22.791Z] 
[2026-04-05T10:05:22.791Z] PLAY [advanced server tests] ***************************************************
[2026-04-05T10:05:22.791Z] 
[2026-04-05T10:05:22.791Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T10:05:24.613Z] ok: [an-a01n01]
[2026-04-05T10:05:24.613Z] ok: [an-a01n02]
[2026-04-05T10:05:24.613Z] 
[2026-04-05T10:05:24.613Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T10:05:24.613Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T10:05:24.613Z] 
[2026-04-05T10:05:24.613Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T10:05:28.679Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T10:05:28.679Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T10:05:28.679Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T10:05:28.679Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T10:05:28.679Z] 
[2026-04-05T10:05:28.679Z] TASK [check anvil-provision-server storage group] ******************************
[2026-04-05T10:05:28.679Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_storage_group.yml for an-a01n01, an-a01n02
[2026-04-05T10:05:28.679Z] 
[2026-04-05T10:05:28.679Z] TASK [get anvil-provision-server storage group] ********************************
[2026-04-05T10:05:33.759Z] changed: [an-a01n01]
[2026-04-05T10:05:33.759Z] changed: [an-a01n02]
[2026-04-05T10:05:33.759Z] 
[2026-04-05T10:05:33.759Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-05T10:05:34.129Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_install_iso.yml for an-a01n01, an-a01n02
[2026-04-05T10:05:34.129Z] 
[2026-04-05T10:05:34.129Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-05T10:05:39.208Z] changed: [an-a01n02]
[2026-04-05T10:05:39.208Z] changed: [an-a01n01]
[2026-04-05T10:05:39.208Z] 
[2026-04-05T10:05:39.208Z] TASK [deploy servers] **********************************************************
[2026-04-05T10:05:39.208Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-05T10:05:39.208Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-05T10:05:39.208Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-05T10:05:39.208Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-05T10:05:39.208Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-05T10:05:39.208Z] 
[2026-04-05T10:05:39.208Z] TASK [check if server an-test-deploy1 exists] **********************************
[2026-04-05T10:05:40.476Z] skipping: [an-a01n02]
[2026-04-05T10:05:40.476Z] fatal: [an-a01n01]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy1\n", "delta": "0:00:00.861155", "end": "2026-04-05 06:05:40.107466", "msg": "non-zero return code", "rc": 1, "start": "2026-04-05 06:05:39.246311", "stderr": "Error: resource or tag id 'an-test-deploy1' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy1' not found"], "stdout": "", "stdout_lines": []}
[2026-04-05T10:05:40.476Z] ...ignoring
[2026-04-05T10:05:40.476Z] 
[2026-04-05T10:05:40.476Z] TASK [deploy server an-test-deploy1 from an-a01n01 on storage 70316199-07b5-4a00-a81f-d24be730e79c] ***
[2026-04-05T10:05:49.815Z] skipping: [an-a01n02]
[2026-04-05T10:05:49.815Z] changed: [an-a01n01]
[2026-04-05T10:05:49.815Z] 
[2026-04-05T10:05:49.815Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to be started in pacemaker] ***
[2026-04-05T10:05:49.815Z] skipping: [an-a01n01]
[2026-04-05T10:05:49.815Z] skipping: [an-a01n02]
[2026-04-05T10:05:49.815Z] 
[2026-04-05T10:05:49.815Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} drbd storage to be synced] ***
[2026-04-05T10:05:49.815Z] skipping: [an-a01n01]
[2026-04-05T10:05:49.815Z] skipping: [an-a01n02]
[2026-04-05T10:05:49.815Z] 
[2026-04-05T10:05:49.815Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to start installation] ***
[2026-04-05T10:05:49.815Z] skipping: [an-a01n01]
[2026-04-05T10:05:49.815Z] skipping: [an-a01n02]
[2026-04-05T10:05:49.815Z] 
[2026-04-05T10:05:49.815Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to complete installation] ***
[2026-04-05T10:05:49.815Z] skipping: [an-a01n01]
[2026-04-05T10:05:49.815Z] skipping: [an-a01n02]
[2026-04-05T10:05:49.815Z] 
[2026-04-05T10:05:49.815Z] TASK [check if server an-test-deploy2 exists] **********************************
[2026-04-05T10:05:50.606Z] skipping: [an-a01n02]
[2026-04-05T10:05:50.606Z] fatal: [an-a01n01]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy2\n", "delta": "0:00:00.902274", "end": "2026-04-05 06:05:50.261310", "msg": "non-zero return code", "rc": 1, "start": "2026-04-05 06:05:49.359036", "stderr": "Error: resource or tag id 'an-test-deploy2' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy2' not found"], "stdout": "", "stdout_lines": []}
[2026-04-05T10:05:50.606Z] ...ignoring
[2026-04-05T10:05:50.606Z] 
[2026-04-05T10:05:50.606Z] TASK [deploy server an-test-deploy2 from an-a01n01 on storage 70316199-07b5-4a00-a81f-d24be730e79c] ***
[2026-04-05T10:05:59.967Z] skipping: [an-a01n02]
[2026-04-05T10:05:59.967Z] changed: [an-a01n01]
[2026-04-05T10:05:59.967Z] 
[2026-04-05T10:05:59.967Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to be started in pacemaker] ***
[2026-04-05T10:05:59.967Z] skipping: [an-a01n01]
[2026-04-05T10:05:59.967Z] skipping: [an-a01n02]
[2026-04-05T10:05:59.967Z] 
[2026-04-05T10:05:59.967Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} drbd storage to be synced] ***
[2026-04-05T10:05:59.967Z] skipping: [an-a01n01]
[2026-04-05T10:05:59.967Z] skipping: [an-a01n02]
[2026-04-05T10:05:59.967Z] 
[2026-04-05T10:05:59.967Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to start installation] ***
[2026-04-05T10:05:59.967Z] skipping: [an-a01n01]
[2026-04-05T10:05:59.967Z] skipping: [an-a01n02]
[2026-04-05T10:05:59.967Z] 
[2026-04-05T10:05:59.967Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to complete installation] ***
[2026-04-05T10:05:59.967Z] skipping: [an-a01n01]
[2026-04-05T10:05:59.967Z] skipping: [an-a01n02]
[2026-04-05T10:05:59.967Z] 
[2026-04-05T10:05:59.967Z] TASK [check if server an-test-deploy3 exists] **********************************
[2026-04-05T10:06:00.828Z] skipping: [an-a01n02]
[2026-04-05T10:06:00.828Z] fatal: [an-a01n01]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy3\n", "delta": "0:00:00.882894", "end": "2026-04-05 06:06:00.638331", "msg": "non-zero return code", "rc": 1, "start": "2026-04-05 06:05:59.755437", "stderr": "Error: resource or tag id 'an-test-deploy3' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy3' not found"], "stdout": "", "stdout_lines": []}
[2026-04-05T10:06:00.828Z] ...ignoring
[2026-04-05T10:06:00.828Z] 
[2026-04-05T10:06:00.828Z] TASK [deploy server an-test-deploy3 from an-a01n01 on storage 70316199-07b5-4a00-a81f-d24be730e79c] ***
[2026-04-05T10:06:09.095Z] skipping: [an-a01n02]
[2026-04-05T10:06:09.095Z] changed: [an-a01n01]
[2026-04-05T10:06:09.095Z] 
[2026-04-05T10:06:09.095Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to be started in pacemaker] ***
[2026-04-05T10:06:09.095Z] skipping: [an-a01n01]
[2026-04-05T10:06:09.095Z] skipping: [an-a01n02]
[2026-04-05T10:06:09.095Z] 
[2026-04-05T10:06:09.095Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} drbd storage to be synced] ***
[2026-04-05T10:06:09.095Z] skipping: [an-a01n01]
[2026-04-05T10:06:09.095Z] skipping: [an-a01n02]
[2026-04-05T10:06:09.095Z] 
[2026-04-05T10:06:09.095Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to start installation] ***
[2026-04-05T10:06:09.467Z] skipping: [an-a01n01]
[2026-04-05T10:06:09.467Z] skipping: [an-a01n02]
[2026-04-05T10:06:09.467Z] 
[2026-04-05T10:06:09.467Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to complete installation] ***
[2026-04-05T10:06:09.467Z] skipping: [an-a01n01]
[2026-04-05T10:06:09.467Z] skipping: [an-a01n02]
[2026-04-05T10:06:09.467Z] 
[2026-04-05T10:06:09.467Z] TASK [check if server an-test-deploy4 exists] **********************************
[2026-04-05T10:06:10.742Z] skipping: [an-a01n02]
[2026-04-05T10:06:10.742Z] fatal: [an-a01n01]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy4\n", "delta": "0:00:00.831620", "end": "2026-04-05 06:06:10.506423", "msg": "non-zero return code", "rc": 1, "start": "2026-04-05 06:06:09.674803", "stderr": "Error: resource or tag id 'an-test-deploy4' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy4' not found"], "stdout": "", "stdout_lines": []}
[2026-04-05T10:06:10.742Z] ...ignoring
[2026-04-05T10:06:10.742Z] 
[2026-04-05T10:06:10.742Z] TASK [deploy server an-test-deploy4 from an-a01n01 on storage 70316199-07b5-4a00-a81f-d24be730e79c] ***
[2026-04-05T10:06:20.379Z] skipping: [an-a01n02]
[2026-04-05T10:06:20.379Z] changed: [an-a01n01]
[2026-04-05T10:06:20.379Z] 
[2026-04-05T10:06:20.379Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to be started in pacemaker] ***
[2026-04-05T10:06:20.379Z] skipping: [an-a01n01]
[2026-04-05T10:06:20.379Z] skipping: [an-a01n02]
[2026-04-05T10:06:20.379Z] 
[2026-04-05T10:06:20.379Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} drbd storage to be synced] ***
[2026-04-05T10:06:20.379Z] skipping: [an-a01n01]
[2026-04-05T10:06:20.379Z] skipping: [an-a01n02]
[2026-04-05T10:06:20.379Z] 
[2026-04-05T10:06:20.379Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to start installation] ***
[2026-04-05T10:06:20.379Z] skipping: [an-a01n01]
[2026-04-05T10:06:20.379Z] skipping: [an-a01n02]
[2026-04-05T10:06:20.379Z] 
[2026-04-05T10:06:20.379Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to complete installation] ***
[2026-04-05T10:06:20.379Z] skipping: [an-a01n01]
[2026-04-05T10:06:20.379Z] skipping: [an-a01n02]
[2026-04-05T10:06:20.379Z] 
[2026-04-05T10:06:20.379Z] TASK [check if server an-test-deploy5 exists] **********************************
[2026-04-05T10:06:21.177Z] skipping: [an-a01n02]
[2026-04-05T10:06:21.177Z] fatal: [an-a01n01]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy5\n", "delta": "0:00:00.870801", "end": "2026-04-05 06:06:20.846641", "msg": "non-zero return code", "rc": 1, "start": "2026-04-05 06:06:19.975840", "stderr": "Error: resource or tag id 'an-test-deploy5' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy5' not found"], "stdout": "", "stdout_lines": []}
[2026-04-05T10:06:21.177Z] ...ignoring
[2026-04-05T10:06:21.177Z] 
[2026-04-05T10:06:21.177Z] TASK [deploy server an-test-deploy5 from an-a01n01 on storage 70316199-07b5-4a00-a81f-d24be730e79c] ***
[2026-04-05T10:06:32.601Z] skipping: [an-a01n02]
[2026-04-05T10:06:32.601Z] changed: [an-a01n01]
[2026-04-05T10:06:32.601Z] 
[2026-04-05T10:06:32.601Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to be started in pacemaker] ***
[2026-04-05T10:06:32.601Z] skipping: [an-a01n01]
[2026-04-05T10:06:32.601Z] skipping: [an-a01n02]
[2026-04-05T10:06:32.601Z] 
[2026-04-05T10:06:32.601Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} drbd storage to be synced] ***
[2026-04-05T10:06:32.601Z] skipping: [an-a01n01]
[2026-04-05T10:06:32.601Z] skipping: [an-a01n02]
[2026-04-05T10:06:32.601Z] 
[2026-04-05T10:06:32.601Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to start installation] ***
[2026-04-05T10:06:32.601Z] skipping: [an-a01n01]
[2026-04-05T10:06:32.601Z] skipping: [an-a01n02]
[2026-04-05T10:06:32.601Z] 
[2026-04-05T10:06:32.601Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to complete installation] ***
[2026-04-05T10:06:32.601Z] skipping: [an-a01n01]
[2026-04-05T10:06:32.601Z] skipping: [an-a01n02]
[2026-04-05T10:06:32.601Z] 
[2026-04-05T10:06:32.601Z] TASK [wait servers to be started in pacemaker] *********************************
[2026-04-05T10:06:32.601Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n02, an-a01n01 => (item=1)
[2026-04-05T10:06:32.601Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n02, an-a01n01 => (item=2)
[2026-04-05T10:06:32.601Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n02, an-a01n01 => (item=3)
[2026-04-05T10:06:32.601Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n02, an-a01n01 => (item=4)
[2026-04-05T10:06:32.601Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n02, an-a01n01 => (item=5)
[2026-04-05T10:06:32.601Z] 
[2026-04-05T10:06:32.601Z] TASK [Wait up to 60 minutes for an-test-deploy1 to be started in pacemaker] ****
[2026-04-05T10:07:01.145Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 to be started in pacemaker (360 retries left).
[2026-04-05T10:07:01.145Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 to be started in pacemaker (360 retries left).
[2026-04-05T10:07:01.145Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 to be started in pacemaker (359 retries left).
[2026-04-05T10:07:01.145Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 to be started in pacemaker (359 retries left).
[2026-04-05T10:07:01.145Z] ok: [an-a01n02]
[2026-04-05T10:07:01.145Z] ok: [an-a01n01]
[2026-04-05T10:07:01.145Z] 
[2026-04-05T10:07:01.145Z] TASK [Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker] ****
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (360 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (360 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (359 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (359 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (358 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (358 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (357 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (357 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (356 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (356 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (355 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (355 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (354 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (354 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (353 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (353 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (352 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (352 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (351 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (351 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (350 retries left).
[2026-04-05T10:09:10.572Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (350 retries left).
[2026-04-05T10:09:10.572Z] ok: [an-a01n02]
[2026-04-05T10:09:10.572Z] ok: [an-a01n01]
[2026-04-05T10:09:10.572Z] 
[2026-04-05T10:09:10.572Z] TASK [Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker] ****
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (360 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (360 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (359 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (359 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (358 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (358 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (357 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (357 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (356 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (356 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (355 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (355 retries left).
[2026-04-05T10:11:35.123Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (354 retries left).
[2026-04-05T10:11:35.124Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (354 retries left).
[2026-04-05T10:11:35.124Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (353 retries left).
[2026-04-05T10:11:35.124Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (353 retries left).
[2026-04-05T10:11:35.124Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (352 retries left).
[2026-04-05T10:11:35.124Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (352 retries left).
[2026-04-05T10:11:35.124Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (351 retries left).
[2026-04-05T10:11:35.124Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (351 retries left).
[2026-04-05T10:11:35.124Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (350 retries left).
[2026-04-05T10:11:35.124Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (350 retries left).
[2026-04-05T10:11:35.124Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (349 retries left).
[2026-04-05T10:11:35.124Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (349 retries left).
[2026-04-05T10:11:35.124Z] ok: [an-a01n02]
[2026-04-05T10:11:35.124Z] ok: [an-a01n01]
[2026-04-05T10:11:35.124Z] 
[2026-04-05T10:11:35.124Z] TASK [Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker] ****
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (360 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (360 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (359 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (359 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (358 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (358 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (357 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (357 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (356 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (356 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (355 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (355 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (354 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (354 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (353 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (353 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (352 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (352 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (351 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (351 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (350 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (350 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (349 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (349 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (348 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (348 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (347 retries left).
[2026-04-05T10:14:31.266Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (347 retries left).
[2026-04-05T10:14:31.266Z] ok: [an-a01n02]
[2026-04-05T10:14:31.266Z] ok: [an-a01n01]
[2026-04-05T10:14:31.266Z] 
[2026-04-05T10:14:31.266Z] TASK [Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker] ****
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (360 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (360 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (359 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (359 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (358 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (358 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (357 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (357 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (356 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (356 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (355 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (355 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (354 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (354 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (353 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (353 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (352 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (352 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (351 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (351 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (350 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (350 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (349 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (349 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (348 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (348 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (347 retries left).
[2026-04-05T10:17:10.848Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (347 retries left).
[2026-04-05T10:17:10.848Z] ok: [an-a01n02]
[2026-04-05T10:17:10.848Z] ok: [an-a01n01]
[2026-04-05T10:17:10.848Z] 
[2026-04-05T10:17:10.848Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-05T10:17:10.848Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-05T10:17:10.848Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-05T10:17:10.848Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-05T10:17:10.848Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-05T10:17:10.848Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-05T10:17:10.848Z] 
[2026-04-05T10:17:10.848Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-05T10:17:10.848Z] ok: [an-a01n02]
[2026-04-05T10:17:10.848Z] ok: [an-a01n01]
[2026-04-05T10:17:10.848Z] 
[2026-04-05T10:17:10.848Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-05T10:17:11.637Z] ok: [an-a01n02]
[2026-04-05T10:17:11.637Z] ok: [an-a01n01]
[2026-04-05T10:17:11.637Z] 
[2026-04-05T10:17:11.637Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-05T10:17:12.426Z] ok: [an-a01n01]
[2026-04-05T10:17:12.426Z] ok: [an-a01n02]
[2026-04-05T10:17:12.426Z] 
[2026-04-05T10:17:12.426Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-05T10:17:13.695Z] ok: [an-a01n02]
[2026-04-05T10:17:13.695Z] ok: [an-a01n01]
[2026-04-05T10:17:13.695Z] 
[2026-04-05T10:17:13.695Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-05T10:18:23.830Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (360 retries left).
[2026-04-05T10:18:23.831Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (360 retries left).
[2026-04-05T10:18:23.831Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (359 retries left).
[2026-04-05T10:18:23.831Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (359 retries left).
[2026-04-05T10:18:23.831Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (358 retries left).
[2026-04-05T10:18:23.831Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (358 retries left).
[2026-04-05T10:18:23.831Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (357 retries left).
[2026-04-05T10:18:23.831Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (357 retries left).
[2026-04-05T10:18:23.831Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (356 retries left).
[2026-04-05T10:18:23.831Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (356 retries left).
[2026-04-05T10:18:23.831Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (355 retries left).
[2026-04-05T10:18:23.831Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (355 retries left).
[2026-04-05T10:18:23.831Z] ok: [an-a01n02]
[2026-04-05T10:18:23.831Z] ok: [an-a01n01]
[2026-04-05T10:18:23.831Z] 
[2026-04-05T10:18:23.831Z] TASK [wait servers to respond to ping] *****************************************
[2026-04-05T10:18:23.831Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-05T10:18:23.831Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-05T10:18:23.831Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-05T10:18:23.831Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-05T10:18:23.831Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-05T10:18:23.831Z] 
[2026-04-05T10:18:23.831Z] TASK [Wait up to 60 minutes for an-test-deploy1 to respond to ping] ************
[2026-04-05T10:18:23.831Z] ok: [an-a01n02]
[2026-04-05T10:18:23.831Z] ok: [an-a01n01]
[2026-04-05T10:18:23.831Z] 
[2026-04-05T10:18:23.831Z] TASK [Wait up to 60 minutes for an-test-deploy2 to respond to ping] ************
[2026-04-05T10:18:23.831Z] ok: [an-a01n02]
[2026-04-05T10:18:23.831Z] ok: [an-a01n01]
[2026-04-05T10:18:23.831Z] 
[2026-04-05T10:18:23.831Z] TASK [Wait up to 60 minutes for an-test-deploy3 to respond to ping] ************
[2026-04-05T10:19:12.751Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (360 retries left).
[2026-04-05T10:19:12.751Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (360 retries left).
[2026-04-05T10:19:12.751Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (359 retries left).
[2026-04-05T10:19:12.751Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (359 retries left).
[2026-04-05T10:19:12.751Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (358 retries left).
[2026-04-05T10:19:12.751Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (358 retries left).
[2026-04-05T10:19:12.751Z] ok: [an-a01n02]
[2026-04-05T10:19:12.751Z] ok: [an-a01n01]
[2026-04-05T10:19:12.751Z] 
[2026-04-05T10:19:12.751Z] TASK [Wait up to 60 minutes for an-test-deploy4 to respond to ping] ************
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (360 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (360 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (359 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (359 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (358 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (358 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (357 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (357 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (356 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (356 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (355 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (355 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (354 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (354 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (353 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (353 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (352 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (352 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (351 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (351 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (350 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (350 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (349 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (349 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (348 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (348 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (347 retries left).
[2026-04-05T10:22:22.584Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (347 retries left).
[2026-04-05T10:22:22.584Z] ok: [an-a01n02]
[2026-04-05T10:22:22.584Z] ok: [an-a01n01]
[2026-04-05T10:22:22.584Z] 
[2026-04-05T10:22:22.584Z] TASK [Wait up to 60 minutes for an-test-deploy5 to respond to ping] ************
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (360 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (360 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (359 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (359 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (358 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (358 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (357 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (357 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (356 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (356 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (355 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (355 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (354 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (354 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (353 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (353 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (352 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (352 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (351 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (351 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (350 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (350 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (349 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (349 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (348 retries left).
[2026-04-05T10:25:32.435Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (348 retries left).
[2026-04-05T10:25:32.435Z] ok: [an-a01n02]
[2026-04-05T10:25:32.435Z] ok: [an-a01n01]
[2026-04-05T10:25:32.435Z] 
[2026-04-05T10:25:32.435Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-05T10:25:32.435Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n02, an-a01n01 => (item=1)
[2026-04-05T10:25:32.435Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n02, an-a01n01 => (item=2)
[2026-04-05T10:25:32.435Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n02, an-a01n01 => (item=3)
[2026-04-05T10:25:32.435Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n02, an-a01n01 => (item=4)
[2026-04-05T10:25:32.435Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n02, an-a01n01 => (item=5)
[2026-04-05T10:25:32.435Z] 
[2026-04-05T10:25:32.435Z] TASK [Wait up to 60 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-05T10:25:32.435Z] ok: [an-a01n01]
[2026-04-05T10:25:32.435Z] ok: [an-a01n02]
[2026-04-05T10:25:32.435Z] 
[2026-04-05T10:25:32.435Z] TASK [Wait up to 60 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-05T10:26:21.332Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (360 retries left).
[2026-04-05T10:26:21.332Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (360 retries left).
[2026-04-05T10:26:21.332Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (359 retries left).
[2026-04-05T10:26:21.332Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (359 retries left).
[2026-04-05T10:26:21.332Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (358 retries left).
[2026-04-05T10:26:21.332Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (358 retries left).
[2026-04-05T10:26:21.332Z] ok: [an-a01n02]
[2026-04-05T10:26:21.332Z] ok: [an-a01n01]
[2026-04-05T10:26:21.332Z] 
[2026-04-05T10:26:21.332Z] TASK [Wait up to 60 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-05T10:29:31.185Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (360 retries left).
[2026-04-05T10:29:31.185Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (360 retries left).
[2026-04-05T10:29:31.185Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (359 retries left).
[2026-04-05T10:29:31.185Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (359 retries left).
[2026-04-05T10:29:31.185Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (358 retries left).
[2026-04-05T10:29:31.185Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (358 retries left).
[2026-04-05T10:29:31.185Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (357 retries left).
[2026-04-05T10:29:31.185Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (357 retries left).
[2026-04-05T10:29:31.185Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (356 retries left).
[2026-04-05T10:29:31.185Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (356 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (355 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (355 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (354 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (354 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (353 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (353 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (352 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (352 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (351 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (351 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (350 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (350 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (349 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (349 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (348 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (348 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (347 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (347 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (346 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (346 retries left).
[2026-04-05T10:29:31.186Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (345 retries left).
[2026-04-05T10:29:31.186Z] ok: [an-a01n02]
[2026-04-05T10:29:31.186Z] ok: [an-a01n01]
[2026-04-05T10:29:31.186Z] 
[2026-04-05T10:29:31.186Z] TASK [Wait up to 60 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (360 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (360 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (359 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (359 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (358 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (358 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (357 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (357 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (356 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (356 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (355 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (355 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (354 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (354 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (353 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (353 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (352 retries left).
[2026-04-05T10:31:25.403Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (352 retries left).
[2026-04-05T10:31:25.403Z] ok: [an-a01n02]
[2026-04-05T10:31:25.403Z] ok: [an-a01n01]
[2026-04-05T10:31:25.403Z] 
[2026-04-05T10:31:25.403Z] TASK [Wait up to 60 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (360 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (360 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (359 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (359 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (358 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (358 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (357 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (357 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (356 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (356 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (355 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (355 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (354 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (354 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (353 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (353 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (352 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (352 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (351 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (351 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (350 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (350 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (349 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (349 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (348 retries left).
[2026-04-05T10:34:04.988Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (348 retries left).
[2026-04-05T10:34:04.988Z] ok: [an-a01n02]
[2026-04-05T10:34:04.988Z] ok: [an-a01n01]
[2026-04-05T10:34:04.988Z] 
[2026-04-05T10:34:04.988Z] TASK [fence with servers test] *************************************************
[2026-04-05T10:34:04.988Z] included: /srv/anvil/anvil-test-suite/playbooks/common/fence_with_servers.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-05T10:34:04.988Z] included: /srv/anvil/anvil-test-suite/playbooks/common/fence_with_servers.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-05T10:34:04.988Z] included: /srv/anvil/anvil-test-suite/playbooks/common/fence_with_servers.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-05T10:34:04.988Z] included: /srv/anvil/anvil-test-suite/playbooks/common/fence_with_servers.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-05T10:34:04.988Z] included: /srv/anvil/anvil-test-suite/playbooks/common/fence_with_servers.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-05T10:34:04.988Z] 
[2026-04-05T10:34:04.988Z] TASK [fence with servers iteration 1] ******************************************
[2026-04-05T10:34:04.988Z] ok: [an-a01n01]
[2026-04-05T10:34:04.988Z] ok: [an-a01n02]
[2026-04-05T10:34:04.988Z] 
[2026-04-05T10:34:04.988Z] TASK [check servers status] ****************************************************
[2026-04-05T10:34:04.988Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01, an-a01n02
[2026-04-05T10:34:04.988Z] 
[2026-04-05T10:34:04.988Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T10:34:04.988Z] ok: [an-a01n02]
[2026-04-05T10:34:04.988Z] ok: [an-a01n01]
[2026-04-05T10:34:04.988Z] 
[2026-04-05T10:34:04.988Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T10:34:04.988Z] ok: [an-a01n02]
[2026-04-05T10:34:04.988Z] ok: [an-a01n01]
[2026-04-05T10:34:04.988Z] 
[2026-04-05T10:34:04.988Z] TASK [set target] **************************************************************
[2026-04-05T10:34:04.988Z] ok: [an-a01n01]
[2026-04-05T10:34:04.988Z] ok: [an-a01n02]
[2026-04-05T10:34:04.988Z] 
[2026-04-05T10:34:04.988Z] TASK [will kill node with servers] *********************************************
[2026-04-05T10:34:04.988Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./kill_node.yml for an-a01n01, an-a01n02
[2026-04-05T10:34:04.988Z] 
[2026-04-05T10:34:04.988Z] TASK [kill node "an-a01n01"] ***************************************************
[2026-04-05T10:34:04.988Z] skipping: [an-a01n02]
[2026-04-05T10:34:04.988Z] changed: [an-a01n01]
[2026-04-05T10:34:04.988Z] 
[2026-04-05T10:34:04.988Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-05T10:34:04.988Z] skipping: [an-a01n01]
[2026-04-05T10:34:04.988Z] skipping: [an-a01n02]
[2026-04-05T10:34:04.988Z] 
[2026-04-05T10:34:04.988Z] TASK [wait for node to be offline] *********************************************
[2026-04-05T10:34:04.988Z] skipping: [an-a01n01]
[2026-04-05T10:34:04.988Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./offline_node.yml for an-a01n02
[2026-04-05T10:34:04.988Z] 
[2026-04-05T10:34:04.988Z] TASK [wait up to 10 minutes for node an-a01n01 to be offline] ******************
[2026-04-05T10:35:03.554Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (60 retries left).
[2026-04-05T10:35:03.554Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (59 retries left).
[2026-04-05T10:35:03.554Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (58 retries left).
[2026-04-05T10:35:03.554Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (57 retries left).
[2026-04-05T10:35:03.554Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (56 retries left).
[2026-04-05T10:35:03.554Z] ok: [an-a01n02]
[2026-04-05T10:35:03.554Z] 
[2026-04-05T10:35:03.554Z] TASK [check servers status] ****************************************************
[2026-04-05T10:35:03.554Z] skipping: [an-a01n01]
[2026-04-05T10:35:03.554Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n02
[2026-04-05T10:35:03.554Z] 
[2026-04-05T10:35:03.554Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T10:36:42.649Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (120 retries left).
[2026-04-05T10:36:42.649Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (119 retries left).
[2026-04-05T10:36:42.649Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (118 retries left).
[2026-04-05T10:36:42.649Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (117 retries left).
[2026-04-05T10:36:42.649Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (116 retries left).
[2026-04-05T10:36:42.649Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (115 retries left).
[2026-04-05T10:36:42.649Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (114 retries left).
[2026-04-05T10:36:42.649Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (113 retries left).
[2026-04-05T10:36:42.649Z] ok: [an-a01n02]
[2026-04-05T10:36:42.649Z] 
[2026-04-05T10:36:42.649Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T10:36:44.468Z] ok: [an-a01n02]
[2026-04-05T10:36:44.468Z] 
[2026-04-05T10:36:44.468Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-05T10:36:44.468Z] skipping: [an-a01n01] => (item=1) 
[2026-04-05T10:36:44.468Z] skipping: [an-a01n01] => (item=2) 
[2026-04-05T10:36:44.468Z] skipping: [an-a01n01] => (item=3) 
[2026-04-05T10:36:44.468Z] skipping: [an-a01n01] => (item=4) 
[2026-04-05T10:36:44.468Z] skipping: [an-a01n01] => (item=5) 
[2026-04-05T10:36:44.468Z] skipping: [an-a01n01]
[2026-04-05T10:36:44.468Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=1)
[2026-04-05T10:36:44.468Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=2)
[2026-04-05T10:36:44.468Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=3)
[2026-04-05T10:36:44.468Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=4)
[2026-04-05T10:36:44.468Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=5)
[2026-04-05T10:36:44.468Z] 
[2026-04-05T10:36:44.468Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-05T10:36:45.256Z] ok: [an-a01n02]
[2026-04-05T10:36:45.256Z] 
[2026-04-05T10:36:45.256Z] TASK [Wait up to 10 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-05T10:36:46.044Z] ok: [an-a01n02]
[2026-04-05T10:36:46.044Z] 
[2026-04-05T10:36:46.044Z] TASK [Wait up to 10 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-05T10:36:46.832Z] ok: [an-a01n02]
[2026-04-05T10:36:46.832Z] 
[2026-04-05T10:36:46.832Z] TASK [Wait up to 10 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-05T10:36:47.621Z] ok: [an-a01n02]
[2026-04-05T10:36:47.621Z] 
[2026-04-05T10:36:47.621Z] TASK [Wait up to 10 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-05T10:36:48.410Z] ok: [an-a01n02]
[2026-04-05T10:36:48.410Z] 
[2026-04-05T10:36:48.410Z] TASK [wait for node to be back online] *****************************************
[2026-04-05T10:36:48.779Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01, an-a01n02
[2026-04-05T10:36:48.779Z] 
[2026-04-05T10:36:48.779Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-05T10:40:13.742Z] ok: [an-a01n01 -> localhost]
[2026-04-05T10:40:13.742Z] ok: [an-a01n02 -> localhost]
[2026-04-05T10:40:13.742Z] 
[2026-04-05T10:40:13.742Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T10:40:13.742Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T10:40:13.742Z] 
[2026-04-05T10:40:13.742Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T10:40:13.742Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T10:40:13.742Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T10:40:13.742Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T10:40:13.742Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T10:40:13.742Z] 
[2026-04-05T10:40:13.742Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-05T10:40:13.742Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-05T10:40:13.742Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-05T10:40:13.742Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-05T10:40:13.742Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-05T10:40:13.742Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-05T10:40:13.742Z] 
[2026-04-05T10:40:13.742Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-05T10:40:47.806Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (360 retries left).
[2026-04-05T10:40:47.806Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (360 retries left).
[2026-04-05T10:40:47.806Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (359 retries left).
[2026-04-05T10:40:47.806Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (359 retries left).
[2026-04-05T10:40:47.806Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (358 retries left).
[2026-04-05T10:40:47.806Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (358 retries left).
[2026-04-05T10:40:47.806Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (357 retries left).
[2026-04-05T10:40:47.806Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (357 retries left).
[2026-04-05T10:40:47.806Z] ok: [an-a01n02]
[2026-04-05T10:40:47.806Z] ok: [an-a01n01]
[2026-04-05T10:40:47.806Z] 
[2026-04-05T10:40:47.806Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-05T10:40:48.594Z] ok: [an-a01n02]
[2026-04-05T10:40:48.594Z] ok: [an-a01n01]
[2026-04-05T10:40:48.594Z] 
[2026-04-05T10:40:48.594Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-05T10:40:49.398Z] ok: [an-a01n02]
[2026-04-05T10:40:49.398Z] ok: [an-a01n01]
[2026-04-05T10:40:49.398Z] 
[2026-04-05T10:40:49.398Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-05T10:40:49.767Z] ok: [an-a01n02]
[2026-04-05T10:40:49.767Z] ok: [an-a01n01]
[2026-04-05T10:40:49.767Z] 
[2026-04-05T10:40:49.767Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-05T10:40:50.556Z] ok: [an-a01n02]
[2026-04-05T10:40:50.556Z] ok: [an-a01n01]
[2026-04-05T10:40:50.556Z] 
[2026-04-05T10:40:50.556Z] TASK [fence with servers iteration 2] ******************************************
[2026-04-05T10:40:50.556Z] ok: [an-a01n01]
[2026-04-05T10:40:50.556Z] ok: [an-a01n02]
[2026-04-05T10:40:50.556Z] 
[2026-04-05T10:40:50.556Z] TASK [check servers status] ****************************************************
[2026-04-05T10:40:50.556Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01, an-a01n02
[2026-04-05T10:40:50.556Z] 
[2026-04-05T10:40:50.556Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T10:40:53.012Z] ok: [an-a01n02]
[2026-04-05T10:40:53.012Z] ok: [an-a01n01]
[2026-04-05T10:40:53.012Z] 
[2026-04-05T10:40:53.012Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T10:40:54.830Z] ok: [an-a01n02]
[2026-04-05T10:40:54.830Z] ok: [an-a01n01]
[2026-04-05T10:40:54.830Z] 
[2026-04-05T10:40:54.830Z] TASK [set target] **************************************************************
[2026-04-05T10:40:54.830Z] ok: [an-a01n01]
[2026-04-05T10:40:54.830Z] ok: [an-a01n02]
[2026-04-05T10:40:54.830Z] 
[2026-04-05T10:40:54.830Z] TASK [will kill node with servers] *********************************************
[2026-04-05T10:40:54.830Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./kill_node.yml for an-a01n01, an-a01n02
[2026-04-05T10:40:54.830Z] 
[2026-04-05T10:40:54.830Z] TASK [kill node "an-a01n02"] ***************************************************
[2026-04-05T10:40:55.617Z] skipping: [an-a01n01]
[2026-04-05T10:40:55.617Z] changed: [an-a01n02]
[2026-04-05T10:40:55.617Z] 
[2026-04-05T10:40:55.617Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-05T10:40:55.617Z] skipping: [an-a01n01]
[2026-04-05T10:40:55.617Z] skipping: [an-a01n02]
[2026-04-05T10:40:55.617Z] 
[2026-04-05T10:40:55.617Z] TASK [wait for node to be offline] *********************************************
[2026-04-05T10:40:55.617Z] skipping: [an-a01n02]
[2026-04-05T10:40:55.617Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./offline_node.yml for an-a01n01
[2026-04-05T10:40:55.617Z] 
[2026-04-05T10:40:55.617Z] TASK [wait up to 10 minutes for node an-a01n02 to be offline] ******************
[2026-04-05T10:42:05.724Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (60 retries left).
[2026-04-05T10:42:05.724Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (59 retries left).
[2026-04-05T10:42:05.724Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (58 retries left).
[2026-04-05T10:42:05.724Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (57 retries left).
[2026-04-05T10:42:05.724Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (56 retries left).
[2026-04-05T10:42:05.724Z] ok: [an-a01n01]
[2026-04-05T10:42:05.724Z] 
[2026-04-05T10:42:05.724Z] TASK [check servers status] ****************************************************
[2026-04-05T10:42:05.724Z] skipping: [an-a01n02]
[2026-04-05T10:42:05.724Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01
[2026-04-05T10:42:05.724Z] 
[2026-04-05T10:42:05.724Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T10:43:15.843Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (120 retries left).
[2026-04-05T10:43:15.843Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (119 retries left).
[2026-04-05T10:43:15.843Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (118 retries left).
[2026-04-05T10:43:15.843Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (117 retries left).
[2026-04-05T10:43:15.843Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (116 retries left).
[2026-04-05T10:43:15.843Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (115 retries left).
[2026-04-05T10:43:15.843Z] ok: [an-a01n01]
[2026-04-05T10:43:15.843Z] 
[2026-04-05T10:43:15.843Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T10:43:15.843Z] ok: [an-a01n01]
[2026-04-05T10:43:15.843Z] 
[2026-04-05T10:43:15.843Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-05T10:43:15.843Z] skipping: [an-a01n02] => (item=1) 
[2026-04-05T10:43:15.843Z] skipping: [an-a01n02] => (item=2) 
[2026-04-05T10:43:15.843Z] skipping: [an-a01n02] => (item=3) 
[2026-04-05T10:43:15.843Z] skipping: [an-a01n02] => (item=4) 
[2026-04-05T10:43:15.843Z] skipping: [an-a01n02] => (item=5) 
[2026-04-05T10:43:15.843Z] skipping: [an-a01n02]
[2026-04-05T10:43:15.843Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=1)
[2026-04-05T10:43:15.843Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=2)
[2026-04-05T10:43:15.843Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=3)
[2026-04-05T10:43:15.843Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=4)
[2026-04-05T10:43:15.843Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=5)
[2026-04-05T10:43:15.843Z] 
[2026-04-05T10:43:15.843Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-05T10:43:16.631Z] ok: [an-a01n01]
[2026-04-05T10:43:16.631Z] 
[2026-04-05T10:43:16.631Z] TASK [Wait up to 10 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-05T10:43:17.897Z] ok: [an-a01n01]
[2026-04-05T10:43:17.897Z] 
[2026-04-05T10:43:17.897Z] TASK [Wait up to 10 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-05T10:43:19.166Z] ok: [an-a01n01]
[2026-04-05T10:43:19.166Z] 
[2026-04-05T10:43:19.166Z] TASK [Wait up to 10 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-05T10:43:19.954Z] ok: [an-a01n01]
[2026-04-05T10:43:19.954Z] 
[2026-04-05T10:43:19.954Z] TASK [Wait up to 10 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-05T10:43:20.744Z] ok: [an-a01n01]
[2026-04-05T10:43:20.744Z] 
[2026-04-05T10:43:20.744Z] TASK [wait for node to be back online] *****************************************
[2026-04-05T10:43:21.113Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01, an-a01n02
[2026-04-05T10:43:21.113Z] 
[2026-04-05T10:43:21.113Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-05T10:46:46.082Z] ok: [an-a01n01 -> localhost]
[2026-04-05T10:46:46.082Z] ok: [an-a01n02 -> localhost]
[2026-04-05T10:46:46.082Z] 
[2026-04-05T10:46:46.082Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T10:46:46.082Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T10:46:46.082Z] 
[2026-04-05T10:46:46.082Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T10:46:46.082Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T10:46:46.082Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T10:46:46.082Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T10:46:46.082Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T10:46:46.082Z] 
[2026-04-05T10:46:46.082Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-05T10:46:46.082Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-05T10:46:46.082Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-05T10:46:46.082Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-05T10:46:46.082Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-05T10:46:46.082Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-05T10:46:46.082Z] 
[2026-04-05T10:46:46.082Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-05T10:47:34.966Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (360 retries left).
[2026-04-05T10:47:34.966Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (360 retries left).
[2026-04-05T10:47:34.966Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (359 retries left).
[2026-04-05T10:47:34.966Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (359 retries left).
[2026-04-05T10:47:34.966Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (358 retries left).
[2026-04-05T10:47:34.966Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (358 retries left).
[2026-04-05T10:47:34.966Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (357 retries left).
[2026-04-05T10:47:34.966Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (357 retries left).
[2026-04-05T10:47:34.966Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (356 retries left).
[2026-04-05T10:47:34.966Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (356 retries left).
[2026-04-05T10:47:34.966Z] ok: [an-a01n02]
[2026-04-05T10:47:34.966Z] ok: [an-a01n01]
[2026-04-05T10:47:34.966Z] 
[2026-04-05T10:47:34.966Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-05T10:48:09.028Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced (360 retries left).
[2026-04-05T10:48:09.028Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced (360 retries left).
[2026-04-05T10:48:09.028Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced (359 retries left).
[2026-04-05T10:48:09.028Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced (359 retries left).
[2026-04-05T10:48:09.028Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced (358 retries left).
[2026-04-05T10:48:09.028Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced (358 retries left).
[2026-04-05T10:48:09.028Z] ok: [an-a01n01]
[2026-04-05T10:48:09.028Z] ok: [an-a01n02]
[2026-04-05T10:48:09.028Z] 
[2026-04-05T10:48:09.028Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-05T10:48:09.028Z] ok: [an-a01n02]
[2026-04-05T10:48:09.028Z] ok: [an-a01n01]
[2026-04-05T10:48:09.028Z] 
[2026-04-05T10:48:09.028Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-05T10:48:09.028Z] ok: [an-a01n01]
[2026-04-05T10:48:09.028Z] ok: [an-a01n02]
[2026-04-05T10:48:09.028Z] 
[2026-04-05T10:48:09.028Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-05T10:48:09.029Z] ok: [an-a01n01]
[2026-04-05T10:48:09.029Z] ok: [an-a01n02]
[2026-04-05T10:48:09.029Z] 
[2026-04-05T10:48:09.029Z] TASK [fence with servers iteration 3] ******************************************
[2026-04-05T10:48:09.029Z] ok: [an-a01n01]
[2026-04-05T10:48:09.029Z] ok: [an-a01n02]
[2026-04-05T10:48:09.029Z] 
[2026-04-05T10:48:09.029Z] TASK [check servers status] ****************************************************
[2026-04-05T10:48:09.029Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01, an-a01n02
[2026-04-05T10:48:09.029Z] 
[2026-04-05T10:48:09.029Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T10:48:10.846Z] ok: [an-a01n02]
[2026-04-05T10:48:10.846Z] ok: [an-a01n01]
[2026-04-05T10:48:10.846Z] 
[2026-04-05T10:48:10.846Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T10:48:12.665Z] ok: [an-a01n02]
[2026-04-05T10:48:12.665Z] ok: [an-a01n01]
[2026-04-05T10:48:12.665Z] 
[2026-04-05T10:48:12.665Z] TASK [set target] **************************************************************
[2026-04-05T10:48:12.665Z] ok: [an-a01n01]
[2026-04-05T10:48:12.665Z] ok: [an-a01n02]
[2026-04-05T10:48:12.665Z] 
[2026-04-05T10:48:12.665Z] TASK [will kill node with servers] *********************************************
[2026-04-05T10:48:12.665Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./kill_node.yml for an-a01n01, an-a01n02
[2026-04-05T10:48:12.665Z] 
[2026-04-05T10:48:12.665Z] TASK [kill node "an-a01n01"] ***************************************************
[2026-04-05T10:48:13.453Z] skipping: [an-a01n02]
[2026-04-05T10:48:13.453Z] changed: [an-a01n01]
[2026-04-05T10:48:13.453Z] 
[2026-04-05T10:48:13.453Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-05T10:48:13.453Z] skipping: [an-a01n01]
[2026-04-05T10:48:13.453Z] skipping: [an-a01n02]
[2026-04-05T10:48:13.453Z] 
[2026-04-05T10:48:13.453Z] TASK [wait for node to be offline] *********************************************
[2026-04-05T10:48:13.453Z] skipping: [an-a01n01]
[2026-04-05T10:48:13.453Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./offline_node.yml for an-a01n02
[2026-04-05T10:48:13.453Z] 
[2026-04-05T10:48:13.453Z] TASK [wait up to 10 minutes for node an-a01n01 to be offline] ******************
[2026-04-05T10:49:23.564Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (60 retries left).
[2026-04-05T10:49:23.564Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (59 retries left).
[2026-04-05T10:49:23.564Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (58 retries left).
[2026-04-05T10:49:23.564Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (57 retries left).
[2026-04-05T10:49:23.564Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (56 retries left).
[2026-04-05T10:49:23.564Z] ok: [an-a01n02]
[2026-04-05T10:49:23.564Z] 
[2026-04-05T10:49:23.564Z] TASK [check servers status] ****************************************************
[2026-04-05T10:49:23.564Z] skipping: [an-a01n01]
[2026-04-05T10:49:23.564Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n02
[2026-04-05T10:49:23.564Z] 
[2026-04-05T10:49:23.564Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T10:50:22.135Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (120 retries left).
[2026-04-05T10:50:22.135Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (119 retries left).
[2026-04-05T10:50:22.135Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (118 retries left).
[2026-04-05T10:50:22.135Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (117 retries left).
[2026-04-05T10:50:22.135Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (116 retries left).
[2026-04-05T10:50:22.135Z] ok: [an-a01n02]
[2026-04-05T10:50:22.135Z] 
[2026-04-05T10:50:22.135Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T10:50:22.135Z] ok: [an-a01n02]
[2026-04-05T10:50:22.135Z] 
[2026-04-05T10:50:22.135Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-05T10:50:22.135Z] skipping: [an-a01n01] => (item=1) 
[2026-04-05T10:50:22.135Z] skipping: [an-a01n01] => (item=2) 
[2026-04-05T10:50:22.135Z] skipping: [an-a01n01] => (item=3) 
[2026-04-05T10:50:22.135Z] skipping: [an-a01n01] => (item=4) 
[2026-04-05T10:50:22.135Z] skipping: [an-a01n01] => (item=5) 
[2026-04-05T10:50:22.135Z] skipping: [an-a01n01]
[2026-04-05T10:50:22.135Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=1)
[2026-04-05T10:50:22.135Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=2)
[2026-04-05T10:50:22.135Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=3)
[2026-04-05T10:50:22.135Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=4)
[2026-04-05T10:50:22.135Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=5)
[2026-04-05T10:50:22.135Z] 
[2026-04-05T10:50:22.135Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-05T10:50:35.778Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to respond to ssh (60 retries left).
[2026-04-05T10:50:35.778Z] ok: [an-a01n02]
[2026-04-05T10:50:35.778Z] 
[2026-04-05T10:50:35.778Z] TASK [Wait up to 10 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-05T10:50:35.778Z] ok: [an-a01n02]
[2026-04-05T10:50:35.778Z] 
[2026-04-05T10:50:35.778Z] TASK [Wait up to 10 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-05T10:50:36.148Z] ok: [an-a01n02]
[2026-04-05T10:50:36.148Z] 
[2026-04-05T10:50:36.148Z] TASK [Wait up to 10 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-05T10:50:36.935Z] ok: [an-a01n02]
[2026-04-05T10:50:36.935Z] 
[2026-04-05T10:50:36.935Z] TASK [Wait up to 10 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-05T10:50:37.723Z] ok: [an-a01n02]
[2026-04-05T10:50:37.723Z] 
[2026-04-05T10:50:37.723Z] TASK [wait for node to be back online] *****************************************
[2026-04-05T10:50:38.092Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01, an-a01n02
[2026-04-05T10:50:38.092Z] 
[2026-04-05T10:50:38.092Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-05T10:54:03.019Z] ok: [an-a01n01 -> localhost]
[2026-04-05T10:54:03.019Z] ok: [an-a01n02 -> localhost]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T10:54:03.019Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T10:54:03.019Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T10:54:03.019Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T10:54:03.019Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T10:54:03.019Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-05T10:54:03.019Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-05T10:54:03.019Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-05T10:54:03.019Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-05T10:54:03.019Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-05T10:54:03.019Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-05T10:54:03.019Z] ok: [an-a01n01]
[2026-04-05T10:54:03.019Z] ok: [an-a01n02]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-05T10:54:03.019Z] ok: [an-a01n01]
[2026-04-05T10:54:03.019Z] ok: [an-a01n02]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-05T10:54:03.019Z] ok: [an-a01n02]
[2026-04-05T10:54:03.019Z] ok: [an-a01n01]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-05T10:54:03.019Z] ok: [an-a01n01]
[2026-04-05T10:54:03.019Z] ok: [an-a01n02]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-05T10:54:03.019Z] ok: [an-a01n02]
[2026-04-05T10:54:03.019Z] ok: [an-a01n01]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [fence with servers iteration 4] ******************************************
[2026-04-05T10:54:03.019Z] ok: [an-a01n01]
[2026-04-05T10:54:03.019Z] ok: [an-a01n02]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [check servers status] ****************************************************
[2026-04-05T10:54:03.019Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01, an-a01n02
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T10:54:03.019Z] ok: [an-a01n02]
[2026-04-05T10:54:03.019Z] ok: [an-a01n01]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T10:54:03.019Z] ok: [an-a01n01]
[2026-04-05T10:54:03.019Z] ok: [an-a01n02]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [set target] **************************************************************
[2026-04-05T10:54:03.019Z] ok: [an-a01n01]
[2026-04-05T10:54:03.019Z] ok: [an-a01n02]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [will kill node with servers] *********************************************
[2026-04-05T10:54:03.019Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./kill_node.yml for an-a01n01, an-a01n02
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [kill node "an-a01n02"] ***************************************************
[2026-04-05T10:54:03.019Z] skipping: [an-a01n01]
[2026-04-05T10:54:03.019Z] changed: [an-a01n02]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-05T10:54:03.019Z] skipping: [an-a01n01]
[2026-04-05T10:54:03.019Z] skipping: [an-a01n02]
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [wait for node to be offline] *********************************************
[2026-04-05T10:54:03.019Z] skipping: [an-a01n02]
[2026-04-05T10:54:03.019Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./offline_node.yml for an-a01n01
[2026-04-05T10:54:03.019Z] 
[2026-04-05T10:54:03.019Z] TASK [wait up to 10 minutes for node an-a01n02 to be offline] ******************
[2026-04-05T10:55:13.158Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (60 retries left).
[2026-04-05T10:55:13.158Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (59 retries left).
[2026-04-05T10:55:13.158Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (58 retries left).
[2026-04-05T10:55:13.158Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (57 retries left).
[2026-04-05T10:55:13.158Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (56 retries left).
[2026-04-05T10:55:13.158Z] ok: [an-a01n01]
[2026-04-05T10:55:13.158Z] 
[2026-04-05T10:55:13.158Z] TASK [check servers status] ****************************************************
[2026-04-05T10:55:13.158Z] skipping: [an-a01n02]
[2026-04-05T10:55:13.158Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01
[2026-04-05T10:55:13.158Z] 
[2026-04-05T10:55:13.158Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T10:56:23.275Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (120 retries left).
[2026-04-05T10:56:23.275Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (119 retries left).
[2026-04-05T10:56:23.275Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (118 retries left).
[2026-04-05T10:56:23.275Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (117 retries left).
[2026-04-05T10:56:23.275Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (116 retries left).
[2026-04-05T10:56:23.275Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (115 retries left).
[2026-04-05T10:56:23.275Z] ok: [an-a01n01]
[2026-04-05T10:56:23.275Z] 
[2026-04-05T10:56:23.275Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T10:56:23.275Z] ok: [an-a01n01]
[2026-04-05T10:56:23.275Z] 
[2026-04-05T10:56:23.275Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-05T10:56:23.275Z] skipping: [an-a01n02] => (item=1) 
[2026-04-05T10:56:23.275Z] skipping: [an-a01n02] => (item=2) 
[2026-04-05T10:56:23.275Z] skipping: [an-a01n02] => (item=3) 
[2026-04-05T10:56:23.275Z] skipping: [an-a01n02] => (item=4) 
[2026-04-05T10:56:23.275Z] skipping: [an-a01n02] => (item=5) 
[2026-04-05T10:56:23.275Z] skipping: [an-a01n02]
[2026-04-05T10:56:23.275Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=1)
[2026-04-05T10:56:23.275Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=2)
[2026-04-05T10:56:23.275Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=3)
[2026-04-05T10:56:23.275Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=4)
[2026-04-05T10:56:23.275Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=5)
[2026-04-05T10:56:23.275Z] 
[2026-04-05T10:56:23.275Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-05T10:56:36.918Z] FAILED - RETRYING: [an-a01n01]: Wait up to 10 minutes for an-test-deploy1 to respond to ssh (60 retries left).
[2026-04-05T10:56:36.918Z] ok: [an-a01n01]
[2026-04-05T10:56:36.918Z] 
[2026-04-05T10:56:36.918Z] TASK [Wait up to 10 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-05T10:56:36.918Z] ok: [an-a01n01]
[2026-04-05T10:56:36.918Z] 
[2026-04-05T10:56:36.918Z] TASK [Wait up to 10 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-05T10:56:36.918Z] ok: [an-a01n01]
[2026-04-05T10:56:36.918Z] 
[2026-04-05T10:56:36.918Z] TASK [Wait up to 10 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-05T10:56:37.287Z] ok: [an-a01n01]
[2026-04-05T10:56:37.287Z] 
[2026-04-05T10:56:37.287Z] TASK [Wait up to 10 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-05T10:56:37.657Z] ok: [an-a01n01]
[2026-04-05T10:56:37.657Z] 
[2026-04-05T10:56:37.657Z] TASK [wait for node to be back online] *****************************************
[2026-04-05T10:56:38.026Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01, an-a01n02
[2026-04-05T10:56:38.026Z] 
[2026-04-05T10:56:38.026Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-05T11:00:02.986Z] ok: [an-a01n01 -> localhost]
[2026-04-05T11:00:02.986Z] ok: [an-a01n02 -> localhost]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T11:00:02.986Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T11:00:02.986Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T11:00:02.986Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T11:00:02.986Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T11:00:02.986Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-05T11:00:02.986Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-05T11:00:02.986Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-05T11:00:02.986Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-05T11:00:02.986Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-05T11:00:02.986Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-05T11:00:02.986Z] ok: [an-a01n01]
[2026-04-05T11:00:02.986Z] ok: [an-a01n02]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-05T11:00:02.986Z] ok: [an-a01n01]
[2026-04-05T11:00:02.986Z] ok: [an-a01n02]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-05T11:00:02.986Z] ok: [an-a01n02]
[2026-04-05T11:00:02.986Z] ok: [an-a01n01]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-05T11:00:02.986Z] ok: [an-a01n01]
[2026-04-05T11:00:02.986Z] ok: [an-a01n02]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-05T11:00:02.986Z] ok: [an-a01n01]
[2026-04-05T11:00:02.986Z] ok: [an-a01n02]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [fence with servers iteration 5] ******************************************
[2026-04-05T11:00:02.986Z] ok: [an-a01n01]
[2026-04-05T11:00:02.986Z] ok: [an-a01n02]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [check servers status] ****************************************************
[2026-04-05T11:00:02.986Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01, an-a01n02
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T11:00:02.986Z] ok: [an-a01n02]
[2026-04-05T11:00:02.986Z] ok: [an-a01n01]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T11:00:02.986Z] ok: [an-a01n02]
[2026-04-05T11:00:02.986Z] ok: [an-a01n01]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [set target] **************************************************************
[2026-04-05T11:00:02.986Z] ok: [an-a01n01]
[2026-04-05T11:00:02.986Z] ok: [an-a01n02]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [will kill node with servers] *********************************************
[2026-04-05T11:00:02.986Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./kill_node.yml for an-a01n01, an-a01n02
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [kill node "an-a01n01"] ***************************************************
[2026-04-05T11:00:02.986Z] skipping: [an-a01n02]
[2026-04-05T11:00:02.986Z] changed: [an-a01n01]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-05T11:00:02.986Z] skipping: [an-a01n01]
[2026-04-05T11:00:02.986Z] skipping: [an-a01n02]
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [wait for node to be offline] *********************************************
[2026-04-05T11:00:02.986Z] skipping: [an-a01n01]
[2026-04-05T11:00:02.986Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./offline_node.yml for an-a01n02
[2026-04-05T11:00:02.986Z] 
[2026-04-05T11:00:02.986Z] TASK [wait up to 10 minutes for node an-a01n01 to be offline] ******************
[2026-04-05T11:01:01.527Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (60 retries left).
[2026-04-05T11:01:01.527Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (59 retries left).
[2026-04-05T11:01:01.527Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (58 retries left).
[2026-04-05T11:01:01.527Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (57 retries left).
[2026-04-05T11:01:01.527Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (56 retries left).
[2026-04-05T11:01:01.527Z] ok: [an-a01n02]
[2026-04-05T11:01:01.527Z] 
[2026-04-05T11:01:01.527Z] TASK [check servers status] ****************************************************
[2026-04-05T11:01:01.895Z] skipping: [an-a01n01]
[2026-04-05T11:01:01.895Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n02
[2026-04-05T11:01:01.895Z] 
[2026-04-05T11:01:01.895Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T11:02:25.875Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (120 retries left).
[2026-04-05T11:02:25.875Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (119 retries left).
[2026-04-05T11:02:25.875Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (118 retries left).
[2026-04-05T11:02:25.875Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (117 retries left).
[2026-04-05T11:02:25.875Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (116 retries left).
[2026-04-05T11:02:25.875Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (115 retries left).
[2026-04-05T11:02:25.875Z] ok: [an-a01n02]
[2026-04-05T11:02:25.875Z] 
[2026-04-05T11:02:25.875Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T11:02:25.875Z] ok: [an-a01n02]
[2026-04-05T11:02:25.875Z] 
[2026-04-05T11:02:25.875Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-05T11:02:25.875Z] skipping: [an-a01n01] => (item=1) 
[2026-04-05T11:02:25.875Z] skipping: [an-a01n01] => (item=2) 
[2026-04-05T11:02:25.875Z] skipping: [an-a01n01] => (item=3) 
[2026-04-05T11:02:25.875Z] skipping: [an-a01n01] => (item=4) 
[2026-04-05T11:02:25.875Z] skipping: [an-a01n01] => (item=5) 
[2026-04-05T11:02:25.875Z] skipping: [an-a01n01]
[2026-04-05T11:02:25.875Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=1)
[2026-04-05T11:02:25.875Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=2)
[2026-04-05T11:02:25.875Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=3)
[2026-04-05T11:02:25.875Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=4)
[2026-04-05T11:02:25.875Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=5)
[2026-04-05T11:02:25.875Z] 
[2026-04-05T11:02:25.875Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-05T11:02:35.204Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to respond to ssh (60 retries left).
[2026-04-05T11:02:35.204Z] ok: [an-a01n02]
[2026-04-05T11:02:35.204Z] 
[2026-04-05T11:02:35.204Z] TASK [Wait up to 10 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-05T11:02:35.204Z] ok: [an-a01n02]
[2026-04-05T11:02:35.204Z] 
[2026-04-05T11:02:35.204Z] TASK [Wait up to 10 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-05T11:02:35.204Z] ok: [an-a01n02]
[2026-04-05T11:02:35.204Z] 
[2026-04-05T11:02:35.204Z] TASK [Wait up to 10 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-05T11:02:35.992Z] ok: [an-a01n02]
[2026-04-05T11:02:35.992Z] 
[2026-04-05T11:02:35.992Z] TASK [Wait up to 10 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-05T11:02:36.361Z] ok: [an-a01n02]
[2026-04-05T11:02:36.361Z] 
[2026-04-05T11:02:36.361Z] TASK [wait for node to be back online] *****************************************
[2026-04-05T11:02:36.730Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01, an-a01n02
[2026-04-05T11:02:36.730Z] 
[2026-04-05T11:02:36.730Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-05T11:06:01.687Z] ok: [an-a01n01 -> localhost]
[2026-04-05T11:06:01.687Z] ok: [an-a01n02 -> localhost]
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T11:06:01.687Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T11:06:01.687Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T11:06:01.687Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T11:06:01.687Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T11:06:01.687Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-05T11:06:01.687Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-05T11:06:01.687Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-05T11:06:01.687Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-05T11:06:01.687Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-05T11:06:01.687Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-05T11:06:01.687Z] ok: [an-a01n01]
[2026-04-05T11:06:01.687Z] ok: [an-a01n02]
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-05T11:06:01.687Z] ok: [an-a01n01]
[2026-04-05T11:06:01.687Z] ok: [an-a01n02]
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-05T11:06:01.687Z] ok: [an-a01n02]
[2026-04-05T11:06:01.687Z] ok: [an-a01n01]
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-05T11:06:01.687Z] ok: [an-a01n01]
[2026-04-05T11:06:01.687Z] ok: [an-a01n02]
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-05T11:06:01.687Z] ok: [an-a01n01]
[2026-04-05T11:06:01.687Z] ok: [an-a01n02]
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] PLAY [basic server tests] ******************************************************
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:06:01.687Z] ok: [an-a01n01]
[2026-04-05T11:06:01.687Z] ok: [an-a01n02]
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T11:06:01.687Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T11:06:01.687Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T11:06:01.687Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T11:06:01.687Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T11:06:01.687Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [check anvil-provision-server storage group] ******************************
[2026-04-05T11:06:01.687Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_storage_group.yml for an-a01n01, an-a01n02
[2026-04-05T11:06:01.687Z] 
[2026-04-05T11:06:01.687Z] TASK [get anvil-provision-server storage group] ********************************
[2026-04-05T11:06:05.759Z] changed: [an-a01n01]
[2026-04-05T11:06:05.759Z] changed: [an-a01n02]
[2026-04-05T11:06:05.759Z] 
[2026-04-05T11:06:05.759Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-05T11:06:05.759Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_install_iso.yml for an-a01n01, an-a01n02
[2026-04-05T11:06:05.759Z] 
[2026-04-05T11:06:05.759Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-05T11:06:10.830Z] changed: [an-a01n01]
[2026-04-05T11:06:10.830Z] changed: [an-a01n02]
[2026-04-05T11:06:10.830Z] 
[2026-04-05T11:06:10.830Z] TASK [deploy server 1 from "an-a01n01"] ****************************************
[2026-04-05T11:06:10.830Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02
[2026-04-05T11:06:10.830Z] 
[2026-04-05T11:06:10.830Z] TASK [check if server an-test-deploy1 exists] **********************************
[2026-04-05T11:06:12.096Z] skipping: [an-a01n02]
[2026-04-05T11:06:12.096Z] ok: [an-a01n01]
[2026-04-05T11:06:12.096Z] 
[2026-04-05T11:06:12.096Z] TASK [deploy server an-test-deploy1 from an-a01n01 on storage 70316199-07b5-4a00-a81f-d24be730e79c] ***
[2026-04-05T11:06:12.096Z] skipping: [an-a01n01]
[2026-04-05T11:06:12.096Z] skipping: [an-a01n02]
[2026-04-05T11:06:12.096Z] 
[2026-04-05T11:06:12.096Z] TASK [Wait up to 10 minutes for an-test-deploy1 to be started in pacemaker] ****
[2026-04-05T11:06:12.466Z] skipping: [an-a01n01]
[2026-04-05T11:06:12.466Z] skipping: [an-a01n02]
[2026-04-05T11:06:12.466Z] 
[2026-04-05T11:06:12.466Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-05T11:06:12.466Z] skipping: [an-a01n01]
[2026-04-05T11:06:12.466Z] skipping: [an-a01n02]
[2026-04-05T11:06:12.466Z] 
[2026-04-05T11:06:12.466Z] TASK [Wait up to 10 minutes for an-test-deploy1 to start installation] *********
[2026-04-05T11:06:12.466Z] skipping: [an-a01n01]
[2026-04-05T11:06:12.466Z] skipping: [an-a01n02]
[2026-04-05T11:06:12.466Z] 
[2026-04-05T11:06:12.466Z] TASK [Wait up to 10 minutes for an-test-deploy1 to complete installation] ******
[2026-04-05T11:06:12.466Z] skipping: [an-a01n01]
[2026-04-05T11:06:12.466Z] skipping: [an-a01n02]
[2026-04-05T11:06:12.466Z] 
[2026-04-05T11:06:12.466Z] TASK [deploy server 2 from "an-a01n02"] ****************************************
[2026-04-05T11:06:12.466Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02
[2026-04-05T11:06:12.466Z] 
[2026-04-05T11:06:12.466Z] TASK [check if server an-test-deploy2 exists] **********************************
[2026-04-05T11:06:14.282Z] skipping: [an-a01n01]
[2026-04-05T11:06:14.282Z] ok: [an-a01n02]
[2026-04-05T11:06:14.282Z] 
[2026-04-05T11:06:14.282Z] TASK [deploy server an-test-deploy2 from an-a01n02 on storage 70316199-07b5-4a00-a81f-d24be730e79c] ***
[2026-04-05T11:06:14.282Z] skipping: [an-a01n01]
[2026-04-05T11:06:14.282Z] skipping: [an-a01n02]
[2026-04-05T11:06:14.282Z] 
[2026-04-05T11:06:14.282Z] TASK [Wait up to 10 minutes for an-test-deploy2 to be started in pacemaker] ****
[2026-04-05T11:06:14.282Z] skipping: [an-a01n01]
[2026-04-05T11:06:14.282Z] skipping: [an-a01n02]
[2026-04-05T11:06:14.282Z] 
[2026-04-05T11:06:14.282Z] TASK [Wait up to 10 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-05T11:06:14.282Z] skipping: [an-a01n01]
[2026-04-05T11:06:14.282Z] skipping: [an-a01n02]
[2026-04-05T11:06:14.282Z] 
[2026-04-05T11:06:14.282Z] TASK [Wait up to 10 minutes for an-test-deploy2 to start installation] *********
[2026-04-05T11:06:14.282Z] skipping: [an-a01n01]
[2026-04-05T11:06:14.282Z] skipping: [an-a01n02]
[2026-04-05T11:06:14.282Z] 
[2026-04-05T11:06:14.282Z] TASK [Wait up to 10 minutes for an-test-deploy2 to complete installation] ******
[2026-04-05T11:06:14.282Z] skipping: [an-a01n01]
[2026-04-05T11:06:14.282Z] skipping: [an-a01n02]
[2026-04-05T11:06:14.282Z] 
[2026-04-05T11:06:14.282Z] TASK [check servers status] ****************************************************
[2026-04-05T11:06:14.650Z] included: /srv/anvil/anvil-test-suite/playbooks/common/server_started.yml for an-a01n01, an-a01n02
[2026-04-05T11:06:14.650Z] 
[2026-04-05T11:06:14.650Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T11:06:16.469Z] ok: [an-a01n01]
[2026-04-05T11:06:16.469Z] ok: [an-a01n02]
[2026-04-05T11:06:16.469Z] 
[2026-04-05T11:06:16.469Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T11:06:18.924Z] ok: [an-a01n01]
[2026-04-05T11:06:18.924Z] ok: [an-a01n02]
[2026-04-05T11:06:18.924Z] 
[2026-04-05T11:06:18.924Z] PLAY [delete extra server tests] ***********************************************
[2026-04-05T11:06:18.924Z] 
[2026-04-05T11:06:18.924Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:06:20.758Z] ok: [an-a01n01]
[2026-04-05T11:06:20.758Z] ok: [an-a01n02]
[2026-04-05T11:06:20.758Z] 
[2026-04-05T11:06:20.758Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T11:06:20.758Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T11:06:20.758Z] 
[2026-04-05T11:06:20.758Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T11:06:24.817Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T11:06:24.817Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T11:06:24.817Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T11:06:24.817Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T11:06:24.817Z] 
[2026-04-05T11:06:24.817Z] TASK [delete servers] **********************************************************
[2026-04-05T11:06:24.817Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-05T11:06:24.817Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-05T11:06:24.817Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-05T11:06:24.817Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-05T11:06:24.817Z] 
[2026-04-05T11:06:24.817Z] TASK [check if server an-test-deploy2 exists] **********************************
[2026-04-05T11:06:26.082Z] skipping: [an-a01n01]
[2026-04-05T11:06:26.082Z] ok: [an-a01n02]
[2026-04-05T11:06:26.082Z] 
[2026-04-05T11:06:26.082Z] TASK [delete server an-test-deploy2 from an-a01n02] ****************************
[2026-04-05T11:06:32.353Z] skipping: [an-a01n01]
[2026-04-05T11:06:32.353Z] changed: [an-a01n02]
[2026-04-05T11:06:32.353Z] 
[2026-04-05T11:06:32.353Z] TASK [Wait up to 10 minutes for an-test-deploy2 to be delete in pacemaker] *****
[2026-04-05T11:07:13.166Z] skipping: [an-a01n01]
[2026-04-05T11:07:13.166Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy2 to be delete in pacemaker (60 retries left).
[2026-04-05T11:07:13.166Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy2 to be delete in pacemaker (59 retries left).
[2026-04-05T11:07:13.166Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy2 to be delete in pacemaker (58 retries left).
[2026-04-05T11:07:13.166Z] ok: [an-a01n02]
[2026-04-05T11:07:13.166Z] 
[2026-04-05T11:07:13.166Z] TASK [Wait up to 10 minutes for an-test-deploy2 drbd storage to be removed] ****
[2026-04-05T11:07:13.166Z] skipping: [an-a01n01]
[2026-04-05T11:07:13.166Z] ok: [an-a01n02]
[2026-04-05T11:07:13.166Z] 
[2026-04-05T11:07:13.166Z] TASK [Wait up to 10 minutes for an-test-deploy2 lvm volume to be removed] ******
[2026-04-05T11:07:13.166Z] skipping: [an-a01n01]
[2026-04-05T11:07:13.166Z] ok: [an-a01n02]
[2026-04-05T11:07:13.166Z] 
[2026-04-05T11:07:13.166Z] TASK [check if server an-test-deploy3 exists] **********************************
[2026-04-05T11:07:13.166Z] skipping: [an-a01n01]
[2026-04-05T11:07:13.166Z] ok: [an-a01n02]
[2026-04-05T11:07:13.166Z] 
[2026-04-05T11:07:13.166Z] TASK [delete server an-test-deploy3 from an-a01n02] ****************************
[2026-04-05T11:07:17.224Z] skipping: [an-a01n01]
[2026-04-05T11:07:17.224Z] changed: [an-a01n02]
[2026-04-05T11:07:17.224Z] 
[2026-04-05T11:07:17.224Z] TASK [Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker] *****
[2026-04-05T11:08:27.338Z] skipping: [an-a01n01]
[2026-04-05T11:08:27.338Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker (60 retries left).
[2026-04-05T11:08:27.339Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker (59 retries left).
[2026-04-05T11:08:27.339Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker (58 retries left).
[2026-04-05T11:08:27.339Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker (57 retries left).
[2026-04-05T11:08:27.339Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker (56 retries left).
[2026-04-05T11:08:27.339Z] ok: [an-a01n02]
[2026-04-05T11:08:27.339Z] 
[2026-04-05T11:08:27.339Z] TASK [Wait up to 10 minutes for an-test-deploy3 drbd storage to be removed] ****
[2026-04-05T11:08:27.339Z] skipping: [an-a01n01]
[2026-04-05T11:08:27.339Z] ok: [an-a01n02]
[2026-04-05T11:08:27.339Z] 
[2026-04-05T11:08:27.339Z] TASK [Wait up to 10 minutes for an-test-deploy3 lvm volume to be removed] ******
[2026-04-05T11:08:27.339Z] skipping: [an-a01n01]
[2026-04-05T11:08:27.339Z] ok: [an-a01n02]
[2026-04-05T11:08:27.339Z] 
[2026-04-05T11:08:27.339Z] TASK [check if server an-test-deploy4 exists] **********************************
[2026-04-05T11:08:27.339Z] skipping: [an-a01n01]
[2026-04-05T11:08:27.339Z] ok: [an-a01n02]
[2026-04-05T11:08:27.339Z] 
[2026-04-05T11:08:27.339Z] TASK [delete server an-test-deploy4 from an-a01n02] ****************************
[2026-04-05T11:08:27.339Z] skipping: [an-a01n01]
[2026-04-05T11:08:27.339Z] changed: [an-a01n02]
[2026-04-05T11:08:27.339Z] 
[2026-04-05T11:08:27.339Z] TASK [Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker] *****
[2026-04-05T11:09:25.889Z] skipping: [an-a01n01]
[2026-04-05T11:09:25.889Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker (60 retries left).
[2026-04-05T11:09:25.889Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker (59 retries left).
[2026-04-05T11:09:25.889Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker (58 retries left).
[2026-04-05T11:09:25.889Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker (57 retries left).
[2026-04-05T11:09:25.889Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker (56 retries left).
[2026-04-05T11:09:25.889Z] ok: [an-a01n02]
[2026-04-05T11:09:25.889Z] 
[2026-04-05T11:09:25.889Z] TASK [Wait up to 10 minutes for an-test-deploy4 drbd storage to be removed] ****
[2026-04-05T11:09:25.889Z] skipping: [an-a01n01]
[2026-04-05T11:09:25.889Z] ok: [an-a01n02]
[2026-04-05T11:09:25.889Z] 
[2026-04-05T11:09:25.889Z] TASK [Wait up to 10 minutes for an-test-deploy4 lvm volume to be removed] ******
[2026-04-05T11:09:26.258Z] skipping: [an-a01n01]
[2026-04-05T11:09:26.258Z] ok: [an-a01n02]
[2026-04-05T11:09:26.258Z] 
[2026-04-05T11:09:26.258Z] TASK [check if server an-test-deploy5 exists] **********************************
[2026-04-05T11:09:28.076Z] skipping: [an-a01n01]
[2026-04-05T11:09:28.076Z] ok: [an-a01n02]
[2026-04-05T11:09:28.076Z] 
[2026-04-05T11:09:28.076Z] TASK [delete server an-test-deploy5 from an-a01n02] ****************************
[2026-04-05T11:09:34.345Z] skipping: [an-a01n01]
[2026-04-05T11:09:34.345Z] changed: [an-a01n02]
[2026-04-05T11:09:34.345Z] 
[2026-04-05T11:09:34.345Z] TASK [Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker] *****
[2026-04-05T11:10:44.490Z] skipping: [an-a01n01]
[2026-04-05T11:10:44.490Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker (60 retries left).
[2026-04-05T11:10:44.490Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker (59 retries left).
[2026-04-05T11:10:44.490Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker (58 retries left).
[2026-04-05T11:10:44.490Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker (57 retries left).
[2026-04-05T11:10:44.490Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker (56 retries left).
[2026-04-05T11:10:44.490Z] ok: [an-a01n02]
[2026-04-05T11:10:44.490Z] 
[2026-04-05T11:10:44.490Z] TASK [Wait up to 10 minutes for an-test-deploy5 drbd storage to be removed] ****
[2026-04-05T11:10:44.490Z] skipping: [an-a01n01]
[2026-04-05T11:10:44.490Z] ok: [an-a01n02]
[2026-04-05T11:10:44.490Z] 
[2026-04-05T11:10:44.490Z] TASK [Wait up to 10 minutes for an-test-deploy5 lvm volume to be removed] ******
[2026-04-05T11:10:44.490Z] skipping: [an-a01n01]
[2026-04-05T11:10:44.490Z] ok: [an-a01n02]
[2026-04-05T11:10:44.490Z] 
[2026-04-05T11:10:44.490Z] PLAY [create server for DR testing] ********************************************
[2026-04-05T11:10:44.490Z] 
[2026-04-05T11:10:44.490Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:10:44.490Z] ok: [an-a01n02]
[2026-04-05T11:10:44.490Z] ok: [an-a01n01]
[2026-04-05T11:10:44.490Z] 
[2026-04-05T11:10:44.490Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T11:10:44.490Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T11:10:44.490Z] 
[2026-04-05T11:10:44.490Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T11:10:44.490Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T11:10:44.490Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T11:10:44.490Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T11:10:44.490Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T11:10:44.490Z] 
[2026-04-05T11:10:44.490Z] TASK [check anvil-provision-server storage group] ******************************
[2026-04-05T11:10:44.490Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_storage_group.yml for an-a01n01, an-a01n02
[2026-04-05T11:10:44.490Z] 
[2026-04-05T11:10:44.490Z] TASK [get anvil-provision-server storage group] ********************************
[2026-04-05T11:10:46.328Z] changed: [an-a01n02]
[2026-04-05T11:10:46.328Z] changed: [an-a01n01]
[2026-04-05T11:10:46.328Z] 
[2026-04-05T11:10:46.328Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-05T11:10:46.328Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_install_iso.yml for an-a01n01, an-a01n02
[2026-04-05T11:10:46.328Z] 
[2026-04-05T11:10:46.328Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-05T11:10:51.661Z] changed: [an-a01n01]
[2026-04-05T11:10:51.661Z] changed: [an-a01n02]
[2026-04-05T11:10:51.661Z] 
[2026-04-05T11:10:51.661Z] TASK [deploy server 1 from "an-a01n01"] ****************************************
[2026-04-05T11:10:51.661Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02
[2026-04-05T11:10:51.661Z] 
[2026-04-05T11:10:51.661Z] TASK [check if server an-test-deploy1 exists] **********************************
[2026-04-05T11:10:53.543Z] skipping: [an-a01n02]
[2026-04-05T11:10:53.543Z] ok: [an-a01n01]
[2026-04-05T11:10:53.543Z] 
[2026-04-05T11:10:53.543Z] TASK [deploy server an-test-deploy1 from an-a01n01 on storage 70316199-07b5-4a00-a81f-d24be730e79c] ***
[2026-04-05T11:10:53.543Z] skipping: [an-a01n01]
[2026-04-05T11:10:53.543Z] skipping: [an-a01n02]
[2026-04-05T11:10:53.543Z] 
[2026-04-05T11:10:53.543Z] TASK [Wait up to 10 minutes for an-test-deploy1 to be started in pacemaker] ****
[2026-04-05T11:10:53.543Z] skipping: [an-a01n01]
[2026-04-05T11:10:53.543Z] skipping: [an-a01n02]
[2026-04-05T11:10:53.543Z] 
[2026-04-05T11:10:53.543Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-05T11:10:53.543Z] skipping: [an-a01n01]
[2026-04-05T11:10:53.543Z] skipping: [an-a01n02]
[2026-04-05T11:10:53.543Z] 
[2026-04-05T11:10:53.543Z] TASK [Wait up to 10 minutes for an-test-deploy1 to start installation] *********
[2026-04-05T11:10:53.543Z] skipping: [an-a01n01]
[2026-04-05T11:10:53.543Z] skipping: [an-a01n02]
[2026-04-05T11:10:53.543Z] 
[2026-04-05T11:10:53.543Z] TASK [Wait up to 10 minutes for an-test-deploy1 to complete installation] ******
[2026-04-05T11:10:53.543Z] skipping: [an-a01n01]
[2026-04-05T11:10:53.543Z] skipping: [an-a01n02]
[2026-04-05T11:10:53.543Z] 
[2026-04-05T11:10:53.543Z] TASK [check servers status] ****************************************************
[2026-04-05T11:10:53.925Z] included: /srv/anvil/anvil-test-suite/playbooks/common/server_started.yml for an-a01n01, an-a01n02
[2026-04-05T11:10:53.925Z] 
[2026-04-05T11:10:53.925Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T11:10:55.777Z] ok: [an-a01n02]
[2026-04-05T11:10:55.777Z] ok: [an-a01n01]
[2026-04-05T11:10:55.777Z] 
[2026-04-05T11:10:55.777Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T11:10:58.231Z] ok: [an-a01n02]
[2026-04-05T11:10:58.231Z] ok: [an-a01n01]
[2026-04-05T11:10:58.231Z] 
[2026-04-05T11:10:58.231Z] PLAY [DR tests] ****************************************************************
[2026-04-05T11:10:58.231Z] skipping: no hosts matched
[2026-04-05T11:10:58.231Z] 
[2026-04-05T11:10:58.231Z] PLAY [upgrade server test] *****************************************************
[2026-04-05T11:10:58.231Z] 
[2026-04-05T11:10:58.231Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:10:59.496Z] ok: [an-a01n02]
[2026-04-05T11:10:59.496Z] ok: [an-a01n01]
[2026-04-05T11:10:59.496Z] 
[2026-04-05T11:10:59.496Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T11:10:59.496Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T11:10:59.496Z] 
[2026-04-05T11:10:59.496Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T11:11:03.551Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T11:11:03.551Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T11:11:03.551Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T11:11:03.551Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T11:11:03.551Z] 
[2026-04-05T11:11:03.551Z] TASK [check anvil-provision-server storage group] ******************************
[2026-04-05T11:11:03.551Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_storage_group.yml for an-a01n01, an-a01n02
[2026-04-05T11:11:03.551Z] 
[2026-04-05T11:11:03.551Z] TASK [get anvil-provision-server storage group] ********************************
[2026-04-05T11:11:09.812Z] changed: [an-a01n01]
[2026-04-05T11:11:09.812Z] changed: [an-a01n02]
[2026-04-05T11:11:09.812Z] 
[2026-04-05T11:11:09.812Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-05T11:11:09.812Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_install_iso.yml for an-a01n01, an-a01n02
[2026-04-05T11:11:09.812Z] 
[2026-04-05T11:11:09.812Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-05T11:11:14.879Z] changed: [an-a01n02]
[2026-04-05T11:11:14.879Z] changed: [an-a01n01]
[2026-04-05T11:11:14.879Z] 
[2026-04-05T11:11:14.879Z] TASK [deploy server 1 from "an-a01n01"] ****************************************
[2026-04-05T11:11:14.879Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02
[2026-04-05T11:11:14.879Z] 
[2026-04-05T11:11:14.879Z] TASK [check if server an-test-deploy1 exists] **********************************
[2026-04-05T11:11:15.665Z] skipping: [an-a01n02]
[2026-04-05T11:11:15.665Z] ok: [an-a01n01]
[2026-04-05T11:11:15.665Z] 
[2026-04-05T11:11:15.665Z] TASK [deploy server an-test-deploy1 from an-a01n01 on storage 70316199-07b5-4a00-a81f-d24be730e79c] ***
[2026-04-05T11:11:15.665Z] skipping: [an-a01n01]
[2026-04-05T11:11:15.665Z] skipping: [an-a01n02]
[2026-04-05T11:11:15.665Z] 
[2026-04-05T11:11:15.665Z] TASK [Wait up to 10 minutes for an-test-deploy1 to be started in pacemaker] ****
[2026-04-05T11:11:15.665Z] skipping: [an-a01n01]
[2026-04-05T11:11:15.665Z] skipping: [an-a01n02]
[2026-04-05T11:11:15.665Z] 
[2026-04-05T11:11:15.665Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-05T11:11:15.665Z] skipping: [an-a01n01]
[2026-04-05T11:11:15.665Z] skipping: [an-a01n02]
[2026-04-05T11:11:15.665Z] 
[2026-04-05T11:11:15.665Z] TASK [Wait up to 10 minutes for an-test-deploy1 to start installation] *********
[2026-04-05T11:11:16.034Z] skipping: [an-a01n01]
[2026-04-05T11:11:16.034Z] skipping: [an-a01n02]
[2026-04-05T11:11:16.034Z] 
[2026-04-05T11:11:16.034Z] TASK [Wait up to 10 minutes for an-test-deploy1 to complete installation] ******
[2026-04-05T11:11:16.034Z] skipping: [an-a01n01]
[2026-04-05T11:11:16.034Z] skipping: [an-a01n02]
[2026-04-05T11:11:16.034Z] 
[2026-04-05T11:11:16.034Z] TASK [check servers status] ****************************************************
[2026-04-05T11:11:16.034Z] included: /srv/anvil/anvil-test-suite/playbooks/common/server_started.yml for an-a01n01, an-a01n02
[2026-04-05T11:11:16.034Z] 
[2026-04-05T11:11:16.034Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-05T11:11:17.848Z] ok: [an-a01n02]
[2026-04-05T11:11:17.848Z] ok: [an-a01n01]
[2026-04-05T11:11:17.848Z] 
[2026-04-05T11:11:17.848Z] TASK [check servers are started all on the same node] **************************
[2026-04-05T11:11:19.664Z] ok: [an-a01n02]
[2026-04-05T11:11:19.664Z] ok: [an-a01n01]
[2026-04-05T11:11:19.664Z] 
[2026-04-05T11:11:19.664Z] PLAY [enable upgrade repo] *****************************************************
[2026-04-05T11:11:19.664Z] 
[2026-04-05T11:11:19.664Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:11:22.120Z] ok: [an-a01n02]
[2026-04-05T11:11:22.120Z] ok: [an-a01n01]
[2026-04-05T11:11:22.120Z] ok: [an-striker02]
[2026-04-05T11:11:22.120Z] ok: [an-striker01]
[2026-04-05T11:11:22.120Z] 
[2026-04-05T11:11:22.120Z] TASK [enable anvil test-update repo] *******************************************
[2026-04-05T11:11:31.449Z] changed: [an-a01n02]
[2026-04-05T11:11:31.449Z] changed: [an-a01n01]
[2026-04-05T11:11:31.449Z] changed: [an-striker02]
[2026-04-05T11:11:31.449Z] changed: [an-striker01]
[2026-04-05T11:11:31.449Z] 
[2026-04-05T11:11:31.449Z] TASK [store current uptime] ****************************************************
[2026-04-05T11:11:31.449Z] ok: [an-striker01]
[2026-04-05T11:11:31.449Z] ok: [an-striker02]
[2026-04-05T11:11:31.449Z] ok: [an-a01n01]
[2026-04-05T11:11:31.449Z] ok: [an-a01n02]
[2026-04-05T11:11:31.449Z] 
[2026-04-05T11:11:31.449Z] PLAY [start upgrade process] ***************************************************
[2026-04-05T11:11:31.449Z] 
[2026-04-05T11:11:31.449Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:11:33.265Z] ok: [an-striker01]
[2026-04-05T11:11:33.265Z] 
[2026-04-05T11:11:33.265Z] TASK [upgrade and reboot all nodes but an-striker01] ***************************
[2026-04-05T11:32:54.582Z] changed: [an-striker01]
[2026-04-05T11:32:54.582Z] 
[2026-04-05T11:32:54.582Z] TASK [reboot striker01] ********************************************************
[2026-04-05T11:33:18.282Z] changed: [an-striker01]
[2026-04-05T11:33:18.282Z] 
[2026-04-05T11:33:18.282Z] PLAY [reconnect all nodes] *****************************************************
[2026-04-05T11:33:18.282Z] 
[2026-04-05T11:33:18.282Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:33:20.102Z] ok: [an-striker02]
[2026-04-05T11:33:20.102Z] ok: [an-a01n01]
[2026-04-05T11:33:20.102Z] ok: [an-a01n02]
[2026-04-05T11:33:20.102Z] ok: [an-striker01]
[2026-04-05T11:33:20.102Z] 
[2026-04-05T11:33:20.102Z] TASK [wait for node "an-striker01" to be back online] **************************
[2026-04-05T11:33:20.471Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_node.yml for an-striker01, an-striker02, an-a01n01, an-a01n02
[2026-04-05T11:33:20.471Z] 
[2026-04-05T11:33:20.471Z] TASK [wait for node "all" to be back online] ***********************************
[2026-04-05T11:36:45.442Z] ok: [an-a01n01 -> localhost]
[2026-04-05T11:36:45.442Z] ok: [an-striker02 -> localhost]
[2026-04-05T11:36:45.442Z] ok: [an-striker01 -> localhost]
[2026-04-05T11:36:45.442Z] ok: [an-a01n02 -> localhost]
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] TASK [check nodes uptime after reboot] *****************************************
[2026-04-05T11:36:45.442Z] fatal: [an-a01n01]: FAILED! => {"changed": true, "cmd": "if [ \"644\" -ge \"623\" ]; then\n  echo \"Node current uptime higher than expected! tool failed to reboot nodes?\"\n  exit 1\nfi\n", "delta": "0:00:00.007732", "end": "2026-04-05 07:36:31.442988", "msg": "non-zero return code", "rc": 1, "start": "2026-04-05 07:36:31.435256", "stderr": "", "stderr_lines": [], "stdout": "Node current uptime higher than expected! tool failed to reboot nodes?", "stdout_lines": ["Node current uptime higher than expected! tool failed to reboot nodes?"]}
[2026-04-05T11:36:45.442Z] ...ignoring
[2026-04-05T11:36:45.442Z] changed: [an-a01n02]
[2026-04-05T11:36:45.442Z] changed: [an-striker02]
[2026-04-05T11:36:45.442Z] changed: [an-striker01]
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] PLAY [check all nodes status after upgrade] ************************************
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:36:45.442Z] ok: [an-striker02]
[2026-04-05T11:36:45.442Z] ok: [an-a01n02]
[2026-04-05T11:36:45.442Z] ok: [an-a01n01]
[2026-04-05T11:36:45.442Z] ok: [an-striker01]
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] TASK [check perl-Log-Journald package version] *********************************
[2026-04-05T11:36:45.442Z] changed: [an-a01n01]
[2026-04-05T11:36:45.442Z] changed: [an-a01n02]
[2026-04-05T11:36:45.442Z] changed: [an-striker02]
[2026-04-05T11:36:45.442Z] changed: [an-striker01]
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] PLAY [check nodes status after upgrade] ****************************************
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:36:45.442Z] ok: [an-a01n02]
[2026-04-05T11:36:45.442Z] ok: [an-a01n01]
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] TASK [check kmod-drdb package version] *****************************************
[2026-04-05T11:36:45.442Z] changed: [an-a01n02]
[2026-04-05T11:36:45.442Z] changed: [an-a01n01]
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] PLAY [check hosted node status after upgrade] **********************************
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:36:45.442Z] ok: [an-a01n02]
[2026-04-05T11:36:45.442Z] ok: [an-a01n01]
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T11:36:45.442Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T11:36:45.442Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T11:36:45.442Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T11:36:45.442Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T11:36:45.442Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] TASK [wait servers to be started in pacemaker] *********************************
[2026-04-05T11:36:45.442Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n01, an-a01n02
[2026-04-05T11:36:45.442Z] 
[2026-04-05T11:36:45.442Z] TASK [Wait up to 10 minutes for an-test-deploy1 to be started in pacemaker] ****
[2026-04-05T11:36:47.260Z] ok: [an-a01n02]
[2026-04-05T11:36:47.260Z] ok: [an-a01n01]
[2026-04-05T11:36:47.260Z] 
[2026-04-05T11:36:47.260Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-05T11:36:47.260Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02
[2026-04-05T11:36:47.260Z] 
[2026-04-05T11:36:47.260Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-05T11:36:47.629Z] ok: [an-a01n02]
[2026-04-05T11:36:47.629Z] ok: [an-a01n01]
[2026-04-05T11:36:47.629Z] 
[2026-04-05T11:36:47.629Z] TASK [wait servers to respond to ping] *****************************************
[2026-04-05T11:36:47.997Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02
[2026-04-05T11:36:47.997Z] 
[2026-04-05T11:36:47.997Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ping] ************
[2026-04-05T11:36:48.784Z] ok: [an-a01n01]
[2026-04-05T11:36:48.784Z] ok: [an-a01n02]
[2026-04-05T11:36:48.784Z] 
[2026-04-05T11:36:48.784Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-05T11:36:48.784Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n01, an-a01n02
[2026-04-05T11:36:48.784Z] 
[2026-04-05T11:36:48.784Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-05T11:36:49.572Z] ok: [an-a01n02]
[2026-04-05T11:36:49.572Z] ok: [an-a01n01]
[2026-04-05T11:36:49.572Z] 
[2026-04-05T11:36:49.572Z] PLAY [test anvil package self update] ******************************************
[2026-04-05T11:36:49.572Z] 
[2026-04-05T11:36:49.572Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:36:51.388Z] ok: [an-a01n02]
[2026-04-05T11:36:51.388Z] ok: [an-a01n01]
[2026-04-05T11:36:51.388Z] ok: [an-striker01]
[2026-04-05T11:36:51.388Z] ok: [an-striker02]
[2026-04-05T11:36:51.388Z] 
[2026-04-05T11:36:51.388Z] TASK [stop anvil and scan-core services] ***************************************
[2026-04-05T11:36:55.445Z] changed: [an-a01n02] => (item=anvil-daemon.service)
[2026-04-05T11:36:55.445Z] changed: [an-a01n01] => (item=anvil-daemon.service)
[2026-04-05T11:36:55.445Z] changed: [an-striker02] => (item=anvil-daemon.service)
[2026-04-05T11:36:55.445Z] changed: [an-a01n02] => (item=scancore.service)
[2026-04-05T11:36:55.445Z] changed: [an-a01n01] => (item=scancore.service)
[2026-04-05T11:36:55.445Z] changed: [an-striker02] => (item=scancore.service)
[2026-04-05T11:36:55.445Z] changed: [an-striker01] => (item=anvil-daemon.service)
[2026-04-05T11:36:55.445Z] changed: [an-striker01] => (item=scancore.service)
[2026-04-05T11:36:55.445Z] 
[2026-04-05T11:36:55.445Z] PLAY [test striker package self update] ****************************************
[2026-04-05T11:36:55.445Z] 
[2026-04-05T11:36:55.445Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:36:56.712Z] ok: [an-striker02]
[2026-04-05T11:36:56.712Z] ok: [an-striker01]
[2026-04-05T11:36:56.712Z] 
[2026-04-05T11:36:56.712Z] TASK [reinstall anvil-core and anvil-striker] **********************************
[2026-04-05T11:37:20.426Z] changed: [an-striker02]
[2026-04-05T11:37:20.426Z] changed: [an-striker01]
[2026-04-05T11:37:20.426Z] 
[2026-04-05T11:37:20.426Z] PLAY [test node package self update] *******************************************
[2026-04-05T11:37:20.426Z] 
[2026-04-05T11:37:20.426Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:37:21.213Z] ok: [an-a01n02]
[2026-04-05T11:37:21.213Z] ok: [an-a01n01]
[2026-04-05T11:37:21.213Z] 
[2026-04-05T11:37:21.213Z] TASK [reinstall anvil-core and anvil-node] *************************************
[2026-04-05T11:37:44.907Z] changed: [an-a01n02]
[2026-04-05T11:37:44.907Z] changed: [an-a01n01]
[2026-04-05T11:37:44.907Z] 
[2026-04-05T11:37:44.907Z] PLAY [test dr package self update] *********************************************
[2026-04-05T11:37:44.907Z] skipping: no hosts matched
[2026-04-05T11:37:44.907Z] 
[2026-04-05T11:37:44.907Z] PLAY [test anvil package self update] ******************************************
[2026-04-05T11:37:44.907Z] 
[2026-04-05T11:37:44.907Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:37:46.183Z] ok: [an-striker02]
[2026-04-05T11:37:46.183Z] ok: [an-a01n02]
[2026-04-05T11:37:46.183Z] ok: [an-striker01]
[2026-04-05T11:37:46.183Z] ok: [an-a01n01]
[2026-04-05T11:37:46.183Z] 
[2026-04-05T11:37:46.183Z] TASK [start anvil and scan-core services] **************************************
[2026-04-05T11:37:48.639Z] changed: [an-a01n01] => (item=anvil-daemon.service)
[2026-04-05T11:37:48.639Z] changed: [an-a01n02] => (item=anvil-daemon.service)
[2026-04-05T11:37:48.639Z] changed: [an-striker02] => (item=anvil-daemon.service)
[2026-04-05T11:37:48.639Z] changed: [an-striker01] => (item=anvil-daemon.service)
[2026-04-05T11:37:48.639Z] changed: [an-a01n02] => (item=scancore.service)
[2026-04-05T11:37:48.639Z] changed: [an-a01n01] => (item=scancore.service)
[2026-04-05T11:37:48.639Z] changed: [an-striker02] => (item=scancore.service)
[2026-04-05T11:37:48.639Z] changed: [an-striker01] => (item=scancore.service)
[2026-04-05T11:37:48.639Z] 
[2026-04-05T11:37:48.639Z] PLAY [delete server tests] *****************************************************
[2026-04-05T11:37:48.639Z] 
[2026-04-05T11:37:48.639Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:37:50.456Z] ok: [an-a01n02]
[2026-04-05T11:37:50.456Z] ok: [an-a01n01]
[2026-04-05T11:37:50.456Z] 
[2026-04-05T11:37:50.456Z] TASK [check cluster is in good state] ******************************************
[2026-04-05T11:37:50.456Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-05T11:37:50.456Z] 
[2026-04-05T11:37:50.456Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-05T11:37:54.514Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-05T11:37:54.514Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-05T11:37:54.514Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-05T11:37:54.514Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-05T11:37:54.514Z] 
[2026-04-05T11:37:54.514Z] TASK [delete servers] **********************************************************
[2026-04-05T11:37:54.883Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-05T11:37:54.883Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-05T11:37:54.883Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-05T11:37:54.883Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-05T11:37:54.883Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-05T11:37:54.883Z] 
[2026-04-05T11:37:54.883Z] TASK [check if server an-test-deploy1 exists] **********************************
[2026-04-05T11:37:56.701Z] skipping: [an-a01n01]
[2026-04-05T11:37:56.701Z] ok: [an-a01n02]
[2026-04-05T11:37:56.701Z] 
[2026-04-05T11:37:56.701Z] TASK [delete server an-test-deploy1 from an-a01n02] ****************************
[2026-04-05T11:38:02.963Z] skipping: [an-a01n01]
[2026-04-05T11:38:02.963Z] changed: [an-a01n02]
[2026-04-05T11:38:02.963Z] 
[2026-04-05T11:38:02.963Z] TASK [Wait up to 10 minutes for an-test-deploy1 to be delete in pacemaker] *****
[2026-04-05T11:38:51.847Z] skipping: [an-a01n01]
[2026-04-05T11:38:51.847Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to be delete in pacemaker (60 retries left).
[2026-04-05T11:38:51.847Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to be delete in pacemaker (59 retries left).
[2026-04-05T11:38:51.847Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to be delete in pacemaker (58 retries left).
[2026-04-05T11:38:51.847Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to be delete in pacemaker (57 retries left).
[2026-04-05T11:38:51.847Z] ok: [an-a01n02]
[2026-04-05T11:38:51.847Z] 
[2026-04-05T11:38:51.847Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be removed] ****
[2026-04-05T11:38:51.847Z] skipping: [an-a01n01]
[2026-04-05T11:38:51.847Z] ok: [an-a01n02]
[2026-04-05T11:38:51.847Z] 
[2026-04-05T11:38:51.847Z] TASK [Wait up to 10 minutes for an-test-deploy1 lvm volume to be removed] ******
[2026-04-05T11:38:51.847Z] skipping: [an-a01n01]
[2026-04-05T11:38:51.847Z] ok: [an-a01n02]
[2026-04-05T11:38:51.847Z] 
[2026-04-05T11:38:51.847Z] TASK [check if server an-test-deploy2 exists] **********************************
[2026-04-05T11:38:53.113Z] skipping: [an-a01n01]
[2026-04-05T11:38:53.113Z] fatal: [an-a01n02]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy2\n", "delta": "0:00:01.021024", "end": "2026-04-05 07:38:52.830216", "msg": "non-zero return code", "rc": 1, "start": "2026-04-05 07:38:51.809192", "stderr": "Error: resource or tag id 'an-test-deploy2' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy2' not found"], "stdout": "", "stdout_lines": []}
[2026-04-05T11:38:53.113Z] ...ignoring
[2026-04-05T11:38:53.113Z] 
[2026-04-05T11:38:53.113Z] TASK [delete server an-test-deploy2 from an-a01n02] ****************************
[2026-04-05T11:38:53.113Z] skipping: [an-a01n01]
[2026-04-05T11:38:53.113Z] skipping: [an-a01n02]
[2026-04-05T11:38:53.113Z] 
[2026-04-05T11:38:53.113Z] TASK [Wait up to 10 minutes for an-test-deploy2 to be delete in pacemaker] *****
[2026-04-05T11:38:53.113Z] skipping: [an-a01n01]
[2026-04-05T11:38:53.113Z] skipping: [an-a01n02]
[2026-04-05T11:38:53.113Z] 
[2026-04-05T11:38:53.113Z] TASK [Wait up to 10 minutes for an-test-deploy2 drbd storage to be removed] ****
[2026-04-05T11:38:53.481Z] skipping: [an-a01n01]
[2026-04-05T11:38:53.481Z] skipping: [an-a01n02]
[2026-04-05T11:38:53.481Z] 
[2026-04-05T11:38:53.481Z] TASK [Wait up to 10 minutes for an-test-deploy2 lvm volume to be removed] ******
[2026-04-05T11:38:53.481Z] skipping: [an-a01n01]
[2026-04-05T11:38:53.481Z] skipping: [an-a01n02]
[2026-04-05T11:38:53.481Z] 
[2026-04-05T11:38:53.481Z] TASK [check if server an-test-deploy3 exists] **********************************
[2026-04-05T11:38:54.749Z] skipping: [an-a01n01]
[2026-04-05T11:38:54.749Z] fatal: [an-a01n02]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy3\n", "delta": "0:00:00.903314", "end": "2026-04-05 07:38:54.591450", "msg": "non-zero return code", "rc": 1, "start": "2026-04-05 07:38:53.688136", "stderr": "Error: resource or tag id 'an-test-deploy3' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy3' not found"], "stdout": "", "stdout_lines": []}
[2026-04-05T11:38:54.749Z] ...ignoring
[2026-04-05T11:38:54.749Z] 
[2026-04-05T11:38:54.749Z] TASK [delete server an-test-deploy3 from an-a01n02] ****************************
[2026-04-05T11:38:55.118Z] skipping: [an-a01n01]
[2026-04-05T11:38:55.118Z] skipping: [an-a01n02]
[2026-04-05T11:38:55.118Z] 
[2026-04-05T11:38:55.118Z] TASK [Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker] *****
[2026-04-05T11:38:55.118Z] skipping: [an-a01n01]
[2026-04-05T11:38:55.118Z] skipping: [an-a01n02]
[2026-04-05T11:38:55.118Z] 
[2026-04-05T11:38:55.118Z] TASK [Wait up to 10 minutes for an-test-deploy3 drbd storage to be removed] ****
[2026-04-05T11:38:55.118Z] skipping: [an-a01n01]
[2026-04-05T11:38:55.118Z] skipping: [an-a01n02]
[2026-04-05T11:38:55.118Z] 
[2026-04-05T11:38:55.118Z] TASK [Wait up to 10 minutes for an-test-deploy3 lvm volume to be removed] ******
[2026-04-05T11:38:55.118Z] skipping: [an-a01n01]
[2026-04-05T11:38:55.118Z] skipping: [an-a01n02]
[2026-04-05T11:38:55.118Z] 
[2026-04-05T11:38:55.118Z] TASK [check if server an-test-deploy4 exists] **********************************
[2026-04-05T11:38:56.936Z] skipping: [an-a01n01]
[2026-04-05T11:38:56.936Z] fatal: [an-a01n02]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy4\n", "delta": "0:00:00.920515", "end": "2026-04-05 07:38:56.510829", "msg": "non-zero return code", "rc": 1, "start": "2026-04-05 07:38:55.590314", "stderr": "Error: resource or tag id 'an-test-deploy4' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy4' not found"], "stdout": "", "stdout_lines": []}
[2026-04-05T11:38:56.936Z] ...ignoring
[2026-04-05T11:38:56.936Z] 
[2026-04-05T11:38:56.936Z] TASK [delete server an-test-deploy4 from an-a01n02] ****************************
[2026-04-05T11:38:56.936Z] skipping: [an-a01n01]
[2026-04-05T11:38:56.936Z] skipping: [an-a01n02]
[2026-04-05T11:38:56.936Z] 
[2026-04-05T11:38:56.936Z] TASK [Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker] *****
[2026-04-05T11:38:56.936Z] skipping: [an-a01n01]
[2026-04-05T11:38:56.936Z] skipping: [an-a01n02]
[2026-04-05T11:38:56.936Z] 
[2026-04-05T11:38:56.936Z] TASK [Wait up to 10 minutes for an-test-deploy4 drbd storage to be removed] ****
[2026-04-05T11:38:56.936Z] skipping: [an-a01n01]
[2026-04-05T11:38:56.936Z] skipping: [an-a01n02]
[2026-04-05T11:38:56.936Z] 
[2026-04-05T11:38:56.936Z] TASK [Wait up to 10 minutes for an-test-deploy4 lvm volume to be removed] ******
[2026-04-05T11:38:56.936Z] skipping: [an-a01n01]
[2026-04-05T11:38:56.936Z] skipping: [an-a01n02]
[2026-04-05T11:38:56.936Z] 
[2026-04-05T11:38:56.936Z] TASK [check if server an-test-deploy5 exists] **********************************
[2026-04-05T11:38:58.753Z] skipping: [an-a01n01]
[2026-04-05T11:38:58.753Z] fatal: [an-a01n02]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy5\n", "delta": "0:00:00.848203", "end": "2026-04-05 07:38:58.243502", "msg": "non-zero return code", "rc": 1, "start": "2026-04-05 07:38:57.395299", "stderr": "Error: resource or tag id 'an-test-deploy5' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy5' not found"], "stdout": "", "stdout_lines": []}
[2026-04-05T11:38:58.753Z] ...ignoring
[2026-04-05T11:38:58.753Z] 
[2026-04-05T11:38:58.753Z] TASK [delete server an-test-deploy5 from an-a01n02] ****************************
[2026-04-05T11:38:58.753Z] skipping: [an-a01n01]
[2026-04-05T11:38:58.753Z] skipping: [an-a01n02]
[2026-04-05T11:38:58.753Z] 
[2026-04-05T11:38:58.753Z] TASK [Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker] *****
[2026-04-05T11:38:58.753Z] skipping: [an-a01n01]
[2026-04-05T11:38:58.753Z] skipping: [an-a01n02]
[2026-04-05T11:38:58.753Z] 
[2026-04-05T11:38:58.753Z] TASK [Wait up to 10 minutes for an-test-deploy5 drbd storage to be removed] ****
[2026-04-05T11:38:58.753Z] skipping: [an-a01n01]
[2026-04-05T11:38:58.753Z] skipping: [an-a01n02]
[2026-04-05T11:38:58.753Z] 
[2026-04-05T11:38:58.753Z] TASK [Wait up to 10 minutes for an-test-deploy5 lvm volume to be removed] ******
[2026-04-05T11:38:58.753Z] skipping: [an-a01n01]
[2026-04-05T11:38:58.753Z] skipping: [an-a01n02]
[2026-04-05T11:38:58.753Z] 
[2026-04-05T11:38:58.753Z] PLAY [Check anvil available resources from striker01] **************************
[2026-04-05T11:38:58.753Z] 
[2026-04-05T11:38:58.753Z] TASK [Gathering Facts] *********************************************************
[2026-04-05T11:39:01.206Z] ok: [an-striker01]
[2026-04-05T11:39:01.206Z] 
[2026-04-05T11:39:01.206Z] TASK [Check anvil available resources] *****************************************
[2026-04-05T11:44:12.130Z] ok: [an-striker01]
[2026-04-05T11:44:12.130Z] 
[2026-04-05T11:44:12.130Z] PLAY RECAP *********************************************************************
[2026-04-05T11:44:12.130Z] an-a01n01                  : ok=314  changed=28   unreachable=0    failed=0    skipped=109  rescued=0    ignored=6   
[2026-04-05T11:44:12.130Z] an-a01n02                  : ok=345  changed=27   unreachable=0    failed=0    skipped=90   rescued=0    ignored=4   
[2026-04-05T11:44:12.130Z] an-striker01               : ok=57   changed=19   unreachable=0    failed=0    skipped=3    rescued=0    ignored=0   
[2026-04-05T11:44:12.130Z] an-striker02               : ok=42   changed=12   unreachable=0    failed=0    skipped=3    rescued=0    ignored=0   
[2026-04-05T11:44:12.130Z] 
[2026-04-05T11:44:12.130Z] tar: Removing leading `/' from member names
[2026-04-05T11:44:12.130Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-05T11:44:12.130Z] /root/ci-tools
[2026-04-05T11:44:12.130Z] Complete anvil logs available here for download: http://anvil-ci-repo.ci.alteeve.com/testing-logs/210.tar.gz
[2026-04-05T11:44:12.130Z] Disconnecting an-striker01.ci.alteeve.com from Red Hat.
[2026-04-05T11:44:12.130Z] This might take a few seconds.
[2026-04-05T11:44:12.130Z] 
[2026-04-05T11:44:12.130Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-05T11:44:13.950Z] 𐄂 Disconnected from Red Hat Insights
[2026-04-05T11:44:15.221Z] 𐄂 Disconnected from Red Hat Subscription Management
[2026-04-05T11:44:15.221Z] 
[2026-04-05T11:44:15.221Z] Manage your connected systems: https://red.ht/connector
[2026-04-05T11:44:20.305Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-05T11:44:20.305Z] 
[2026-04-05T11:44:20.305Z] 1. Register with subscription-manager
[2026-04-05T11:44:20.305Z] # subscription-manager register
[2026-04-05T11:44:20.305Z] 
[2026-04-05T11:44:20.305Z] 2. Register with insights-client
[2026-04-05T11:44:20.305Z] # insights-client --register
[2026-04-05T11:44:21.093Z] This system is currently not registered.
[2026-04-05T11:44:21.881Z] Disconnecting an-striker02.ci.alteeve.com from Red Hat.
[2026-04-05T11:44:21.881Z] This might take a few seconds.
[2026-04-05T11:44:21.881Z] 
[2026-04-05T11:44:21.881Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-05T11:44:29.556Z] 𐄂 Disconnected from Red Hat Insights
[2026-04-05T11:44:29.556Z] 𐄂 Disconnected from Red Hat Subscription Management
[2026-04-05T11:44:29.556Z] 
[2026-04-05T11:44:29.556Z] Manage your connected systems: https://red.ht/connector
[2026-04-05T11:44:34.630Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-05T11:44:34.630Z] 
[2026-04-05T11:44:34.630Z] 1. Register with subscription-manager
[2026-04-05T11:44:34.630Z] # subscription-manager register
[2026-04-05T11:44:34.630Z] 
[2026-04-05T11:44:34.630Z] 2. Register with insights-client
[2026-04-05T11:44:34.630Z] # insights-client --register
[2026-04-05T11:44:35.899Z] This system is currently not registered.
[2026-04-05T11:44:36.269Z] Disconnecting an-a01n01.ci.alteeve.com from Red Hat.
[2026-04-05T11:44:36.269Z] This might take a few seconds.
[2026-04-05T11:44:36.269Z] 
[2026-04-05T11:44:36.269Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-05T11:44:42.536Z] 𐄂 Disconnected from Red Hat Insights
[2026-04-05T11:44:42.536Z] 𐄂 Disconnected from Red Hat Subscription Management
[2026-04-05T11:44:42.536Z] 
[2026-04-05T11:44:42.536Z] Manage your connected systems: https://red.ht/connector
[2026-04-05T11:44:47.610Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-05T11:44:47.610Z] 
[2026-04-05T11:44:47.610Z] 1. Register with subscription-manager
[2026-04-05T11:44:47.610Z] # subscription-manager register
[2026-04-05T11:44:47.610Z] 
[2026-04-05T11:44:47.610Z] 2. Register with insights-client
[2026-04-05T11:44:47.610Z] # insights-client --register
[2026-04-05T11:44:47.979Z] This system is currently not registered.
[2026-04-05T11:44:48.348Z] Disconnecting an-a01n02.ci.alteeve.com from Red Hat.
[2026-04-05T11:44:48.348Z] This might take a few seconds.
[2026-04-05T11:44:48.348Z] 
[2026-04-05T11:44:48.348Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-05T11:44:54.617Z] 𐄂 Disconnected from Red Hat Insights
[2026-04-05T11:44:54.617Z] 𐄂 Disconnected from Red Hat Subscription Management
[2026-04-05T11:44:54.617Z] 
[2026-04-05T11:44:54.617Z] Manage your connected systems: https://red.ht/connector
[2026-04-05T11:44:59.690Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-05T11:44:59.690Z] 
[2026-04-05T11:44:59.690Z] 1. Register with subscription-manager
[2026-04-05T11:44:59.690Z] # subscription-manager register
[2026-04-05T11:44:59.690Z] 
[2026-04-05T11:44:59.690Z] 2. Register with insights-client
[2026-04-05T11:44:59.690Z] # insights-client --register
[2026-04-05T11:44:59.690Z] This system is currently not registered.
[2026-04-05T11:45:00.478Z] Disconnecting an-a01dr01.ci.alteeve.com from Red Hat.
[2026-04-05T11:45:00.478Z] This might take a few seconds.
[2026-04-05T11:45:00.478Z] 
[2026-04-05T11:45:00.478Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-05T11:45:06.743Z] 𐄂 Disconnected from Red Hat Insights
[2026-04-05T11:45:06.743Z] 𐄂 Disconnected from Red Hat Subscription Management
[2026-04-05T11:45:06.743Z] 
[2026-04-05T11:45:06.743Z] Manage your connected systems: https://red.ht/connector
[2026-04-05T11:45:11.818Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-05T11:45:11.818Z] 
[2026-04-05T11:45:11.818Z] 1. Register with subscription-manager
[2026-04-05T11:45:11.818Z] # subscription-manager register
[2026-04-05T11:45:11.818Z] 
[2026-04-05T11:45:11.818Z] 2. Register with insights-client
[2026-04-05T11:45:11.818Z] # insights-client --register
[2026-04-05T11:45:11.818Z] This system is currently not registered.
[2026-04-05T11:45:11.908Z] [Pipeline] }
[2026-04-05T11:45:11.910Z] [Pipeline] // timeout
[2026-04-05T11:45:11.914Z] [Pipeline] }
[2026-04-05T11:45:11.915Z] [Pipeline] // script
[2026-04-05T11:45:11.918Z] [Pipeline] }
[2026-04-05T11:45:11.920Z] [Pipeline] // stage
[2026-04-05T11:45:11.923Z] [Pipeline] stage
[2026-04-05T11:45:11.923Z] [Pipeline] { (Declarative: Post Actions)
[2026-04-05T11:45:11.929Z] [Pipeline] cleanWs
[2026-04-05T11:45:11.990Z] [WS-CLEANUP] Deleting project workspace...
[2026-04-05T11:45:11.990Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-04-05T11:45:12.021Z] [WS-CLEANUP] done
[2026-04-05T11:45:12.024Z] [Pipeline] script
[2026-04-05T11:45:12.027Z] [Pipeline] {
[2026-04-05T11:45:12.048Z] [Pipeline] stage
[2026-04-05T11:45:12.049Z] [Pipeline] { (Copying logs)
[2026-04-05T11:45:12.081Z] [Pipeline] node
[2026-04-05T11:45:12.129Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-functional-testing
[2026-04-05T11:45:12.130Z] [Pipeline] {
[2026-04-05T11:45:12.172Z] [Pipeline] node
[2026-04-05T11:45:12.216Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-functional-testing@2
[2026-04-05T11:45:12.217Z] [Pipeline] {
[2026-04-05T11:45:12.220Z] [Pipeline] sh
[2026-04-05T11:45:12.475Z] + mkdir -p /var/lib/jenkins/locks
[2026-04-05T11:45:12.488Z] [Pipeline] echo
[2026-04-05T11:45:12.489Z] RWLock: FD for lock log_archive in stage projectFinishUp is 699
[2026-04-05T11:45:12.493Z] [Pipeline] sleep
[2026-04-05T11:45:12.493Z] No need to sleep any longer
[2026-04-05T11:45:12.496Z] [Pipeline] echo
[2026-04-05T11:45:12.496Z] RWLock: log_archive in stage projectFinishUp locked for WRITE
[2026-04-05T11:45:12.500Z] [Pipeline] }
[2026-04-05T11:45:12.504Z] [Pipeline] // node
[2026-04-05T11:45:12.509Z] [Pipeline] echo
[2026-04-05T11:45:12.510Z] Copying logs to /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-functional-testing/210/
[2026-04-05T11:45:12.511Z] [Pipeline] sh
[2026-04-05T11:45:12.767Z] + mkdir -p /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-functional-testing/210/
[2026-04-05T11:45:12.769Z] [Pipeline] sh