[2026-04-12T22:20:54.498Z] Started by upstream project "anvil/anvil-pipeline/PR-999" build number 20
[2026-04-12T22:20:54.498Z] originally caused by:
[2026-04-12T22:20:54.499Z]  GitHub pull request comment by ylei-tsubame: retest this please (https://github.com/ClusterLabs/anvil/pull/999#issuecomment-4232861890)
[2026-04-12T22:20:55.062Z] Obtained pipelines/projects/anvil/build-functional-testing/Jenkinsfile from git https://github.com/kronosnet/ci-tools
[2026-04-12T22:20:55.070Z] Loading library ProjectLib@main:pipelines/projects/anvil/
[2026-04-12T22:20:55.229Z] Examining kronosnet/ci-tools
[2026-04-12T22:20:55.229Z] Attempting to resolve main as a branch
[2026-04-12T22:20:55.330Z] Resolved main as branch main at revision d7dafba0df573c8808b74278a58d8647fff017a9
[2026-04-12T22:20:55.331Z] Selected Git installation does not exist. Using Default
[2026-04-12T22:20:55.331Z] The recommended git tool is: NONE
[2026-04-12T22:20:55.331Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-04-12T22:20:55.334Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-04-12T22:20:55.334Z] Wiping out workspace first.
[2026-04-12T22:20:55.361Z] Cloning the remote Git repository
[2026-04-12T22:20:55.361Z] Using shallow clone with depth 1
[2026-04-12T22:20:55.361Z] Avoid fetching tags
[2026-04-12T22:20:55.361Z] Honoring refspec on initial clone
[2026-04-12T22:20:55.361Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-04-12T22:20:55.361Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-functional-testing@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b@3 # timeout=10
[2026-04-12T22:20:55.371Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-04-12T22:20:55.371Z]  > git --version # timeout=10
[2026-04-12T22:20:55.377Z]  > git --version # 'git version 2.47.3'
[2026-04-12T22:20:55.377Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-04-12T22:20:55.384Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-12T22:20:55.559Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-04-12T22:20:55.566Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-12T22:20:55.574Z] Avoid second fetch
[2026-04-12T22:20:55.574Z] Checking out Revision d7dafba0df573c8808b74278a58d8647fff017a9 (main)
[2026-04-12T22:20:55.574Z]  > git config core.sparsecheckout # timeout=10
[2026-04-12T22:20:55.581Z]  > git checkout -f d7dafba0df573c8808b74278a58d8647fff017a9 # timeout=10
[2026-04-12T22:20:55.612Z] Commit message: "run update-all-ci-tools into artifacts"
[2026-04-12T22:20:55.618Z] Loading library GlobalLib@main:./pipelines/libs/global/
[2026-04-12T22:20:55.786Z] Examining kronosnet/ci-tools
[2026-04-12T22:20:55.786Z] Attempting to resolve main as a branch
[2026-04-12T22:20:55.901Z] Resolved main as branch main at revision d7dafba0df573c8808b74278a58d8647fff017a9
[2026-04-12T22:20:55.902Z] Selected Git installation does not exist. Using Default
[2026-04-12T22:20:55.902Z] The recommended git tool is: NONE
[2026-04-12T22:20:55.902Z] using credential 93a393a1-7fe9-45ae-ade0-5a533c87d8d9
[2026-04-12T22:20:55.905Z] DEPRECATED: The 'Wipe out repository & force clone' extension is deprecated for Pipeline jobs. Pipeline users should use the deleteDir() step instead.
[2026-04-12T22:20:55.905Z] Wiping out workspace first.
[2026-04-12T22:20:55.938Z] Cloning the remote Git repository
[2026-04-12T22:20:55.938Z] Using shallow clone with depth 1
[2026-04-12T22:20:55.938Z] Avoid fetching tags
[2026-04-12T22:20:55.938Z] Honoring refspec on initial clone
[2026-04-12T22:20:55.938Z] Cloning repository https://github.com/kronosnet/ci-tools.git
[2026-04-12T22:20:55.938Z]  > git init /var/lib/jenkins/workspace/anvil/anvil-build-functional-testing@libs/a38125f6c167b5c61aae7e05cd041ecf55e96373eebcceb3c4473bea537ec70b@6 # timeout=10
[2026-04-12T22:20:55.954Z] Fetching upstream changes from https://github.com/kronosnet/ci-tools.git
[2026-04-12T22:20:55.955Z]  > git --version # timeout=10
[2026-04-12T22:20:55.968Z]  > git --version # 'git version 2.47.3'
[2026-04-12T22:20:55.968Z] using GIT_ASKPASS to set credentials getCredUUID - github
[2026-04-12T22:20:55.974Z]  > git fetch --no-tags --force --progress --depth=1 -- https://github.com/kronosnet/ci-tools.git +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-12T22:20:56.153Z]  > git config remote.origin.url https://github.com/kronosnet/ci-tools.git # timeout=10
[2026-04-12T22:20:56.160Z]  > git config --add remote.origin.fetch +refs/heads/main:refs/remotes/origin/main # timeout=10
[2026-04-12T22:20:56.168Z] Avoid second fetch
[2026-04-12T22:20:56.168Z] Checking out Revision d7dafba0df573c8808b74278a58d8647fff017a9 (main)
[2026-04-12T22:20:56.168Z]  > git config core.sparsecheckout # timeout=10
[2026-04-12T22:20:56.177Z]  > git checkout -f d7dafba0df573c8808b74278a58d8647fff017a9 # timeout=10
[2026-04-12T22:20:56.223Z] Commit message: "run update-all-ci-tools into artifacts"
[2026-04-12T22:20:56.276Z] [Pipeline] Start of Pipeline
[2026-04-12T22:20:56.408Z] [Pipeline] node
[2026-04-12T22:20:56.410Z] Running on anvil-ci-bm-phy03 in /srv/workspace/anvil/anvil-build-functional-testing
[2026-04-12T22:20:56.411Z] [Pipeline] {
[2026-04-12T22:20:56.419Z] [Pipeline] stage
[2026-04-12T22:20:56.419Z] [Pipeline] { (Prep build env)
[2026-04-12T22:20:56.428Z] [Pipeline] script
[2026-04-12T22:20:56.429Z] [Pipeline] {
[2026-04-12T22:20:56.433Z] [Pipeline] cleanWs
[2026-04-12T22:20:56.494Z] [WS-CLEANUP] Deleting project workspace...
[2026-04-12T22:20:56.494Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-04-12T22:20:56.524Z] [WS-CLEANUP] done
[2026-04-12T22:20:56.544Z] [Pipeline] lock
[2026-04-12T22:20:56.544Z] Trying to acquire lock on [Resource: MAINTENANCE LOCKOUT]
[2026-04-12T22:20:56.544Z] Resource [MAINTENANCE LOCKOUT] did not exist. Created.
[2026-04-12T22:20:56.544Z] Lock acquired on [Resource: MAINTENANCE LOCKOUT]
[2026-04-12T22:20:56.544Z] [Pipeline] {
[2026-04-12T22:20:56.546Z] [Pipeline] }
[2026-04-12T22:20:56.547Z] Lock released on resource [Resource: MAINTENANCE LOCKOUT]
[2026-04-12T22:20:56.549Z] [Pipeline] // lock
[2026-04-12T22:20:56.553Z] [Pipeline] }
[2026-04-12T22:20:56.555Z] [Pipeline] // script
[2026-04-12T22:20:56.557Z] [Pipeline] }
[2026-04-12T22:20:56.559Z] [Pipeline] // stage
[2026-04-12T22:20:56.562Z] [Pipeline] stage
[2026-04-12T22:20:56.562Z] [Pipeline] { (Run functional testing)
[2026-04-12T22:20:56.568Z] [Pipeline] script
[2026-04-12T22:20:56.570Z] [Pipeline] {
[2026-04-12T22:20:56.587Z] [Pipeline] echo
[2026-04-12T22:20:56.587Z] runWithTimeout 360
[2026-04-12T22:20:56.588Z] [Pipeline] timeout
[2026-04-12T22:20:56.588Z] Timeout set to expire in 6 hr 0 min
[2026-04-12T22:20:56.588Z] [Pipeline] {
[2026-04-12T22:20:56.593Z] [Pipeline] sh
[2026-04-12T22:20:57.117Z] + /root/ci-tools/ci-wrap ci-setup-anvil-bm-vm rhel-9 ci pr/999 all
[2026-04-12T22:20:57.117Z] CITBRANCH:  main
[2026-04-12T22:20:57.117Z] CITHOME:    /root/ci-tools
[2026-04-12T22:20:57.117Z] CITCMD:     ci-setup-anvil-bm-vm
[2026-04-12T22:20:57.117Z] CITCMDARGS: rhel-9 ci pr/999 all
[2026-04-12T22:20:57.484Z] Genering ssh keys and ssh config
[2026-04-12T22:20:57.484Z] # github.com:22 SSH-2.0-3992d52
[2026-04-12T22:20:57.484Z] # github.com:22 SSH-2.0-3992d52
[2026-04-12T22:20:57.484Z] # github.com:22 SSH-2.0-3992d52
[2026-04-12T22:20:57.851Z] # github.com:22 SSH-2.0-3992d52
[2026-04-12T22:20:57.851Z] # github.com:22 SSH-2.0-3992d52
[2026-04-12T22:20:57.851Z] Delay build by 180 seconds to allow Jenkins to cancel the job
[2026-04-12T22:24:07.655Z] Destroying VMs (if any)
[2026-04-12T22:24:07.655Z] Warning: Permanently added 'an-striker01' (ED25519) to the list of known hosts.
[2026-04-12T22:24:07.655Z] Disconnecting an-striker01.ci.alteeve.com from Red Hat.
[2026-04-12T22:24:07.655Z] This might take a few seconds.
[2026-04-12T22:24:07.655Z] 
[2026-04-12T22:24:07.655Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-12T22:24:07.655Z] ! Cannot disconnect from Red Hat Insights
[2026-04-12T22:24:07.655Z] ! Cannot disconnect from Red Hat Subscription Management
[2026-04-12T22:24:07.655Z] 
[2026-04-12T22:24:07.655Z] Manage your connected systems: https://red.ht/connector
[2026-04-12T22:24:07.655Z] 
[2026-04-12T22:24:07.655Z] The following errors were encountered during disconnect:
[2026-04-12T22:24:07.655Z] 
[2026-04-12T22:24:07.655Z] TYPE   STEP      ERROR  
[2026-04-12T22:24:07.655Z] ERROR  insights  cannot disconnect from Red Hat Insights: exit status 1
[2026-04-12T22:24:07.655Z] ERROR  rhsm      cannot disconnect from Red Hat Subscription Management: warning: the system is already unregistered
[2026-04-12T22:24:07.655Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-12T22:24:07.655Z] 
[2026-04-12T22:24:07.655Z] 1. Register with subscription-manager
[2026-04-12T22:24:07.655Z] # subscription-manager register
[2026-04-12T22:24:07.655Z] 
[2026-04-12T22:24:07.655Z] 2. Register with insights-client
[2026-04-12T22:24:07.655Z] # insights-client --register
[2026-04-12T22:24:07.655Z] This system is currently not registered.
[2026-04-12T22:24:10.105Z] Domain 'an-striker01' destroyed
[2026-04-12T22:24:10.105Z] 
[2026-04-12T22:24:10.105Z] Domain 'an-striker01' has been undefined
[2026-04-12T22:24:10.105Z] 
[2026-04-12T22:24:10.105Z] Warning: Permanently added 'an-striker02' (ED25519) to the list of known hosts.
[2026-04-12T22:24:10.473Z] Disconnecting an-striker02.ci.alteeve.com from Red Hat.
[2026-04-12T22:24:10.473Z] This might take a few seconds.
[2026-04-12T22:24:10.473Z] 
[2026-04-12T22:24:10.473Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-12T22:24:14.703Z] ! Cannot disconnect from Red Hat Insights
[2026-04-12T22:24:14.703Z] ! Cannot disconnect from Red Hat Subscription Management
[2026-04-12T22:24:14.703Z] 
[2026-04-12T22:24:14.703Z] Manage your connected systems: https://red.ht/connector
[2026-04-12T22:24:14.703Z] 
[2026-04-12T22:24:14.703Z] The following errors were encountered during disconnect:
[2026-04-12T22:24:14.703Z] 
[2026-04-12T22:24:14.703Z] TYPE   STEP      ERROR  
[2026-04-12T22:24:14.703Z] ERROR  insights  cannot disconnect from Red Hat Insights: exit status 1
[2026-04-12T22:24:14.703Z] ERROR  rhsm      cannot disconnect from Red Hat Subscription Management: warning: the system is already unregistered
[2026-04-12T22:24:18.755Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-12T22:24:18.755Z] 
[2026-04-12T22:24:18.755Z] 1. Register with subscription-manager
[2026-04-12T22:24:18.755Z] # subscription-manager register
[2026-04-12T22:24:18.755Z] 
[2026-04-12T22:24:18.755Z] 2. Register with insights-client
[2026-04-12T22:24:18.755Z] # insights-client --register
[2026-04-12T22:24:19.543Z] This system is currently not registered.
[2026-04-12T22:24:21.992Z] Domain 'an-striker02' destroyed
[2026-04-12T22:24:21.992Z] 
[2026-04-12T22:24:21.992Z] Domain 'an-striker02' has been undefined
[2026-04-12T22:24:21.992Z] 
[2026-04-12T22:24:21.992Z] Warning: Permanently added 'an-a01n01' (ED25519) to the list of known hosts.
[2026-04-12T22:24:22.360Z] Disconnecting an-a01n01.ci.alteeve.com from Red Hat.
[2026-04-12T22:24:22.360Z] This might take a few seconds.
[2026-04-12T22:24:22.360Z] 
[2026-04-12T22:24:22.360Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-12T22:24:26.410Z] ! Cannot disconnect from Red Hat Insights
[2026-04-12T22:24:26.410Z] ! Cannot disconnect from Red Hat Subscription Management
[2026-04-12T22:24:26.410Z] 
[2026-04-12T22:24:26.410Z] Manage your connected systems: https://red.ht/connector
[2026-04-12T22:24:26.410Z] 
[2026-04-12T22:24:26.410Z] The following errors were encountered during disconnect:
[2026-04-12T22:24:26.410Z] 
[2026-04-12T22:24:26.410Z] TYPE   STEP      ERROR  
[2026-04-12T22:24:26.410Z] ERROR  rhsm      cannot disconnect from Red Hat Subscription Management: warning: the system is already unregistered
[2026-04-12T22:24:26.410Z] ERROR  insights  cannot disconnect from Red Hat Insights: exit status 1
[2026-04-12T22:24:30.462Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-12T22:24:30.462Z] 
[2026-04-12T22:24:30.462Z] 1. Register with subscription-manager
[2026-04-12T22:24:30.462Z] # subscription-manager register
[2026-04-12T22:24:30.462Z] 
[2026-04-12T22:24:30.462Z] 2. Register with insights-client
[2026-04-12T22:24:30.462Z] # insights-client --register
[2026-04-12T22:24:30.830Z] This system is currently not registered.
[2026-04-12T22:24:34.015Z] Domain 'an-a01n01' destroyed
[2026-04-12T22:24:34.015Z] 
[2026-04-12T22:24:34.015Z] Domain 'an-a01n01' has been undefined
[2026-04-12T22:24:34.015Z] 
[2026-04-12T22:24:34.015Z] Warning: Permanently added 'an-a01n02' (ED25519) to the list of known hosts.
[2026-04-12T22:24:34.383Z] Disconnecting an-a01n02.ci.alteeve.com from Red Hat.
[2026-04-12T22:24:34.383Z] This might take a few seconds.
[2026-04-12T22:24:34.383Z] 
[2026-04-12T22:24:34.383Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-12T22:24:38.431Z] ! Cannot disconnect from Red Hat Insights
[2026-04-12T22:24:38.431Z] ! Cannot disconnect from Red Hat Subscription Management
[2026-04-12T22:24:38.431Z] 
[2026-04-12T22:24:38.431Z] Manage your connected systems: https://red.ht/connector
[2026-04-12T22:24:38.431Z] 
[2026-04-12T22:24:38.431Z] The following errors were encountered during disconnect:
[2026-04-12T22:24:38.431Z] 
[2026-04-12T22:24:38.431Z] TYPE   STEP      ERROR  
[2026-04-12T22:24:38.431Z] ERROR  insights  cannot disconnect from Red Hat Insights: exit status 1
[2026-04-12T22:24:38.431Z] ERROR  rhsm      cannot disconnect from Red Hat Subscription Management: warning: the system is already unregistered
[2026-04-12T22:24:42.480Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-12T22:24:42.480Z] 
[2026-04-12T22:24:42.480Z] 1. Register with subscription-manager
[2026-04-12T22:24:42.480Z] # subscription-manager register
[2026-04-12T22:24:42.480Z] 
[2026-04-12T22:24:42.480Z] 2. Register with insights-client
[2026-04-12T22:24:42.480Z] # insights-client --register
[2026-04-12T22:24:42.848Z] This system is currently not registered.
[2026-04-12T22:24:46.036Z] Domain 'an-a01n02' destroyed
[2026-04-12T22:24:46.036Z] 
[2026-04-12T22:24:46.036Z] Domain 'an-a01n02' has been undefined
[2026-04-12T22:24:46.036Z] 
[2026-04-12T22:24:46.036Z] Warning: Permanently added 'an-a01dr01' (ED25519) to the list of known hosts.
[2026-04-12T22:24:46.036Z] Disconnecting an-a01dr01.ci.alteeve.com from Red Hat.
[2026-04-12T22:24:46.036Z] This might take a few seconds.
[2026-04-12T22:24:46.036Z] 
[2026-04-12T22:24:46.036Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-12T22:24:50.085Z] ! Cannot disconnect from Red Hat Insights
[2026-04-12T22:24:50.085Z] ! Cannot disconnect from Red Hat Subscription Management
[2026-04-12T22:24:50.085Z] 
[2026-04-12T22:24:50.085Z] Manage your connected systems: https://red.ht/connector
[2026-04-12T22:24:50.085Z] 
[2026-04-12T22:24:50.085Z] The following errors were encountered during disconnect:
[2026-04-12T22:24:50.085Z] 
[2026-04-12T22:24:50.085Z] TYPE   STEP      ERROR  
[2026-04-12T22:24:50.085Z] ERROR  insights  cannot disconnect from Red Hat Insights: exit status 1
[2026-04-12T22:24:50.085Z] ERROR  rhsm      cannot disconnect from Red Hat Subscription Management: warning: the system is already unregistered
[2026-04-12T22:24:54.134Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-12T22:24:54.135Z] 
[2026-04-12T22:24:54.135Z] 1. Register with subscription-manager
[2026-04-12T22:24:54.135Z] # subscription-manager register
[2026-04-12T22:24:54.135Z] 
[2026-04-12T22:24:54.135Z] 2. Register with insights-client
[2026-04-12T22:24:54.135Z] # insights-client --register
[2026-04-12T22:24:54.135Z] This system is currently not registered.
[2026-04-12T22:24:57.038Z] Domain 'an-a01dr01' destroyed
[2026-04-12T22:24:57.038Z] 
[2026-04-12T22:24:57.038Z] Domain 'an-a01dr01' has been undefined
[2026-04-12T22:24:57.038Z] 
[2026-04-12T22:24:57.038Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-12T22:24:57.038Z] Genering ssh keys and ssh config
[2026-04-12T22:24:57.038Z] # github.com:22 SSH-2.0-3992d52
[2026-04-12T22:24:57.405Z] # github.com:22 SSH-2.0-3992d52
[2026-04-12T22:24:57.405Z] # github.com:22 SSH-2.0-3992d52
[2026-04-12T22:24:57.405Z] # github.com:22 SSH-2.0-3992d52
[2026-04-12T22:24:57.405Z] # github.com:22 SSH-2.0-3992d52
[2026-04-12T22:24:57.405Z] Cache already updated
[2026-04-12T22:24:57.405Z] Creating nodes disks
[2026-04-12T22:25:05.082Z] Formatting 'an-striker01-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/rhel-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-12T22:25:05.082Z] Formatting 'an-striker01-vol2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 lazy_refcounts=off refcount_bits=16
[2026-04-12T22:25:05.082Z] Formatting 'an-striker02-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/rhel-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-12T22:25:05.082Z] Formatting 'an-striker02-vol2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 lazy_refcounts=off refcount_bits=16
[2026-04-12T22:25:05.082Z] Formatting 'an-a01n01-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/rhel-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-12T22:25:05.082Z] Formatting 'an-a01n01-vol2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 lazy_refcounts=off refcount_bits=16
[2026-04-12T22:25:05.082Z] Formatting 'an-a01n02-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/rhel-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-12T22:25:05.082Z] Formatting 'an-a01n02-vol2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 lazy_refcounts=off refcount_bits=16
[2026-04-12T22:25:05.082Z] Formatting 'an-a01dr01-vol.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 backing_file=/srv/anvil/cache/rhel-9-latest.qcow2 backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
[2026-04-12T22:25:05.082Z] Formatting 'an-a01dr01-vol2.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=214748364800 lazy_refcounts=off refcount_bits=16
[2026-04-12T22:25:05.082Z] Generating an-test-deployX kickstart files/isos
[2026-04-12T22:25:05.082Z] xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.
[2026-04-12T22:25:05.082Z] 
[2026-04-12T22:25:05.082Z] Drive current: -outdev 'stdio:/srv/anvil/snapshots/deploy1.iso'
[2026-04-12T22:25:05.082Z] Media current: stdio file, overwriteable
[2026-04-12T22:25:05.082Z] Media status : is blank
[2026-04-12T22:25:05.082Z] Media summary: 0 sessions, 0 data blocks, 0 data, 1727g free
[2026-04-12T22:25:05.082Z] Added to ISO image: directory '/'='/srv/anvil/configs/ks1'
[2026-04-12T22:25:05.082Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-12T22:25:05.082Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-12T22:25:05.450Z] ISO image produced: 184 sectors
[2026-04-12T22:25:05.450Z] Written to medium : 184 sectors at LBA 0
[2026-04-12T22:25:05.450Z] Writing to 'stdio:/srv/anvil/snapshots/deploy1.iso' completed successfully.
[2026-04-12T22:25:05.450Z] 
[2026-04-12T22:25:05.450Z] xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.
[2026-04-12T22:25:05.450Z] 
[2026-04-12T22:25:05.450Z] Drive current: -outdev 'stdio:/srv/anvil/snapshots/deploy2.iso'
[2026-04-12T22:25:05.450Z] Media current: stdio file, overwriteable
[2026-04-12T22:25:05.450Z] Media status : is blank
[2026-04-12T22:25:05.450Z] Media summary: 0 sessions, 0 data blocks, 0 data, 1729g free
[2026-04-12T22:25:05.450Z] Added to ISO image: directory '/'='/srv/anvil/configs/ks2'
[2026-04-12T22:25:05.450Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-12T22:25:05.450Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-12T22:25:05.450Z] ISO image produced: 184 sectors
[2026-04-12T22:25:05.450Z] Written to medium : 184 sectors at LBA 0
[2026-04-12T22:25:05.450Z] Writing to 'stdio:/srv/anvil/snapshots/deploy2.iso' completed successfully.
[2026-04-12T22:25:05.450Z] 
[2026-04-12T22:25:05.450Z] xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.
[2026-04-12T22:25:05.450Z] 
[2026-04-12T22:25:05.450Z] Drive current: -outdev 'stdio:/srv/anvil/snapshots/deploy3.iso'
[2026-04-12T22:25:05.450Z] Media current: stdio file, overwriteable
[2026-04-12T22:25:05.450Z] Media status : is blank
[2026-04-12T22:25:05.450Z] Media summary: 0 sessions, 0 data blocks, 0 data, 1729g free
[2026-04-12T22:25:05.450Z] Added to ISO image: directory '/'='/srv/anvil/configs/ks3'
[2026-04-12T22:25:05.450Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-12T22:25:05.450Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-12T22:25:05.450Z] ISO image produced: 184 sectors
[2026-04-12T22:25:05.450Z] Written to medium : 184 sectors at LBA 0
[2026-04-12T22:25:05.450Z] Writing to 'stdio:/srv/anvil/snapshots/deploy3.iso' completed successfully.
[2026-04-12T22:25:05.450Z] 
[2026-04-12T22:25:05.450Z] xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.
[2026-04-12T22:25:05.450Z] 
[2026-04-12T22:25:05.450Z] Drive current: -outdev 'stdio:/srv/anvil/snapshots/deploy4.iso'
[2026-04-12T22:25:05.450Z] Media current: stdio file, overwriteable
[2026-04-12T22:25:05.450Z] Media status : is blank
[2026-04-12T22:25:05.450Z] Media summary: 0 sessions, 0 data blocks, 0 data, 1729g free
[2026-04-12T22:25:05.450Z] Added to ISO image: directory '/'='/srv/anvil/configs/ks4'
[2026-04-12T22:25:05.450Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-12T22:25:05.450Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-12T22:25:05.450Z] ISO image produced: 184 sectors
[2026-04-12T22:25:05.450Z] Written to medium : 184 sectors at LBA 0
[2026-04-12T22:25:05.450Z] Writing to 'stdio:/srv/anvil/snapshots/deploy4.iso' completed successfully.
[2026-04-12T22:25:05.450Z] 
[2026-04-12T22:25:05.450Z] xorriso 1.5.4 : RockRidge filesystem manipulator, libburnia project.
[2026-04-12T22:25:05.450Z] 
[2026-04-12T22:25:05.450Z] Drive current: -outdev 'stdio:/srv/anvil/snapshots/deploy5.iso'
[2026-04-12T22:25:05.450Z] Media current: stdio file, overwriteable
[2026-04-12T22:25:05.450Z] Media status : is blank
[2026-04-12T22:25:05.450Z] Media summary: 0 sessions, 0 data blocks, 0 data, 1729g free
[2026-04-12T22:25:05.450Z] Added to ISO image: directory '/'='/srv/anvil/configs/ks5'
[2026-04-12T22:25:05.450Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-12T22:25:05.450Z] xorriso : UPDATE :       1 files added in 1 seconds
[2026-04-12T22:25:05.819Z] ISO image produced: 184 sectors
[2026-04-12T22:25:05.819Z] Written to medium : 184 sectors at LBA 0
[2026-04-12T22:25:05.819Z] Writing to 'stdio:/srv/anvil/snapshots/deploy5.iso' completed successfully.
[2026-04-12T22:25:05.819Z] 
[2026-04-12T22:25:05.819Z] Generating cloud-init configs
[2026-04-12T22:25:05.819Z] Generating cloud-init metadata and seed images
[2026-04-12T22:25:05.819Z] wrote /srv/anvil/snapshots/an-striker01-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-12T22:25:05.819Z] wrote /srv/anvil/snapshots/an-striker02-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-12T22:25:06.187Z] wrote /srv/anvil/snapshots/an-a01n01-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-12T22:25:06.187Z] wrote /srv/anvil/snapshots/an-a01n02-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-12T22:25:06.187Z] wrote /srv/anvil/snapshots/an-a01dr01-seed.img with filesystem=iso9660 and diskformat=raw
[2026-04-12T22:25:06.187Z] Creating VMs
[2026-04-12T22:25:08.007Z] 
[2026-04-12T22:25:08.007Z] Starting install...
[2026-04-12T22:25:08.007Z] 
[2026-04-12T22:25:08.007Z] Creating domain...                                          |         00:00     
[2026-04-12T22:25:08.007Z] Domain creation completed.
[2026-04-12T22:25:09.275Z] 
[2026-04-12T22:25:09.275Z] Starting install...
[2026-04-12T22:25:09.275Z] 
[2026-04-12T22:25:09.275Z] Creating domain...                                          |         00:00     
[2026-04-12T22:25:09.275Z] Domain creation completed.
[2026-04-12T22:25:11.091Z] 
[2026-04-12T22:25:11.091Z] Starting install...
[2026-04-12T22:25:11.091Z] 
[2026-04-12T22:25:11.091Z] Creating domain...                                          |         00:00     
[2026-04-12T22:25:11.091Z] Domain creation completed.
[2026-04-12T22:25:12.908Z] 
[2026-04-12T22:25:12.908Z] Starting install...
[2026-04-12T22:25:12.908Z] 
[2026-04-12T22:25:12.908Z] Creating domain...                                          |         00:00     
[2026-04-12T22:25:12.908Z] Domain creation completed.
[2026-04-12T22:25:14.725Z] 
[2026-04-12T22:25:14.725Z] Starting install...
[2026-04-12T22:25:14.725Z] 
[2026-04-12T22:25:14.725Z] Creating domain...                                          |         00:00     
[2026-04-12T22:25:14.725Z] Domain creation completed.
[2026-04-12T22:25:14.725Z] Waiting for nodes and cloud-init to complete full init
[2026-04-12T22:25:17.918Z] cloud-init status node an-striker01: 
[2026-04-12T22:25:21.115Z] cloud-init status node an-striker02: 
[2026-04-12T22:25:24.308Z] cloud-init status node an-a01n01: 
[2026-04-12T22:25:26.763Z] cloud-init status node an-a01n02: 
[2026-04-12T22:25:29.962Z] cloud-init status node an-a01dr01: 
[2026-04-12T22:25:29.962Z] Sun Apr 12 18:25:29 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:25:41.268Z] cloud-init status node an-striker01: running
[2026-04-12T22:25:42.537Z] cloud-init status node an-striker02: running
[2026-04-12T22:25:43.328Z] cloud-init status node an-a01n01: running
[2026-04-12T22:25:44.598Z] cloud-init status node an-a01n02: running
[2026-04-12T22:25:45.870Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:25:45.870Z] Sun Apr 12 18:25:45 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:25:57.395Z] cloud-init status node an-striker01: running
[2026-04-12T22:25:57.395Z] cloud-init status node an-striker02: running
[2026-04-12T22:25:58.215Z] cloud-init status node an-a01n01: running
[2026-04-12T22:25:59.490Z] cloud-init status node an-a01n02: running
[2026-04-12T22:26:00.284Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:26:00.284Z] Sun Apr 12 18:25:59 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:26:11.595Z] cloud-init status node an-striker01: running
[2026-04-12T22:26:11.966Z] cloud-init status node an-striker02: running
[2026-04-12T22:26:12.755Z] cloud-init status node an-a01n01: running
[2026-04-12T22:26:13.548Z] cloud-init status node an-a01n02: running
[2026-04-12T22:26:14.341Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:26:14.341Z] Sun Apr 12 18:26:14 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:26:26.002Z] cloud-init status node an-striker01: running
[2026-04-12T22:26:26.371Z] cloud-init status node an-striker02: running
[2026-04-12T22:26:27.159Z] cloud-init status node an-a01n01: running
[2026-04-12T22:26:27.960Z] cloud-init status node an-a01n02: running
[2026-04-12T22:26:29.260Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:26:29.260Z] Sun Apr 12 18:26:28 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:26:41.459Z] cloud-init status node an-striker01: running
[2026-04-12T22:26:41.459Z] cloud-init status node an-striker02: running
[2026-04-12T22:26:42.177Z] cloud-init status node an-a01n01: running
[2026-04-12T22:26:42.994Z] cloud-init status node an-a01n02: running
[2026-04-12T22:26:43.882Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:26:43.882Z] Sun Apr 12 18:26:43 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:26:59.648Z] cloud-init status node an-striker01: running
[2026-04-12T22:27:00.617Z] cloud-init status node an-striker02: running
[2026-04-12T22:27:02.042Z] cloud-init status node an-a01n01: running
[2026-04-12T22:27:02.620Z] cloud-init status node an-a01n02: running
[2026-04-12T22:27:03.496Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:27:03.496Z] Sun Apr 12 18:27:03 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:27:15.848Z] cloud-init status node an-striker01: running
[2026-04-12T22:27:15.848Z] cloud-init status node an-striker02: running
[2026-04-12T22:27:16.218Z] cloud-init status node an-a01n01: running
[2026-04-12T22:27:17.502Z] cloud-init status node an-a01n02: running
[2026-04-12T22:27:18.687Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:27:18.687Z] Sun Apr 12 18:27:17 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:27:29.979Z] cloud-init status node an-striker01: running
[2026-04-12T22:27:29.979Z] cloud-init status node an-striker02: running
[2026-04-12T22:27:30.765Z] cloud-init status node an-a01n01: running
[2026-04-12T22:27:32.028Z] cloud-init status node an-a01n02: running
[2026-04-12T22:27:32.818Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:27:32.818Z] Sun Apr 12 18:27:32 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:27:44.129Z] cloud-init status node an-striker01: running
[2026-04-12T22:27:44.499Z] cloud-init status node an-striker02: running
[2026-04-12T22:27:45.300Z] cloud-init status node an-a01n01: running
[2026-04-12T22:27:46.352Z] cloud-init status node an-a01n02: running
[2026-04-12T22:27:47.140Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:27:47.140Z] Sun Apr 12 18:27:46 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:27:58.506Z] cloud-init status node an-striker01: running
[2026-04-12T22:27:58.879Z] cloud-init status node an-striker02: running
[2026-04-12T22:27:59.666Z] cloud-init status node an-a01n01: running
[2026-04-12T22:28:01.010Z] cloud-init status node an-a01n02: running
[2026-04-12T22:28:01.814Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:28:01.814Z] Sun Apr 12 18:28:01 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:28:13.392Z] cloud-init status node an-striker01: running
[2026-04-12T22:28:13.392Z] cloud-init status node an-striker02: running
[2026-04-12T22:28:14.179Z] cloud-init status node an-a01n01: running
[2026-04-12T22:28:15.461Z] cloud-init status node an-a01n02: running
[2026-04-12T22:28:16.266Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:28:16.266Z] Sun Apr 12 18:28:15 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:28:27.556Z] cloud-init status node an-striker01: running
[2026-04-12T22:28:27.925Z] cloud-init status node an-striker02: running
[2026-04-12T22:28:29.191Z] cloud-init status node an-a01n01: running
[2026-04-12T22:28:30.457Z] cloud-init status node an-a01n02: running
[2026-04-12T22:28:31.243Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:28:31.243Z] Sun Apr 12 18:28:30 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:28:42.762Z] cloud-init status node an-striker01: running
[2026-04-12T22:28:43.131Z] cloud-init status node an-striker02: running
[2026-04-12T22:28:43.916Z] cloud-init status node an-a01n01: running
[2026-04-12T22:28:44.703Z] cloud-init status node an-a01n02: running
[2026-04-12T22:28:45.967Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:28:45.967Z] Sun Apr 12 18:28:45 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:28:57.265Z] cloud-init status node an-striker01: running
[2026-04-12T22:28:57.633Z] cloud-init status node an-striker02: running
[2026-04-12T22:28:58.900Z] cloud-init status node an-a01n01: running
[2026-04-12T22:28:59.686Z] cloud-init status node an-a01n02: running
[2026-04-12T22:29:00.473Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:29:00.473Z] Sun Apr 12 18:29:00 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:29:11.934Z] cloud-init status node an-striker01: running
[2026-04-12T22:29:12.302Z] cloud-init status node an-striker02: running
[2026-04-12T22:29:13.566Z] cloud-init status node an-a01n01: running
[2026-04-12T22:29:14.352Z] cloud-init status node an-a01n02: running
[2026-04-12T22:29:15.138Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:29:15.138Z] Sun Apr 12 18:29:15 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:29:26.424Z] cloud-init status node an-striker01: running
[2026-04-12T22:29:27.210Z] cloud-init status node an-striker02: running
[2026-04-12T22:29:28.474Z] cloud-init status node an-a01n01: running
[2026-04-12T22:29:29.260Z] cloud-init status node an-a01n02: running
[2026-04-12T22:29:30.044Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:29:30.044Z] Sun Apr 12 18:29:29 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:29:41.336Z] cloud-init status node an-striker01: running
[2026-04-12T22:29:42.121Z] cloud-init status node an-striker02: running
[2026-04-12T22:29:42.911Z] cloud-init status node an-a01n01: running
[2026-04-12T22:29:44.174Z] cloud-init status node an-a01n02: running
[2026-04-12T22:29:44.960Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:29:44.960Z] Sun Apr 12 18:29:44 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:29:56.249Z] cloud-init status node an-striker01: running
[2026-04-12T22:29:57.035Z] cloud-init status node an-striker02: running
[2026-04-12T22:29:57.821Z] cloud-init status node an-a01n01: running
[2026-04-12T22:29:58.607Z] cloud-init status node an-a01n02: running
[2026-04-12T22:29:59.872Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:29:59.872Z] Sun Apr 12 18:29:59 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:30:11.161Z] cloud-init status node an-striker01: running
[2026-04-12T22:30:11.529Z] cloud-init status node an-striker02: running
[2026-04-12T22:30:12.793Z] cloud-init status node an-a01n01: running
[2026-04-12T22:30:13.579Z] cloud-init status node an-a01n02: running
[2026-04-12T22:30:14.843Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:30:14.843Z] Sun Apr 12 18:30:14 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:30:26.331Z] cloud-init status node an-striker01: running
[2026-04-12T22:30:26.331Z] cloud-init status node an-striker02: running
[2026-04-12T22:30:27.594Z] cloud-init status node an-a01n01: running
[2026-04-12T22:30:28.379Z] cloud-init status node an-a01n02: running
[2026-04-12T22:30:29.164Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:30:29.164Z] Sun Apr 12 18:30:29 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:30:40.450Z] cloud-init status node an-striker01: running
[2026-04-12T22:30:41.235Z] cloud-init status node an-striker02: running
[2026-04-12T22:30:42.021Z] cloud-init status node an-a01n01: running
[2026-04-12T22:30:42.815Z] cloud-init status node an-a01n02: running
[2026-04-12T22:30:44.109Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:30:44.109Z] Sun Apr 12 18:30:43 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:30:55.395Z] cloud-init status node an-striker01: running
[2026-04-12T22:30:55.763Z] cloud-init status node an-striker02: running
[2026-04-12T22:30:56.549Z] cloud-init status node an-a01n01: running
[2026-04-12T22:30:57.335Z] cloud-init status node an-a01n02: running
[2026-04-12T22:30:58.627Z] cloud-init status node an-a01dr01: running
[2026-04-12T22:30:58.627Z] Sun Apr 12 18:30:58 EDT 2026 actual: 0 expected: 5
[2026-04-12T22:31:12.259Z] cloud-init status node an-striker01: running
[2026-04-12T22:31:12.259Z] cloud-init status node an-striker02: running
[2026-04-12T22:31:13.697Z] cloud-init status node an-a01n01: running
[2026-04-12T22:31:14.964Z] cloud-init status node an-a01n02: done
[2026-04-12T22:31:16.231Z] cloud-init status node an-a01dr01: done
[2026-04-12T22:31:16.231Z] Sun Apr 12 18:31:15 EDT 2026 actual: 2 expected: 5
[2026-04-12T22:31:27.528Z] cloud-init status node an-striker01: running
[2026-04-12T22:31:29.982Z] cloud-init status node an-striker02: running
[2026-04-12T22:31:31.246Z] cloud-init status node an-a01n01: done
[2026-04-12T22:31:32.511Z] cloud-init status node an-a01n02: done
[2026-04-12T22:31:33.776Z] cloud-init status node an-a01dr01: done
[2026-04-12T22:31:33.776Z] Sun Apr 12 18:31:33 EDT 2026 actual: 3 expected: 5
[2026-04-12T22:31:45.242Z] cloud-init status node an-striker01: running
[2026-04-12T22:31:45.242Z] cloud-init status node an-striker02: done
[2026-04-12T22:31:46.506Z] cloud-init status node an-a01n01: done
[2026-04-12T22:31:47.292Z] cloud-init status node an-a01n02: done
[2026-04-12T22:31:48.077Z] cloud-init status node an-a01dr01: done
[2026-04-12T22:31:48.077Z] Sun Apr 12 18:31:47 EDT 2026 actual: 4 expected: 5
[2026-04-12T22:31:59.366Z] cloud-init status node an-striker01: done
[2026-04-12T22:32:00.152Z] cloud-init status node an-striker02: done
[2026-04-12T22:32:01.416Z] cloud-init status node an-a01n01: done
[2026-04-12T22:32:02.348Z] cloud-init status node an-a01n02: done
[2026-04-12T22:32:03.134Z] cloud-init status node an-a01dr01: done
[2026-04-12T22:32:03.134Z] Sun Apr 12 18:32:02 EDT 2026 actual: 5 expected: 5
[2026-04-12T22:32:03.134Z] Building simengine model
[2026-04-12T22:32:16.763Z] Generating anvil striker auto config
[2026-04-12T22:32:18.578Z] Cloning into 'anvil-test-suite'...
[2026-04-12T22:32:19.364Z] generated ansible config:
[2026-04-12T22:32:19.364Z] =========================
[2026-04-12T22:32:19.364Z] [defaults]
[2026-04-12T22:32:19.364Z] inventory = ansible-inventory
[2026-04-12T22:32:19.364Z] forks = 500
[2026-04-12T22:32:19.364Z] any_errors_fatal = true
[2026-04-12T22:32:19.364Z] 
[2026-04-12T22:32:19.364Z] [ssh_connection]
[2026-04-12T22:32:19.364Z] retries = 20
[2026-04-12T22:32:19.364Z] =========================
[2026-04-12T22:32:19.364Z] 
[2026-04-12T22:32:19.364Z] generated ansible inventory:
[2026-04-12T22:32:19.364Z] ============================
[2026-04-12T22:32:19.364Z] [strikers]
[2026-04-12T22:32:19.364Z] an-striker01
[2026-04-12T22:32:19.364Z] an-striker02
[2026-04-12T22:32:19.364Z] 
[2026-04-12T22:32:19.364Z] [nodes]
[2026-04-12T22:32:19.364Z] an-a01n01
[2026-04-12T22:32:19.364Z] an-a01n02
[2026-04-12T22:32:19.364Z] 
[2026-04-12T22:32:19.364Z] [drnodes]
[2026-04-12T22:32:19.364Z] an-a01dr01
[2026-04-12T22:32:19.364Z] ============================
[2026-04-12T22:32:19.364Z] 
[2026-04-12T22:32:19.364Z] Checking connectivity to nodes
[2026-04-12T22:32:19.364Z] 
[2026-04-12T22:32:21.813Z] an-a01n02 | SUCCESS => {
[2026-04-12T22:32:21.813Z]     "ansible_facts": {
[2026-04-12T22:32:21.813Z]         "discovered_interpreter_python": "/usr/bin/python3"
[2026-04-12T22:32:21.813Z]     },
[2026-04-12T22:32:21.813Z]     "changed": false,
[2026-04-12T22:32:21.813Z]     "ping": "pong"
[2026-04-12T22:32:21.813Z] }
[2026-04-12T22:32:21.813Z] an-a01n01 | SUCCESS => {
[2026-04-12T22:32:21.813Z]     "ansible_facts": {
[2026-04-12T22:32:21.813Z]         "discovered_interpreter_python": "/usr/bin/python3"
[2026-04-12T22:32:21.813Z]     },
[2026-04-12T22:32:21.813Z]     "changed": false,
[2026-04-12T22:32:21.813Z]     "ping": "pong"
[2026-04-12T22:32:21.813Z] }
[2026-04-12T22:32:21.813Z] an-a01dr01 | SUCCESS => {
[2026-04-12T22:32:21.813Z]     "ansible_facts": {
[2026-04-12T22:32:21.813Z]         "discovered_interpreter_python": "/usr/bin/python3"
[2026-04-12T22:32:21.813Z]     },
[2026-04-12T22:32:21.813Z]     "changed": false,
[2026-04-12T22:32:21.813Z]     "ping": "pong"
[2026-04-12T22:32:21.813Z] }
[2026-04-12T22:32:21.813Z] an-striker02 | SUCCESS => {
[2026-04-12T22:32:21.813Z]     "ansible_facts": {
[2026-04-12T22:32:21.813Z]         "discovered_interpreter_python": "/usr/bin/python3"
[2026-04-12T22:32:21.813Z]     },
[2026-04-12T22:32:21.813Z]     "changed": false,
[2026-04-12T22:32:21.813Z]     "ping": "pong"
[2026-04-12T22:32:21.813Z] }
[2026-04-12T22:32:21.813Z] an-striker01 | SUCCESS => {
[2026-04-12T22:32:21.813Z]     "ansible_facts": {
[2026-04-12T22:32:21.813Z]         "discovered_interpreter_python": "/usr/bin/python3"
[2026-04-12T22:32:21.813Z]     },
[2026-04-12T22:32:21.813Z]     "changed": false,
[2026-04-12T22:32:21.813Z]     "ping": "pong"
[2026-04-12T22:32:21.813Z] }
[2026-04-12T22:32:21.813Z] 
[2026-04-12T22:32:22.599Z] 
[2026-04-12T22:32:22.599Z] PLAY [Setup all nodes] *********************************************************
[2026-04-12T22:32:22.599Z] 
[2026-04-12T22:32:22.599Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:32:25.050Z] ok: [an-a01n02]
[2026-04-12T22:32:25.050Z] ok: [an-a01n01]
[2026-04-12T22:32:25.050Z] ok: [an-a01dr01]
[2026-04-12T22:32:25.050Z] ok: [an-striker01]
[2026-04-12T22:32:25.050Z] ok: [an-striker02]
[2026-04-12T22:32:25.050Z] 
[2026-04-12T22:32:25.050Z] TASK [create /opt/MegaRAID/storcli/] *******************************************
[2026-04-12T22:32:25.837Z] changed: [an-a01n02]
[2026-04-12T22:32:25.837Z] changed: [an-a01dr01]
[2026-04-12T22:32:25.837Z] changed: [an-a01n01]
[2026-04-12T22:32:25.837Z] changed: [an-striker02]
[2026-04-12T22:32:25.837Z] changed: [an-striker01]
[2026-04-12T22:32:25.837Z] 
[2026-04-12T22:32:25.837Z] TASK [wipe old storcli64] ******************************************************
[2026-04-12T22:32:26.624Z] ok: [an-a01n02]
[2026-04-12T22:32:26.624Z] ok: [an-a01dr01]
[2026-04-12T22:32:26.624Z] ok: [an-a01n01]
[2026-04-12T22:32:26.624Z] ok: [an-striker01]
[2026-04-12T22:32:26.624Z] ok: [an-striker02]
[2026-04-12T22:32:26.624Z] 
[2026-04-12T22:32:26.624Z] TASK [get latest Simengine storcli64] ******************************************
[2026-04-12T22:32:28.438Z] changed: [an-a01n01]
[2026-04-12T22:32:28.438Z] changed: [an-a01dr01]
[2026-04-12T22:32:28.438Z] changed: [an-a01n02]
[2026-04-12T22:32:28.438Z] changed: [an-striker01]
[2026-04-12T22:32:28.438Z] changed: [an-striker02]
[2026-04-12T22:32:28.438Z] 
[2026-04-12T22:32:28.438Z] PLAY [setup strikers] **********************************************************
[2026-04-12T22:32:28.438Z] 
[2026-04-12T22:32:28.438Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:32:29.703Z] ok: [an-striker01]
[2026-04-12T22:32:29.703Z] ok: [an-striker02]
[2026-04-12T22:32:29.703Z] 
[2026-04-12T22:32:29.703Z] TASK [copy striker autogenerated config] ***************************************
[2026-04-12T22:32:31.516Z] changed: [an-striker01]
[2026-04-12T22:32:31.516Z] changed: [an-striker02]
[2026-04-12T22:32:31.516Z] 
[2026-04-12T22:32:31.516Z] PLAY [setup lvm storage] *******************************************************
[2026-04-12T22:32:31.516Z] 
[2026-04-12T22:32:31.516Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:32:32.781Z] ok: [an-a01n02]
[2026-04-12T22:32:32.781Z] ok: [an-a01n01]
[2026-04-12T22:32:32.781Z] ok: [an-a01dr01]
[2026-04-12T22:32:32.781Z] 
[2026-04-12T22:32:32.781Z] TASK [check if vg already exists] **********************************************
[2026-04-12T22:32:33.567Z] ok: [an-a01dr01]
[2026-04-12T22:32:33.567Z] ok: [an-a01n02]
[2026-04-12T22:32:33.568Z] ok: [an-a01n01]
[2026-04-12T22:32:33.568Z] 
[2026-04-12T22:32:33.568Z] TASK [vgcheck] *****************************************************************
[2026-04-12T22:32:33.568Z] skipping: [an-a01n01]
[2026-04-12T22:32:33.568Z] 
[2026-04-12T22:32:33.568Z] TASK [create anvil-test-vg] ****************************************************
[2026-04-12T22:32:34.833Z] changed: [an-a01dr01]
[2026-04-12T22:32:34.833Z] changed: [an-a01n02]
[2026-04-12T22:32:34.833Z] changed: [an-a01n01]
[2026-04-12T22:32:34.833Z] 
[2026-04-12T22:32:34.833Z] PLAY [deploy anvil] ************************************************************
[2026-04-12T22:32:34.833Z] 
[2026-04-12T22:32:34.833Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:32:36.648Z] ok: [an-striker01]
[2026-04-12T22:32:36.648Z] ok: [an-striker02]
[2026-04-12T22:32:36.648Z] 
[2026-04-12T22:32:36.648Z] TASK [check if installation is in progress] ************************************
[2026-04-12T22:32:37.016Z] ok: [an-striker01]
[2026-04-12T22:32:37.016Z] ok: [an-striker02]
[2026-04-12T22:32:37.016Z] 
[2026-04-12T22:32:37.016Z] TASK [register installation is in progress] ************************************
[2026-04-12T22:32:38.831Z] changed: [an-striker02]
[2026-04-12T22:32:38.831Z] changed: [an-striker01]
[2026-04-12T22:32:38.831Z] 
[2026-04-12T22:32:38.831Z] TASK [start anvil deployment and configuration] ********************************
[2026-04-12T22:32:42.170Z] changed: [an-striker02]
[2026-04-12T22:32:42.170Z] changed: [an-striker01]
[2026-04-12T22:32:42.170Z] 
[2026-04-12T22:32:42.170Z] TASK [Wait up to 20 minutes for DB to be running on both strikers] *************
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (120 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (120 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (119 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (119 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (118 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (118 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (117 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (117 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (116 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (116 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (115 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (115 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (114 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (114 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (113 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (113 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (112 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (112 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (111 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (111 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (110 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (110 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (109 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (109 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (108 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (108 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (107 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (107 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (106 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (106 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (105 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (105 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (104 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (104 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (103 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (103 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (102 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (102 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (101 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (101 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (100 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (100 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (99 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (99 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (98 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (98 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (97 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (97 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (96 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (95 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (94 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (93 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (96 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (92 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (95 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (91 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (94 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (90 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (93 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (89 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (92 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (88 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (91 retries left).
[2026-04-12T22:38:38.444Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (87 retries left).
[2026-04-12T22:39:48.534Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (90 retries left).
[2026-04-12T22:39:48.534Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (89 retries left).
[2026-04-12T22:39:48.534Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (88 retries left).
[2026-04-12T22:39:48.534Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (87 retries left).
[2026-04-12T22:39:48.534Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (86 retries left).
[2026-04-12T22:39:48.534Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (86 retries left).
[2026-04-12T22:39:48.534Z] FAILED - RETRYING: [an-striker01 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (85 retries left).
[2026-04-12T22:39:48.534Z] FAILED - RETRYING: [an-striker02 -> localhost]: Wait up to 20 minutes for DB to be running on both strikers (85 retries left).
[2026-04-12T22:39:48.534Z] ok: [an-striker01 -> localhost]
[2026-04-12T22:39:48.534Z] ok: [an-striker02 -> localhost]
[2026-04-12T22:39:48.534Z] 
[2026-04-12T22:39:48.534Z] TASK [Wait up to 20 minutes for both striker-db-status to sync] ****************
[2026-04-12T22:39:56.195Z] FAILED - RETRYING: [an-striker02]: Wait up to 20 minutes for both striker-db-status to sync (120 retries left).
[2026-04-12T22:39:56.195Z] ok: [an-striker01]
[2026-04-12T22:39:56.195Z] ok: [an-striker02]
[2026-04-12T22:39:56.195Z] 
[2026-04-12T22:39:56.195Z] TASK [wait up to 20 minutes for nodes to join the database] ********************
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (120 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (120 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (119 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (119 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (118 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (118 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (117 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (117 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (116 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (116 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (115 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (115 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (114 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (114 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (113 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (113 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (112 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (112 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (111 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (111 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (110 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (110 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (109 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (109 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (108 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (108 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (107 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (107 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (106 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (106 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (105 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (105 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (104 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (104 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (103 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (103 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (102 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (102 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (101 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (101 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (100 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (100 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (99 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (99 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (98 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (98 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (97 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (97 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (96 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (96 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (95 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (95 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (94 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (94 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (93 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (93 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (92 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (92 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (91 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (91 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (90 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (90 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (89 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (89 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (88 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (88 retries left).
[2026-04-12T22:45:37.343Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (87 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (87 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (86 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (86 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (85 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (85 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (84 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (84 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (83 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (83 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (82 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (82 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (81 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (81 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (80 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (80 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (79 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (79 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (78 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (78 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (77 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (77 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (76 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (76 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (75 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (75 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (74 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (74 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (73 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (73 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (72 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (72 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (71 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (71 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (70 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (70 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (69 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (69 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (68 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (68 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for nodes to join the database (67 retries left).
[2026-04-12T22:49:17.528Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for nodes to join the database (67 retries left).
[2026-04-12T22:49:17.528Z] ok: [an-striker02 -> localhost]
[2026-04-12T22:49:17.528Z] ok: [an-striker01 -> localhost]
[2026-04-12T22:49:17.528Z] 
[2026-04-12T22:49:17.528Z] TASK [wait up to 20 minutes for anvil nodes to form] ***************************
[2026-04-12T22:49:17.528Z] ok: [an-striker01 -> localhost]
[2026-04-12T22:49:17.528Z] ok: [an-striker02 -> localhost]
[2026-04-12T22:49:17.528Z] 
[2026-04-12T22:49:17.528Z] TASK [wait up to 60 minutes for pacemaker to be configured] ********************
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (360 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (360 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (359 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (359 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (358 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (358 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (357 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (357 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (356 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (356 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (355 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (355 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (354 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (354 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (353 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (353 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (352 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (352 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (351 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (351 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (350 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (350 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (349 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (349 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 60 minutes for pacemaker to be configured (348 retries left).
[2026-04-12T22:51:26.847Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 60 minutes for pacemaker to be configured (348 retries left).
[2026-04-12T22:51:26.847Z] ok: [an-striker01 -> localhost]
[2026-04-12T22:51:26.847Z] ok: [an-striker02 -> localhost]
[2026-04-12T22:51:26.847Z] 
[2026-04-12T22:51:26.847Z] TASK [wait up to 20 minutes for drbd to be configured] *************************
[2026-04-12T22:52:00.923Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for drbd to be configured (120 retries left).
[2026-04-12T22:52:00.923Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for drbd to be configured (120 retries left).
[2026-04-12T22:52:00.923Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for drbd to be configured (119 retries left).
[2026-04-12T22:52:00.923Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for drbd to be configured (119 retries left).
[2026-04-12T22:52:00.923Z] FAILED - RETRYING: [an-striker02 -> localhost]: wait up to 20 minutes for drbd to be configured (118 retries left).
[2026-04-12T22:52:00.923Z] FAILED - RETRYING: [an-striker01 -> localhost]: wait up to 20 minutes for drbd to be configured (118 retries left).
[2026-04-12T22:52:00.923Z] ok: [an-striker02 -> localhost]
[2026-04-12T22:52:00.923Z] ok: [an-striker01 -> localhost]
[2026-04-12T22:52:00.923Z] 
[2026-04-12T22:52:00.923Z] TASK [wait up to 20 minutes for any other configuration job to complete] *******
[2026-04-12T22:52:00.923Z] ok: [an-striker01 -> localhost]
[2026-04-12T22:52:00.923Z] ok: [an-striker02 -> localhost]
[2026-04-12T22:52:00.923Z] 
[2026-04-12T22:52:00.923Z] PLAY [check network status after deployment] ***********************************
[2026-04-12T22:52:00.923Z] 
[2026-04-12T22:52:00.923Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:52:00.923Z] ok: [an-a01n02]
[2026-04-12T22:52:00.923Z] ok: [an-a01n01]
[2026-04-12T22:52:00.923Z] ok: [an-a01dr01]
[2026-04-12T22:52:00.923Z] ok: [an-striker02]
[2026-04-12T22:52:00.923Z] ok: [an-striker01]
[2026-04-12T22:52:00.923Z] 
[2026-04-12T22:52:00.923Z] TASK [check network manager autoconnect is set to yes] *************************
[2026-04-12T22:52:10.243Z] changed: [an-a01n01] => (item=bcn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-a01dr01] => (item=sn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n02] => (item=sn1_link2)
[2026-04-12T22:52:10.243Z] changed: [an-striker01] => (item=ifn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n01] => (item=ifn1_link2)
[2026-04-12T22:52:10.243Z] changed: [an-a01n02] => (item=bcn1_link2)
[2026-04-12T22:52:10.243Z] changed: [an-a01dr01] => (item=ifn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-striker02] => (item=ifn1_bond1)
[2026-04-12T22:52:10.243Z] skipping: [an-striker02] => (item=lo) 
[2026-04-12T22:52:10.243Z] changed: [an-a01n01] => (item=ifn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n02] => (item=bcn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-a01dr01] => (item=ifn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-striker01] => (item=bcn1_link2)
[2026-04-12T22:52:10.243Z] changed: [an-a01n01] => (item=bcn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n02] => (item=bcn1_bridge1)
[2026-04-12T22:52:10.243Z] changed: [an-a01dr01] => (item=sn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-striker02] => (item=ifn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n01] => (item=sn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n02] => (item=sn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-a01dr01] => (item=ifn1_bridge1)
[2026-04-12T22:52:10.243Z] changed: [an-striker02] => (item=ifn1_link2)
[2026-04-12T22:52:10.243Z] changed: [an-striker01] => (item=ifn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n01] => (item=sn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-a01dr01] => (item=bcn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n02] => (item=ifn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n01] => (item=sn1_link2)
[2026-04-12T22:52:10.243Z] changed: [an-a01dr01] => (item=bcn1_link2)
[2026-04-12T22:52:10.243Z] changed: [an-a01n02] => (item=ifn1_bridge1)
[2026-04-12T22:52:10.243Z] changed: [an-striker01] => (item=ifn1_link2)
[2026-04-12T22:52:10.243Z] skipping: [an-striker01] => (item=lo) 
[2026-04-12T22:52:10.243Z] changed: [an-a01n01] => (item=bcn1_bridge1)
[2026-04-12T22:52:10.243Z] changed: [an-striker02] => (item=bcn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-a01dr01] => (item=bcn1_bridge1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n02] => (item=ifn1_link1)
[2026-04-12T22:52:10.243Z] skipping: [an-a01n02] => (item=lo) 
[2026-04-12T22:52:10.243Z] changed: [an-striker01] => (item=bcn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n01] => (item=bcn1_link2)
[2026-04-12T22:52:10.243Z] skipping: [an-a01n01] => (item=lo) 
[2026-04-12T22:52:10.243Z] changed: [an-a01dr01] => (item=bcn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n02] => (item=ifn1_link2)
[2026-04-12T22:52:10.243Z] changed: [an-striker02] => (item=bcn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n01] => (item=ifn1_bridge1)
[2026-04-12T22:52:10.243Z] changed: [an-a01dr01] => (item=sn1_link2)
[2026-04-12T22:52:10.243Z] changed: [an-a01n02] => (item=sn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-striker01] => (item=bcn1_link1)
[2026-04-12T22:52:10.243Z] changed: [an-a01n01] => (item=ifn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-a01dr01] => (item=ifn1_link2)
[2026-04-12T22:52:10.243Z] skipping: [an-a01dr01] => (item=lo) 
[2026-04-12T22:52:10.243Z] changed: [an-a01n02] => (item=bcn1_bond1)
[2026-04-12T22:52:10.243Z] changed: [an-striker02] => (item=bcn1_link2)
[2026-04-12T22:52:10.243Z] 
[2026-04-12T22:52:10.243Z] PLAY [check pacemaker status after deployment] *********************************
[2026-04-12T22:52:10.243Z] 
[2026-04-12T22:52:10.243Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:52:10.612Z] ok: [an-a01n01]
[2026-04-12T22:52:10.612Z] ok: [an-a01n02]
[2026-04-12T22:52:10.612Z] 
[2026-04-12T22:52:10.612Z] TASK [check cluster is in good state] ******************************************
[2026-04-12T22:52:10.981Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-12T22:52:10.981Z] 
[2026-04-12T22:52:10.981Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-12T22:52:15.153Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-12T22:52:15.153Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-12T22:52:15.153Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-12T22:52:15.153Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-12T22:52:15.153Z] 
[2026-04-12T22:52:15.153Z] PLAY [check drbd kmod] *********************************************************
[2026-04-12T22:52:15.153Z] 
[2026-04-12T22:52:15.153Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:52:16.418Z] ok: [an-a01n02]
[2026-04-12T22:52:16.418Z] ok: [an-a01n01]
[2026-04-12T22:52:16.418Z] ok: [an-a01dr01]
[2026-04-12T22:52:16.418Z] 
[2026-04-12T22:52:16.418Z] TASK [check if drbd kernel module is already loaded] ***************************
[2026-04-12T22:52:16.786Z] ok: [an-a01n02]
[2026-04-12T22:52:16.786Z] ok: [an-a01dr01]
[2026-04-12T22:52:16.786Z] ok: [an-a01n01]
[2026-04-12T22:52:16.786Z] 
[2026-04-12T22:52:16.786Z] TASK [load drbd kernel module] *************************************************
[2026-04-12T22:52:16.786Z] skipping: [an-a01n01]
[2026-04-12T22:52:16.786Z] skipping: [an-a01n02]
[2026-04-12T22:52:16.786Z] skipping: [an-a01dr01]
[2026-04-12T22:52:16.786Z] 
[2026-04-12T22:52:16.786Z] TASK [check if drbd kernel module is loaded] ***********************************
[2026-04-12T22:52:16.786Z] skipping: [an-a01n01]
[2026-04-12T22:52:17.153Z] skipping: [an-a01n02]
[2026-04-12T22:52:17.153Z] skipping: [an-a01dr01]
[2026-04-12T22:52:17.153Z] 
[2026-04-12T22:52:17.153Z] PLAY [Copy iso images to striker01] ********************************************
[2026-04-12T22:52:17.153Z] 
[2026-04-12T22:52:17.153Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:52:19.604Z] ok: [an-striker01]
[2026-04-12T22:52:19.604Z] 
[2026-04-12T22:52:19.604Z] TASK [Wipe old snapshots dir] **************************************************
[2026-04-12T22:52:20.869Z] ok: [an-striker01]
[2026-04-12T22:52:20.869Z] 
[2026-04-12T22:52:20.869Z] TASK [Create snapshots dir] ****************************************************
[2026-04-12T22:52:22.134Z] changed: [an-striker01]
[2026-04-12T22:52:22.134Z] 
[2026-04-12T22:52:22.134Z] TASK [Copy C9S kickstart images] ***********************************************
[2026-04-12T22:52:33.427Z] changed: [an-striker01] => (item=deploy1.iso)
[2026-04-12T22:52:33.427Z] changed: [an-striker01] => (item=deploy2.iso)
[2026-04-12T22:52:33.427Z] changed: [an-striker01] => (item=deploy3.iso)
[2026-04-12T22:52:33.427Z] changed: [an-striker01] => (item=deploy4.iso)
[2026-04-12T22:52:33.427Z] changed: [an-striker01] => (item=deploy5.iso)
[2026-04-12T22:52:33.427Z] 
[2026-04-12T22:52:33.427Z] TASK [Move install isos to anvil upload dir] ***********************************
[2026-04-12T22:52:34.213Z] changed: [an-striker01]
[2026-04-12T22:52:34.213Z] 
[2026-04-12T22:52:34.213Z] TASK [Wipe old snapshots dir] **************************************************
[2026-04-12T22:52:35.597Z] changed: [an-striker01]
[2026-04-12T22:52:35.597Z] 
[2026-04-12T22:52:35.597Z] PLAY [Check iso images propagation] ********************************************
[2026-04-12T22:52:35.597Z] 
[2026-04-12T22:52:35.597Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:52:37.415Z] ok: [an-a01n01]
[2026-04-12T22:52:37.415Z] ok: [an-a01dr01]
[2026-04-12T22:52:37.415Z] ok: [an-a01n02]
[2026-04-12T22:52:37.415Z] ok: [an-striker02]
[2026-04-12T22:52:37.415Z] ok: [an-striker01]
[2026-04-12T22:52:37.415Z] 
[2026-04-12T22:52:37.415Z] TASK [wait up to 30 minutes for all images to be propagated to all nodes] ******
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (180 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (180 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01dr01]: wait up to 30 minutes for all images to be propagated to all nodes (180 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (180 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (180 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (179 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (179 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01dr01]: wait up to 30 minutes for all images to be propagated to all nodes (179 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (179 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (179 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (178 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (178 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01dr01]: wait up to 30 minutes for all images to be propagated to all nodes (178 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (178 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (178 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (177 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (177 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01dr01]: wait up to 30 minutes for all images to be propagated to all nodes (177 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (177 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (177 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (176 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (176 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01dr01]: wait up to 30 minutes for all images to be propagated to all nodes (176 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (176 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (176 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for all images to be propagated to all nodes (175 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01dr01]: wait up to 30 minutes for all images to be propagated to all nodes (175 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for all images to be propagated to all nodes (175 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker01]: wait up to 30 minutes for all images to be propagated to all nodes (175 retries left).
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for all images to be propagated to all nodes (175 retries left).
[2026-04-12T22:54:01.380Z] ok: [an-a01n02]
[2026-04-12T22:54:01.380Z] ok: [an-a01n01]
[2026-04-12T22:54:01.380Z] FAILED - RETRYING: [an-a01dr01]: wait up to 30 minutes for all images to be propagated to all nodes (174 retries left).
[2026-04-12T22:54:01.380Z] ok: [an-striker02]
[2026-04-12T22:54:01.380Z] ok: [an-striker01]
[2026-04-12T22:54:01.380Z] ok: [an-a01dr01]
[2026-04-12T22:54:01.380Z] 
[2026-04-12T22:54:01.380Z] PLAY [example tests] ***********************************************************
[2026-04-12T22:54:01.380Z] 
[2026-04-12T22:54:01.380Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:54:01.380Z] ok: [an-a01n01]
[2026-04-12T22:54:01.380Z] ok: [an-a01dr01]
[2026-04-12T22:54:01.380Z] ok: [an-a01n02]
[2026-04-12T22:54:01.380Z] ok: [an-striker02]
[2026-04-12T22:54:01.380Z] ok: [an-striker01]
[2026-04-12T22:54:01.380Z] 
[2026-04-12T22:54:01.380Z] TASK [this test should be executed only on strikers] ***************************
[2026-04-12T22:54:01.380Z] skipping: [an-a01n01]
[2026-04-12T22:54:01.380Z] skipping: [an-a01n02]
[2026-04-12T22:54:01.380Z] skipping: [an-a01dr01]
[2026-04-12T22:54:01.380Z] ok: [an-striker01]
[2026-04-12T22:54:01.380Z] ok: [an-striker02]
[2026-04-12T22:54:01.380Z] 
[2026-04-12T22:54:01.380Z] TASK [this test should be executed only on non strikers] ***********************
[2026-04-12T22:54:01.380Z] skipping: [an-striker01]
[2026-04-12T22:54:01.380Z] skipping: [an-striker02]
[2026-04-12T22:54:01.380Z] ok: [an-a01dr01]
[2026-04-12T22:54:01.380Z] ok: [an-a01n01]
[2026-04-12T22:54:01.380Z] ok: [an-a01n02]
[2026-04-12T22:54:01.380Z] 
[2026-04-12T22:54:01.380Z] TASK [inject failure] **********************************************************
[2026-04-12T22:54:01.380Z] skipping: [an-striker01]
[2026-04-12T22:54:01.380Z] skipping: [an-striker02]
[2026-04-12T22:54:01.380Z] skipping: [an-a01n01]
[2026-04-12T22:54:01.380Z] skipping: [an-a01n02]
[2026-04-12T22:54:01.380Z] skipping: [an-a01dr01]
[2026-04-12T22:54:01.380Z] 
[2026-04-12T22:54:01.380Z] PLAY [Record anvil available resources from striker01] *************************
[2026-04-12T22:54:01.380Z] 
[2026-04-12T22:54:01.380Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:54:01.380Z] ok: [an-striker01]
[2026-04-12T22:54:01.380Z] 
[2026-04-12T22:54:01.380Z] TASK [Record anvil available resources] ****************************************
[2026-04-12T22:54:05.432Z] ok: [an-striker01]
[2026-04-12T22:54:05.432Z] 
[2026-04-12T22:54:05.432Z] PLAY [check selinux status] ****************************************************
[2026-04-12T22:54:05.432Z] 
[2026-04-12T22:54:05.432Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:54:08.001Z] ok: [an-a01n01]
[2026-04-12T22:54:08.001Z] ok: [an-a01n02]
[2026-04-12T22:54:08.001Z] ok: [an-a01dr01]
[2026-04-12T22:54:08.001Z] ok: [an-striker02]
[2026-04-12T22:54:08.001Z] ok: [an-striker01]
[2026-04-12T22:54:08.001Z] 
[2026-04-12T22:54:08.001Z] TASK [check selinux status permissive (8)] *************************************
[2026-04-12T22:54:08.001Z] skipping: [an-striker01]
[2026-04-12T22:54:08.001Z] skipping: [an-striker02]
[2026-04-12T22:54:08.001Z] skipping: [an-a01n01]
[2026-04-12T22:54:08.001Z] skipping: [an-a01n02]
[2026-04-12T22:54:08.001Z] skipping: [an-a01dr01]
[2026-04-12T22:54:08.001Z] 
[2026-04-12T22:54:08.001Z] TASK [check selinux status permissive (9)] *************************************
[2026-04-12T22:54:08.369Z] ok: [an-striker01] => {
[2026-04-12T22:54:08.369Z]     "ansible_selinux": {
[2026-04-12T22:54:08.369Z]         "config_mode": "enforcing",
[2026-04-12T22:54:08.369Z]         "mode": "enforcing",
[2026-04-12T22:54:08.369Z]         "policyvers": 33,
[2026-04-12T22:54:08.369Z]         "status": "enabled",
[2026-04-12T22:54:08.369Z]         "type": "targeted"
[2026-04-12T22:54:08.369Z]     },
[2026-04-12T22:54:08.369Z]     "failed_when_result": false
[2026-04-12T22:54:08.369Z] }
[2026-04-12T22:54:08.369Z] ok: [an-striker02] => {
[2026-04-12T22:54:08.369Z]     "ansible_selinux": {
[2026-04-12T22:54:08.369Z]         "config_mode": "enforcing",
[2026-04-12T22:54:08.369Z]         "mode": "enforcing",
[2026-04-12T22:54:08.369Z]         "policyvers": 33,
[2026-04-12T22:54:08.369Z]         "status": "enabled",
[2026-04-12T22:54:08.369Z]         "type": "targeted"
[2026-04-12T22:54:08.369Z]     },
[2026-04-12T22:54:08.369Z]     "failed_when_result": false
[2026-04-12T22:54:08.369Z] }
[2026-04-12T22:54:08.369Z] ok: [an-a01n01] => {
[2026-04-12T22:54:08.369Z]     "ansible_selinux": {
[2026-04-12T22:54:08.369Z]         "config_mode": "enforcing",
[2026-04-12T22:54:08.369Z]         "mode": "enforcing",
[2026-04-12T22:54:08.369Z]         "policyvers": 33,
[2026-04-12T22:54:08.369Z]         "status": "enabled",
[2026-04-12T22:54:08.369Z]         "type": "targeted"
[2026-04-12T22:54:08.369Z]     },
[2026-04-12T22:54:08.369Z]     "failed_when_result": false
[2026-04-12T22:54:08.369Z] }
[2026-04-12T22:54:08.369Z] ok: [an-a01n02] => {
[2026-04-12T22:54:08.369Z]     "ansible_selinux": {
[2026-04-12T22:54:08.369Z]         "config_mode": "enforcing",
[2026-04-12T22:54:08.369Z]         "mode": "enforcing",
[2026-04-12T22:54:08.369Z]         "policyvers": 33,
[2026-04-12T22:54:08.369Z]         "status": "enabled",
[2026-04-12T22:54:08.369Z]         "type": "targeted"
[2026-04-12T22:54:08.369Z]     },
[2026-04-12T22:54:08.369Z]     "failed_when_result": false
[2026-04-12T22:54:08.369Z] }
[2026-04-12T22:54:08.369Z] ok: [an-a01dr01] => {
[2026-04-12T22:54:08.369Z]     "ansible_selinux": {
[2026-04-12T22:54:08.369Z]         "config_mode": "enforcing",
[2026-04-12T22:54:08.369Z]         "mode": "enforcing",
[2026-04-12T22:54:08.369Z]         "policyvers": 33,
[2026-04-12T22:54:08.369Z]         "status": "enabled",
[2026-04-12T22:54:08.369Z]         "type": "targeted"
[2026-04-12T22:54:08.369Z]     },
[2026-04-12T22:54:08.369Z]     "failed_when_result": false
[2026-04-12T22:54:08.369Z] }
[2026-04-12T22:54:08.369Z] 
[2026-04-12T22:54:08.369Z] PLAY [Create random file on striker01] *****************************************
[2026-04-12T22:54:08.369Z] 
[2026-04-12T22:54:08.369Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:54:10.818Z] ok: [an-striker01]
[2026-04-12T22:54:10.818Z] 
[2026-04-12T22:54:10.818Z] TASK [Create archive file] *****************************************************
[2026-04-12T22:54:12.631Z] changed: [an-striker01]
[2026-04-12T22:54:12.631Z] 
[2026-04-12T22:54:12.631Z] PLAY [Check archive file propagation] ******************************************
[2026-04-12T22:54:12.631Z] 
[2026-04-12T22:54:12.631Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:54:15.082Z] ok: [an-a01dr01]
[2026-04-12T22:54:15.082Z] ok: [an-a01n02]
[2026-04-12T22:54:15.082Z] ok: [an-a01n01]
[2026-04-12T22:54:15.082Z] ok: [an-striker02]
[2026-04-12T22:54:15.082Z] ok: [an-striker01]
[2026-04-12T22:54:15.082Z] 
[2026-04-12T22:54:15.082Z] TASK [wait up to 30 minutes for archive to be propagated to all nodes] *********
[2026-04-12T22:54:38.794Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for archive to be propagated to all nodes (180 retries left).
[2026-04-12T22:54:38.794Z] FAILED - RETRYING: [an-a01dr01]: wait up to 30 minutes for archive to be propagated to all nodes (180 retries left).
[2026-04-12T22:54:38.794Z] FAILED - RETRYING: [an-a01n01]: wait up to 30 minutes for archive to be propagated to all nodes (180 retries left).
[2026-04-12T22:54:38.794Z] ok: [an-striker01]
[2026-04-12T22:54:38.794Z] FAILED - RETRYING: [an-striker02]: wait up to 30 minutes for archive to be propagated to all nodes (180 retries left).
[2026-04-12T22:54:38.794Z] FAILED - RETRYING: [an-a01dr01]: wait up to 30 minutes for archive to be propagated to all nodes (179 retries left).
[2026-04-12T22:54:38.794Z] ok: [an-a01n01]
[2026-04-12T22:54:38.794Z] FAILED - RETRYING: [an-a01n02]: wait up to 30 minutes for archive to be propagated to all nodes (179 retries left).
[2026-04-12T22:54:38.794Z] ok: [an-striker02]
[2026-04-12T22:54:38.794Z] ok: [an-a01dr01]
[2026-04-12T22:54:38.794Z] ok: [an-a01n02]
[2026-04-12T22:54:38.794Z] 
[2026-04-12T22:54:38.794Z] PLAY [test fencing] ************************************************************
[2026-04-12T22:54:38.794Z] 
[2026-04-12T22:54:38.794Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T22:54:39.162Z] ok: [an-a01n02]
[2026-04-12T22:54:39.162Z] ok: [an-a01n01]
[2026-04-12T22:54:39.162Z] 
[2026-04-12T22:54:39.162Z] TASK [check cluster is in good state] ******************************************
[2026-04-12T22:54:39.162Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-12T22:54:39.162Z] 
[2026-04-12T22:54:39.162Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-12T22:54:43.217Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-12T22:54:43.217Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-12T22:54:43.217Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-12T22:54:43.217Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-12T22:54:43.217Z] 
[2026-04-12T22:54:43.217Z] TASK [kill node "an-a01n01"] ***************************************************
[2026-04-12T22:54:43.217Z] included: /srv/anvil/anvil-test-suite/playbooks/common/kill_node.yml for an-a01n01, an-a01n02
[2026-04-12T22:54:43.217Z] 
[2026-04-12T22:54:43.217Z] TASK [kill node "an-a01n01"] ***************************************************
[2026-04-12T22:54:44.003Z] skipping: [an-a01n02]
[2026-04-12T22:54:44.003Z] changed: [an-a01n01]
[2026-04-12T22:54:44.003Z] 
[2026-04-12T22:54:44.003Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-12T22:54:44.003Z] skipping: [an-a01n02]
[2026-04-12T22:54:44.003Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01
[2026-04-12T22:54:44.003Z] 
[2026-04-12T22:54:44.003Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-12T22:58:08.936Z] ok: [an-a01n01 -> localhost]
[2026-04-12T22:58:08.936Z] 
[2026-04-12T22:58:08.936Z] TASK [check cluster is in good state] ******************************************
[2026-04-12T22:58:08.936Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-12T22:58:08.936Z] 
[2026-04-12T22:58:08.936Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-12T22:58:08.936Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-12T22:58:08.936Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-12T22:58:08.936Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-12T22:58:08.936Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-12T22:58:08.936Z] 
[2026-04-12T22:58:08.936Z] TASK [kill node "an-a01n02"] ***************************************************
[2026-04-12T22:58:08.936Z] included: /srv/anvil/anvil-test-suite/playbooks/common/kill_node.yml for an-a01n01, an-a01n02
[2026-04-12T22:58:08.936Z] 
[2026-04-12T22:58:08.936Z] TASK [kill node "an-a01n02"] ***************************************************
[2026-04-12T22:58:08.936Z] skipping: [an-a01n01]
[2026-04-12T22:58:08.936Z] changed: [an-a01n02]
[2026-04-12T22:58:08.936Z] 
[2026-04-12T22:58:08.936Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-12T22:58:08.936Z] skipping: [an-a01n01]
[2026-04-12T22:58:08.936Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n02
[2026-04-12T22:58:08.936Z] 
[2026-04-12T22:58:08.936Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-12T23:01:18.842Z] ok: [an-a01n02 -> localhost]
[2026-04-12T23:01:18.842Z] 
[2026-04-12T23:01:18.842Z] TASK [check cluster is in good state] ******************************************
[2026-04-12T23:01:18.842Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-12T23:01:18.842Z] 
[2026-04-12T23:01:18.842Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-12T23:01:18.842Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-12T23:01:18.842Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-12T23:01:18.842Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-12T23:01:18.842Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-12T23:01:18.842Z] 
[2026-04-12T23:01:18.842Z] PLAY [advanced server tests] ***************************************************
[2026-04-12T23:01:18.842Z] 
[2026-04-12T23:01:18.842Z] TASK [Gathering Facts] *********************************************************
[2026-04-12T23:01:18.842Z] ok: [an-a01n01]
[2026-04-12T23:01:18.842Z] ok: [an-a01n02]
[2026-04-12T23:01:18.842Z] 
[2026-04-12T23:01:18.842Z] TASK [check cluster is in good state] ******************************************
[2026-04-12T23:01:18.842Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-12T23:01:18.842Z] 
[2026-04-12T23:01:18.842Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-12T23:01:19.629Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-12T23:01:19.629Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-12T23:01:19.629Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-12T23:01:19.629Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-12T23:01:19.629Z] 
[2026-04-12T23:01:19.629Z] TASK [check anvil-provision-server storage group] ******************************
[2026-04-12T23:01:19.629Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_storage_group.yml for an-a01n01, an-a01n02
[2026-04-12T23:01:19.629Z] 
[2026-04-12T23:01:19.629Z] TASK [get anvil-provision-server storage group] ********************************
[2026-04-12T23:01:25.888Z] changed: [an-a01n01]
[2026-04-12T23:01:25.888Z] changed: [an-a01n02]
[2026-04-12T23:01:25.888Z] 
[2026-04-12T23:01:25.888Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-12T23:01:25.888Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_install_iso.yml for an-a01n01, an-a01n02
[2026-04-12T23:01:25.888Z] 
[2026-04-12T23:01:25.888Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-12T23:01:30.954Z] changed: [an-a01n01]
[2026-04-12T23:01:30.954Z] changed: [an-a01n02]
[2026-04-12T23:01:30.954Z] 
[2026-04-12T23:01:30.954Z] TASK [deploy servers] **********************************************************
[2026-04-12T23:01:30.954Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-12T23:01:30.954Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-12T23:01:30.954Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-12T23:01:30.954Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-12T23:01:30.954Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-12T23:01:30.954Z] 
[2026-04-12T23:01:30.954Z] TASK [check if server an-test-deploy1 exists] **********************************
[2026-04-12T23:01:32.219Z] skipping: [an-a01n02]
[2026-04-12T23:01:32.219Z] fatal: [an-a01n01]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy1\n", "delta": "0:00:00.812522", "end": "2026-04-12 19:01:31.646728", "msg": "non-zero return code", "rc": 1, "start": "2026-04-12 19:01:30.834206", "stderr": "Error: resource or tag id 'an-test-deploy1' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy1' not found"], "stdout": "", "stdout_lines": []}
[2026-04-12T23:01:32.219Z] ...ignoring
[2026-04-12T23:01:32.219Z] 
[2026-04-12T23:01:32.219Z] TASK [deploy server an-test-deploy1 from an-a01n01 on storage 9fafa17d-152b-41c7-9537-6bb636df22f0] ***
[2026-04-12T23:01:41.683Z] skipping: [an-a01n02]
[2026-04-12T23:01:41.683Z] changed: [an-a01n01]
[2026-04-12T23:01:41.683Z] 
[2026-04-12T23:01:41.683Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to be started in pacemaker] ***
[2026-04-12T23:01:41.683Z] skipping: [an-a01n01]
[2026-04-12T23:01:41.683Z] skipping: [an-a01n02]
[2026-04-12T23:01:41.683Z] 
[2026-04-12T23:01:41.683Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} drbd storage to be synced] ***
[2026-04-12T23:01:41.683Z] skipping: [an-a01n01]
[2026-04-12T23:01:41.683Z] skipping: [an-a01n02]
[2026-04-12T23:01:41.683Z] 
[2026-04-12T23:01:41.683Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to start installation] ***
[2026-04-12T23:01:41.683Z] skipping: [an-a01n01]
[2026-04-12T23:01:41.683Z] skipping: [an-a01n02]
[2026-04-12T23:01:41.683Z] 
[2026-04-12T23:01:41.683Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to complete installation] ***
[2026-04-12T23:01:41.683Z] skipping: [an-a01n01]
[2026-04-12T23:01:41.683Z] skipping: [an-a01n02]
[2026-04-12T23:01:41.683Z] 
[2026-04-12T23:01:41.683Z] TASK [check if server an-test-deploy2 exists] **********************************
[2026-04-12T23:01:42.052Z] skipping: [an-a01n02]
[2026-04-12T23:01:42.052Z] fatal: [an-a01n01]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy2\n", "delta": "0:00:00.901089", "end": "2026-04-12 19:01:41.738877", "msg": "non-zero return code", "rc": 1, "start": "2026-04-12 19:01:40.837788", "stderr": "Error: resource or tag id 'an-test-deploy2' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy2' not found"], "stdout": "", "stdout_lines": []}
[2026-04-12T23:01:42.052Z] ...ignoring
[2026-04-12T23:01:42.052Z] 
[2026-04-12T23:01:42.052Z] TASK [deploy server an-test-deploy2 from an-a01n01 on storage 9fafa17d-152b-41c7-9537-6bb636df22f0] ***
[2026-04-12T23:01:51.375Z] skipping: [an-a01n02]
[2026-04-12T23:01:51.375Z] changed: [an-a01n01]
[2026-04-12T23:01:51.375Z] 
[2026-04-12T23:01:51.375Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to be started in pacemaker] ***
[2026-04-12T23:01:51.375Z] skipping: [an-a01n01]
[2026-04-12T23:01:51.375Z] skipping: [an-a01n02]
[2026-04-12T23:01:51.375Z] 
[2026-04-12T23:01:51.375Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} drbd storage to be synced] ***
[2026-04-12T23:01:51.375Z] skipping: [an-a01n01]
[2026-04-12T23:01:51.375Z] skipping: [an-a01n02]
[2026-04-12T23:01:51.375Z] 
[2026-04-12T23:01:51.375Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to start installation] ***
[2026-04-12T23:01:51.375Z] skipping: [an-a01n01]
[2026-04-12T23:01:51.375Z] skipping: [an-a01n02]
[2026-04-12T23:01:51.375Z] 
[2026-04-12T23:01:51.375Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to complete installation] ***
[2026-04-12T23:01:51.375Z] skipping: [an-a01n01]
[2026-04-12T23:01:51.375Z] skipping: [an-a01n02]
[2026-04-12T23:01:51.375Z] 
[2026-04-12T23:01:51.375Z] TASK [check if server an-test-deploy3 exists] **********************************
[2026-04-12T23:01:52.161Z] skipping: [an-a01n02]
[2026-04-12T23:01:52.161Z] fatal: [an-a01n01]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy3\n", "delta": "0:00:00.877884", "end": "2026-04-12 19:01:52.004902", "msg": "non-zero return code", "rc": 1, "start": "2026-04-12 19:01:51.127018", "stderr": "Error: resource or tag id 'an-test-deploy3' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy3' not found"], "stdout": "", "stdout_lines": []}
[2026-04-12T23:01:52.161Z] ...ignoring
[2026-04-12T23:01:52.161Z] 
[2026-04-12T23:01:52.161Z] TASK [deploy server an-test-deploy3 from an-a01n01 on storage 9fafa17d-152b-41c7-9537-6bb636df22f0] ***
[2026-04-12T23:02:01.485Z] skipping: [an-a01n02]
[2026-04-12T23:02:01.485Z] changed: [an-a01n01]
[2026-04-12T23:02:01.485Z] 
[2026-04-12T23:02:01.485Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to be started in pacemaker] ***
[2026-04-12T23:02:01.485Z] skipping: [an-a01n01]
[2026-04-12T23:02:01.485Z] skipping: [an-a01n02]
[2026-04-12T23:02:01.485Z] 
[2026-04-12T23:02:01.485Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} drbd storage to be synced] ***
[2026-04-12T23:02:01.485Z] skipping: [an-a01n01]
[2026-04-12T23:02:01.485Z] skipping: [an-a01n02]
[2026-04-12T23:02:01.485Z] 
[2026-04-12T23:02:01.485Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to start installation] ***
[2026-04-12T23:02:01.485Z] skipping: [an-a01n01]
[2026-04-12T23:02:01.485Z] skipping: [an-a01n02]
[2026-04-12T23:02:01.485Z] 
[2026-04-12T23:02:01.485Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to complete installation] ***
[2026-04-12T23:02:01.485Z] skipping: [an-a01n01]
[2026-04-12T23:02:01.485Z] skipping: [an-a01n02]
[2026-04-12T23:02:01.485Z] 
[2026-04-12T23:02:01.485Z] TASK [check if server an-test-deploy4 exists] **********************************
[2026-04-12T23:02:03.298Z] skipping: [an-a01n02]
[2026-04-12T23:02:03.298Z] fatal: [an-a01n01]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy4\n", "delta": "0:00:00.928067", "end": "2026-04-12 19:02:02.729105", "msg": "non-zero return code", "rc": 1, "start": "2026-04-12 19:02:01.801038", "stderr": "Error: resource or tag id 'an-test-deploy4' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy4' not found"], "stdout": "", "stdout_lines": []}
[2026-04-12T23:02:03.298Z] ...ignoring
[2026-04-12T23:02:03.298Z] 
[2026-04-12T23:02:03.298Z] TASK [deploy server an-test-deploy4 from an-a01n01 on storage 9fafa17d-152b-41c7-9537-6bb636df22f0] ***
[2026-04-12T23:02:12.620Z] skipping: [an-a01n02]
[2026-04-12T23:02:12.620Z] changed: [an-a01n01]
[2026-04-12T23:02:12.620Z] 
[2026-04-12T23:02:12.620Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to be started in pacemaker] ***
[2026-04-12T23:02:12.620Z] skipping: [an-a01n01]
[2026-04-12T23:02:12.620Z] skipping: [an-a01n02]
[2026-04-12T23:02:12.620Z] 
[2026-04-12T23:02:12.620Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} drbd storage to be synced] ***
[2026-04-12T23:02:12.620Z] skipping: [an-a01n01]
[2026-04-12T23:02:12.620Z] skipping: [an-a01n02]
[2026-04-12T23:02:12.620Z] 
[2026-04-12T23:02:12.620Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to start installation] ***
[2026-04-12T23:02:12.620Z] skipping: [an-a01n01]
[2026-04-12T23:02:12.620Z] skipping: [an-a01n02]
[2026-04-12T23:02:12.620Z] 
[2026-04-12T23:02:12.620Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to complete installation] ***
[2026-04-12T23:02:12.620Z] skipping: [an-a01n01]
[2026-04-12T23:02:12.620Z] skipping: [an-a01n02]
[2026-04-12T23:02:12.620Z] 
[2026-04-12T23:02:12.620Z] TASK [check if server an-test-deploy5 exists] **********************************
[2026-04-12T23:02:13.883Z] skipping: [an-a01n02]
[2026-04-12T23:02:13.883Z] fatal: [an-a01n01]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy5\n", "delta": "0:00:00.957786", "end": "2026-04-12 19:02:13.633870", "msg": "non-zero return code", "rc": 1, "start": "2026-04-12 19:02:12.676084", "stderr": "Error: resource or tag id 'an-test-deploy5' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy5' not found"], "stdout": "", "stdout_lines": []}
[2026-04-12T23:02:13.883Z] ...ignoring
[2026-04-12T23:02:13.883Z] 
[2026-04-12T23:02:13.883Z] TASK [deploy server an-test-deploy5 from an-a01n01 on storage 9fafa17d-152b-41c7-9537-6bb636df22f0] ***
[2026-04-12T23:02:25.176Z] skipping: [an-a01n02]
[2026-04-12T23:02:25.176Z] changed: [an-a01n01]
[2026-04-12T23:02:25.176Z] 
[2026-04-12T23:02:25.176Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to be started in pacemaker] ***
[2026-04-12T23:02:25.176Z] skipping: [an-a01n01]
[2026-04-12T23:02:25.176Z] skipping: [an-a01n02]
[2026-04-12T23:02:25.176Z] 
[2026-04-12T23:02:25.176Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} drbd storage to be synced] ***
[2026-04-12T23:02:25.176Z] skipping: [an-a01n01]
[2026-04-12T23:02:25.176Z] skipping: [an-a01n02]
[2026-04-12T23:02:25.176Z] 
[2026-04-12T23:02:25.176Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to start installation] ***
[2026-04-12T23:02:25.176Z] skipping: [an-a01n01]
[2026-04-12T23:02:25.176Z] skipping: [an-a01n02]
[2026-04-12T23:02:25.176Z] 
[2026-04-12T23:02:25.176Z] TASK [Wait up to {{ timeout }} minutes for an-test-deploy{{ servernum }} to complete installation] ***
[2026-04-12T23:02:25.176Z] skipping: [an-a01n01]
[2026-04-12T23:02:25.176Z] skipping: [an-a01n02]
[2026-04-12T23:02:25.176Z] 
[2026-04-12T23:02:25.176Z] TASK [wait servers to be started in pacemaker] *********************************
[2026-04-12T23:02:25.176Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-12T23:02:25.176Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-12T23:02:25.176Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-12T23:02:25.176Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-12T23:02:25.176Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-12T23:02:25.176Z] 
[2026-04-12T23:02:25.176Z] TASK [Wait up to 60 minutes for an-test-deploy1 to be started in pacemaker] ****
[2026-04-12T23:02:53.699Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 to be started in pacemaker (360 retries left).
[2026-04-12T23:02:53.699Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 to be started in pacemaker (360 retries left).
[2026-04-12T23:02:53.699Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 to be started in pacemaker (359 retries left).
[2026-04-12T23:02:53.699Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 to be started in pacemaker (359 retries left).
[2026-04-12T23:02:53.699Z] ok: [an-a01n02]
[2026-04-12T23:02:53.699Z] ok: [an-a01n01]
[2026-04-12T23:02:53.699Z] 
[2026-04-12T23:02:53.699Z] TASK [Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker] ****
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (360 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (360 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (359 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (359 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (358 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (358 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (357 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (357 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (356 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (356 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (355 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (355 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (354 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (354 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (353 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (353 retries left).
[2026-04-12T23:05:03.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (352 retries left).
[2026-04-12T23:05:03.017Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (352 retries left).
[2026-04-12T23:05:03.017Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (351 retries left).
[2026-04-12T23:05:03.017Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to be started in pacemaker (351 retries left).
[2026-04-12T23:05:03.017Z] ok: [an-a01n02]
[2026-04-12T23:05:03.017Z] ok: [an-a01n01]
[2026-04-12T23:05:03.017Z] 
[2026-04-12T23:05:03.017Z] TASK [Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker] ****
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (360 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (360 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (359 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (359 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (358 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (358 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (357 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (357 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (356 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (356 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (355 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (355 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (354 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (354 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (353 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (353 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (352 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (352 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (351 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (351 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (350 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (350 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (349 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (349 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (348 retries left).
[2026-04-12T23:07:27.596Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to be started in pacemaker (348 retries left).
[2026-04-12T23:07:27.596Z] ok: [an-a01n02]
[2026-04-12T23:07:27.596Z] ok: [an-a01n01]
[2026-04-12T23:07:27.596Z] 
[2026-04-12T23:07:27.596Z] TASK [Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker] ****
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (360 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (360 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (359 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (359 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (358 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (358 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (357 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (357 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (356 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (356 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (355 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (355 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (354 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (354 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (353 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (353 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (352 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (352 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (351 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (351 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (350 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (350 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (349 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (349 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (348 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (348 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (347 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (347 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (346 retries left).
[2026-04-12T23:10:37.502Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to be started in pacemaker (346 retries left).
[2026-04-12T23:10:37.502Z] ok: [an-a01n02]
[2026-04-12T23:10:37.502Z] ok: [an-a01n01]
[2026-04-12T23:10:37.502Z] 
[2026-04-12T23:10:37.502Z] TASK [Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker] ****
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (360 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (360 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (359 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (359 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (358 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (358 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (357 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (357 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (356 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (356 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (355 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (355 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (354 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (354 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (353 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (353 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (352 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (352 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (351 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (351 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (350 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (350 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (349 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (349 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (348 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (348 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (347 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (347 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (346 retries left).
[2026-04-12T23:13:47.327Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to be started in pacemaker (346 retries left).
[2026-04-12T23:13:47.327Z] ok: [an-a01n02]
[2026-04-12T23:13:47.327Z] ok: [an-a01n01]
[2026-04-12T23:13:47.327Z] 
[2026-04-12T23:13:47.327Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-12T23:13:47.327Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-12T23:13:47.327Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-12T23:13:47.327Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-12T23:13:47.327Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-12T23:13:47.327Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-12T23:13:47.327Z] 
[2026-04-12T23:13:47.327Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-12T23:13:47.327Z] ok: [an-a01n02]
[2026-04-12T23:13:47.327Z] ok: [an-a01n01]
[2026-04-12T23:13:47.327Z] 
[2026-04-12T23:13:47.327Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-12T23:13:47.327Z] ok: [an-a01n02]
[2026-04-12T23:13:47.327Z] ok: [an-a01n01]
[2026-04-12T23:13:47.327Z] 
[2026-04-12T23:13:47.327Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-12T23:13:47.327Z] ok: [an-a01n02]
[2026-04-12T23:13:47.327Z] ok: [an-a01n01]
[2026-04-12T23:13:47.327Z] 
[2026-04-12T23:13:47.327Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-12T23:13:47.327Z] ok: [an-a01n02]
[2026-04-12T23:13:47.327Z] ok: [an-a01n01]
[2026-04-12T23:13:47.327Z] 
[2026-04-12T23:13:47.327Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-12T23:14:11.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (360 retries left).
[2026-04-12T23:14:11.016Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (360 retries left).
[2026-04-12T23:14:11.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (359 retries left).
[2026-04-12T23:14:11.016Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (359 retries left).
[2026-04-12T23:14:11.016Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (358 retries left).
[2026-04-12T23:14:11.016Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced (358 retries left).
[2026-04-12T23:14:11.016Z] ok: [an-a01n02]
[2026-04-12T23:14:11.016Z] ok: [an-a01n01]
[2026-04-12T23:14:11.016Z] 
[2026-04-12T23:14:11.016Z] TASK [wait servers to respond to ping] *****************************************
[2026-04-12T23:14:11.016Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-12T23:14:11.016Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-12T23:14:11.016Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-12T23:14:11.016Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-12T23:14:11.016Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-12T23:14:11.016Z] 
[2026-04-12T23:14:11.016Z] TASK [Wait up to 60 minutes for an-test-deploy1 to respond to ping] ************
[2026-04-12T23:14:12.280Z] ok: [an-a01n02]
[2026-04-12T23:14:12.280Z] ok: [an-a01n01]
[2026-04-12T23:14:12.280Z] 
[2026-04-12T23:14:12.280Z] TASK [Wait up to 60 minutes for an-test-deploy2 to respond to ping] ************
[2026-04-12T23:14:13.067Z] ok: [an-a01n02]
[2026-04-12T23:14:13.067Z] ok: [an-a01n01]
[2026-04-12T23:14:13.067Z] 
[2026-04-12T23:14:13.067Z] TASK [Wait up to 60 minutes for an-test-deploy3 to respond to ping] ************
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (360 retries left).
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (360 retries left).
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (359 retries left).
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (359 retries left).
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (358 retries left).
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (358 retries left).
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (357 retries left).
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (357 retries left).
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (356 retries left).
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (356 retries left).
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (355 retries left).
[2026-04-12T23:15:52.267Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ping (355 retries left).
[2026-04-12T23:15:52.267Z] ok: [an-a01n02]
[2026-04-12T23:15:52.267Z] ok: [an-a01n01]
[2026-04-12T23:15:52.267Z] 
[2026-04-12T23:15:52.267Z] TASK [Wait up to 60 minutes for an-test-deploy4 to respond to ping] ************
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (360 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (360 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (359 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (359 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (358 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (358 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (357 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (357 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (356 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (356 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (355 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (355 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (354 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (354 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (353 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (353 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (352 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (352 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (351 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (351 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (350 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (350 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (349 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (349 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (348 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (348 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (347 retries left).
[2026-04-12T23:19:04.157Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ping (347 retries left).
[2026-04-12T23:19:04.157Z] ok: [an-a01n01]
[2026-04-12T23:19:04.157Z] ok: [an-a01n02]
[2026-04-12T23:19:04.157Z] 
[2026-04-12T23:19:04.157Z] TASK [Wait up to 60 minutes for an-test-deploy5 to respond to ping] ************
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (360 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (360 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (359 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (359 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (358 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (358 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (357 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (357 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (356 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (356 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (355 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (355 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (354 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (354 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (353 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (353 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (352 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (352 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (351 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (351 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (350 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (350 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (349 retries left).
[2026-04-12T23:21:43.745Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ping (349 retries left).
[2026-04-12T23:21:43.745Z] ok: [an-a01n02]
[2026-04-12T23:21:43.745Z] ok: [an-a01n01]
[2026-04-12T23:21:43.745Z] 
[2026-04-12T23:21:43.745Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-12T23:21:43.745Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-12T23:21:43.745Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-12T23:21:43.745Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-12T23:21:43.745Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-12T23:21:43.745Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-12T23:21:43.745Z] 
[2026-04-12T23:21:43.745Z] TASK [Wait up to 60 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-12T23:21:43.745Z] ok: [an-a01n02]
[2026-04-12T23:21:43.745Z] ok: [an-a01n01]
[2026-04-12T23:21:43.745Z] 
[2026-04-12T23:21:43.745Z] TASK [Wait up to 60 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-12T23:22:53.894Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (360 retries left).
[2026-04-12T23:22:53.894Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (360 retries left).
[2026-04-12T23:22:53.894Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (359 retries left).
[2026-04-12T23:22:53.894Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (359 retries left).
[2026-04-12T23:22:53.894Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (358 retries left).
[2026-04-12T23:22:53.894Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (358 retries left).
[2026-04-12T23:22:53.894Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (357 retries left).
[2026-04-12T23:22:53.894Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 to respond to ssh (357 retries left).
[2026-04-12T23:22:53.894Z] ok: [an-a01n02]
[2026-04-12T23:22:53.894Z] ok: [an-a01n01]
[2026-04-12T23:22:53.894Z] 
[2026-04-12T23:22:53.894Z] TASK [Wait up to 60 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (360 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (360 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (359 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (359 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (358 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (358 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (357 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (357 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (356 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (356 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (355 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (355 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (354 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (354 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (353 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (353 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (352 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (352 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (351 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (351 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (350 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (350 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (349 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (349 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (348 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (348 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (347 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (347 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (346 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (346 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (345 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (345 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (344 retries left).
[2026-04-12T23:26:04.100Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy3 to respond to ssh (344 retries left).
[2026-04-12T23:26:04.100Z] ok: [an-a01n01]
[2026-04-12T23:26:04.100Z] ok: [an-a01n02]
[2026-04-12T23:26:04.100Z] 
[2026-04-12T23:26:04.100Z] TASK [Wait up to 60 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (360 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (360 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (359 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (359 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (358 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (358 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (357 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (357 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (356 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (356 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (355 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (355 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (354 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (354 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (353 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (353 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (352 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (352 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (351 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (351 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (350 retries left).
[2026-04-12T23:28:13.448Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy4 to respond to ssh (350 retries left).
[2026-04-12T23:28:13.448Z] ok: [an-a01n02]
[2026-04-12T23:28:13.448Z] ok: [an-a01n01]
[2026-04-12T23:28:13.448Z] 
[2026-04-12T23:28:13.448Z] TASK [Wait up to 60 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (360 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (360 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (359 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (359 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (358 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (358 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (357 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (357 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (356 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (356 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (355 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (355 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (354 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (354 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (353 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (353 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (352 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (352 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (351 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (351 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (350 retries left).
[2026-04-12T23:30:38.132Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy5 to respond to ssh (350 retries left).
[2026-04-12T23:30:38.132Z] ok: [an-a01n01]
[2026-04-12T23:30:38.132Z] ok: [an-a01n02]
[2026-04-12T23:30:38.132Z] 
[2026-04-12T23:30:38.132Z] TASK [fence with servers test] *************************************************
[2026-04-12T23:30:38.132Z] included: /srv/anvil/anvil-test-suite/playbooks/common/fence_with_servers.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-12T23:30:38.132Z] included: /srv/anvil/anvil-test-suite/playbooks/common/fence_with_servers.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-12T23:30:38.132Z] included: /srv/anvil/anvil-test-suite/playbooks/common/fence_with_servers.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-12T23:30:38.132Z] included: /srv/anvil/anvil-test-suite/playbooks/common/fence_with_servers.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-12T23:30:38.132Z] included: /srv/anvil/anvil-test-suite/playbooks/common/fence_with_servers.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-12T23:30:38.132Z] 
[2026-04-12T23:30:38.132Z] TASK [fence with servers iteration 1] ******************************************
[2026-04-12T23:30:38.132Z] ok: [an-a01n01]
[2026-04-12T23:30:38.132Z] ok: [an-a01n02]
[2026-04-12T23:30:38.132Z] 
[2026-04-12T23:30:38.132Z] TASK [check servers status] ****************************************************
[2026-04-12T23:30:38.132Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01, an-a01n02
[2026-04-12T23:30:38.132Z] 
[2026-04-12T23:30:38.132Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-12T23:30:38.132Z] ok: [an-a01n02]
[2026-04-12T23:30:38.132Z] ok: [an-a01n01]
[2026-04-12T23:30:38.132Z] 
[2026-04-12T23:30:38.132Z] TASK [check servers are started all on the same node] **************************
[2026-04-12T23:30:38.132Z] ok: [an-a01n01]
[2026-04-12T23:30:38.132Z] ok: [an-a01n02]
[2026-04-12T23:30:38.132Z] 
[2026-04-12T23:30:38.132Z] TASK [set target] **************************************************************
[2026-04-12T23:30:38.132Z] ok: [an-a01n01]
[2026-04-12T23:30:38.132Z] ok: [an-a01n02]
[2026-04-12T23:30:38.132Z] 
[2026-04-12T23:30:38.132Z] TASK [will kill node with servers] *********************************************
[2026-04-12T23:30:38.132Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./kill_node.yml for an-a01n01, an-a01n02
[2026-04-12T23:30:38.132Z] 
[2026-04-12T23:30:38.132Z] TASK [kill node "an-a01n01"] ***************************************************
[2026-04-12T23:30:38.132Z] skipping: [an-a01n02]
[2026-04-12T23:30:38.132Z] changed: [an-a01n01]
[2026-04-12T23:30:38.132Z] 
[2026-04-12T23:30:38.132Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-12T23:30:38.132Z] skipping: [an-a01n01]
[2026-04-12T23:30:38.132Z] skipping: [an-a01n02]
[2026-04-12T23:30:38.132Z] 
[2026-04-12T23:30:38.132Z] TASK [wait for node to be offline] *********************************************
[2026-04-12T23:30:38.132Z] skipping: [an-a01n01]
[2026-04-12T23:30:38.132Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./offline_node.yml for an-a01n02
[2026-04-12T23:30:38.132Z] 
[2026-04-12T23:30:38.132Z] TASK [wait up to 10 minutes for node an-a01n01 to be offline] ******************
[2026-04-12T23:31:36.658Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (60 retries left).
[2026-04-12T23:31:36.658Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (59 retries left).
[2026-04-12T23:31:36.658Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (58 retries left).
[2026-04-12T23:31:36.658Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (57 retries left).
[2026-04-12T23:31:36.658Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (56 retries left).
[2026-04-12T23:31:36.658Z] ok: [an-a01n02]
[2026-04-12T23:31:36.658Z] 
[2026-04-12T23:31:36.658Z] TASK [check servers status] ****************************************************
[2026-04-12T23:31:36.658Z] skipping: [an-a01n01]
[2026-04-12T23:31:36.658Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n02
[2026-04-12T23:31:36.658Z] 
[2026-04-12T23:31:36.658Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-12T23:33:16.058Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (120 retries left).
[2026-04-12T23:33:16.058Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (119 retries left).
[2026-04-12T23:33:16.058Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (118 retries left).
[2026-04-12T23:33:16.058Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (117 retries left).
[2026-04-12T23:33:16.058Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (116 retries left).
[2026-04-12T23:33:16.058Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (115 retries left).
[2026-04-12T23:33:16.058Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (114 retries left).
[2026-04-12T23:33:16.058Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (113 retries left).
[2026-04-12T23:33:16.058Z] ok: [an-a01n02]
[2026-04-12T23:33:16.058Z] 
[2026-04-12T23:33:16.058Z] TASK [check servers are started all on the same node] **************************
[2026-04-12T23:33:16.058Z] ok: [an-a01n02]
[2026-04-12T23:33:16.058Z] 
[2026-04-12T23:33:16.058Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-12T23:33:16.058Z] skipping: [an-a01n01] => (item=1) 
[2026-04-12T23:33:16.058Z] skipping: [an-a01n01] => (item=2) 
[2026-04-12T23:33:16.058Z] skipping: [an-a01n01] => (item=3) 
[2026-04-12T23:33:16.058Z] skipping: [an-a01n01] => (item=4) 
[2026-04-12T23:33:16.058Z] skipping: [an-a01n01] => (item=5) 
[2026-04-12T23:33:16.058Z] skipping: [an-a01n01]
[2026-04-12T23:33:16.058Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=1)
[2026-04-12T23:33:16.058Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=2)
[2026-04-12T23:33:16.058Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=3)
[2026-04-12T23:33:16.058Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=4)
[2026-04-12T23:33:16.058Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=5)
[2026-04-12T23:33:16.058Z] 
[2026-04-12T23:33:16.058Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-12T23:33:16.845Z] ok: [an-a01n02]
[2026-04-12T23:33:16.845Z] 
[2026-04-12T23:33:16.845Z] TASK [Wait up to 10 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-12T23:33:17.632Z] ok: [an-a01n02]
[2026-04-12T23:33:17.632Z] 
[2026-04-12T23:33:17.632Z] TASK [Wait up to 10 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-12T23:33:18.418Z] ok: [an-a01n02]
[2026-04-12T23:33:18.418Z] 
[2026-04-12T23:33:18.418Z] TASK [Wait up to 10 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-12T23:33:19.204Z] ok: [an-a01n02]
[2026-04-12T23:33:19.204Z] 
[2026-04-12T23:33:19.204Z] TASK [Wait up to 10 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-12T23:33:20.469Z] ok: [an-a01n02]
[2026-04-12T23:33:20.469Z] 
[2026-04-12T23:33:20.469Z] TASK [wait for node to be back online] *****************************************
[2026-04-12T23:33:20.469Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01, an-a01n02
[2026-04-12T23:33:20.469Z] 
[2026-04-12T23:33:20.469Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-12T23:36:45.375Z] ok: [an-a01n01 -> localhost]
[2026-04-12T23:36:45.375Z] ok: [an-a01n02 -> localhost]
[2026-04-12T23:36:45.375Z] 
[2026-04-12T23:36:45.375Z] TASK [check cluster is in good state] ******************************************
[2026-04-12T23:36:45.375Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./cluster_good.yml for an-a01n01, an-a01n02
[2026-04-12T23:36:45.375Z] 
[2026-04-12T23:36:45.375Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-12T23:36:45.375Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-12T23:36:45.375Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-12T23:36:45.375Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-12T23:36:45.375Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-12T23:36:45.375Z] 
[2026-04-12T23:36:45.375Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-12T23:36:45.375Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-12T23:36:45.375Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-12T23:36:45.375Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-12T23:36:45.375Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-12T23:36:45.375Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-12T23:36:45.375Z] 
[2026-04-12T23:36:45.375Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (360 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (360 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (359 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (359 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (358 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (358 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (357 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (357 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (356 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (356 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (355 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (355 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (354 retries left).
[2026-04-12T23:37:55.820Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (354 retries left).
[2026-04-12T23:37:55.820Z] ok: [an-a01n01]
[2026-04-12T23:37:55.820Z] ok: [an-a01n02]
[2026-04-12T23:37:55.820Z] 
[2026-04-12T23:37:55.820Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-12T23:38:03.484Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced (360 retries left).
[2026-04-12T23:38:03.484Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced (360 retries left).
[2026-04-12T23:38:03.484Z] ok: [an-a01n01]
[2026-04-12T23:38:03.484Z] ok: [an-a01n02]
[2026-04-12T23:38:03.484Z] 
[2026-04-12T23:38:03.484Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-12T23:38:04.269Z] ok: [an-a01n01]
[2026-04-12T23:38:04.269Z] ok: [an-a01n02]
[2026-04-12T23:38:04.269Z] 
[2026-04-12T23:38:04.269Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-12T23:38:05.055Z] ok: [an-a01n01]
[2026-04-12T23:38:05.055Z] ok: [an-a01n02]
[2026-04-12T23:38:05.055Z] 
[2026-04-12T23:38:05.055Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-12T23:38:05.842Z] ok: [an-a01n01]
[2026-04-12T23:38:05.842Z] ok: [an-a01n02]
[2026-04-12T23:38:05.842Z] 
[2026-04-12T23:38:05.842Z] TASK [fence with servers iteration 2] ******************************************
[2026-04-12T23:38:05.842Z] ok: [an-a01n01]
[2026-04-12T23:38:05.842Z] ok: [an-a01n02]
[2026-04-12T23:38:05.842Z] 
[2026-04-12T23:38:05.842Z] TASK [check servers status] ****************************************************
[2026-04-12T23:38:05.842Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01, an-a01n02
[2026-04-12T23:38:05.842Z] 
[2026-04-12T23:38:05.842Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-12T23:38:08.295Z] ok: [an-a01n02]
[2026-04-12T23:38:08.295Z] ok: [an-a01n01]
[2026-04-12T23:38:08.295Z] 
[2026-04-12T23:38:08.295Z] TASK [check servers are started all on the same node] **************************
[2026-04-12T23:38:10.110Z] ok: [an-a01n02]
[2026-04-12T23:38:10.110Z] ok: [an-a01n01]
[2026-04-12T23:38:10.110Z] 
[2026-04-12T23:38:10.110Z] TASK [set target] **************************************************************
[2026-04-12T23:38:10.110Z] ok: [an-a01n01]
[2026-04-12T23:38:10.110Z] ok: [an-a01n02]
[2026-04-12T23:38:10.110Z] 
[2026-04-12T23:38:10.110Z] TASK [will kill node with servers] *********************************************
[2026-04-12T23:38:10.479Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./kill_node.yml for an-a01n01, an-a01n02
[2026-04-12T23:38:10.479Z] 
[2026-04-12T23:38:10.479Z] TASK [kill node "an-a01n02"] ***************************************************
[2026-04-12T23:38:10.847Z] skipping: [an-a01n01]
[2026-04-12T23:38:10.847Z] changed: [an-a01n02]
[2026-04-12T23:38:10.847Z] 
[2026-04-12T23:38:10.847Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-12T23:38:11.216Z] skipping: [an-a01n01]
[2026-04-12T23:38:11.216Z] skipping: [an-a01n02]
[2026-04-12T23:38:11.216Z] 
[2026-04-12T23:38:11.216Z] TASK [wait for node to be offline] *********************************************
[2026-04-12T23:38:11.216Z] skipping: [an-a01n02]
[2026-04-12T23:38:11.216Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./offline_node.yml for an-a01n01
[2026-04-12T23:38:11.216Z] 
[2026-04-12T23:38:11.216Z] TASK [wait up to 10 minutes for node an-a01n02 to be offline] ******************
[2026-04-12T23:39:21.315Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (60 retries left).
[2026-04-12T23:39:21.315Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (59 retries left).
[2026-04-12T23:39:21.315Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (58 retries left).
[2026-04-12T23:39:21.315Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (57 retries left).
[2026-04-12T23:39:21.315Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (56 retries left).
[2026-04-12T23:39:21.315Z] ok: [an-a01n01]
[2026-04-12T23:39:21.315Z] 
[2026-04-12T23:39:21.315Z] TASK [check servers status] ****************************************************
[2026-04-12T23:39:21.315Z] skipping: [an-a01n02]
[2026-04-12T23:39:21.315Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01
[2026-04-12T23:39:21.315Z] 
[2026-04-12T23:39:21.315Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-12T23:40:31.476Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (120 retries left).
[2026-04-12T23:40:31.476Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (119 retries left).
[2026-04-12T23:40:31.476Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (118 retries left).
[2026-04-12T23:40:31.476Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (117 retries left).
[2026-04-12T23:40:31.476Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (116 retries left).
[2026-04-12T23:40:31.476Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (115 retries left).
[2026-04-12T23:40:31.476Z] ok: [an-a01n01]
[2026-04-12T23:40:31.476Z] 
[2026-04-12T23:40:31.476Z] TASK [check servers are started all on the same node] **************************
[2026-04-12T23:40:33.292Z] ok: [an-a01n01]
[2026-04-12T23:40:33.292Z] 
[2026-04-12T23:40:33.292Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-12T23:40:33.292Z] skipping: [an-a01n02] => (item=1) 
[2026-04-12T23:40:33.292Z] skipping: [an-a01n02] => (item=2) 
[2026-04-12T23:40:33.292Z] skipping: [an-a01n02] => (item=3) 
[2026-04-12T23:40:33.292Z] skipping: [an-a01n02] => (item=4) 
[2026-04-12T23:40:33.292Z] skipping: [an-a01n02] => (item=5) 
[2026-04-12T23:40:33.292Z] skipping: [an-a01n02]
[2026-04-12T23:40:33.292Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=1)
[2026-04-12T23:40:33.292Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=2)
[2026-04-12T23:40:33.292Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=3)
[2026-04-12T23:40:33.292Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=4)
[2026-04-12T23:40:33.292Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=5)
[2026-04-12T23:40:33.292Z] 
[2026-04-12T23:40:33.292Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-12T23:40:49.708Z] FAILED - RETRYING: [an-a01n01]: Wait up to 10 minutes for an-test-deploy1 to respond to ssh (60 retries left).
[2026-04-12T23:40:49.708Z] ok: [an-a01n01]
[2026-04-12T23:40:49.708Z] 
[2026-04-12T23:40:49.708Z] TASK [Wait up to 10 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-12T23:40:49.708Z] ok: [an-a01n01]
[2026-04-12T23:40:49.708Z] 
[2026-04-12T23:40:49.708Z] TASK [Wait up to 10 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-12T23:40:50.494Z] ok: [an-a01n01]
[2026-04-12T23:40:50.494Z] 
[2026-04-12T23:40:50.494Z] TASK [Wait up to 10 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-12T23:40:51.759Z] ok: [an-a01n01]
[2026-04-12T23:40:51.759Z] 
[2026-04-12T23:40:51.759Z] TASK [Wait up to 10 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-12T23:40:53.024Z] ok: [an-a01n01]
[2026-04-12T23:40:53.024Z] 
[2026-04-12T23:40:53.024Z] TASK [wait for node to be back online] *****************************************
[2026-04-12T23:40:53.024Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01, an-a01n02
[2026-04-12T23:40:53.024Z] 
[2026-04-12T23:40:53.024Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-12T23:44:18.143Z] ok: [an-a01n01 -> localhost]
[2026-04-12T23:44:18.143Z] ok: [an-a01n02 -> localhost]
[2026-04-12T23:44:18.143Z] 
[2026-04-12T23:44:18.143Z] TASK [check cluster is in good state] ******************************************
[2026-04-12T23:44:18.143Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./cluster_good.yml for an-a01n01, an-a01n02
[2026-04-12T23:44:18.143Z] 
[2026-04-12T23:44:18.143Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-12T23:44:18.143Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-12T23:44:18.143Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-12T23:44:18.143Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-12T23:44:18.143Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-12T23:44:18.143Z] 
[2026-04-12T23:44:18.143Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-12T23:44:18.143Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-12T23:44:18.143Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-12T23:44:18.143Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-12T23:44:18.143Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-12T23:44:18.143Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-12T23:44:18.143Z] 
[2026-04-12T23:44:18.143Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-12T23:45:07.012Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (360 retries left).
[2026-04-12T23:45:07.012Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (360 retries left).
[2026-04-12T23:45:07.012Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (359 retries left).
[2026-04-12T23:45:07.012Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (359 retries left).
[2026-04-12T23:45:07.012Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (358 retries left).
[2026-04-12T23:45:07.012Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (358 retries left).
[2026-04-12T23:45:07.012Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (357 retries left).
[2026-04-12T23:45:07.012Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (357 retries left).
[2026-04-12T23:45:07.012Z] FAILED - RETRYING: [an-a01n01]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (356 retries left).
[2026-04-12T23:45:07.012Z] FAILED - RETRYING: [an-a01n02]: Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced (356 retries left).
[2026-04-12T23:45:07.012Z] ok: [an-a01n01]
[2026-04-12T23:45:07.012Z] ok: [an-a01n02]
[2026-04-12T23:45:07.012Z] 
[2026-04-12T23:45:07.012Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-12T23:45:07.012Z] ok: [an-a01n02]
[2026-04-12T23:45:07.012Z] ok: [an-a01n01]
[2026-04-12T23:45:07.012Z] 
[2026-04-12T23:45:07.012Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-12T23:45:07.012Z] ok: [an-a01n01]
[2026-04-12T23:45:07.012Z] ok: [an-a01n02]
[2026-04-12T23:45:07.012Z] 
[2026-04-12T23:45:07.012Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-12T23:45:07.012Z] ok: [an-a01n01]
[2026-04-12T23:45:07.012Z] ok: [an-a01n02]
[2026-04-12T23:45:07.012Z] 
[2026-04-12T23:45:07.012Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-12T23:45:07.012Z] ok: [an-a01n01]
[2026-04-12T23:45:07.012Z] ok: [an-a01n02]
[2026-04-12T23:45:07.012Z] 
[2026-04-12T23:45:07.012Z] TASK [fence with servers iteration 3] ******************************************
[2026-04-12T23:45:07.012Z] ok: [an-a01n01]
[2026-04-12T23:45:07.012Z] ok: [an-a01n02]
[2026-04-12T23:45:07.012Z] 
[2026-04-12T23:45:07.012Z] TASK [check servers status] ****************************************************
[2026-04-12T23:45:07.012Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01, an-a01n02
[2026-04-12T23:45:07.012Z] 
[2026-04-12T23:45:07.012Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-12T23:45:08.828Z] ok: [an-a01n01]
[2026-04-12T23:45:08.828Z] ok: [an-a01n02]
[2026-04-12T23:45:08.828Z] 
[2026-04-12T23:45:08.828Z] TASK [check servers are started all on the same node] **************************
[2026-04-12T23:45:10.643Z] ok: [an-a01n01]
[2026-04-12T23:45:10.643Z] ok: [an-a01n02]
[2026-04-12T23:45:10.643Z] 
[2026-04-12T23:45:10.643Z] TASK [set target] **************************************************************
[2026-04-12T23:45:10.643Z] ok: [an-a01n01]
[2026-04-12T23:45:10.643Z] ok: [an-a01n02]
[2026-04-12T23:45:10.643Z] 
[2026-04-12T23:45:10.643Z] TASK [will kill node with servers] *********************************************
[2026-04-12T23:45:10.643Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./kill_node.yml for an-a01n01, an-a01n02
[2026-04-12T23:45:10.643Z] 
[2026-04-12T23:45:10.643Z] TASK [kill node "an-a01n01"] ***************************************************
[2026-04-12T23:45:11.429Z] skipping: [an-a01n02]
[2026-04-12T23:45:11.430Z] changed: [an-a01n01]
[2026-04-12T23:45:11.430Z] 
[2026-04-12T23:45:11.430Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-12T23:45:11.430Z] skipping: [an-a01n01]
[2026-04-12T23:45:11.430Z] skipping: [an-a01n02]
[2026-04-12T23:45:11.430Z] 
[2026-04-12T23:45:11.430Z] TASK [wait for node to be offline] *********************************************
[2026-04-12T23:45:11.430Z] skipping: [an-a01n01]
[2026-04-12T23:45:11.430Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./offline_node.yml for an-a01n02
[2026-04-12T23:45:11.430Z] 
[2026-04-12T23:45:11.430Z] TASK [wait up to 10 minutes for node an-a01n01 to be offline] ******************
[2026-04-12T23:46:21.693Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (60 retries left).
[2026-04-12T23:46:21.693Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (59 retries left).
[2026-04-12T23:46:21.693Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (58 retries left).
[2026-04-12T23:46:21.693Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (57 retries left).
[2026-04-12T23:46:21.693Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (56 retries left).
[2026-04-12T23:46:21.693Z] ok: [an-a01n02]
[2026-04-12T23:46:21.693Z] 
[2026-04-12T23:46:21.693Z] TASK [check servers status] ****************************************************
[2026-04-12T23:46:21.693Z] skipping: [an-a01n01]
[2026-04-12T23:46:21.693Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n02
[2026-04-12T23:46:21.693Z] 
[2026-04-12T23:46:21.693Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-12T23:47:20.231Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (120 retries left).
[2026-04-12T23:47:20.231Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (119 retries left).
[2026-04-12T23:47:20.231Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (118 retries left).
[2026-04-12T23:47:20.231Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (117 retries left).
[2026-04-12T23:47:20.231Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (116 retries left).
[2026-04-12T23:47:20.231Z] ok: [an-a01n02]
[2026-04-12T23:47:20.231Z] 
[2026-04-12T23:47:20.231Z] TASK [check servers are started all on the same node] **************************
[2026-04-12T23:47:20.231Z] ok: [an-a01n02]
[2026-04-12T23:47:20.231Z] 
[2026-04-12T23:47:20.231Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-12T23:47:20.231Z] skipping: [an-a01n01] => (item=1) 
[2026-04-12T23:47:20.231Z] skipping: [an-a01n01] => (item=2) 
[2026-04-12T23:47:20.231Z] skipping: [an-a01n01] => (item=3) 
[2026-04-12T23:47:20.231Z] skipping: [an-a01n01] => (item=4) 
[2026-04-12T23:47:20.231Z] skipping: [an-a01n01] => (item=5) 
[2026-04-12T23:47:20.231Z] skipping: [an-a01n01]
[2026-04-12T23:47:20.231Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=1)
[2026-04-12T23:47:20.231Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=2)
[2026-04-12T23:47:20.231Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=3)
[2026-04-12T23:47:20.231Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=4)
[2026-04-12T23:47:20.231Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=5)
[2026-04-12T23:47:20.231Z] 
[2026-04-12T23:47:20.231Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-12T23:47:36.678Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to respond to ssh (60 retries left).
[2026-04-12T23:47:36.678Z] ok: [an-a01n02]
[2026-04-12T23:47:36.678Z] 
[2026-04-12T23:47:36.678Z] TASK [Wait up to 10 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-12T23:47:36.678Z] ok: [an-a01n02]
[2026-04-12T23:47:36.678Z] 
[2026-04-12T23:47:36.678Z] TASK [Wait up to 10 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-12T23:47:36.678Z] ok: [an-a01n02]
[2026-04-12T23:47:36.678Z] 
[2026-04-12T23:47:36.678Z] TASK [Wait up to 10 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-12T23:47:37.464Z] ok: [an-a01n02]
[2026-04-12T23:47:37.464Z] 
[2026-04-12T23:47:37.464Z] TASK [Wait up to 10 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-12T23:47:37.833Z] ok: [an-a01n02]
[2026-04-12T23:47:37.833Z] 
[2026-04-12T23:47:37.833Z] TASK [wait for node to be back online] *****************************************
[2026-04-12T23:47:38.201Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01, an-a01n02
[2026-04-12T23:47:38.201Z] 
[2026-04-12T23:47:38.201Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-12T23:51:03.274Z] ok: [an-a01n01 -> localhost]
[2026-04-12T23:51:03.274Z] ok: [an-a01n02 -> localhost]
[2026-04-12T23:51:03.274Z] 
[2026-04-12T23:51:03.274Z] TASK [check cluster is in good state] ******************************************
[2026-04-12T23:51:03.274Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./cluster_good.yml for an-a01n01, an-a01n02
[2026-04-12T23:51:03.274Z] 
[2026-04-12T23:51:03.274Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-12T23:51:03.274Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-12T23:51:03.274Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-12T23:51:03.274Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-12T23:51:03.274Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-12T23:51:03.274Z] 
[2026-04-12T23:51:03.274Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-12T23:51:03.274Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-12T23:51:03.274Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-12T23:51:03.274Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-12T23:51:03.274Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-12T23:51:03.274Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-12T23:51:03.274Z] 
[2026-04-12T23:51:03.274Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-12T23:51:03.274Z] ok: [an-a01n02]
[2026-04-12T23:51:03.274Z] ok: [an-a01n01]
[2026-04-12T23:51:03.274Z] 
[2026-04-12T23:51:03.274Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-12T23:51:03.274Z] ok: [an-a01n02]
[2026-04-12T23:51:03.274Z] ok: [an-a01n01]
[2026-04-12T23:51:03.274Z] 
[2026-04-12T23:51:03.275Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-12T23:51:03.275Z] ok: [an-a01n01]
[2026-04-12T23:51:03.275Z] ok: [an-a01n02]
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-12T23:51:03.275Z] ok: [an-a01n02]
[2026-04-12T23:51:03.275Z] ok: [an-a01n01]
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-12T23:51:03.275Z] ok: [an-a01n02]
[2026-04-12T23:51:03.275Z] ok: [an-a01n01]
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [fence with servers iteration 4] ******************************************
[2026-04-12T23:51:03.275Z] ok: [an-a01n01]
[2026-04-12T23:51:03.275Z] ok: [an-a01n02]
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [check servers status] ****************************************************
[2026-04-12T23:51:03.275Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01, an-a01n02
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-12T23:51:03.275Z] ok: [an-a01n02]
[2026-04-12T23:51:03.275Z] ok: [an-a01n01]
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [check servers are started all on the same node] **************************
[2026-04-12T23:51:03.275Z] ok: [an-a01n01]
[2026-04-12T23:51:03.275Z] ok: [an-a01n02]
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [set target] **************************************************************
[2026-04-12T23:51:03.275Z] ok: [an-a01n01]
[2026-04-12T23:51:03.275Z] ok: [an-a01n02]
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [will kill node with servers] *********************************************
[2026-04-12T23:51:03.275Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./kill_node.yml for an-a01n01, an-a01n02
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [kill node "an-a01n02"] ***************************************************
[2026-04-12T23:51:03.275Z] skipping: [an-a01n01]
[2026-04-12T23:51:03.275Z] changed: [an-a01n02]
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-12T23:51:03.275Z] skipping: [an-a01n01]
[2026-04-12T23:51:03.275Z] skipping: [an-a01n02]
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [wait for node to be offline] *********************************************
[2026-04-12T23:51:03.275Z] skipping: [an-a01n02]
[2026-04-12T23:51:03.275Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./offline_node.yml for an-a01n01
[2026-04-12T23:51:03.275Z] 
[2026-04-12T23:51:03.275Z] TASK [wait up to 10 minutes for node an-a01n02 to be offline] ******************
[2026-04-12T23:52:13.375Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (60 retries left).
[2026-04-12T23:52:13.375Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (59 retries left).
[2026-04-12T23:52:13.375Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (58 retries left).
[2026-04-12T23:52:13.375Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (57 retries left).
[2026-04-12T23:52:13.375Z] FAILED - RETRYING: [an-a01n01]: wait up to 10 minutes for node an-a01n02 to be offline (56 retries left).
[2026-04-12T23:52:13.375Z] ok: [an-a01n01]
[2026-04-12T23:52:13.375Z] 
[2026-04-12T23:52:13.375Z] TASK [check servers status] ****************************************************
[2026-04-12T23:52:13.375Z] skipping: [an-a01n02]
[2026-04-12T23:52:13.375Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01
[2026-04-12T23:52:13.375Z] 
[2026-04-12T23:52:13.375Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-12T23:53:11.935Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (120 retries left).
[2026-04-12T23:53:11.935Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (119 retries left).
[2026-04-12T23:53:11.935Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (118 retries left).
[2026-04-12T23:53:11.935Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (117 retries left).
[2026-04-12T23:53:11.935Z] FAILED - RETRYING: [an-a01n01]: wait up to 20 minutes for servers to be Started (116 retries left).
[2026-04-12T23:53:11.935Z] ok: [an-a01n01]
[2026-04-12T23:53:11.935Z] 
[2026-04-12T23:53:11.935Z] TASK [check servers are started all on the same node] **************************
[2026-04-12T23:53:11.935Z] ok: [an-a01n01]
[2026-04-12T23:53:11.935Z] 
[2026-04-12T23:53:11.935Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-12T23:53:11.935Z] skipping: [an-a01n02] => (item=1) 
[2026-04-12T23:53:11.935Z] skipping: [an-a01n02] => (item=2) 
[2026-04-12T23:53:11.935Z] skipping: [an-a01n02] => (item=3) 
[2026-04-12T23:53:11.935Z] skipping: [an-a01n02] => (item=4) 
[2026-04-12T23:53:11.935Z] skipping: [an-a01n02] => (item=5) 
[2026-04-12T23:53:11.935Z] skipping: [an-a01n02]
[2026-04-12T23:53:11.935Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=1)
[2026-04-12T23:53:11.935Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=2)
[2026-04-12T23:53:11.935Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=3)
[2026-04-12T23:53:11.935Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=4)
[2026-04-12T23:53:11.935Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n01 => (item=5)
[2026-04-12T23:53:11.935Z] 
[2026-04-12T23:53:11.935Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-12T23:53:25.712Z] FAILED - RETRYING: [an-a01n01]: Wait up to 10 minutes for an-test-deploy1 to respond to ssh (60 retries left).
[2026-04-12T23:53:25.712Z] ok: [an-a01n01]
[2026-04-12T23:53:25.712Z] 
[2026-04-12T23:53:25.712Z] TASK [Wait up to 10 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-12T23:53:25.712Z] ok: [an-a01n01]
[2026-04-12T23:53:25.712Z] 
[2026-04-12T23:53:25.712Z] TASK [Wait up to 10 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-12T23:53:26.079Z] ok: [an-a01n01]
[2026-04-12T23:53:26.079Z] 
[2026-04-12T23:53:26.079Z] TASK [Wait up to 10 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-12T23:53:26.865Z] ok: [an-a01n01]
[2026-04-12T23:53:26.865Z] 
[2026-04-12T23:53:26.865Z] TASK [Wait up to 10 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-12T23:53:27.233Z] ok: [an-a01n01]
[2026-04-12T23:53:27.233Z] 
[2026-04-12T23:53:27.233Z] TASK [wait for node to be back online] *****************************************
[2026-04-12T23:53:27.601Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01, an-a01n02
[2026-04-12T23:53:27.601Z] 
[2026-04-12T23:53:27.601Z] TASK [wait for node "an-a01n02" to be back online] *****************************
[2026-04-12T23:56:52.667Z] ok: [an-a01n01 -> localhost]
[2026-04-12T23:56:52.667Z] ok: [an-a01n02 -> localhost]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [check cluster is in good state] ******************************************
[2026-04-12T23:56:52.667Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./cluster_good.yml for an-a01n01, an-a01n02
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-12T23:56:52.667Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-12T23:56:52.667Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-12T23:56:52.667Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-12T23:56:52.667Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-12T23:56:52.667Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-12T23:56:52.667Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-12T23:56:52.667Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-12T23:56:52.667Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-12T23:56:52.667Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-12T23:56:52.667Z] ok: [an-a01n02]
[2026-04-12T23:56:52.667Z] ok: [an-a01n01]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-12T23:56:52.667Z] ok: [an-a01n01]
[2026-04-12T23:56:52.667Z] ok: [an-a01n02]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-12T23:56:52.667Z] ok: [an-a01n02]
[2026-04-12T23:56:52.667Z] ok: [an-a01n01]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-12T23:56:52.667Z] ok: [an-a01n01]
[2026-04-12T23:56:52.667Z] ok: [an-a01n02]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-12T23:56:52.667Z] ok: [an-a01n02]
[2026-04-12T23:56:52.667Z] ok: [an-a01n01]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [fence with servers iteration 5] ******************************************
[2026-04-12T23:56:52.667Z] ok: [an-a01n01]
[2026-04-12T23:56:52.667Z] ok: [an-a01n02]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [check servers status] ****************************************************
[2026-04-12T23:56:52.667Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n01, an-a01n02
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-12T23:56:52.667Z] ok: [an-a01n01]
[2026-04-12T23:56:52.667Z] ok: [an-a01n02]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [check servers are started all on the same node] **************************
[2026-04-12T23:56:52.667Z] ok: [an-a01n02]
[2026-04-12T23:56:52.667Z] ok: [an-a01n01]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [set target] **************************************************************
[2026-04-12T23:56:52.667Z] ok: [an-a01n01]
[2026-04-12T23:56:52.667Z] ok: [an-a01n02]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [will kill node with servers] *********************************************
[2026-04-12T23:56:52.667Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./kill_node.yml for an-a01n01, an-a01n02
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [kill node "an-a01n01"] ***************************************************
[2026-04-12T23:56:52.667Z] skipping: [an-a01n02]
[2026-04-12T23:56:52.667Z] changed: [an-a01n01]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-12T23:56:52.667Z] skipping: [an-a01n01]
[2026-04-12T23:56:52.667Z] skipping: [an-a01n02]
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [wait for node to be offline] *********************************************
[2026-04-12T23:56:52.667Z] skipping: [an-a01n01]
[2026-04-12T23:56:52.667Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./offline_node.yml for an-a01n02
[2026-04-12T23:56:52.667Z] 
[2026-04-12T23:56:52.667Z] TASK [wait up to 10 minutes for node an-a01n01 to be offline] ******************
[2026-04-12T23:58:02.798Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (60 retries left).
[2026-04-12T23:58:02.798Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (59 retries left).
[2026-04-12T23:58:02.798Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (58 retries left).
[2026-04-12T23:58:02.798Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (57 retries left).
[2026-04-12T23:58:02.798Z] FAILED - RETRYING: [an-a01n02]: wait up to 10 minutes for node an-a01n01 to be offline (56 retries left).
[2026-04-12T23:58:02.798Z] ok: [an-a01n02]
[2026-04-12T23:58:02.798Z] 
[2026-04-12T23:58:02.798Z] TASK [check servers status] ****************************************************
[2026-04-12T23:58:02.798Z] skipping: [an-a01n01]
[2026-04-12T23:58:02.798Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./server_started.yml for an-a01n02
[2026-04-12T23:58:02.798Z] 
[2026-04-12T23:58:02.798Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-12T23:59:01.334Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (120 retries left).
[2026-04-12T23:59:01.334Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (119 retries left).
[2026-04-12T23:59:01.334Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (118 retries left).
[2026-04-12T23:59:01.334Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (117 retries left).
[2026-04-12T23:59:01.334Z] FAILED - RETRYING: [an-a01n02]: wait up to 20 minutes for servers to be Started (116 retries left).
[2026-04-12T23:59:01.334Z] ok: [an-a01n02]
[2026-04-12T23:59:01.334Z] 
[2026-04-12T23:59:01.334Z] TASK [check servers are started all on the same node] **************************
[2026-04-12T23:59:01.334Z] ok: [an-a01n02]
[2026-04-12T23:59:01.334Z] 
[2026-04-12T23:59:01.334Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-12T23:59:01.334Z] skipping: [an-a01n01] => (item=1) 
[2026-04-12T23:59:01.334Z] skipping: [an-a01n01] => (item=2) 
[2026-04-12T23:59:01.334Z] skipping: [an-a01n01] => (item=3) 
[2026-04-12T23:59:01.334Z] skipping: [an-a01n01] => (item=4) 
[2026-04-12T23:59:01.334Z] skipping: [an-a01n01] => (item=5) 
[2026-04-12T23:59:01.334Z] skipping: [an-a01n01]
[2026-04-12T23:59:01.334Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=1)
[2026-04-12T23:59:01.334Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=2)
[2026-04-12T23:59:01.334Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=3)
[2026-04-12T23:59:01.334Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=4)
[2026-04-12T23:59:01.334Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_ssh.yml for an-a01n02 => (item=5)
[2026-04-12T23:59:01.334Z] 
[2026-04-12T23:59:01.334Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-12T23:59:14.971Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to respond to ssh (60 retries left).
[2026-04-12T23:59:14.971Z] ok: [an-a01n02]
[2026-04-12T23:59:14.971Z] 
[2026-04-12T23:59:14.971Z] TASK [Wait up to 10 minutes for an-test-deploy2 to respond to ssh] *************
[2026-04-12T23:59:15.339Z] ok: [an-a01n02]
[2026-04-12T23:59:15.339Z] 
[2026-04-12T23:59:15.339Z] TASK [Wait up to 10 minutes for an-test-deploy3 to respond to ssh] *************
[2026-04-12T23:59:16.125Z] ok: [an-a01n02]
[2026-04-12T23:59:16.125Z] 
[2026-04-12T23:59:16.125Z] TASK [Wait up to 10 minutes for an-test-deploy4 to respond to ssh] *************
[2026-04-12T23:59:16.493Z] ok: [an-a01n02]
[2026-04-12T23:59:16.493Z] 
[2026-04-12T23:59:16.493Z] TASK [Wait up to 10 minutes for an-test-deploy5 to respond to ssh] *************
[2026-04-12T23:59:17.279Z] ok: [an-a01n02]
[2026-04-12T23:59:17.279Z] 
[2026-04-12T23:59:17.279Z] TASK [wait for node to be back online] *****************************************
[2026-04-12T23:59:17.279Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_node.yml for an-a01n01, an-a01n02
[2026-04-12T23:59:17.279Z] 
[2026-04-12T23:59:17.279Z] TASK [wait for node "an-a01n01" to be back online] *****************************
[2026-04-13T00:02:42.360Z] ok: [an-a01n01 -> localhost]
[2026-04-13T00:02:42.360Z] ok: [an-a01n02 -> localhost]
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [check cluster is in good state] ******************************************
[2026-04-13T00:02:42.360Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./cluster_good.yml for an-a01n01, an-a01n02
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-13T00:02:42.360Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-13T00:02:42.360Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-13T00:02:42.360Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-13T00:02:42.360Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-13T00:02:42.360Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-13T00:02:42.360Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-13T00:02:42.360Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-13T00:02:42.360Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-13T00:02:42.360Z] included: /srv/anvil/anvil-test-suite/playbooks/tests/functional-tests/../../common/./wait_server_drbd_sync.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [Wait up to 60 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-13T00:02:42.360Z] ok: [an-a01n01]
[2026-04-13T00:02:42.360Z] ok: [an-a01n02]
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [Wait up to 60 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-13T00:02:42.360Z] ok: [an-a01n02]
[2026-04-13T00:02:42.360Z] ok: [an-a01n01]
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [Wait up to 60 minutes for an-test-deploy3 drbd storage to be synced] *****
[2026-04-13T00:02:42.360Z] ok: [an-a01n01]
[2026-04-13T00:02:42.360Z] ok: [an-a01n02]
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [Wait up to 60 minutes for an-test-deploy4 drbd storage to be synced] *****
[2026-04-13T00:02:42.360Z] ok: [an-a01n02]
[2026-04-13T00:02:42.360Z] ok: [an-a01n01]
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [Wait up to 60 minutes for an-test-deploy5 drbd storage to be synced] *****
[2026-04-13T00:02:42.360Z] ok: [an-a01n01]
[2026-04-13T00:02:42.360Z] ok: [an-a01n02]
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] PLAY [basic server tests] ******************************************************
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:02:42.360Z] ok: [an-a01n01]
[2026-04-13T00:02:42.360Z] ok: [an-a01n02]
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [check cluster is in good state] ******************************************
[2026-04-13T00:02:42.360Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-13T00:02:42.360Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-13T00:02:42.360Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-13T00:02:42.360Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-13T00:02:42.360Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-13T00:02:42.360Z] 
[2026-04-13T00:02:42.360Z] TASK [check anvil-provision-server storage group] ******************************
[2026-04-13T00:02:42.728Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_storage_group.yml for an-a01n01, an-a01n02
[2026-04-13T00:02:42.728Z] 
[2026-04-13T00:02:42.728Z] TASK [get anvil-provision-server storage group] ********************************
[2026-04-13T00:02:49.017Z] changed: [an-a01n01]
[2026-04-13T00:02:49.017Z] changed: [an-a01n02]
[2026-04-13T00:02:49.017Z] 
[2026-04-13T00:02:49.017Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-13T00:02:49.017Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_install_iso.yml for an-a01n01, an-a01n02
[2026-04-13T00:02:49.017Z] 
[2026-04-13T00:02:49.017Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-13T00:02:54.083Z] changed: [an-a01n01]
[2026-04-13T00:02:54.083Z] changed: [an-a01n02]
[2026-04-13T00:02:54.083Z] 
[2026-04-13T00:02:54.083Z] TASK [deploy server 1 from "an-a01n01"] ****************************************
[2026-04-13T00:02:54.083Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02
[2026-04-13T00:02:54.083Z] 
[2026-04-13T00:02:54.083Z] TASK [check if server an-test-deploy1 exists] **********************************
[2026-04-13T00:02:55.350Z] skipping: [an-a01n02]
[2026-04-13T00:02:55.350Z] ok: [an-a01n01]
[2026-04-13T00:02:55.350Z] 
[2026-04-13T00:02:55.350Z] TASK [deploy server an-test-deploy1 from an-a01n01 on storage 9fafa17d-152b-41c7-9537-6bb636df22f0] ***
[2026-04-13T00:02:55.350Z] skipping: [an-a01n01]
[2026-04-13T00:02:55.350Z] skipping: [an-a01n02]
[2026-04-13T00:02:55.350Z] 
[2026-04-13T00:02:55.350Z] TASK [Wait up to 10 minutes for an-test-deploy1 to be started in pacemaker] ****
[2026-04-13T00:02:55.350Z] skipping: [an-a01n01]
[2026-04-13T00:02:55.350Z] skipping: [an-a01n02]
[2026-04-13T00:02:55.350Z] 
[2026-04-13T00:02:55.350Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-13T00:02:55.718Z] skipping: [an-a01n01]
[2026-04-13T00:02:55.718Z] skipping: [an-a01n02]
[2026-04-13T00:02:55.718Z] 
[2026-04-13T00:02:55.718Z] TASK [Wait up to 10 minutes for an-test-deploy1 to start installation] *********
[2026-04-13T00:02:55.718Z] skipping: [an-a01n01]
[2026-04-13T00:02:55.718Z] skipping: [an-a01n02]
[2026-04-13T00:02:55.718Z] 
[2026-04-13T00:02:55.718Z] TASK [Wait up to 10 minutes for an-test-deploy1 to complete installation] ******
[2026-04-13T00:02:55.718Z] skipping: [an-a01n01]
[2026-04-13T00:02:55.718Z] skipping: [an-a01n02]
[2026-04-13T00:02:55.718Z] 
[2026-04-13T00:02:55.718Z] TASK [deploy server 2 from "an-a01n02"] ****************************************
[2026-04-13T00:02:55.718Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02
[2026-04-13T00:02:55.718Z] 
[2026-04-13T00:02:55.718Z] TASK [check if server an-test-deploy2 exists] **********************************
[2026-04-13T00:02:57.532Z] skipping: [an-a01n01]
[2026-04-13T00:02:57.532Z] ok: [an-a01n02]
[2026-04-13T00:02:57.532Z] 
[2026-04-13T00:02:57.532Z] TASK [deploy server an-test-deploy2 from an-a01n02 on storage 9fafa17d-152b-41c7-9537-6bb636df22f0] ***
[2026-04-13T00:02:57.532Z] skipping: [an-a01n01]
[2026-04-13T00:02:57.532Z] skipping: [an-a01n02]
[2026-04-13T00:02:57.532Z] 
[2026-04-13T00:02:57.532Z] TASK [Wait up to 10 minutes for an-test-deploy2 to be started in pacemaker] ****
[2026-04-13T00:02:57.532Z] skipping: [an-a01n01]
[2026-04-13T00:02:57.532Z] skipping: [an-a01n02]
[2026-04-13T00:02:57.532Z] 
[2026-04-13T00:02:57.532Z] TASK [Wait up to 10 minutes for an-test-deploy2 drbd storage to be synced] *****
[2026-04-13T00:02:57.532Z] skipping: [an-a01n01]
[2026-04-13T00:02:57.532Z] skipping: [an-a01n02]
[2026-04-13T00:02:57.532Z] 
[2026-04-13T00:02:57.532Z] TASK [Wait up to 10 minutes for an-test-deploy2 to start installation] *********
[2026-04-13T00:02:57.532Z] skipping: [an-a01n01]
[2026-04-13T00:02:57.532Z] skipping: [an-a01n02]
[2026-04-13T00:02:57.532Z] 
[2026-04-13T00:02:57.532Z] TASK [Wait up to 10 minutes for an-test-deploy2 to complete installation] ******
[2026-04-13T00:02:57.900Z] skipping: [an-a01n01]
[2026-04-13T00:02:57.900Z] skipping: [an-a01n02]
[2026-04-13T00:02:57.900Z] 
[2026-04-13T00:02:57.900Z] TASK [check servers status] ****************************************************
[2026-04-13T00:02:57.900Z] included: /srv/anvil/anvil-test-suite/playbooks/common/server_started.yml for an-a01n01, an-a01n02
[2026-04-13T00:02:57.900Z] 
[2026-04-13T00:02:57.900Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-13T00:02:59.717Z] ok: [an-a01n02]
[2026-04-13T00:02:59.717Z] ok: [an-a01n01]
[2026-04-13T00:02:59.717Z] 
[2026-04-13T00:02:59.717Z] TASK [check servers are started all on the same node] **************************
[2026-04-13T00:03:02.168Z] ok: [an-a01n01]
[2026-04-13T00:03:02.168Z] ok: [an-a01n02]
[2026-04-13T00:03:02.168Z] 
[2026-04-13T00:03:02.168Z] PLAY [delete extra server tests] ***********************************************
[2026-04-13T00:03:02.168Z] 
[2026-04-13T00:03:02.168Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:03:03.985Z] ok: [an-a01n01]
[2026-04-13T00:03:03.985Z] ok: [an-a01n02]
[2026-04-13T00:03:03.985Z] 
[2026-04-13T00:03:03.985Z] TASK [check cluster is in good state] ******************************************
[2026-04-13T00:03:04.496Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-13T00:03:04.496Z] 
[2026-04-13T00:03:04.496Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-13T00:03:08.550Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-13T00:03:08.550Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-13T00:03:08.550Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-13T00:03:08.550Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-13T00:03:08.550Z] 
[2026-04-13T00:03:08.550Z] TASK [delete servers] **********************************************************
[2026-04-13T00:03:08.550Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-13T00:03:08.550Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-13T00:03:08.550Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-13T00:03:08.550Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-13T00:03:08.550Z] 
[2026-04-13T00:03:08.550Z] TASK [check if server an-test-deploy2 exists] **********************************
[2026-04-13T00:03:10.365Z] skipping: [an-a01n01]
[2026-04-13T00:03:10.365Z] ok: [an-a01n02]
[2026-04-13T00:03:10.365Z] 
[2026-04-13T00:03:10.365Z] TASK [delete server an-test-deploy2 from an-a01n02] ****************************
[2026-04-13T00:03:18.025Z] skipping: [an-a01n01]
[2026-04-13T00:03:18.025Z] changed: [an-a01n02]
[2026-04-13T00:03:18.025Z] 
[2026-04-13T00:03:18.025Z] TASK [Wait up to 10 minutes for an-test-deploy2 to be delete in pacemaker] *****
[2026-04-13T00:03:46.579Z] skipping: [an-a01n01]
[2026-04-13T00:03:46.579Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy2 to be delete in pacemaker (60 retries left).
[2026-04-13T00:03:46.579Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy2 to be delete in pacemaker (59 retries left).
[2026-04-13T00:03:46.579Z] ok: [an-a01n02]
[2026-04-13T00:03:46.579Z] 
[2026-04-13T00:03:46.579Z] TASK [Wait up to 10 minutes for an-test-deploy2 drbd storage to be removed] ****
[2026-04-13T00:03:46.579Z] skipping: [an-a01n01]
[2026-04-13T00:03:46.579Z] ok: [an-a01n02]
[2026-04-13T00:03:46.579Z] 
[2026-04-13T00:03:46.579Z] TASK [Wait up to 10 minutes for an-test-deploy2 lvm volume to be removed] ******
[2026-04-13T00:03:46.579Z] skipping: [an-a01n01]
[2026-04-13T00:03:46.579Z] ok: [an-a01n02]
[2026-04-13T00:03:46.579Z] 
[2026-04-13T00:03:46.579Z] TASK [check if server an-test-deploy3 exists] **********************************
[2026-04-13T00:03:46.579Z] skipping: [an-a01n01]
[2026-04-13T00:03:46.579Z] ok: [an-a01n02]
[2026-04-13T00:03:46.579Z] 
[2026-04-13T00:03:46.579Z] TASK [delete server an-test-deploy3 from an-a01n02] ****************************
[2026-04-13T00:03:54.240Z] skipping: [an-a01n01]
[2026-04-13T00:03:54.240Z] changed: [an-a01n02]
[2026-04-13T00:03:54.240Z] 
[2026-04-13T00:03:54.240Z] TASK [Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker] *****
[2026-04-13T00:04:52.766Z] skipping: [an-a01n01]
[2026-04-13T00:04:52.766Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker (60 retries left).
[2026-04-13T00:04:52.766Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker (59 retries left).
[2026-04-13T00:04:52.766Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker (58 retries left).
[2026-04-13T00:04:52.766Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker (57 retries left).
[2026-04-13T00:04:52.766Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker (56 retries left).
[2026-04-13T00:04:52.766Z] ok: [an-a01n02]
[2026-04-13T00:04:52.766Z] 
[2026-04-13T00:04:52.766Z] TASK [Wait up to 10 minutes for an-test-deploy3 drbd storage to be removed] ****
[2026-04-13T00:04:53.694Z] skipping: [an-a01n01]
[2026-04-13T00:04:53.694Z] ok: [an-a01n02]
[2026-04-13T00:04:53.694Z] 
[2026-04-13T00:04:53.694Z] TASK [Wait up to 10 minutes for an-test-deploy3 lvm volume to be removed] ******
[2026-04-13T00:04:54.481Z] skipping: [an-a01n01]
[2026-04-13T00:04:54.481Z] ok: [an-a01n02]
[2026-04-13T00:04:54.481Z] 
[2026-04-13T00:04:54.481Z] TASK [check if server an-test-deploy4 exists] **********************************
[2026-04-13T00:04:56.297Z] skipping: [an-a01n01]
[2026-04-13T00:04:56.298Z] ok: [an-a01n02]
[2026-04-13T00:04:56.298Z] 
[2026-04-13T00:04:56.298Z] TASK [delete server an-test-deploy4 from an-a01n02] ****************************
[2026-04-13T00:05:03.957Z] skipping: [an-a01n01]
[2026-04-13T00:05:03.957Z] changed: [an-a01n02]
[2026-04-13T00:05:03.957Z] 
[2026-04-13T00:05:03.957Z] TASK [Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker] *****
[2026-04-13T00:06:28.045Z] skipping: [an-a01n01]
[2026-04-13T00:06:28.045Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker (60 retries left).
[2026-04-13T00:06:28.045Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker (59 retries left).
[2026-04-13T00:06:28.045Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker (58 retries left).
[2026-04-13T00:06:28.045Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker (57 retries left).
[2026-04-13T00:06:28.045Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker (56 retries left).
[2026-04-13T00:06:28.045Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker (55 retries left).
[2026-04-13T00:06:28.045Z] ok: [an-a01n02]
[2026-04-13T00:06:28.045Z] 
[2026-04-13T00:06:28.045Z] TASK [Wait up to 10 minutes for an-test-deploy4 drbd storage to be removed] ****
[2026-04-13T00:06:28.045Z] skipping: [an-a01n01]
[2026-04-13T00:06:28.045Z] ok: [an-a01n02]
[2026-04-13T00:06:28.045Z] 
[2026-04-13T00:06:28.045Z] TASK [Wait up to 10 minutes for an-test-deploy4 lvm volume to be removed] ******
[2026-04-13T00:06:28.045Z] skipping: [an-a01n01]
[2026-04-13T00:06:28.045Z] ok: [an-a01n02]
[2026-04-13T00:06:28.045Z] 
[2026-04-13T00:06:28.045Z] TASK [check if server an-test-deploy5 exists] **********************************
[2026-04-13T00:06:28.045Z] skipping: [an-a01n01]
[2026-04-13T00:06:28.045Z] ok: [an-a01n02]
[2026-04-13T00:06:28.045Z] 
[2026-04-13T00:06:28.045Z] TASK [delete server an-test-deploy5 from an-a01n02] ****************************
[2026-04-13T00:06:28.045Z] skipping: [an-a01n01]
[2026-04-13T00:06:28.045Z] changed: [an-a01n02]
[2026-04-13T00:06:28.045Z] 
[2026-04-13T00:06:28.045Z] TASK [Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker] *****
[2026-04-13T00:07:38.142Z] skipping: [an-a01n01]
[2026-04-13T00:07:38.142Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker (60 retries left).
[2026-04-13T00:07:38.142Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker (59 retries left).
[2026-04-13T00:07:38.142Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker (58 retries left).
[2026-04-13T00:07:38.142Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker (57 retries left).
[2026-04-13T00:07:38.142Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker (56 retries left).
[2026-04-13T00:07:38.142Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker (55 retries left).
[2026-04-13T00:07:38.142Z] ok: [an-a01n02]
[2026-04-13T00:07:38.142Z] 
[2026-04-13T00:07:38.142Z] TASK [Wait up to 10 minutes for an-test-deploy5 drbd storage to be removed] ****
[2026-04-13T00:07:38.142Z] skipping: [an-a01n01]
[2026-04-13T00:07:38.142Z] ok: [an-a01n02]
[2026-04-13T00:07:38.142Z] 
[2026-04-13T00:07:38.142Z] TASK [Wait up to 10 minutes for an-test-deploy5 lvm volume to be removed] ******
[2026-04-13T00:07:38.957Z] skipping: [an-a01n01]
[2026-04-13T00:07:38.957Z] ok: [an-a01n02]
[2026-04-13T00:07:38.957Z] 
[2026-04-13T00:07:38.957Z] PLAY [create server for DR testing] ********************************************
[2026-04-13T00:07:38.957Z] 
[2026-04-13T00:07:38.957Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:07:41.409Z] ok: [an-a01n02]
[2026-04-13T00:07:41.409Z] ok: [an-a01n01]
[2026-04-13T00:07:41.409Z] 
[2026-04-13T00:07:41.409Z] TASK [check cluster is in good state] ******************************************
[2026-04-13T00:07:41.409Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-13T00:07:41.409Z] 
[2026-04-13T00:07:41.409Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-13T00:07:45.578Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-13T00:07:45.578Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-13T00:07:45.578Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-13T00:07:45.578Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-13T00:07:45.578Z] 
[2026-04-13T00:07:45.578Z] TASK [check anvil-provision-server storage group] ******************************
[2026-04-13T00:07:45.578Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_storage_group.yml for an-a01n01, an-a01n02
[2026-04-13T00:07:45.578Z] 
[2026-04-13T00:07:45.578Z] TASK [get anvil-provision-server storage group] ********************************
[2026-04-13T00:07:51.831Z] changed: [an-a01n01]
[2026-04-13T00:07:51.831Z] changed: [an-a01n02]
[2026-04-13T00:07:51.831Z] 
[2026-04-13T00:07:51.831Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-13T00:07:51.831Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_install_iso.yml for an-a01n01, an-a01n02
[2026-04-13T00:07:51.831Z] 
[2026-04-13T00:07:51.831Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-13T00:07:56.896Z] changed: [an-a01n02]
[2026-04-13T00:07:56.896Z] changed: [an-a01n01]
[2026-04-13T00:07:56.896Z] 
[2026-04-13T00:07:56.896Z] TASK [deploy server 1 from "an-a01n01"] ****************************************
[2026-04-13T00:07:56.896Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02
[2026-04-13T00:07:56.896Z] 
[2026-04-13T00:07:56.896Z] TASK [check if server an-test-deploy1 exists] **********************************
[2026-04-13T00:07:58.711Z] skipping: [an-a01n02]
[2026-04-13T00:07:58.711Z] ok: [an-a01n01]
[2026-04-13T00:07:58.711Z] 
[2026-04-13T00:07:58.711Z] TASK [deploy server an-test-deploy1 from an-a01n01 on storage 9fafa17d-152b-41c7-9537-6bb636df22f0] ***
[2026-04-13T00:07:58.711Z] skipping: [an-a01n01]
[2026-04-13T00:07:58.711Z] skipping: [an-a01n02]
[2026-04-13T00:07:58.711Z] 
[2026-04-13T00:07:58.711Z] TASK [Wait up to 10 minutes for an-test-deploy1 to be started in pacemaker] ****
[2026-04-13T00:07:58.711Z] skipping: [an-a01n01]
[2026-04-13T00:07:58.711Z] skipping: [an-a01n02]
[2026-04-13T00:07:58.711Z] 
[2026-04-13T00:07:58.711Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-13T00:07:58.711Z] skipping: [an-a01n01]
[2026-04-13T00:07:58.711Z] skipping: [an-a01n02]
[2026-04-13T00:07:58.711Z] 
[2026-04-13T00:07:58.711Z] TASK [Wait up to 10 minutes for an-test-deploy1 to start installation] *********
[2026-04-13T00:07:58.711Z] skipping: [an-a01n01]
[2026-04-13T00:07:58.711Z] skipping: [an-a01n02]
[2026-04-13T00:07:58.711Z] 
[2026-04-13T00:07:58.711Z] TASK [Wait up to 10 minutes for an-test-deploy1 to complete installation] ******
[2026-04-13T00:07:58.711Z] skipping: [an-a01n01]
[2026-04-13T00:07:58.711Z] skipping: [an-a01n02]
[2026-04-13T00:07:58.711Z] 
[2026-04-13T00:07:58.711Z] TASK [check servers status] ****************************************************
[2026-04-13T00:07:58.711Z] included: /srv/anvil/anvil-test-suite/playbooks/common/server_started.yml for an-a01n01, an-a01n02
[2026-04-13T00:07:58.711Z] 
[2026-04-13T00:07:58.711Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-13T00:08:01.162Z] ok: [an-a01n01]
[2026-04-13T00:08:01.162Z] ok: [an-a01n02]
[2026-04-13T00:08:01.162Z] 
[2026-04-13T00:08:01.162Z] TASK [check servers are started all on the same node] **************************
[2026-04-13T00:08:03.094Z] ok: [an-a01n01]
[2026-04-13T00:08:03.094Z] ok: [an-a01n02]
[2026-04-13T00:08:03.094Z] 
[2026-04-13T00:08:03.094Z] PLAY [DR tests] ****************************************************************
[2026-04-13T00:08:03.094Z] 
[2026-04-13T00:08:03.094Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:08:05.545Z] ok: [an-a01dr01]
[2026-04-13T00:08:05.545Z] 
[2026-04-13T00:08:05.545Z] TASK [check DR is correctly deployed] ******************************************
[2026-04-13T00:08:08.735Z] changed: [an-a01dr01]
[2026-04-13T00:08:08.735Z] 
[2026-04-13T00:08:08.735Z] TASK [check DR host is registered in the DB] ***********************************
[2026-04-13T00:08:13.801Z] changed: [an-a01dr01]
[2026-04-13T00:08:13.801Z] 
[2026-04-13T00:08:13.801Z] TASK [check no DR hosts are linked] ********************************************
[2026-04-13T00:08:17.854Z] changed: [an-a01dr01]
[2026-04-13T00:08:17.854Z] 
[2026-04-13T00:08:17.854Z] TASK [link DR host to anvil] ***************************************************
[2026-04-13T00:08:22.917Z] changed: [an-a01dr01]
[2026-04-13T00:08:22.917Z] 
[2026-04-13T00:08:22.917Z] TASK [verify DR host is linked to anvil] ***************************************
[2026-04-13T00:08:27.087Z] changed: [an-a01dr01]
[2026-04-13T00:08:27.087Z] 
[2026-04-13T00:08:27.087Z] TASK [link DR vg to storage group] *********************************************
[2026-04-13T00:08:33.342Z] changed: [an-a01dr01]
[2026-04-13T00:08:33.342Z] 
[2026-04-13T00:08:33.342Z] TASK [verify DR vg is linked to storage group] *********************************
[2026-04-13T00:08:36.532Z] changed: [an-a01dr01]
[2026-04-13T00:08:36.532Z] 
[2026-04-13T00:08:36.532Z] TASK [protect an-test-deploy1 short-throw] *************************************
[2026-04-13T00:08:42.787Z] changed: [an-a01dr01]
[2026-04-13T00:08:42.787Z] 
[2026-04-13T00:08:42.787Z] TASK [verify an-test-deploy1 DR drbd status short-throw] ***********************
[2026-04-13T00:08:42.787Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_dr_sync.yml for an-a01dr01
[2026-04-13T00:08:42.787Z] 
[2026-04-13T00:08:42.787Z] TASK [Wait up to 20 minutes for an-test-deploy1 drbd storage resource to be created] ***
[2026-04-13T00:09:41.432Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage resource to be created (120 retries left).
[2026-04-13T00:09:41.432Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage resource to be created (119 retries left).
[2026-04-13T00:09:41.432Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage resource to be created (118 retries left).
[2026-04-13T00:09:41.432Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage resource to be created (117 retries left).
[2026-04-13T00:09:41.432Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage resource to be created (116 retries left).
[2026-04-13T00:09:41.432Z] ok: [an-a01dr01]
[2026-04-13T00:09:41.432Z] 
[2026-04-13T00:09:41.432Z] TASK [Check correct drbd protocol is configured] *******************************
[2026-04-13T00:09:41.432Z] changed: [an-a01dr01]
[2026-04-13T00:09:41.432Z] 
[2026-04-13T00:09:41.432Z] TASK [Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-13T00:11:35.743Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (120 retries left).
[2026-04-13T00:11:35.743Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (119 retries left).
[2026-04-13T00:11:35.743Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (118 retries left).
[2026-04-13T00:11:35.743Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (117 retries left).
[2026-04-13T00:11:35.743Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (116 retries left).
[2026-04-13T00:11:35.743Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (115 retries left).
[2026-04-13T00:11:35.743Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (114 retries left).
[2026-04-13T00:11:35.743Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (113 retries left).
[2026-04-13T00:11:35.743Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (112 retries left).
[2026-04-13T00:11:35.743Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (111 retries left).
[2026-04-13T00:11:35.743Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (110 retries left).
[2026-04-13T00:11:35.743Z] ok: [an-a01dr01]
[2026-04-13T00:11:35.743Z] 
[2026-04-13T00:11:35.743Z] TASK [verify disconnect/update/connect features short-throw] *******************
[2026-04-13T00:11:35.743Z] included: /srv/anvil/anvil-test-suite/playbooks/common/check_server_drbd_dr_management.yml for an-a01dr01
[2026-04-13T00:11:35.743Z] 
[2026-04-13T00:11:35.743Z] TASK [Verify an-test-deploy1 drbd storage is synced] ***************************
[2026-04-13T00:11:35.743Z] ok: [an-a01dr01]
[2026-04-13T00:11:35.743Z] 
[2026-04-13T00:11:35.743Z] TASK [Disconnect an-test-deploy1 drbd storage] *********************************
[2026-04-13T00:11:39.796Z] changed: [an-a01dr01]
[2026-04-13T00:11:39.796Z] 
[2026-04-13T00:11:39.796Z] TASK [Wait up to 20 minutes for an-test-deploy1 drbd storage to be disconnected] ***
[2026-04-13T00:12:03.550Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be disconnected (120 retries left).
[2026-04-13T00:12:03.550Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be disconnected (119 retries left).
[2026-04-13T00:12:03.550Z] ok: [an-a01dr01]
[2026-04-13T00:12:03.550Z] 
[2026-04-13T00:12:03.550Z] TASK [Inject random data to an-test-deploy1] ***********************************
[2026-04-13T00:14:13.768Z] changed: [an-a01dr01 -> localhost]
[2026-04-13T00:14:13.768Z] 
[2026-04-13T00:14:13.768Z] TASK [Update an-test-deploy1 drbd storage] *************************************
[2026-04-13T00:14:13.768Z] changed: [an-a01dr01]
[2026-04-13T00:14:13.768Z] 
[2026-04-13T00:14:13.768Z] TASK [Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected] ***
[2026-04-13T00:14:37.910Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected (120 retries left).
[2026-04-13T00:14:37.910Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected (119 retries left).
[2026-04-13T00:14:37.910Z] ok: [an-a01dr01]
[2026-04-13T00:14:37.910Z] 
[2026-04-13T00:14:37.910Z] TASK [Wait up to 20 minutes for an-test-deploy1 drbd storage to be disconnected] ***
[2026-04-13T00:14:47.261Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be disconnected (120 retries left).
[2026-04-13T00:14:47.261Z] ok: [an-a01dr01]
[2026-04-13T00:14:47.261Z] 
[2026-04-13T00:14:47.261Z] TASK [Reconnect an-test-deploy1 drbd storage] **********************************
[2026-04-13T00:14:52.326Z] changed: [an-a01dr01]
[2026-04-13T00:14:52.326Z] 
[2026-04-13T00:14:52.326Z] TASK [Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected] ***
[2026-04-13T00:15:16.011Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected (120 retries left).
[2026-04-13T00:15:16.011Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected (119 retries left).
[2026-04-13T00:15:16.011Z] ok: [an-a01dr01]
[2026-04-13T00:15:16.011Z] 
[2026-04-13T00:15:16.011Z] TASK [remove an-test-deploy1 protection short-throw] ***************************
[2026-04-13T00:15:19.199Z] changed: [an-a01dr01]
[2026-04-13T00:15:19.199Z] 
[2026-04-13T00:15:19.199Z] TASK [verify an-test-deploy1 DR drbd clean status short-throw] *****************
[2026-04-13T00:15:19.199Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_dr_clean.yml for an-a01dr01
[2026-04-13T00:15:19.199Z] 
[2026-04-13T00:15:19.199Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be removed] ****
[2026-04-13T00:15:53.318Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage to be removed (60 retries left).
[2026-04-13T00:15:53.318Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage to be removed (59 retries left).
[2026-04-13T00:15:53.318Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage to be removed (58 retries left).
[2026-04-13T00:15:53.318Z] ok: [an-a01dr01]
[2026-04-13T00:15:53.318Z] 
[2026-04-13T00:15:53.318Z] TASK [Wait up to 10 minutes for an-test-deploy1 lvm volume to be removed] ******
[2026-04-13T00:15:53.318Z] ok: [an-a01dr01]
[2026-04-13T00:15:53.318Z] 
[2026-04-13T00:15:53.318Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage resource to be removed] ***
[2026-04-13T00:16:27.378Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage resource to be removed (60 retries left).
[2026-04-13T00:16:27.378Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage resource to be removed (59 retries left).
[2026-04-13T00:16:27.378Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage resource to be removed (58 retries left).
[2026-04-13T00:16:27.378Z] ok: [an-a01dr01]
[2026-04-13T00:16:27.379Z] 
[2026-04-13T00:16:27.379Z] TASK [check drbd sync status on nodes after disconnect short-throw] ************
[2026-04-13T00:16:27.379Z] changed: [an-a01dr01 -> an-a01n01]
[2026-04-13T00:16:27.379Z] 
[2026-04-13T00:16:27.379Z] TASK [protect an-test-deploy1 sync] ********************************************
[2026-04-13T00:16:30.569Z] changed: [an-a01dr01]
[2026-04-13T00:16:30.569Z] 
[2026-04-13T00:16:30.569Z] TASK [verify an-test-deploy1 DR drbd status sync] ******************************
[2026-04-13T00:16:30.569Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_dr_sync.yml for an-a01dr01
[2026-04-13T00:16:30.569Z] 
[2026-04-13T00:16:30.569Z] TASK [Wait up to 20 minutes for an-test-deploy1 drbd storage resource to be created] ***
[2026-04-13T00:17:04.740Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage resource to be created (120 retries left).
[2026-04-13T00:17:04.740Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage resource to be created (119 retries left).
[2026-04-13T00:17:04.740Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage resource to be created (118 retries left).
[2026-04-13T00:17:04.740Z] ok: [an-a01dr01]
[2026-04-13T00:17:04.740Z] 
[2026-04-13T00:17:04.740Z] TASK [Check correct drbd protocol is configured] *******************************
[2026-04-13T00:17:04.740Z] changed: [an-a01dr01]
[2026-04-13T00:17:04.740Z] 
[2026-04-13T00:17:04.740Z] TASK [Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (120 retries left).
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (119 retries left).
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (118 retries left).
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (117 retries left).
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (116 retries left).
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (115 retries left).
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (114 retries left).
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (113 retries left).
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (112 retries left).
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (111 retries left).
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (110 retries left).
[2026-04-13T00:19:15.509Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be synced (109 retries left).
[2026-04-13T00:19:15.509Z] ok: [an-a01dr01]
[2026-04-13T00:19:15.509Z] 
[2026-04-13T00:19:15.509Z] TASK [verify disconnect/update/connect features sync] **************************
[2026-04-13T00:19:15.509Z] included: /srv/anvil/anvil-test-suite/playbooks/common/check_server_drbd_dr_management.yml for an-a01dr01
[2026-04-13T00:19:15.509Z] 
[2026-04-13T00:19:15.509Z] TASK [Verify an-test-deploy1 drbd storage is synced] ***************************
[2026-04-13T00:19:15.509Z] ok: [an-a01dr01]
[2026-04-13T00:19:15.509Z] 
[2026-04-13T00:19:15.509Z] TASK [Disconnect an-test-deploy1 drbd storage] *********************************
[2026-04-13T00:19:17.320Z] changed: [an-a01dr01]
[2026-04-13T00:19:17.320Z] 
[2026-04-13T00:19:17.320Z] TASK [Wait up to 20 minutes for an-test-deploy1 drbd storage to be disconnected] ***
[2026-04-13T00:19:41.006Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be disconnected (120 retries left).
[2026-04-13T00:19:41.006Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be disconnected (119 retries left).
[2026-04-13T00:19:41.006Z] ok: [an-a01dr01]
[2026-04-13T00:19:41.006Z] 
[2026-04-13T00:19:41.006Z] TASK [Inject random data to an-test-deploy1] ***********************************
[2026-04-13T00:22:05.808Z] changed: [an-a01dr01 -> localhost]
[2026-04-13T00:22:05.808Z] 
[2026-04-13T00:22:05.808Z] TASK [Update an-test-deploy1 drbd storage] *************************************
[2026-04-13T00:22:05.808Z] changed: [an-a01dr01]
[2026-04-13T00:22:05.808Z] 
[2026-04-13T00:22:05.808Z] TASK [Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected] ***
[2026-04-13T00:22:19.461Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected (120 retries left).
[2026-04-13T00:22:19.461Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected (119 retries left).
[2026-04-13T00:22:19.461Z] ok: [an-a01dr01]
[2026-04-13T00:22:19.461Z] 
[2026-04-13T00:22:19.461Z] TASK [Wait up to 20 minutes for an-test-deploy1 drbd storage to be disconnected] ***
[2026-04-13T00:22:43.158Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be disconnected (120 retries left).
[2026-04-13T00:22:43.158Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 minutes for an-test-deploy1 drbd storage to be disconnected (119 retries left).
[2026-04-13T00:22:43.158Z] ok: [an-a01dr01]
[2026-04-13T00:22:43.158Z] 
[2026-04-13T00:22:43.158Z] TASK [Reconnect an-test-deploy1 drbd storage] **********************************
[2026-04-13T00:22:45.641Z] changed: [an-a01dr01]
[2026-04-13T00:22:45.641Z] 
[2026-04-13T00:22:45.641Z] TASK [Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected] ***
[2026-04-13T00:23:20.019Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected (120 retries left).
[2026-04-13T00:23:20.019Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected (119 retries left).
[2026-04-13T00:23:20.019Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 20 timeout for an-test-deploy1 drbd storage to be (re)connected (118 retries left).
[2026-04-13T00:23:20.019Z] ok: [an-a01dr01]
[2026-04-13T00:23:20.019Z] 
[2026-04-13T00:23:20.019Z] TASK [remove an-test-deploy1 protection sync] **********************************
[2026-04-13T00:23:23.214Z] changed: [an-a01dr01]
[2026-04-13T00:23:23.214Z] 
[2026-04-13T00:23:23.214Z] TASK [verify an-test-deploy1 DR drbd clean status sync] ************************
[2026-04-13T00:23:23.214Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_dr_clean.yml for an-a01dr01
[2026-04-13T00:23:23.214Z] 
[2026-04-13T00:23:23.214Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be removed] ****
[2026-04-13T00:23:57.266Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage to be removed (60 retries left).
[2026-04-13T00:23:57.266Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage to be removed (59 retries left).
[2026-04-13T00:23:57.266Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage to be removed (58 retries left).
[2026-04-13T00:23:57.266Z] ok: [an-a01dr01]
[2026-04-13T00:23:57.266Z] 
[2026-04-13T00:23:57.266Z] TASK [Wait up to 10 minutes for an-test-deploy1 lvm volume to be removed] ******
[2026-04-13T00:23:57.266Z] ok: [an-a01dr01]
[2026-04-13T00:23:57.266Z] 
[2026-04-13T00:23:57.266Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage resource to be removed] ***
[2026-04-13T00:24:31.538Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage resource to be removed (60 retries left).
[2026-04-13T00:24:31.538Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage resource to be removed (59 retries left).
[2026-04-13T00:24:31.538Z] FAILED - RETRYING: [an-a01dr01]: Wait up to 10 minutes for an-test-deploy1 drbd storage resource to be removed (58 retries left).
[2026-04-13T00:24:31.538Z] ok: [an-a01dr01]
[2026-04-13T00:24:31.538Z] 
[2026-04-13T00:24:31.538Z] TASK [check drbd sync status on nodes after disconnect sync] *******************
[2026-04-13T00:24:31.538Z] changed: [an-a01dr01 -> an-a01n01]
[2026-04-13T00:24:31.538Z] 
[2026-04-13T00:24:31.538Z] TASK [unlink DR vg from storage group] *****************************************
[2026-04-13T00:24:35.590Z] changed: [an-a01dr01]
[2026-04-13T00:24:35.591Z] 
[2026-04-13T00:24:35.591Z] TASK [verify DR vg is unlinked from storage group] *****************************
[2026-04-13T00:24:39.644Z] changed: [an-a01dr01]
[2026-04-13T00:24:39.644Z] 
[2026-04-13T00:24:39.644Z] TASK [unlink DR host from anvil] ***********************************************
[2026-04-13T00:24:44.707Z] changed: [an-a01dr01]
[2026-04-13T00:24:44.707Z] 
[2026-04-13T00:24:44.707Z] TASK [check no DR hosts are linked] ********************************************
[2026-04-13T00:24:48.956Z] changed: [an-a01dr01]
[2026-04-13T00:24:48.956Z] 
[2026-04-13T00:24:48.956Z] PLAY [upgrade server test] *****************************************************
[2026-04-13T00:24:48.956Z] 
[2026-04-13T00:24:48.956Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:24:50.770Z] ok: [an-a01n01]
[2026-04-13T00:24:50.770Z] ok: [an-a01n02]
[2026-04-13T00:24:50.770Z] 
[2026-04-13T00:24:50.770Z] TASK [check cluster is in good state] ******************************************
[2026-04-13T00:24:50.770Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-13T00:24:50.770Z] 
[2026-04-13T00:24:50.770Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-13T00:24:54.822Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-13T00:24:54.822Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-13T00:24:54.822Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-13T00:24:54.822Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-13T00:24:54.822Z] 
[2026-04-13T00:24:54.822Z] TASK [check anvil-provision-server storage group] ******************************
[2026-04-13T00:24:54.822Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_storage_group.yml for an-a01n01, an-a01n02
[2026-04-13T00:24:54.822Z] 
[2026-04-13T00:24:54.822Z] TASK [get anvil-provision-server storage group] ********************************
[2026-04-13T00:24:59.887Z] changed: [an-a01n01]
[2026-04-13T00:24:59.887Z] changed: [an-a01n02]
[2026-04-13T00:24:59.887Z] 
[2026-04-13T00:24:59.887Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-13T00:24:59.887Z] included: /srv/anvil/anvil-test-suite/playbooks/common/get_install_iso.yml for an-a01n01, an-a01n02
[2026-04-13T00:24:59.887Z] 
[2026-04-13T00:24:59.887Z] TASK [check anvil-provision-server install isos] *******************************
[2026-04-13T00:25:06.143Z] changed: [an-a01n01]
[2026-04-13T00:25:06.143Z] changed: [an-a01n02]
[2026-04-13T00:25:06.143Z] 
[2026-04-13T00:25:06.143Z] TASK [deploy server 1 from "an-a01n01"] ****************************************
[2026-04-13T00:25:06.143Z] included: /srv/anvil/anvil-test-suite/playbooks/common/deploy_server.yml for an-a01n01, an-a01n02
[2026-04-13T00:25:06.143Z] 
[2026-04-13T00:25:06.143Z] TASK [check if server an-test-deploy1 exists] **********************************
[2026-04-13T00:25:06.929Z] skipping: [an-a01n02]
[2026-04-13T00:25:06.929Z] ok: [an-a01n01]
[2026-04-13T00:25:06.929Z] 
[2026-04-13T00:25:06.929Z] TASK [deploy server an-test-deploy1 from an-a01n01 on storage 9fafa17d-152b-41c7-9537-6bb636df22f0] ***
[2026-04-13T00:25:06.929Z] skipping: [an-a01n01]
[2026-04-13T00:25:06.929Z] skipping: [an-a01n02]
[2026-04-13T00:25:06.929Z] 
[2026-04-13T00:25:06.929Z] TASK [Wait up to 10 minutes for an-test-deploy1 to be started in pacemaker] ****
[2026-04-13T00:25:06.929Z] skipping: [an-a01n01]
[2026-04-13T00:25:06.929Z] skipping: [an-a01n02]
[2026-04-13T00:25:06.929Z] 
[2026-04-13T00:25:06.929Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-13T00:25:06.929Z] skipping: [an-a01n01]
[2026-04-13T00:25:06.929Z] skipping: [an-a01n02]
[2026-04-13T00:25:06.929Z] 
[2026-04-13T00:25:06.929Z] TASK [Wait up to 10 minutes for an-test-deploy1 to start installation] *********
[2026-04-13T00:25:06.929Z] skipping: [an-a01n01]
[2026-04-13T00:25:06.929Z] skipping: [an-a01n02]
[2026-04-13T00:25:06.929Z] 
[2026-04-13T00:25:06.929Z] TASK [Wait up to 10 minutes for an-test-deploy1 to complete installation] ******
[2026-04-13T00:25:07.297Z] skipping: [an-a01n01]
[2026-04-13T00:25:07.297Z] skipping: [an-a01n02]
[2026-04-13T00:25:07.297Z] 
[2026-04-13T00:25:07.297Z] TASK [check servers status] ****************************************************
[2026-04-13T00:25:07.297Z] included: /srv/anvil/anvil-test-suite/playbooks/common/server_started.yml for an-a01n01, an-a01n02
[2026-04-13T00:25:07.297Z] 
[2026-04-13T00:25:07.297Z] TASK [wait up to 20 minutes for servers to be Started] *************************
[2026-04-13T00:25:09.749Z] ok: [an-a01n01]
[2026-04-13T00:25:09.749Z] ok: [an-a01n02]
[2026-04-13T00:25:09.749Z] 
[2026-04-13T00:25:09.749Z] TASK [check servers are started all on the same node] **************************
[2026-04-13T00:25:11.564Z] ok: [an-a01n02]
[2026-04-13T00:25:11.564Z] ok: [an-a01n01]
[2026-04-13T00:25:11.564Z] 
[2026-04-13T00:25:11.564Z] PLAY [enable upgrade repo] *****************************************************
[2026-04-13T00:25:11.564Z] 
[2026-04-13T00:25:11.564Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:25:14.751Z] ok: [an-a01dr01]
[2026-04-13T00:25:14.751Z] ok: [an-a01n01]
[2026-04-13T00:25:14.751Z] ok: [an-a01n02]
[2026-04-13T00:25:14.751Z] ok: [an-striker02]
[2026-04-13T00:25:14.751Z] ok: [an-striker01]
[2026-04-13T00:25:14.751Z] 
[2026-04-13T00:25:14.751Z] TASK [enable anvil test-update repo] *******************************************
[2026-04-13T00:25:28.377Z] changed: [an-a01n01]
[2026-04-13T00:25:28.377Z] changed: [an-a01dr01]
[2026-04-13T00:25:28.377Z] changed: [an-a01n02]
[2026-04-13T00:25:28.377Z] changed: [an-striker01]
[2026-04-13T00:25:28.377Z] changed: [an-striker02]
[2026-04-13T00:25:28.377Z] 
[2026-04-13T00:25:28.377Z] TASK [store current uptime] ****************************************************
[2026-04-13T00:25:28.377Z] ok: [an-striker01]
[2026-04-13T00:25:28.377Z] ok: [an-striker02]
[2026-04-13T00:25:28.377Z] ok: [an-a01n01]
[2026-04-13T00:25:28.377Z] ok: [an-a01n02]
[2026-04-13T00:25:28.377Z] ok: [an-a01dr01]
[2026-04-13T00:25:28.377Z] 
[2026-04-13T00:25:28.377Z] PLAY [start upgrade process] ***************************************************
[2026-04-13T00:25:28.377Z] 
[2026-04-13T00:25:28.377Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:25:28.745Z] ok: [an-striker01]
[2026-04-13T00:25:28.745Z] 
[2026-04-13T00:25:28.745Z] TASK [upgrade and reboot all nodes but an-striker01] ***************************
[2026-04-13T00:54:06.219Z] changed: [an-striker01]
[2026-04-13T00:54:06.219Z] 
[2026-04-13T00:54:06.219Z] TASK [reboot striker01] ********************************************************
[2026-04-13T00:54:29.907Z] changed: [an-striker01]
[2026-04-13T00:54:29.907Z] 
[2026-04-13T00:54:29.907Z] PLAY [reconnect all nodes] *****************************************************
[2026-04-13T00:54:29.907Z] 
[2026-04-13T00:54:29.907Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:54:33.097Z] ok: [an-striker02]
[2026-04-13T00:54:33.097Z] ok: [an-a01dr01]
[2026-04-13T00:54:33.097Z] ok: [an-a01n02]
[2026-04-13T00:54:33.097Z] ok: [an-a01n01]
[2026-04-13T00:54:33.097Z] ok: [an-striker01]
[2026-04-13T00:54:33.097Z] 
[2026-04-13T00:54:33.097Z] TASK [wait for node "an-striker01" to be back online] **************************
[2026-04-13T00:54:33.465Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_node.yml for an-striker01, an-striker02, an-a01n01, an-a01n02, an-a01dr01
[2026-04-13T00:54:33.465Z] 
[2026-04-13T00:54:33.465Z] TASK [wait for node "all" to be back online] ***********************************
[2026-04-13T00:57:58.363Z] ok: [an-striker01 -> localhost]
[2026-04-13T00:57:58.363Z] ok: [an-striker02 -> localhost]
[2026-04-13T00:57:58.363Z] ok: [an-a01dr01 -> localhost]
[2026-04-13T00:57:58.363Z] ok: [an-a01n01 -> localhost]
[2026-04-13T00:57:58.363Z] ok: [an-a01n02 -> localhost]
[2026-04-13T00:57:58.363Z] 
[2026-04-13T00:57:58.363Z] TASK [check nodes uptime after reboot] *****************************************
[2026-04-13T00:57:58.363Z] changed: [an-a01n02]
[2026-04-13T00:57:58.363Z] changed: [an-a01n01]
[2026-04-13T00:57:58.363Z] changed: [an-striker02]
[2026-04-13T00:57:58.363Z] changed: [an-a01dr01]
[2026-04-13T00:57:58.363Z] changed: [an-striker01]
[2026-04-13T00:57:58.363Z] 
[2026-04-13T00:57:58.363Z] PLAY [check all nodes status after upgrade] ************************************
[2026-04-13T00:57:58.363Z] 
[2026-04-13T00:57:58.363Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:57:58.363Z] ok: [an-a01n02]
[2026-04-13T00:57:58.363Z] ok: [an-a01n01]
[2026-04-13T00:57:58.363Z] ok: [an-a01dr01]
[2026-04-13T00:57:58.363Z] ok: [an-striker02]
[2026-04-13T00:57:58.363Z] ok: [an-striker01]
[2026-04-13T00:57:58.363Z] 
[2026-04-13T00:57:58.363Z] TASK [check perl-Log-Journald package version] *********************************
[2026-04-13T00:57:58.363Z] changed: [an-a01n02]
[2026-04-13T00:57:58.363Z] changed: [an-a01dr01]
[2026-04-13T00:57:58.363Z] changed: [an-a01n01]
[2026-04-13T00:57:58.363Z] changed: [an-striker02]
[2026-04-13T00:57:58.363Z] changed: [an-striker01]
[2026-04-13T00:57:58.363Z] 
[2026-04-13T00:57:58.363Z] PLAY [check nodes status after upgrade] ****************************************
[2026-04-13T00:57:58.363Z] 
[2026-04-13T00:57:58.363Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:57:58.363Z] ok: [an-a01n02]
[2026-04-13T00:57:58.363Z] ok: [an-a01dr01]
[2026-04-13T00:57:58.363Z] ok: [an-a01n01]
[2026-04-13T00:57:58.363Z] 
[2026-04-13T00:57:58.363Z] TASK [check kmod-drdb package version] *****************************************
[2026-04-13T00:57:58.363Z] changed: [an-a01n02]
[2026-04-13T00:57:58.363Z] changed: [an-a01n01]
[2026-04-13T00:57:58.363Z] changed: [an-a01dr01]
[2026-04-13T00:57:58.363Z] 
[2026-04-13T00:57:58.363Z] PLAY [check hosted node status after upgrade] **********************************
[2026-04-13T00:57:58.363Z] 
[2026-04-13T00:57:58.363Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:57:58.363Z] ok: [an-a01n02]
[2026-04-13T00:57:58.363Z] ok: [an-a01n01]
[2026-04-13T00:57:58.363Z] 
[2026-04-13T00:57:58.363Z] TASK [check cluster is in good state] ******************************************
[2026-04-13T00:57:58.363Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-13T00:57:58.363Z] 
[2026-04-13T00:57:58.363Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-13T00:58:00.813Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-13T00:58:00.813Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-13T00:58:00.813Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-13T00:58:00.813Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-13T00:58:00.813Z] 
[2026-04-13T00:58:00.813Z] TASK [wait servers to be started in pacemaker] *********************************
[2026-04-13T00:58:00.813Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_in_pacemaker.yml for an-a01n01, an-a01n02
[2026-04-13T00:58:00.813Z] 
[2026-04-13T00:58:00.813Z] TASK [Wait up to 10 minutes for an-test-deploy1 to be started in pacemaker] ****
[2026-04-13T00:58:02.629Z] ok: [an-a01n01]
[2026-04-13T00:58:02.629Z] ok: [an-a01n02]
[2026-04-13T00:58:02.629Z] 
[2026-04-13T00:58:02.629Z] TASK [wait servers drbd storage to be synced] **********************************
[2026-04-13T00:58:02.629Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_drbd_sync.yml for an-a01n01, an-a01n02
[2026-04-13T00:58:02.629Z] 
[2026-04-13T00:58:02.629Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be synced] *****
[2026-04-13T00:58:02.997Z] ok: [an-a01n02]
[2026-04-13T00:58:02.997Z] ok: [an-a01n01]
[2026-04-13T00:58:02.997Z] 
[2026-04-13T00:58:02.997Z] TASK [wait servers to respond to ping] *****************************************
[2026-04-13T00:58:03.365Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ping.yml for an-a01n01, an-a01n02
[2026-04-13T00:58:03.365Z] 
[2026-04-13T00:58:03.365Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ping] ************
[2026-04-13T00:58:04.150Z] ok: [an-a01n02]
[2026-04-13T00:58:04.150Z] ok: [an-a01n01]
[2026-04-13T00:58:04.150Z] 
[2026-04-13T00:58:04.150Z] TASK [wait servers to respond to ssh] ******************************************
[2026-04-13T00:58:04.150Z] included: /srv/anvil/anvil-test-suite/playbooks/common/wait_server_ssh.yml for an-a01n01, an-a01n02
[2026-04-13T00:58:04.150Z] 
[2026-04-13T00:58:04.150Z] TASK [Wait up to 10 minutes for an-test-deploy1 to respond to ssh] *************
[2026-04-13T00:58:04.937Z] ok: [an-a01n01]
[2026-04-13T00:58:04.937Z] ok: [an-a01n02]
[2026-04-13T00:58:04.937Z] 
[2026-04-13T00:58:04.937Z] PLAY [test anvil package self update] ******************************************
[2026-04-13T00:58:04.937Z] 
[2026-04-13T00:58:04.937Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:58:07.417Z] ok: [an-a01n01]
[2026-04-13T00:58:07.417Z] ok: [an-a01n02]
[2026-04-13T00:58:07.417Z] ok: [an-a01dr01]
[2026-04-13T00:58:07.417Z] ok: [an-striker02]
[2026-04-13T00:58:07.417Z] ok: [an-striker01]
[2026-04-13T00:58:07.417Z] 
[2026-04-13T00:58:07.417Z] TASK [stop anvil and scan-core services] ***************************************
[2026-04-13T00:58:10.607Z] changed: [an-a01n01] => (item=anvil-daemon.service)
[2026-04-13T00:58:10.607Z] changed: [an-a01n02] => (item=anvil-daemon.service)
[2026-04-13T00:58:10.607Z] changed: [an-a01dr01] => (item=anvil-daemon.service)
[2026-04-13T00:58:10.607Z] changed: [an-striker02] => (item=anvil-daemon.service)
[2026-04-13T00:58:10.607Z] changed: [an-striker01] => (item=anvil-daemon.service)
[2026-04-13T00:58:10.607Z] changed: [an-a01dr01] => (item=scancore.service)
[2026-04-13T00:58:10.607Z] changed: [an-a01n01] => (item=scancore.service)
[2026-04-13T00:58:10.607Z] changed: [an-a01n02] => (item=scancore.service)
[2026-04-13T00:58:10.607Z] changed: [an-striker02] => (item=scancore.service)
[2026-04-13T00:58:10.607Z] changed: [an-striker01] => (item=scancore.service)
[2026-04-13T00:58:10.607Z] 
[2026-04-13T00:58:10.607Z] PLAY [test striker package self update] ****************************************
[2026-04-13T00:58:10.607Z] 
[2026-04-13T00:58:10.607Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:58:12.072Z] ok: [an-striker02]
[2026-04-13T00:58:12.072Z] ok: [an-striker01]
[2026-04-13T00:58:12.072Z] 
[2026-04-13T00:58:12.072Z] TASK [reinstall anvil-core and anvil-striker] **********************************
[2026-04-13T00:58:35.787Z] changed: [an-striker01]
[2026-04-13T00:58:35.787Z] changed: [an-striker02]
[2026-04-13T00:58:35.787Z] 
[2026-04-13T00:58:35.787Z] PLAY [test node package self update] *******************************************
[2026-04-13T00:58:35.787Z] 
[2026-04-13T00:58:35.787Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:58:37.051Z] ok: [an-a01n02]
[2026-04-13T00:58:37.051Z] ok: [an-a01n01]
[2026-04-13T00:58:37.051Z] 
[2026-04-13T00:58:37.051Z] TASK [reinstall anvil-core and anvil-node] *************************************
[2026-04-13T00:59:00.731Z] changed: [an-a01n02]
[2026-04-13T00:59:00.731Z] changed: [an-a01n01]
[2026-04-13T00:59:00.731Z] 
[2026-04-13T00:59:00.731Z] PLAY [test dr package self update] *********************************************
[2026-04-13T00:59:00.731Z] 
[2026-04-13T00:59:00.731Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:59:01.518Z] ok: [an-a01dr01]
[2026-04-13T00:59:01.518Z] 
[2026-04-13T00:59:01.518Z] TASK [reinstall anvil-core and anvil-dr] ***************************************
[2026-04-13T00:59:25.197Z] changed: [an-a01dr01]
[2026-04-13T00:59:25.197Z] 
[2026-04-13T00:59:25.197Z] PLAY [test anvil package self update] ******************************************
[2026-04-13T00:59:25.197Z] 
[2026-04-13T00:59:25.197Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:59:25.565Z] ok: [an-striker02]
[2026-04-13T00:59:25.565Z] ok: [an-striker01]
[2026-04-13T00:59:25.565Z] ok: [an-a01dr01]
[2026-04-13T00:59:25.565Z] ok: [an-a01n02]
[2026-04-13T00:59:25.565Z] ok: [an-a01n01]
[2026-04-13T00:59:25.565Z] 
[2026-04-13T00:59:25.565Z] TASK [start anvil and scan-core services] **************************************
[2026-04-13T00:59:28.757Z] changed: [an-a01n01] => (item=anvil-daemon.service)
[2026-04-13T00:59:28.757Z] changed: [an-a01n02] => (item=anvil-daemon.service)
[2026-04-13T00:59:28.757Z] changed: [an-a01dr01] => (item=anvil-daemon.service)
[2026-04-13T00:59:28.757Z] changed: [an-striker02] => (item=anvil-daemon.service)
[2026-04-13T00:59:28.757Z] changed: [an-striker01] => (item=anvil-daemon.service)
[2026-04-13T00:59:28.757Z] changed: [an-a01n01] => (item=scancore.service)
[2026-04-13T00:59:28.757Z] changed: [an-a01n02] => (item=scancore.service)
[2026-04-13T00:59:28.757Z] changed: [an-a01dr01] => (item=scancore.service)
[2026-04-13T00:59:28.757Z] changed: [an-striker02] => (item=scancore.service)
[2026-04-13T00:59:28.757Z] changed: [an-striker01] => (item=scancore.service)
[2026-04-13T00:59:28.757Z] 
[2026-04-13T00:59:28.757Z] PLAY [delete server tests] *****************************************************
[2026-04-13T00:59:28.757Z] 
[2026-04-13T00:59:28.757Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T00:59:30.022Z] ok: [an-a01n02]
[2026-04-13T00:59:30.022Z] ok: [an-a01n01]
[2026-04-13T00:59:30.022Z] 
[2026-04-13T00:59:30.022Z] TASK [check cluster is in good state] ******************************************
[2026-04-13T00:59:30.022Z] included: /srv/anvil/anvil-test-suite/playbooks/common/cluster_good.yml for an-a01n01, an-a01n02
[2026-04-13T00:59:30.022Z] 
[2026-04-13T00:59:30.022Z] TASK [wait up to 10 minutes for nodes to be in pacemaker cluster] **************
[2026-04-13T00:59:35.293Z] ok: [an-a01n02] => (item=an-a01n01)
[2026-04-13T00:59:35.293Z] ok: [an-a01n01] => (item=an-a01n01)
[2026-04-13T00:59:35.293Z] ok: [an-a01n02] => (item=an-a01n02)
[2026-04-13T00:59:35.293Z] ok: [an-a01n01] => (item=an-a01n02)
[2026-04-13T00:59:35.293Z] 
[2026-04-13T00:59:35.293Z] TASK [delete servers] **********************************************************
[2026-04-13T00:59:35.293Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=1)
[2026-04-13T00:59:35.293Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=2)
[2026-04-13T00:59:35.293Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=3)
[2026-04-13T00:59:35.293Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=4)
[2026-04-13T00:59:35.293Z] included: /srv/anvil/anvil-test-suite/playbooks/common/delete_server.yml for an-a01n01, an-a01n02 => (item=5)
[2026-04-13T00:59:35.293Z] 
[2026-04-13T00:59:35.293Z] TASK [check if server an-test-deploy1 exists] **********************************
[2026-04-13T00:59:36.081Z] skipping: [an-a01n01]
[2026-04-13T00:59:36.081Z] ok: [an-a01n02]
[2026-04-13T00:59:36.081Z] 
[2026-04-13T00:59:36.081Z] TASK [delete server an-test-deploy1 from an-a01n02] ****************************
[2026-04-13T00:59:43.744Z] skipping: [an-a01n01]
[2026-04-13T00:59:43.744Z] changed: [an-a01n02]
[2026-04-13T00:59:43.744Z] 
[2026-04-13T00:59:43.744Z] TASK [Wait up to 10 minutes for an-test-deploy1 to be delete in pacemaker] *****
[2026-04-13T01:00:32.618Z] skipping: [an-a01n01]
[2026-04-13T01:00:32.618Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to be delete in pacemaker (60 retries left).
[2026-04-13T01:00:32.618Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to be delete in pacemaker (59 retries left).
[2026-04-13T01:00:32.618Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to be delete in pacemaker (58 retries left).
[2026-04-13T01:00:32.618Z] FAILED - RETRYING: [an-a01n02]: Wait up to 10 minutes for an-test-deploy1 to be delete in pacemaker (57 retries left).
[2026-04-13T01:00:32.618Z] ok: [an-a01n02]
[2026-04-13T01:00:32.618Z] 
[2026-04-13T01:00:32.618Z] TASK [Wait up to 10 minutes for an-test-deploy1 drbd storage to be removed] ****
[2026-04-13T01:00:32.618Z] skipping: [an-a01n01]
[2026-04-13T01:00:32.618Z] ok: [an-a01n02]
[2026-04-13T01:00:32.618Z] 
[2026-04-13T01:00:32.618Z] TASK [Wait up to 10 minutes for an-test-deploy1 lvm volume to be removed] ******
[2026-04-13T01:00:33.405Z] skipping: [an-a01n01]
[2026-04-13T01:00:33.405Z] ok: [an-a01n02]
[2026-04-13T01:00:33.405Z] 
[2026-04-13T01:00:33.405Z] TASK [check if server an-test-deploy2 exists] **********************************
[2026-04-13T01:00:34.925Z] skipping: [an-a01n01]
[2026-04-13T01:00:34.925Z] fatal: [an-a01n02]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy2\n", "delta": "0:00:00.925684", "end": "2026-04-12 21:00:34.480250", "msg": "non-zero return code", "rc": 1, "start": "2026-04-12 21:00:33.554566", "stderr": "Error: resource or tag id 'an-test-deploy2' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy2' not found"], "stdout": "", "stdout_lines": []}
[2026-04-13T01:00:34.925Z] ...ignoring
[2026-04-13T01:00:34.925Z] 
[2026-04-13T01:00:34.925Z] TASK [delete server an-test-deploy2 from an-a01n02] ****************************
[2026-04-13T01:00:34.925Z] skipping: [an-a01n01]
[2026-04-13T01:00:34.925Z] skipping: [an-a01n02]
[2026-04-13T01:00:34.925Z] 
[2026-04-13T01:00:34.925Z] TASK [Wait up to 10 minutes for an-test-deploy2 to be delete in pacemaker] *****
[2026-04-13T01:00:34.925Z] skipping: [an-a01n01]
[2026-04-13T01:00:34.925Z] skipping: [an-a01n02]
[2026-04-13T01:00:34.925Z] 
[2026-04-13T01:00:34.925Z] TASK [Wait up to 10 minutes for an-test-deploy2 drbd storage to be removed] ****
[2026-04-13T01:00:34.925Z] skipping: [an-a01n01]
[2026-04-13T01:00:34.925Z] skipping: [an-a01n02]
[2026-04-13T01:00:34.925Z] 
[2026-04-13T01:00:34.925Z] TASK [Wait up to 10 minutes for an-test-deploy2 lvm volume to be removed] ******
[2026-04-13T01:00:34.925Z] skipping: [an-a01n01]
[2026-04-13T01:00:34.925Z] skipping: [an-a01n02]
[2026-04-13T01:00:34.925Z] 
[2026-04-13T01:00:34.925Z] TASK [check if server an-test-deploy3 exists] **********************************
[2026-04-13T01:00:36.740Z] skipping: [an-a01n01]
[2026-04-13T01:00:36.740Z] fatal: [an-a01n02]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy3\n", "delta": "0:00:00.927192", "end": "2026-04-12 21:00:36.344870", "msg": "non-zero return code", "rc": 1, "start": "2026-04-12 21:00:35.417678", "stderr": "Error: resource or tag id 'an-test-deploy3' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy3' not found"], "stdout": "", "stdout_lines": []}
[2026-04-13T01:00:36.740Z] ...ignoring
[2026-04-13T01:00:36.740Z] 
[2026-04-13T01:00:36.740Z] TASK [delete server an-test-deploy3 from an-a01n02] ****************************
[2026-04-13T01:00:36.740Z] skipping: [an-a01n01]
[2026-04-13T01:00:36.740Z] skipping: [an-a01n02]
[2026-04-13T01:00:36.740Z] 
[2026-04-13T01:00:36.740Z] TASK [Wait up to 10 minutes for an-test-deploy3 to be delete in pacemaker] *****
[2026-04-13T01:00:36.740Z] skipping: [an-a01n01]
[2026-04-13T01:00:36.740Z] skipping: [an-a01n02]
[2026-04-13T01:00:36.740Z] 
[2026-04-13T01:00:36.740Z] TASK [Wait up to 10 minutes for an-test-deploy3 drbd storage to be removed] ****
[2026-04-13T01:00:36.740Z] skipping: [an-a01n01]
[2026-04-13T01:00:36.740Z] skipping: [an-a01n02]
[2026-04-13T01:00:36.740Z] 
[2026-04-13T01:00:36.740Z] TASK [Wait up to 10 minutes for an-test-deploy3 lvm volume to be removed] ******
[2026-04-13T01:00:36.740Z] skipping: [an-a01n01]
[2026-04-13T01:00:36.740Z] skipping: [an-a01n02]
[2026-04-13T01:00:36.740Z] 
[2026-04-13T01:00:36.740Z] TASK [check if server an-test-deploy4 exists] **********************************
[2026-04-13T01:00:38.555Z] skipping: [an-a01n01]
[2026-04-13T01:00:38.555Z] fatal: [an-a01n02]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy4\n", "delta": "0:00:00.922132", "end": "2026-04-12 21:00:38.231326", "msg": "non-zero return code", "rc": 1, "start": "2026-04-12 21:00:37.309194", "stderr": "Error: resource or tag id 'an-test-deploy4' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy4' not found"], "stdout": "", "stdout_lines": []}
[2026-04-13T01:00:38.555Z] ...ignoring
[2026-04-13T01:00:38.555Z] 
[2026-04-13T01:00:38.555Z] TASK [delete server an-test-deploy4 from an-a01n02] ****************************
[2026-04-13T01:00:38.555Z] skipping: [an-a01n01]
[2026-04-13T01:00:38.555Z] skipping: [an-a01n02]
[2026-04-13T01:00:38.555Z] 
[2026-04-13T01:00:38.555Z] TASK [Wait up to 10 minutes for an-test-deploy4 to be delete in pacemaker] *****
[2026-04-13T01:00:38.555Z] skipping: [an-a01n01]
[2026-04-13T01:00:38.555Z] skipping: [an-a01n02]
[2026-04-13T01:00:38.555Z] 
[2026-04-13T01:00:38.555Z] TASK [Wait up to 10 minutes for an-test-deploy4 drbd storage to be removed] ****
[2026-04-13T01:00:38.555Z] skipping: [an-a01n01]
[2026-04-13T01:00:38.555Z] skipping: [an-a01n02]
[2026-04-13T01:00:38.555Z] 
[2026-04-13T01:00:38.555Z] TASK [Wait up to 10 minutes for an-test-deploy4 lvm volume to be removed] ******
[2026-04-13T01:00:38.923Z] skipping: [an-a01n01]
[2026-04-13T01:00:38.923Z] skipping: [an-a01n02]
[2026-04-13T01:00:38.923Z] 
[2026-04-13T01:00:38.923Z] TASK [check if server an-test-deploy5 exists] **********************************
[2026-04-13T01:00:40.187Z] skipping: [an-a01n01]
[2026-04-13T01:00:40.187Z] fatal: [an-a01n02]: FAILED! => {"changed": false, "cmd": "pcs status resources an-test-deploy5\n", "delta": "0:00:00.842434", "end": "2026-04-12 21:00:39.999716", "msg": "non-zero return code", "rc": 1, "start": "2026-04-12 21:00:39.157282", "stderr": "Error: resource or tag id 'an-test-deploy5' not found", "stderr_lines": ["Error: resource or tag id 'an-test-deploy5' not found"], "stdout": "", "stdout_lines": []}
[2026-04-13T01:00:40.187Z] ...ignoring
[2026-04-13T01:00:40.187Z] 
[2026-04-13T01:00:40.187Z] TASK [delete server an-test-deploy5 from an-a01n02] ****************************
[2026-04-13T01:00:40.187Z] skipping: [an-a01n01]
[2026-04-13T01:00:40.187Z] skipping: [an-a01n02]
[2026-04-13T01:00:40.187Z] 
[2026-04-13T01:00:40.187Z] TASK [Wait up to 10 minutes for an-test-deploy5 to be delete in pacemaker] *****
[2026-04-13T01:00:40.556Z] skipping: [an-a01n01]
[2026-04-13T01:00:40.556Z] skipping: [an-a01n02]
[2026-04-13T01:00:40.556Z] 
[2026-04-13T01:00:40.556Z] TASK [Wait up to 10 minutes for an-test-deploy5 drbd storage to be removed] ****
[2026-04-13T01:00:40.556Z] skipping: [an-a01n01]
[2026-04-13T01:00:40.556Z] skipping: [an-a01n02]
[2026-04-13T01:00:40.556Z] 
[2026-04-13T01:00:40.556Z] TASK [Wait up to 10 minutes for an-test-deploy5 lvm volume to be removed] ******
[2026-04-13T01:00:40.556Z] skipping: [an-a01n01]
[2026-04-13T01:00:40.556Z] skipping: [an-a01n02]
[2026-04-13T01:00:40.556Z] 
[2026-04-13T01:00:40.556Z] PLAY [Check anvil available resources from striker01] **************************
[2026-04-13T01:00:40.556Z] 
[2026-04-13T01:00:40.556Z] TASK [Gathering Facts] *********************************************************
[2026-04-13T01:00:43.006Z] ok: [an-striker01]
[2026-04-13T01:00:43.006Z] 
[2026-04-13T01:00:43.006Z] TASK [Check anvil available resources] *****************************************
[2026-04-13T01:05:58.996Z] ok: [an-striker01]
[2026-04-13T01:05:58.996Z] 
[2026-04-13T01:05:58.996Z] PLAY RECAP *********************************************************************
[2026-04-13T01:05:58.996Z] an-a01dr01                 : ok=90   changed=38   unreachable=0    failed=0    skipped=5    rescued=0    ignored=0   
[2026-04-13T01:05:58.996Z] an-a01n01                  : ok=314  changed=28   unreachable=0    failed=0    skipped=109  rescued=0    ignored=5   
[2026-04-13T01:05:58.996Z] an-a01n02                  : ok=345  changed=27   unreachable=0    failed=0    skipped=90   rescued=0    ignored=4   
[2026-04-13T01:05:58.996Z] an-striker01               : ok=57   changed=19   unreachable=0    failed=0    skipped=3    rescued=0    ignored=0   
[2026-04-13T01:05:58.996Z] an-striker02               : ok=42   changed=12   unreachable=0    failed=0    skipped=3    rescued=0    ignored=0   
[2026-04-13T01:05:58.996Z] 
[2026-04-13T01:05:58.996Z] tar: Removing leading `/' from member names
[2026-04-13T01:05:58.996Z] Warning: Permanently added '[anvil-ci-proxy]:8086' (ED25519) to the list of known hosts.
[2026-04-13T01:05:58.996Z] /root/ci-tools
[2026-04-13T01:05:58.996Z] Complete anvil logs available here for download: http://anvil-ci-repo.ci.alteeve.com/testing-logs/290.tar.gz
[2026-04-13T01:05:58.996Z] Disconnecting an-striker01.ci.alteeve.com from Red Hat.
[2026-04-13T01:05:58.996Z] This might take a few seconds.
[2026-04-13T01:05:58.996Z] 
[2026-04-13T01:05:58.996Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-13T01:05:58.996Z] 𐄂 Disconnected from Red Hat Insights
[2026-04-13T01:05:58.996Z] 𐄂 Disconnected from Red Hat Subscription Management
[2026-04-13T01:05:58.996Z] 
[2026-04-13T01:05:58.996Z] Manage your connected systems: https://red.ht/connector
[2026-04-13T01:06:04.063Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-13T01:06:04.063Z] 
[2026-04-13T01:06:04.063Z] 1. Register with subscription-manager
[2026-04-13T01:06:04.063Z] # subscription-manager register
[2026-04-13T01:06:04.063Z] 
[2026-04-13T01:06:04.063Z] 2. Register with insights-client
[2026-04-13T01:06:04.063Z] # insights-client --register
[2026-04-13T01:06:04.432Z] This system is currently not registered.
[2026-04-13T01:06:05.616Z] Disconnecting an-striker02.ci.alteeve.com from Red Hat.
[2026-04-13T01:06:05.616Z] This might take a few seconds.
[2026-04-13T01:06:05.616Z] 
[2026-04-13T01:06:05.616Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-13T01:06:12.133Z] 𐄂 Disconnected from Red Hat Insights
[2026-04-13T01:06:12.524Z] 𐄂 Disconnected from Red Hat Subscription Management
[2026-04-13T01:06:12.524Z] 
[2026-04-13T01:06:12.524Z] Manage your connected systems: https://red.ht/connector
[2026-04-13T01:06:17.645Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-13T01:06:17.645Z] 
[2026-04-13T01:06:17.645Z] 1. Register with subscription-manager
[2026-04-13T01:06:17.645Z] # subscription-manager register
[2026-04-13T01:06:17.645Z] 
[2026-04-13T01:06:17.645Z] 2. Register with insights-client
[2026-04-13T01:06:17.645Z] # insights-client --register
[2026-04-13T01:06:18.014Z] This system is currently not registered.
[2026-04-13T01:06:18.801Z] Disconnecting an-a01n01.ci.alteeve.com from Red Hat.
[2026-04-13T01:06:18.801Z] This might take a few seconds.
[2026-04-13T01:06:18.801Z] 
[2026-04-13T01:06:18.801Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-13T01:06:25.057Z] 𐄂 Disconnected from Red Hat Insights
[2026-04-13T01:06:25.058Z] 𐄂 Disconnected from Red Hat Subscription Management
[2026-04-13T01:06:25.058Z] 
[2026-04-13T01:06:25.058Z] Manage your connected systems: https://red.ht/connector
[2026-04-13T01:06:29.508Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-13T01:06:29.508Z] 
[2026-04-13T01:06:29.508Z] 1. Register with subscription-manager
[2026-04-13T01:06:29.508Z] # subscription-manager register
[2026-04-13T01:06:29.508Z] 
[2026-04-13T01:06:29.508Z] 2. Register with insights-client
[2026-04-13T01:06:29.508Z] # insights-client --register
[2026-04-13T01:06:30.294Z] This system is currently not registered.
[2026-04-13T01:06:30.662Z] Disconnecting an-a01n02.ci.alteeve.com from Red Hat.
[2026-04-13T01:06:30.662Z] This might take a few seconds.
[2026-04-13T01:06:30.662Z] 
[2026-04-13T01:06:30.662Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-13T01:06:36.919Z] 𐄂 Disconnected from Red Hat Insights
[2026-04-13T01:06:36.919Z] 𐄂 Disconnected from Red Hat Subscription Management
[2026-04-13T01:06:36.919Z] 
[2026-04-13T01:06:36.919Z] Manage your connected systems: https://red.ht/connector
[2026-04-13T01:06:41.986Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-13T01:06:41.986Z] 
[2026-04-13T01:06:41.986Z] 1. Register with subscription-manager
[2026-04-13T01:06:41.986Z] # subscription-manager register
[2026-04-13T01:06:41.986Z] 
[2026-04-13T01:06:41.986Z] 2. Register with insights-client
[2026-04-13T01:06:41.986Z] # insights-client --register
[2026-04-13T01:06:42.354Z] This system is currently not registered.
[2026-04-13T01:06:43.141Z] Disconnecting an-a01dr01.ci.alteeve.com from Red Hat.
[2026-04-13T01:06:43.141Z] This might take a few seconds.
[2026-04-13T01:06:43.141Z] 
[2026-04-13T01:06:43.141Z] 𐄂 Deactivated the Remote Host Configuration daemon
[2026-04-13T01:06:48.210Z] 𐄂 Disconnected from Red Hat Insights
[2026-04-13T01:06:49.704Z] 𐄂 Disconnected from Red Hat Subscription Management
[2026-04-13T01:06:49.704Z] 
[2026-04-13T01:06:49.704Z] Manage your connected systems: https://red.ht/connector
[2026-04-13T01:06:53.842Z] This host is unregistered, please ensure that the system is registered with subscription-manager and then with insights-client.
[2026-04-13T01:06:53.842Z] 
[2026-04-13T01:06:53.842Z] 1. Register with subscription-manager
[2026-04-13T01:06:53.842Z] # subscription-manager register
[2026-04-13T01:06:53.842Z] 
[2026-04-13T01:06:53.842Z] 2. Register with insights-client
[2026-04-13T01:06:53.842Z] # insights-client --register
[2026-04-13T01:06:54.628Z] This system is currently not registered.
[2026-04-13T01:06:54.720Z] [Pipeline] }
[2026-04-13T01:06:54.723Z] [Pipeline] // timeout
[2026-04-13T01:06:54.728Z] [Pipeline] }
[2026-04-13T01:06:54.730Z] [Pipeline] // script
[2026-04-13T01:06:54.732Z] [Pipeline] }
[2026-04-13T01:06:54.734Z] [Pipeline] // stage
[2026-04-13T01:06:54.738Z] [Pipeline] stage
[2026-04-13T01:06:54.738Z] [Pipeline] { (Declarative: Post Actions)
[2026-04-13T01:06:54.744Z] [Pipeline] cleanWs
[2026-04-13T01:06:54.805Z] [WS-CLEANUP] Deleting project workspace...
[2026-04-13T01:06:54.805Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[2026-04-13T01:06:54.835Z] [WS-CLEANUP] done
[2026-04-13T01:06:54.838Z] [Pipeline] script
[2026-04-13T01:06:54.839Z] [Pipeline] {
[2026-04-13T01:06:54.854Z] [Pipeline] stage
[2026-04-13T01:06:54.854Z] [Pipeline] { (Copying logs)
[2026-04-13T01:06:54.886Z] [Pipeline] node
[2026-04-13T01:06:54.891Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-functional-testing
[2026-04-13T01:06:54.892Z] [Pipeline] {
[2026-04-13T01:06:54.929Z] [Pipeline] node
[2026-04-13T01:06:54.934Z] Running on Jenkins in /var/lib/jenkins/workspace/anvil/anvil-build-functional-testing@2
[2026-04-13T01:06:54.935Z] [Pipeline] {
[2026-04-13T01:06:54.937Z] [Pipeline] sh
[2026-04-13T01:06:55.193Z] + mkdir -p /var/lib/jenkins/locks
[2026-04-13T01:06:55.202Z] [Pipeline] echo
[2026-04-13T01:06:55.202Z] RWLock: FD for lock log_archive in stage projectFinishUp is 421
[2026-04-13T01:06:55.204Z] [Pipeline] sleep
[2026-04-13T01:06:55.205Z] No need to sleep any longer
[2026-04-13T01:06:55.207Z] [Pipeline] echo
[2026-04-13T01:06:55.207Z] RWLock: log_archive in stage projectFinishUp locked for WRITE
[2026-04-13T01:06:55.210Z] [Pipeline] }
[2026-04-13T01:06:55.212Z] [Pipeline] // node
[2026-04-13T01:06:55.216Z] [Pipeline] echo
[2026-04-13T01:06:55.216Z] Copying logs to /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-functional-testing/290/
[2026-04-13T01:06:55.217Z] [Pipeline] sh
[2026-04-13T01:06:55.472Z] + mkdir -p /var/www/ci.kronosnet.org/job/anvil/job/anvil-build-functional-testing/290/
[2026-04-13T01:06:55.474Z] [Pipeline] sh