+ sudo -E -H -u github ./ya make --test-size=small --test-size=medium --test-type=unittest --test-type=clang_tidy --test-type=gtest --test-type=py3test --test-type=py2test --test-type=pytest --test-type=flake8 --test-type=black --test-type=py2_flake8 --test-type=go_test --test-type=gofmt --run-all-tests --keep-going --retest --test-threads 16 --link-threads 32 --cache-size 512G --do-not-output-stderrs -T --nice 0 --stat --output /home/github/tmp/out --build relwithdebinfo --target=cloud/blockstore/ --target=cloud/filestore/ --target=cloud/disk_manager/ --target=cloud/tasks/ --target=cloud/storage/ --bazel-remote-store --bazel-remote-base-uri http://195.242.10.243:9090 --bazel-remote-username bazel-remote-user --bazel-remote-password-file /tmp/tmp.bHELfWT92b --junit /home/github/tmp/junit.xml.3 --log-file /home/github/tmp/logs/ya_log.3.txt --evlog-file /home/github/tmp/logs/ya_evlog.3.jsonl -X Configuring dependencies for platform default-linux-x86_64-relwithdebinfo Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-kikimr-test/cloud-blockstore-tests-fio-qemu-vhost-kikimr-test: Requirement cpu is redefined 8 -> 4 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-kikimr-test/cloud-blockstore-tests-fio-qemu-vhost-kikimr-test: Requirement ram is redefined 30 -> 16 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-local-test/cloud-blockstore-tests-fio-qemu-vhost-local-test: Requirement cpu is redefined 8 -> 4 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-local-test/cloud-blockstore-tests-fio-qemu-vhost-local-test: Requirement ram is redefined 30 -> 16 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-null-test/cloud-blockstore-tests-fio-qemu-vhost-null-test: Requirement cpu is redefined 8 -> 4 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-null-test/cloud-blockstore-tests-fio-qemu-vhost-null-test: Requirement ram is redefined 30 -> 16 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-encryption/cloud-blockstore-tests-loadtest-local-encryption: Requirement ram is redefined 30 -> 48 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-mirror/cloud-blockstore-tests-loadtest-local-mirror: Requirement cpu is redefined 8 -> all Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-mirror-lagging/blockstore-tests-loadtest-local-mirror-lagging: Requirement cpu is redefined 8 -> all Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/tests-loadtest-local-nonrepl-dedicated: Requirement cpu is redefined 8 -> 16 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/tests-loadtest-local-nonrepl-dedicated: Requirement ram is redefined 30 -> 96 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-nonrepl/cloud-blockstore-tests-loadtest-local-nonrepl: Requirement cpu is redefined 8 -> 16 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-nonrepl/cloud-blockstore-tests-loadtest-local-nonrepl: Requirement ram is redefined 30 -> 96 Configuring dependencies for platform tools Configuring dependencies for platform test_tool_tc1-global Configuring tests execution Configuring local and dist store caches Configuration done. Preparing for execution |17.2%| PREPARE $(TEST_TOOL_HOST-sbr:8580453620) |19.4%| PREPARE $(LLD_ROOT-3334042713) |20.4%| PREPARE $(GO_TOOLS-2652799470) |21.5%| PREPARE $(WITH_JDK-sbr:4561486916) |22.6%| PREPARE $(FLAKE8_PY3-3680676150) |23.7%| PREPARE $(WITH_JDK11-sbr:3064614561) |24.7%| PREPARE $(JDK_DEFAULT-1779948129) |25.8%| PREPARE $(JDK11-2842640623) |26.9%| PREPARE $(WITH_JDK17-sbr:4561486916) |28.0%| PREPARE $(JDK17-1779948129) |29.0%| PREPARE $(PYTHON) |30.1%| PREPARE $(CLANG-2185927622) |31.2%| PREPARE $(GDB) |32.3%| PREPARE $(YMAKE_PYTHON3-1415908779) |33.3%| PREPARE $(OS_SDK_ROOT-sbr:1966560555) |35.5%| COMPACTING CACHE 409.7GiB |37.6%| CLEANING SYMRES |38.7%| [LD] {RESULT} $(B)/contrib/python/moto/bin/moto_server |39.8%| [LD] {RESULT} $(B)/cloud/disk_manager/cmd/disk-manager/disk-manager |40.9%| [LD] {RESULT} $(B)/cloud/disk_manager/test/mocks/compute/compute-mock |41.9%| [LD] {RESULT} $(B)/cloud/disk_manager/cmd/disk-manager-init-db/disk-manager-init-db |43.0%| [GO] {RESULT} $(B)/cloud/disk_manager/internal/pkg/clients/nbs/tests/tests |44.1%| [LD] {RESULT} $(B)/cloud/tasks/test/nemesis/nemesis |45.2%| [LD] {RESULT} $(B)/cloud/disk_manager/test/mocks/kms/kms-mock |46.2%| [LD] {RESULT} $(B)/cloud/disk_manager/test/recipe/recipe |47.3%| [LD] {RESULT} $(B)/cloud/blockstore/tests/direct_device_acquire/cloud-blockstore-tests-direct_device_acquire |48.4%| [LD] {RESULT} $(B)/cloud/disk_manager/test/mocks/metadata/metadata-mock |49.5%| [PK] {RESULT} $(B)/cloud/storage/core/tools/testing/ydb/bin/{testing-ydb-bin.final.pkg.fake ... cloud/storage/core/tools/testing/ydb/bin/ydbd} |50.5%| [LD] {RESULT} $(B)/cloud/blockstore/apps/client/{blockstore-client ... liblibidn-dynamic.so} |51.6%| [LD] {RESULT} $(B)/cloud/blockstore/apps/server/{nbsd ... liblibaio-dynamic.so} |52.7%| [LD] {RESULT} $(B)/cloud/blockstore/libs/storage/service/ut/{cloud-blockstore-libs-storage-service-ut ... liblibaio-dynamic.so} |53.8%| [LD] {RESULT} $(B)/cloud/blockstore/libs/storage/volume/ut/{cloud-blockstore-libs-storage-volume-ut ... liblibaio-dynamic.so} |54.8%| [LD] {RESULT} $(B)/cloud/blockstore/apps/disk_agent/{diskagentd ... liblibaio-dynamic.so} |55.9%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |57.0%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |58.1%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest >> TServiceMountVolumeTest::ShouldNot >> TServiceMountVolumeTest::ShouldStopVolumeTabletAfterLocalStartBecauseOfTimedoutAddClient >> TVolumeTest::ShouldAggregateMetricsFromTabletsAndSendThemToHive >> TServiceMountVolumeTest::ShouldCauseRemountWhenVolumeIsStolen |59.1%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |60.2%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/service/ut/unittest |61.3%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/service/ut/unittest |62.4%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |63.4%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |64.5%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/service/ut/unittest |65.6%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |66.7%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |68.8%| [LD] {RESULT} $(B)/cloud/filestore/apps/server/{filestore-server ... liblibaio-dynamic.so} >> TVolumeTest::ShouldAggregateMetricsFromTabletsAndSendThemToHive [FAIL] >> TServiceMountVolumeTest::ShouldReturnTabletBackIfLocalMountedAndTabletWasStolen |69.9%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |71.0%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/service/ut/unittest |72.0%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/service/ut/unittest |73.1%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/service/ut/unittest |75.3%| [LD] {RESULT} $(B)/contrib/ydb/apps/ydbd/{ydbd ... liblibaio-dynamic.so} >> test.py::test_should_mount_volume_with_unknown_devices[60000] >> test.py::test_should_mount_volume_with_unavailable_agents[6000] >> test.py::test_should_mount_volume_without_dr[6000] |76.3%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/direct_device_acquire/py3test |77.4%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/direct_device_acquire/py3test |78.5%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/direct_device_acquire/py3test |79.6%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/direct_device_acquire/py3test |80.6%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/direct_device_acquire/py3test |81.7%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/direct_device_acquire/py3test |82.8%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/direct_device_acquire/py3test >> TServiceMountVolumeTest::ShouldStopVolumeTabletAfterLocalStartBecauseOfTimedoutAddClient [FAIL] >> TServiceMountVolumeTest::ShouldNot [FAIL] >> TServiceMountVolumeTest::ShouldCauseRemountWhenVolumeIsStolen [FAIL] >> TServiceMountVolumeTest::ShouldReturnTabletBackIfLocalMountedAndTabletWasStolen [FAIL] |83.9%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest >> TVolumeTest::ShouldAggregateMetricsFromTabletsAndSendThemToHive [FAIL] |84.9%| [TA] $(B)/cloud/blockstore/libs/storage/volume/ut/test-results/unittest/{meta.json ... results_accumulator.log} |86.0%| [TA] {RESULT} $(B)/cloud/blockstore/libs/storage/volume/ut/test-results/unittest/{meta.json ... results_accumulator.log} |87.1%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/service/ut/unittest >> TServiceMountVolumeTest::ShouldNot [FAIL] |88.2%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/service/ut/unittest >> TServiceMountVolumeTest::ShouldStopVolumeTabletAfterLocalStartBecauseOfTimedoutAddClient [FAIL] |89.2%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/service/ut/unittest >> TServiceMountVolumeTest::ShouldCauseRemountWhenVolumeIsStolen [FAIL] |90.3%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/service/ut/unittest >> TServiceMountVolumeTest::ShouldReturnTabletBackIfLocalMountedAndTabletWasStolen [FAIL] |91.4%| [TA] $(B)/cloud/blockstore/libs/storage/service/ut/test-results/unittest/{meta.json ... results_accumulator.log} |92.5%| [TA] {RESULT} $(B)/cloud/blockstore/libs/storage/service/ut/test-results/unittest/{meta.json ... results_accumulator.log} >> tests::TestCreateDisk [GOOD] >> tests::TestDeleteDisk [GOOD] >> tests::TestDeleteDiskSync [GOOD] >> tests::TestCreateDeleteCheckpoint [GOOD] >> tests::TestDeleteCheckpointOnUnexistingDisk [GOOD] >> tests::TestDeleteCheckpointData [GOOD] >> tests::TestResizeDisk [GOOD] >> tests::TestResizeDiskConcurrently [GOOD] >> tests::TestResizeDiskFailureBecauseSizeIsNotDivisibleByBlockSize [GOOD] >> tests::TestResizeDiskFailureBecauseSizeDecreaseIsForbidden [GOOD] >> tests::TestResizeDiskFailureWhileChekpointing [GOOD] >> tests::TestAlterDisk [GOOD] >> tests::TestAlterDiskConcurrently [GOOD] >> tests::TestAlterDiskFailureWhileCheckpointing [GOOD] >> tests::TestRebaseDisk [GOOD] >> tests::TestAssignDisk [GOOD] >> tests::TestUnassignDisk [GOOD] >> tests::TestUnassignDeletedDisk [GOOD] >> tests::TestTokenErrorsShouldBeRetriable [GOOD] |93.5%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/disk_manager/internal/pkg/clients/nbs/tests/tests >> tests::TestTokenErrorsShouldBeRetriable [GOOD] |94.6%| [TM] {RESULT} cloud/disk_manager/internal/pkg/clients/nbs/tests/tests >> test.py::test_should_mount_volume_with_unknown_devices[60000] [FAIL] >> test.py::test_should_mount_volume_without_dr[6000] [FAIL] >> test.py::test_should_mount_volume_with_unavailable_agents[6000] [FAIL] |95.7%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/direct_device_acquire/py3test >> test.py::test_should_mount_volume_with_unknown_devices[60000] [FAIL] |96.8%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/direct_device_acquire/py3test >> test.py::test_should_mount_volume_without_dr[6000] [FAIL] |97.8%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/direct_device_acquire/py3test >> test.py::test_should_mount_volume_with_unavailable_agents[6000] [FAIL] |98.9%| [TA] $(B)/cloud/blockstore/tests/direct_device_acquire/test-results/py3test/{meta.json ... results_accumulator.log} |99.9%| [TA] {RESULT} $(B)/cloud/blockstore/tests/direct_device_acquire/test-results/py3test/{meta.json ... results_accumulator.log} |99.9%| CLEANING BUILD ROOT ------ sole chunk ran 40 tests (total:199.45s - recipes:72.54s test:126.14s recipes:0.73s) Info: Test run has exceeded 8.0G (8388608K) memory limit with 8.2G (8564356K) used. This may lead to test failure on the Autocheck/CI You can increase test's ram requirement using REQUIREMENTS(ram:X) in the ya.make pid rss ref pdirt 647826 44.8M 44.8M 6.1M test_tool run_test @/home/github/tmp/out/ya_command_file_0.args 647829 32.4M 20.3M 8.2M ├─ test_tool run_test @/home/github/tmp/out/ya_command_file_0.args 652531 45.9M 46.3M 23.4M │ └─ test_tool run_go_test @/home/github/tmp/out/cloud/disk_manager/internal/pkg/clients/nbs/tests/test-results/tests/testing_out_stuff/test_tool. 652539 83.9M 88.9M 71.7M │ └─ tests -test.v 647923 82.8M 82.9M 40.9M ├─ moto_server s3 --port 26524 647924 1.2G 1.2G 635M ├─ ydbd server --suppress-version-check --node=1 --log-file-name=/home/github/tmp/out/cloud/disk_manager/internal/pkg/clients/nbs/tests/test-result 649210 388M 392M 214M ├─ nbsd --service kikimr --server-port 65088 --data-server-port 29923 --secure-server-port 18197 --mon-port 20253 --profile-file /home/github/.ya/build/build_root/yk35/0000 649672 263M 266M 119M ├─ diskagentd --domain Root --node-broker localhost:62981 --ic-port 2354 --mon-port 9657 --scheme-shard-dir nbs --load-configs-from-cms --node-type disk-agent --diag-file / 649813 270M 273M 125M ├─ diskagentd --domain Root --node-broker localhost:62981 --ic-port 26525 --mon-port 6674 --scheme-shard-dir nbs --load-configs-from-cms --node-type disk-agent-1 --diag-fil 650268 260M 262M 114M ├─ diskagentd --domain Root --node-broker localhost:62981 --ic-port 2005 --mon-port 2956 --scheme-shard-dir nbs --load-configs-from-cms --node-type disk-agent-2 --diag-file 650627 274M 277M 130M ├─ diskagentd --domain Root --node-broker localhost:62981 --ic-port 7853 --mon-port 4380 --scheme-shard-dir nbs --load-configs-from-cms --node-type disk-agent-3 --diag-file 650870 1.4G 1.4G 1.3G ├─ diskagentd --domain Root --node-broker localhost:62981 --ic-port 19660 --mon-port 31764 --scheme-shard-dir nbs --load-configs-from-cms --node-type disk-agent-4 --diag-fi 651016 265M 268M 121M ├─ diskagentd --domain Root --node-broker localhost:62981 --ic-port 4379 --mon-port 12159 --scheme-shard-dir nbs --load-configs-from-cms --node-type disk-agent-5 --diag-fil 651162 258M 261M 114M ├─ diskagentd --domain Root --node-broker localhost:62981 --ic-port 29913 --mon-port 14917 --scheme-shard-dir nbs --load-configs-from-cms --node-type disk-agent-6 --diag-fi 651346 1.1G 1.1G 566M ├─ ydbd server --suppress-version-check --node=1 --log-file-name=/home/github/tmp/out/cloud/disk_manager/internal/pkg/clients/nbs/tests/test-result 651699 302M 306M 131M ├─ nbsd --service kikimr --server-port 4247 --data-server-port 22343 --secure-server-port 18936 --mon-port 24144 --profile-file /home/github/.ya/build/build_root/yk35/00001 651827 254M 257M 110M ├─ diskagentd --domain Root --node-broker localhost:27181 --ic-port 11546 --mon-port 9214 --scheme-shard-dir nbs --load-configs-from-cms --node-type disk-agent --diag-file 651947 1.1G 1.1G 569M ├─ ydbd server --suppress-version-check --node=1 --log-file-name=/home/github/tmp/out/cloud/disk_manager/internal/pkg/clients/nbs/tests/test-result 652287 279M 284M 111M ├─ nbsd --service kikimr --server-port 13900 --data-server-port 29806 --secure-server-port 8065 --mon-port 8116 --profile-file /home/github/tmp/out 652416 252M 255M 108M └─ diskagentd --domain Root --node-broker localhost:25945 --ic-port 24754 --mon-port 1478 --scheme-shard-dir nbs --load-configs-from-cms --node-type disk-agent --diag-file Log: /home/github/tmp/out/cloud/disk_manager/internal/pkg/clients/nbs/tests/test-results/tests/run_test.log Logsdir: /home/github/tmp/out/cloud/disk_manager/internal/pkg/clients/nbs/tests/test-results/tests/testing_out_stuff Stderr: /home/github/tmp/out/cloud/disk_manager/internal/pkg/clients/nbs/tests/test-results/tests/testing_out_stuff/stderr cloud/blockstore/tests/direct_device_acquire [size:medium] nchunks:10 [tags: sb:MULTISLOT, sb:cores=8, sb:logs_ttl=3, sb:ssd, sb:ttl=3] ------ [0/10] chunk ran 1 test (total:348.38s - test:348.37s) [fail] test.py::test_should_mount_volume_with_unavailable_agents[6000] [default-linux-x86_64-relwithdebinfo] (347.29s) cloud/blockstore/tests/direct_device_acquire/test.py:388: in test_should_mount_volume_with_unavailable_agents session.mount_volume() cloud/blockstore/public/sdk/python/client/error.py:15: in wrapper return f(*args, **kw) cloud/blockstore/public/sdk/python/client/session.py:174: in mount_volume request_timeout).result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:456: in result return self.__get_result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:401: in __get_result raise self._exception cloud/blockstore/public/sdk/python/client/session.py:151: in process_mount_response r = f.result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:449: in result return self.__get_result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:401: in __get_result raise self._exception cloud/blockstore/public/sdk/python/client/durable.py:130: in _process_response raise exception cloud/blockstore/public/sdk/python/client/grpc_client.py:149: in set_result response = extract_response(f) cloud/blockstore/public/sdk/python/client/error.py:15: in wrapper return f(*args, **kw) cloud/blockstore/public/sdk/python/client/grpc_client.py:141: in extract_response return self._process_response( cloud/blockstore/public/sdk/python/client/error.py:15: in wrapper return f(*args, **kw) cloud/blockstore/public/sdk/python/client/base_client.py:186: in _process_response raise error E cloud.blockstore.public.sdk.python.client.error.ClientError: SEVERITY_ERROR FACILITY_NULL status:2 Failed to deliver AddClient request to volume Log: /home/github/tmp/out/cloud/blockstore/tests/direct_device_acquire/test-results/py3test/chunk0/testing_out_stuff/test.py.test_should_mount_volume_with_unavailable_agents.6000.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/direct_device_acquire/test-results/py3test/chunk0/testing_out_stuff ------ [1/10] chunk ran 1 test (total:341.47s - test:341.46s) [fail] test.py::test_should_mount_volume_with_unknown_devices[60000] [default-linux-x86_64-relwithdebinfo] (340.45s) cloud/blockstore/tests/direct_device_acquire/test.py:254: in test_should_mount_volume_with_unknown_devices session.mount_volume() cloud/blockstore/public/sdk/python/client/error.py:15: in wrapper return f(*args, **kw) cloud/blockstore/public/sdk/python/client/session.py:174: in mount_volume request_timeout).result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:456: in result return self.__get_result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:401: in __get_result raise self._exception cloud/blockstore/public/sdk/python/client/session.py:151: in process_mount_response r = f.result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:449: in result return self.__get_result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:401: in __get_result raise self._exception cloud/blockstore/public/sdk/python/client/durable.py:130: in _process_response raise exception cloud/blockstore/public/sdk/python/client/grpc_client.py:149: in set_result response = extract_response(f) cloud/blockstore/public/sdk/python/client/error.py:15: in wrapper return f(*args, **kw) cloud/blockstore/public/sdk/python/client/grpc_client.py:141: in extract_response return self._process_response( cloud/blockstore/public/sdk/python/client/error.py:15: in wrapper return f(*args, **kw) cloud/blockstore/public/sdk/python/client/base_client.py:186: in _process_response raise error E cloud.blockstore.public.sdk.python.client.error.ClientError: SEVERITY_ERROR FACILITY_NULL status:2 Failed to deliver AddClient request to volume Log: /home/github/tmp/out/cloud/blockstore/tests/direct_device_acquire/test-results/py3test/chunk1/testing_out_stuff/test.py.test_should_mount_volume_with_unknown_devices.60000.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/direct_device_acquire/test-results/py3test/chunk1/testing_out_stuff ------ [2/10] chunk ran 1 test (total:342.26s - test:342.25s) [fail] test.py::test_should_mount_volume_without_dr[6000] [default-linux-x86_64-relwithdebinfo] (341.23s) cloud/blockstore/tests/direct_device_acquire/test.py:316: in test_should_mount_volume_without_dr session.mount_volume() cloud/blockstore/public/sdk/python/client/error.py:15: in wrapper return f(*args, **kw) cloud/blockstore/public/sdk/python/client/session.py:174: in mount_volume request_timeout).result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:456: in result return self.__get_result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:401: in __get_result raise self._exception cloud/blockstore/public/sdk/python/client/session.py:151: in process_mount_response r = f.result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:449: in result return self.__get_result() contrib/tools/python3/src/Lib/concurrent/futures/_base.py:401: in __get_result raise self._exception cloud/blockstore/public/sdk/python/client/durable.py:130: in _process_response raise exception cloud/blockstore/public/sdk/python/client/grpc_client.py:149: in set_result response = extract_response(f) cloud/blockstore/public/sdk/python/client/error.py:15: in wrapper return f(*args, **kw) cloud/blockstore/public/sdk/python/client/grpc_client.py:141: in extract_response return self._process_response( cloud/blockstore/public/sdk/python/client/error.py:15: in wrapper return f(*args, **kw) cloud/blockstore/public/sdk/python/client/base_client.py:186: in _process_response raise error E cloud.blockstore.public.sdk.python.client.error.ClientError: SEVERITY_ERROR FACILITY_NULL status:2 Failed to deliver AddClient request to volume Log: /home/github/tmp/out/cloud/blockstore/tests/direct_device_acquire/test-results/py3test/chunk2/testing_out_stuff/test.py.test_should_mount_volume_without_dr.6000.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/direct_device_acquire/test-results/py3test/chunk2/testing_out_stuff ------ FAIL: 3 - FAIL cloud/blockstore/tests/direct_device_acquire cloud/blockstore/libs/storage/service/ut [size:medium] nchunks:10 [tags: sb:MULTISLOT, sb:cores=8, sb:logs_ttl=3, sb:ssd, sb:ttl=3] ------ [0/10] chunk ran 1 test (total:20.66s - test:20.65s) [fail] TServiceMountVolumeTest::ShouldCauseRemountWhenVolumeIsStolen [default-linux-x86_64-relwithdebinfo] (8.06s) assertion failed at cloud/blockstore/libs/storage/testlib/service_client.h:294, std::unique_ptr NCloud::NBlockStore::NStorage::TServiceClient::MountVolume(Args &&...) [Args = &, const char (&)[4], const char (&)[4]>]: (SUCCEEDED(response->GetStatus())) Failed to deliver AddClient request to volume TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::Get() const at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:560:16 TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::UnRef() at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:598:13 std::__y1::unique_ptr, std::__y1::default_delete>>::operator->[abi:v15000]() const at /actions-runner/_work/nbs/nbs/contrib/libs/cxxsupp/libcxx/include/__memory/unique_ptr.h:275:19 NCloud::NBlockStore::NStorage::NTestSuiteTServiceMountVolumeTest::TTestCaseShouldCauseRemountWhenVolumeIsStolen::Execute_(NUnitTest::TTestContext&) at /actions-runner/_work/nbs/nbs/cloud/blockstore/libs/storage/service/service_ut_mount.cpp:0:9 NCloud::NBlockStore::NStorage::NTestSuiteTServiceMountVolumeTest::TCurrentTest::Execute()::'lambda'()::operator()() const at /actions-runner/_work/nbs/nbs/cloud/blockstore/libs/storage/service/service_ut_mount.cpp:0:1 std::__y1::__function::__value_func::~__value_func[abi:v15000]() at /actions-runner/_work/nbs/nbs/contrib/libs/cxxsupp/libcxx/include/__functional/function.h:470:20 TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::UnRef() at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:598:13 NUnitTest::TTestFactory::Execute() at /actions-runner/_work/nbs/nbs/library/cpp/testing/unittest/registar.cpp:0:19 NUnitTest::RunMain(int, char**) at /actions-runner/_work/nbs/nbs/library/cpp/testing/unittest/utmain.cpp:0:0 ?? at ??:0:0 ?? at ??:0:0 _start at ??:0:0 Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk0/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk0/testing_out_stuff/TServiceMountVolumeTest.ShouldCauseRemountWhenVolumeIsStolen.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk0/testing_out_stuff/TServiceMountVolumeTest.ShouldCauseRemountWhenVolumeIsStolen.out ------ [1/10] chunk ran 1 test (total:15.05s - test:15.03s) [fail] TServiceMountVolumeTest::ShouldNot [default-linux-x86_64-relwithdebinfo] (2.45s) assertion failed at cloud/blockstore/libs/storage/service/service_ut_mount.cpp:3054, virtual void NCloud::NBlockStore::NStorage::NTestSuiteTServiceMountVolumeTest::TTestCaseShouldNot::Execute_(NUnitTest::TTestContext &): (!((ui32(response->GetStatus()) & 0x80000000u) != 0)) TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::Get() const at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:560:16 TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::UnRef() at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:598:13 NCloud::NBlockStore::NStorage::NTestSuiteTServiceMountVolumeTest::TCurrentTest::Execute()::'lambda'()::operator()() const at /actions-runner/_work/nbs/nbs/cloud/blockstore/libs/storage/service/service_ut_mount.cpp:0:1 std::__y1::__function::__value_func::~__value_func[abi:v15000]() at /actions-runner/_work/nbs/nbs/contrib/libs/cxxsupp/libcxx/include/__functional/function.h:470:20 TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::UnRef() at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:598:13 NUnitTest::TTestFactory::Execute() at /actions-runner/_work/nbs/nbs/library/cpp/testing/unittest/registar.cpp:0:19 NUnitTest::RunMain(int, char**) at /actions-runner/_work/nbs/nbs/library/cpp/testing/unittest/utmain.cpp:0:0 ?? at ??:0:0 ?? at ??:0:0 _start at ??:0:0 Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk1/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk1/testing_out_stuff/TServiceMountVolumeTest.ShouldNot.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk1/testing_out_stuff/TServiceMountVolumeTest.ShouldNot.out ------ [2/10] chunk ran 1 test (total:20.46s - test:20.45s) [fail] TServiceMountVolumeTest::ShouldReturnTabletBackIfLocalMountedAndTabletWasStolen [default-linux-x86_64-relwithdebinfo] (7.86s) assertion failed at cloud/blockstore/libs/storage/testlib/service_client.h:294, std::unique_ptr NCloud::NBlockStore::NStorage::TServiceClient::MountVolume(Args &&...) [Args = <>]: (SUCCEEDED(response->GetStatus())) Failed to deliver AddClient request to volume TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::Get() const at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:560:16 TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::UnRef() at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:598:13 std::__y1::unique_ptr, std::__y1::default_delete>>::reset[abi:v15000](NCloud::NBlockStore::TProtoResponseEvent*) at /actions-runner/_work/nbs/nbs/contrib/libs/cxxsupp/libcxx/include/__memory/unique_ptr.h:303:28 NCloud::NBlockStore::NStorage::NTestSuiteTServiceMountVolumeTest::TCurrentTest::Execute()::'lambda'()::operator()() const at /actions-runner/_work/nbs/nbs/cloud/blockstore/libs/storage/service/service_ut_mount.cpp:0:1 std::__y1::__function::__value_func::~__value_func[abi:v15000]() at /actions-runner/_work/nbs/nbs/contrib/libs/cxxsupp/libcxx/include/__functional/function.h:470:20 TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::UnRef() at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:598:13 NUnitTest::TTestFactory::Execute() at /actions-runner/_work/nbs/nbs/library/cpp/testing/unittest/registar.cpp:0:19 NUnitTest::RunMain(int, char**) at /actions-runner/_work/nbs/nbs/library/cpp/testing/unittest/utmain.cpp:0:0 ?? at ??:0:0 ?? at ??:0:0 _start at ??:0:0 Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk2/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk2/testing_out_stuff/TServiceMountVolumeTest.ShouldReturnTabletBackIfLocalMountedAndTabletWasStolen.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk2/testing_out_stuff/TServiceMountVolumeTest.ShouldReturnTabletBackIfLocalMountedAndTabletWasStolen.out ------ [3/10] chunk ran 1 test (total:15.05s - test:15.04s) [fail] TServiceMountVolumeTest::ShouldStopVolumeTabletAfterLocalStartBecauseOfTimedoutAddClient [default-linux-x86_64-relwithdebinfo] (2.45s) assertion failed at cloud/blockstore/libs/storage/testlib/service_client.h:294, std::unique_ptr NCloud::NBlockStore::NStorage::TServiceClient::MountVolume(Args &&...) [Args = &, const char (&)[4], const char (&)[4], NCloud::NBlockStore::NProto::EClientIpcType, NCloud::NBlockStore::NProto::EVolumeAccessMode, NCloud::NBlockStore::NProto::EVolumeMountMode>]: (SUCCEEDED(response->GetStatus())) Failed to deliver AddClient request to volume TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::Get() const at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:560:16 TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::UnRef() at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:598:13 std::__y1::unique_ptr, std::__y1::default_delete>>::operator->[abi:v15000]() const at /actions-runner/_work/nbs/nbs/contrib/libs/cxxsupp/libcxx/include/__memory/unique_ptr.h:275:19 NCloud::NBlockStore::NStorage::NTestSuiteTServiceMountVolumeTest::TCurrentTest::Execute()::'lambda'()::operator()() const at /actions-runner/_work/nbs/nbs/cloud/blockstore/libs/storage/service/service_ut_mount.cpp:0:1 std::__y1::__function::__value_func::~__value_func[abi:v15000]() at /actions-runner/_work/nbs/nbs/contrib/libs/cxxsupp/libcxx/include/__functional/function.h:470:20 TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::UnRef() at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:598:13 NUnitTest::TTestFactory::Execute() at /actions-runner/_work/nbs/nbs/library/cpp/testing/unittest/registar.cpp:0:19 NUnitTest::RunMain(int, char**) at /actions-runner/_work/nbs/nbs/library/cpp/testing/unittest/utmain.cpp:0:0 ?? at ??:0:0 ?? at ??:0:0 _start at ??:0:0 Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk3/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk3/testing_out_stuff/TServiceMountVolumeTest.ShouldStopVolumeTabletAfterLocalStartBecauseOfTimedoutAddClient.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/service/ut/test-results/unittest/chunk3/testing_out_stuff/TServiceMountVolumeTest.ShouldStopVolumeTabletAfterLocalStartBecauseOfTimedoutAddClient.out ------ FAIL: 4 - FAIL cloud/blockstore/libs/storage/service/ut cloud/blockstore/libs/storage/volume/ut [size:medium] nchunks:10 [tags: sb:MULTISLOT, sb:cores=8, sb:logs_ttl=3, sb:ssd, sb:ttl=3] ------ [0/10] chunk ran 1 test (total:13.05s - test:13.03s) [fail] TVolumeTest::ShouldAggregateMetricsFromTabletsAndSendThemToHive [default-linux-x86_64-relwithdebinfo] (0.64s) assertion failed at cloud/blockstore/libs/storage/volume/volume_ut.cpp:9082, virtual void NCloud::NBlockStore::NStorage::NTestSuiteTVolumeTest::TTestCaseShouldAggregateMetricsFromTabletsAndSendThemToHive::Execute_(NUnitTest::TTestContext &): (0 != metrics.GetNetwork()) failed: (0 == 0) TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::Get() const at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:560:16 NPrivateStringBuilder::TStringBuilder::~TStringBuilder() at /actions-runner/_work/nbs/nbs/util/string/builder.h:8:11 NCloud::NBlockStore::NStorage::NTestSuiteTVolumeTest::TCurrentTest::Execute()::'lambda'()::operator()() const at /actions-runner/_work/nbs/nbs/cloud/blockstore/libs/storage/volume/volume_ut.cpp:0:1 std::__y1::__function::__value_func::~__value_func[abi:v15000]() at /actions-runner/_work/nbs/nbs/contrib/libs/cxxsupp/libcxx/include/__functional/function.h:470:20 TIntrusivePtr, std::__y1::allocator>>, TStringPtrOps, std::__y1::allocator>>>>::UnRef() at /actions-runner/_work/nbs/nbs/util/generic/ptr.h:598:13 NUnitTest::TTestFactory::Execute() at /actions-runner/_work/nbs/nbs/library/cpp/testing/unittest/registar.cpp:0:19 NUnitTest::RunMain(int, char**) at /actions-runner/_work/nbs/nbs/library/cpp/testing/unittest/utmain.cpp:0:0 ?? at ??:0:0 ?? at ??:0:0 _start at ??:0:0 Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut/test-results/unittest/chunk0/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut/test-results/unittest/chunk0/testing_out_stuff/TVolumeTest.ShouldAggregateMetricsFromTabletsAndSendThemToHive.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut/test-results/unittest/chunk0/testing_out_stuff/TVolumeTest.ShouldAggregateMetricsFromTabletsAndSendThemToHive.out ------ FAIL: 1 - FAIL cloud/blockstore/libs/storage/volume/ut Total 4 suites: 1 - GOOD 3 - FAIL Total 48 tests: 40 - GOOD 8 - FAIL Cache efficiency ratio is 99.89% (29673 of 29707). Local: 19 (0.06%), dist: 0 (0.00%), by dynamic uids: 0 (0.00%), avoided: 29654 (99.82%) Dist cache download: count=0, size=0 bytes, speed=0.0 bytes/s Disk usage for tools/sdk 2.34 GiB Additional disk space consumed for build cache 0 bytes Critical path: [348588 ms] [TM] [rnd-8917190638250003588 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/direct_device_acquire/py3test [started: 0 (1750259042985), finished: 348588 (1750259391573)] [ 69822 ms] [TA] [rnd-95xaxfec72ccruz4]: $(BUILD_ROOT)/cloud/blockstore/tests/direct_device_acquire/test-results/py3test/{meta.json ... results_accumulator.log} [started: 348615 (1750259391600), finished: 418437 (1750259461422)] Time from start: 427298.5051269531 ms, time elapsed by graph 418410 ms, time diff 8888.505126953125 ms. The longest 10 tasks: [348588 ms] [TM] [rnd-8917190638250003588 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/direct_device_acquire/py3test [started: 1750259042985, finished: 1750259391573] [342465 ms] [TM] [rnd-5734966195954700487 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/direct_device_acquire/py3test [started: 1750259042994, finished: 1750259385459] [341676 ms] [TM] [rnd-2656974136989509955 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/direct_device_acquire/py3test [started: 1750259042981, finished: 1750259384657] [199669 ms] [TM] [rnd-1jtzrtm71rj67fcn default-linux-x86_64 relwithdebinfo]: cloud/disk_manager/internal/pkg/clients/nbs/tests/tests [started: 1750259042619, finished: 1750259242288] [ 69822 ms] [TA] [rnd-95xaxfec72ccruz4]: $(BUILD_ROOT)/cloud/blockstore/tests/direct_device_acquire/test-results/py3test/{meta.json ... results_accumulator.log} [started: 1750259391600, finished: 1750259461422] [ 20861 ms] [TM] [rnd-6425208155814207864 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/libs/storage/service/ut/unittest [started: 1750259041519, finished: 1750259062380] [ 20659 ms] [TM] [rnd-4403057091833509412 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/libs/storage/service/ut/unittest [started: 1750259042166, finished: 1750259062825] [ 15266 ms] [TM] [rnd-11101770413114207739 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/libs/storage/service/ut/unittest [started: 1750259041525, finished: 1750259056791] [ 15242 ms] [TM] [rnd-13267669077023370284 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/libs/storage/service/ut/unittest [started: 1750259041516, finished: 1750259056758] [ 13256 ms] [TM] [rnd-15596513113370898193 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/libs/storage/volume/ut/unittest [started: 1750259041534, finished: 1750259054790] Total time by type: [1335653 ms] [TM] [count: 31, ave time 43085.58 msec] [ 70127 ms] [TA] [count: 3, ave time 23375.67 msec] [ 2967 ms] [prepare:get from local cache] [count: 19, ave time 156.16 msec] [ 477 ms] [prepare:clean] [count: 3, ave time 159.00 msec] [ 290 ms] [prepare:tools] [count: 15, ave time 19.33 msec] [ 77 ms] [prepare:AC] [count: 2, ave time 38.50 msec] Total tasks times: Total failed tasks time - 0 ms (0.00%) Total tests tasks time - 1405780 ms (100.00%) Total run tasks time - 1405780 ms Configure time - 10.5 s Statistics overhead 509 ms Info: Dump junit report to /home/github/tmp/junit.xml.3 Failed + echo 10