+ sudo -E -H -u github ./ya make --test-size=small --test-size=medium --test-size=large --test-type=unittest --test-type=clang_tidy --test-type=gtest --test-type=py3test --test-type=py2test --test-type=pytest --test-type=flake8 --test-type=black --test-type=py2_flake8 --test-type=go_test --test-type=gofmt --run-all-tests --keep-going --retest --test-threads 14 --link-threads 28 --cache-size 512G --do-not-output-stderrs -T --nice 0 --stat --output /home/github/tmp/out -DGITHUB_CI=yes --build relwithdebinfo --target=cloud/blockstore/ --bazel-remote-store --bazel-remote-base-uri http://195.242.10.243:9090 --bazel-remote-username bazel-remote-user --bazel-remote-password-file /tmp/tmp.QfO2wPDJmM --junit /home/github/tmp/junit.xml.2 --log-file /home/github/tmp/logs/2/ya_log.txt --evlog-file /home/github/tmp/logs/2/ya_evlog.jsonl -X Configuring dependencies for platform default-linux-x86_64-relwithdebinfo Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-kikimr-test/cloud-blockstore-tests-fio-qemu-vhost-kikimr-test: Requirement cpu is redefined 8 -> 4 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-kikimr-test/cloud-blockstore-tests-fio-qemu-vhost-kikimr-test: Requirement ram is redefined 30 -> 16 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-local-test/cloud-blockstore-tests-fio-qemu-vhost-local-test: Requirement cpu is redefined 8 -> 4 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-local-test/cloud-blockstore-tests-fio-qemu-vhost-local-test: Requirement ram is redefined 30 -> 16 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-null-test/cloud-blockstore-tests-fio-qemu-vhost-null-test: Requirement cpu is redefined 8 -> 4 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/fio/qemu-vhost-null-test/cloud-blockstore-tests-fio-qemu-vhost-null-test: Requirement ram is redefined 30 -> 16 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-data-integrity/blockstore-tests-loadtest-local-data-integrity: Requirement cpu is redefined 8 -> all Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-encryption/cloud-blockstore-tests-loadtest-local-encryption: Requirement ram is redefined 30 -> 48 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-mirror/cloud-blockstore-tests-loadtest-local-mirror: Requirement cpu is redefined 8 -> all Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-mirror-lagging/blockstore-tests-loadtest-local-mirror-lagging: Requirement cpu is redefined 8 -> all Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/tests-loadtest-local-nonrepl-dedicated: Requirement cpu is redefined 8 -> 16 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/tests-loadtest-local-nonrepl-dedicated: Requirement ram is redefined 30 -> 96 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/tests-loadtest-local-nonrepl-embedded: Requirement cpu is redefined 8 -> 16 Warn[-WPluginErr]: in $B/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/tests-loadtest-local-nonrepl-embedded: Requirement ram is redefined 30 -> 96 Configuring dependencies for platform tools Configuring dependencies for platform test_tool_tc1-global Configuring tests execution Configuring local and dist store caches Configuration done. Preparing for execution |11.9%| PREPARE $(WITH_JDK17-sbr:4561486916) |13.6%| PREPARE $(OS_SDK_ROOT-sbr:1966560555) |14.4%| PREPARE $(TEST_TOOL_HOST-sbr:8580453620) |15.3%| PREPARE $(FLAKE8_PY3-3680676150) |16.1%| PREPARE $(JDK17-1779948129) |16.9%| PREPARE $(WITH_JDK-sbr:4561486916) |17.8%| PREPARE $(CLANG-2185927622) |18.6%| PREPARE $(JDK_DEFAULT-1779948129) |19.5%| PREPARE $(YMAKE_PYTHON3-1415908779) |20.3%| PREPARE $(PYTHON) |22.0%| PREPARE $(LLD_ROOT-3334042713) |22.9%| PREPARE $(GDB) |23.7%| PREPARE $(JDK11-2842640623) |24.6%| PREPARE $(WITH_JDK11-sbr:3064614561) |26.3%| COMPACTING CACHE 309.3GiB |28.0%| CLEANING SYMRES |28.8%| [LD] {RESULT} $(B)/cloud/blockstore/tests/recipes/fake-root-kms/fake-root-kms |29.7%| [LD] {RESULT} $(B)/cloud/storage/core/tools/testing/unstable-process/storage-unstable-process |30.5%| [LD] {RESULT} $(B)/cloud/blockstore/tests/loadtest/local-mirror-lagging/blockstore-tests-loadtest-local-mirror-lagging |31.4%| [LD] {RESULT} $(B)/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/tests-loadtest-local-nonrepl-embedded |32.2%| [LD] {RESULT} $(B)/cloud/blockstore/tests/loadtest/local-mirror/cloud-blockstore-tests-loadtest-local-mirror |33.1%| [LD] {RESULT} $(B)/cloud/blockstore/tools/testing/fake-root-kms/fake-root-kms |33.9%| [LD] {RESULT} $(B)/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/tests-loadtest-local-nonrepl-dedicated |34.7%| [LD] {RESULT} $(B)/contrib/ydb/apps/ydb/{ydb ... libiconv.so} |35.6%| [LD] {RESULT} $(B)/cloud/blockstore/apps/client/{blockstore-client ... liblibidn-dynamic.so} |36.4%| [LD] {RESULT} $(B)/cloud/blockstore/tools/testing/loadtest/bin/{blockstore-loadtest ... liblibaio-dynamic.so} |37.3%| [LD] {RESULT} $(B)/cloud/blockstore/libs/storage/partition_nonrepl/ut/{blockstore-libs-storage-partition_nonrepl-ut ... liblibaio-dynamic.so} |38.1%| [LD] {RESULT} $(B)/cloud/blockstore/apps/server/{nbsd ... liblibaio-dynamic.so} |39.0%| [LD] {RESULT} $(B)/cloud/blockstore/libs/storage/volume/ut_linked/{cloud-blockstore-libs-storage-volume-ut_linked ... liblibaio-dynamic.so} |39.8%| [LD] {RESULT} $(B)/cloud/blockstore/libs/storage/volume/ut/{cloud-blockstore-libs-storage-volume-ut ... liblibaio-dynamic.so} |40.7%| [LD] {RESULT} $(B)/cloud/blockstore/apps/disk_agent/{diskagentd ... liblibaio-dynamic.so} >> TLinkedVolumeTest::ShouldManageFollowerLink >> TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_MIRROR |41.5%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/partition_nonrepl/ut/unittest |42.4%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/partition_nonrepl/ut/unittest |43.2%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut_linked/unittest |44.1%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/partition_nonrepl/ut/unittest |44.9%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/partition_nonrepl/ut/unittest |45.8%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |46.6%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |47.5%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |48.3%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/partition_nonrepl/ut/unittest |49.2%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |50.0%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |50.8%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest >> TMirrorPartitionTest::ShouldNotMigrateIfCantLockMigrationRange >> TMirrorPartitionTest::ShouldNotMigrateIfCantLockMigrationRangeDirectCopy |51.7%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |52.5%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/partition_nonrepl/ut/unittest |53.4%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/partition_nonrepl/ut/unittest |54.2%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut_linked/unittest |55.1%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut_linked/unittest |55.9%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/partition_nonrepl/ut/unittest |56.8%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest >> TVolumeTest::ShouldForwardRequestsToNonreplicatedPartitionDuringMigrationWithFatalDeviceErrors >> TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_NRD_WithReboot |57.6%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest |58.5%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut_linked/unittest >> TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_MIRROR >> TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_NRD >> TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_NRD_WithCheckpointAfter >> TLinkedVolumeTest::ShouldRetryPropagateFollowerLink |59.3%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut_linked/unittest |60.2%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut_linked/unittest >> TLinkedVolumeTest::ShouldManageFollowerLink |61.0%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/partition_nonrepl/ut/unittest >> TMirrorPartitionTest::ShouldNotMigrateIfCantLockMigrationRange |61.9%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/partition_nonrepl/ut/unittest >> TMirrorPartitionTest::ShouldNotMigrateIfCantLockMigrationRangeDirectCopy |62.7%| [TA] $(B)/cloud/blockstore/libs/storage/partition_nonrepl/ut/test-results/unittest/{meta.json ... results_accumulator.log} |63.6%| [TA] {RESULT} $(B)/cloud/blockstore/libs/storage/partition_nonrepl/ut/test-results/unittest/{meta.json ... results_accumulator.log} |64.4%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut/unittest >> TVolumeTest::ShouldForwardRequestsToNonreplicatedPartitionDuringMigrationWithFatalDeviceErrors |65.3%| [TA] $(B)/cloud/blockstore/libs/storage/volume/ut/test-results/unittest/{meta.json ... results_accumulator.log} |66.1%| [TA] {RESULT} $(B)/cloud/blockstore/libs/storage/volume/ut/test-results/unittest/{meta.json ... results_accumulator.log} |66.9%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut_linked/unittest >> TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_NRD_WithReboot |67.8%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut_linked/unittest >> TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_NRD |68.6%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut_linked/unittest >> TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_NRD_WithCheckpointAfter |69.5%| [TM] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/libs/storage/volume/ut_linked/unittest >> TLinkedVolumeTest::ShouldRetryPropagateFollowerLink |70.3%| [TA] $(B)/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/{meta.json ... results_accumulator.log} |71.2%| [TA] {RESULT} $(B)/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/{meta.json ... results_accumulator.log} |72.9%| [LD] {RESULT} $(B)/contrib/ydb/apps/ydbd/{ydbd ... liblibaio-dynamic.so} >> test.py::test_load[mirror2-target-migration-lagging-interconnect] >> test.py::test_load[no-rdma-mirror2-fresh-device-migration] >> test.py::test_load[rdma-mirror2-split-read-column] >> test.py::test_load[mirror2-target-migration-lagging-rdma] |73.7%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror-lagging/py3test |74.6%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror-lagging/py3test |75.4%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror/py3test >> test.py::test_load[rdma-mirror2-split-read-cross] >> test.py::test_load[mirror2-target-migration-lagging-rdma] [GOOD] |76.3%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror-lagging/py3test >> test.py::test_load[mirror2-target-migration-lagging-rdma] [GOOD] >> test.py::test_load[rdma-mirror2-agent-removal] >> test.py::test_load[no-rdma-mirror2-fresh-device-migration] [FAIL] |77.1%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror/py3test >> test.py::test_load[no-rdma-mirror2-fresh-device-migration] [FAIL] |78.0%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror/py3test >> test.py::test_load[mirror2-target-migration-lagging-interconnect] [GOOD] |78.8%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror-lagging/py3test >> test.py::test_load[mirror2-target-migration-lagging-interconnect] [GOOD] |79.7%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror/py3test >> test.py::test_load[rdma-mirror2-split-read-column] [FAIL] |80.5%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror/py3test >> test.py::test_load[rdma-mirror2-split-read-column] [FAIL] >> test.py::test_load[no-rdma-mirror2-split-read-column] |81.4%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror-lagging/py3test >> test.py::test_load[mirror2-fresh-device-migration-rdma] >> test.py::test_load[mirror2-migration-interconnect] >> test.py::test_load[rdma-mirror2-split-read-cross] [FAIL] |82.2%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror/py3test >> test.py::test_load[rdma-mirror2-split-read-cross] [FAIL] >> test.py::test_load[mirror2-migration-rdma] >> test.py::test_load[mirror2-fresh-device-migration-rdma] [FAIL] |83.1%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror-lagging/py3test >> test.py::test_load[mirror2-fresh-device-migration-rdma] [FAIL] >> test.py::test_load_rdma[migration] >> test.py::test_load[mirror2-migration-rdma] [FAIL] |83.9%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror-lagging/py3test >> test.py::test_load[mirror2-migration-rdma] [FAIL] >> test.py::test_load_rdma[migration] >> test.py::test_load[mirror2-migration-interconnect] [FAIL] |84.7%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror-lagging/py3test >> test.py::test_load[mirror2-migration-interconnect] [FAIL] |85.6%| [TA] $(B)/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/{meta.json ... results_accumulator.log} |86.4%| [TA] {RESULT} $(B)/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/{meta.json ... results_accumulator.log} |87.3%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/py3test >> test.py::test_load_rdma[migration-nbd] >> test.py::test_load[no-rdma-mirror2-split-read-column] [FAIL] |88.1%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror/py3test >> test.py::test_load[no-rdma-mirror2-split-read-column] [FAIL] |89.0%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/py3test |89.8%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-nonrepl/embedded/py3test |90.7%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-nonrepl/embedded/py3test >> test.py::test_load_rdma[migration-nbd] >> test.py::test_load[rdma-mirror2-agent-removal] [FAIL] |91.5%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-mirror/py3test >> test.py::test_load[rdma-mirror2-agent-removal] [FAIL] |92.4%| [TA] $(B)/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/{meta.json ... results_accumulator.log} |93.2%| [TA] {RESULT} $(B)/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/{meta.json ... results_accumulator.log} >> test.py::test_load_rdma[migration] [FAIL] |94.1%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/py3test >> test.py::test_load_rdma[migration] [FAIL] >> test.py::test_load_rdma[migration-nbd] [FAIL] |94.9%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/py3test >> test.py::test_load_rdma[migration-nbd] [FAIL] |95.8%| [TA] $(B)/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/{meta.json ... results_accumulator.log} |96.6%| [TA] {RESULT} $(B)/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/{meta.json ... results_accumulator.log} >> test.py::test_load_rdma[migration] [FAIL] |97.5%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-nonrepl/embedded/py3test >> test.py::test_load_rdma[migration] [FAIL] >> test.py::test_load_rdma[migration-nbd] [FAIL] |98.3%| [TL] {default-linux-x86_64, relwithdebinfo} cloud/blockstore/tests/loadtest/local-nonrepl/embedded/py3test >> test.py::test_load_rdma[migration-nbd] [FAIL] |98.3%| [TA] $(B)/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/{meta.json ... results_accumulator.log} |99.2%| [TA] {RESULT} $(B)/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/{meta.json ... results_accumulator.log} |99.9%| CLEANING BUILD ROOT cloud/blockstore/tests/loadtest/local-mirror [size:large] nchunks:8 [tags: sb:cores=16, sb:logs_ttl=3, sb:ssd, sb:ttl=3, ya:fat, ya:force_sandbox] ------ [0/8] chunk ran 1 test (total:158.44s - test:158.42s) [fail] test.py::test_load[no-rdma-mirror2-fresh-device-migration] [default-linux-x86_64-relwithdebinfo] (157.15s) cloud/blockstore/tests/loadtest/local-mirror/test.py:321: in test_load return __run_test(test_case, use_rdma) cloud/blockstore/tests/loadtest/local-mirror/test.py:309: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk0/testing_out_stuff/test.py.test_load.no-rdma-mirror2-fresh-device-migration.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk0/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk0/testing_out_stuff/nbsd.347311.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk0/testing_out_stuff/nbsd.347311.backtrace.html Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk0/testing_out_stuff/nbsd.347311.core ------ [1/8] chunk ran 1 test (total:167.28s - test:167.26s) [fail] test.py::test_load[no-rdma-mirror2-split-read-column] [default-linux-x86_64-relwithdebinfo] (165.92s) cloud/blockstore/tests/loadtest/local-mirror/test.py:321: in test_load return __run_test(test_case, use_rdma) cloud/blockstore/tests/loadtest/local-mirror/test.py:309: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk1/testing_out_stuff/test.py.test_load.no-rdma-mirror2-split-read-column.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk1/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk1/testing_out_stuff/nbsd.354647.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk1/testing_out_stuff/nbsd.354647.backtrace.html Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk1/testing_out_stuff/nbsd.354647.core ------ [2/8] chunk ran 1 test (total:271.96s - test:271.94s) [fail] test.py::test_load[rdma-mirror2-agent-removal] [default-linux-x86_64-relwithdebinfo] (270.48s) cloud/blockstore/tests/loadtest/local-mirror/test.py:296: in __run_test ret = run_test( cloud/blockstore/tests/python/lib/test_base.py:275: in run_test raise Exception( E Exception: Several (1) processes terminated prematurely. During handling of the above exception, another exception occurred: cloud/blockstore/tests/loadtest/local-mirror/test.py:321: in test_load return __run_test(test_case, use_rdma) cloud/blockstore/tests/loadtest/local-mirror/test.py:309: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk2/testing_out_stuff/test.py.test_load.rdma-mirror2-agent-removal.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk2/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk2/testing_out_stuff/nbsd.353553.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk2/testing_out_stuff/nbsd.353553.backtrace.html Nbsd backtrace html2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk2/testing_out_stuff/nbsd.353553.backtrace.0.html Nbsd backtrace2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk2/testing_out_stuff/nbsd.353553.backtrace.0 Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd binary2: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk2/testing_out_stuff/nbsd.353553.core Nbsd core2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk2/testing_out_stuff/nbsd.353553.core.0 ------ [3/8] chunk ran 1 test (total:161.74s - test:161.72s) [fail] test.py::test_load[rdma-mirror2-split-read-column] [default-linux-x86_64-relwithdebinfo] (160.55s) cloud/blockstore/tests/loadtest/local-mirror/test.py:321: in test_load return __run_test(test_case, use_rdma) cloud/blockstore/tests/loadtest/local-mirror/test.py:309: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk3/testing_out_stuff/test.py.test_load.rdma-mirror2-split-read-column.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk3/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk3/testing_out_stuff/nbsd.347236.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk3/testing_out_stuff/nbsd.347236.backtrace.html Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk3/testing_out_stuff/nbsd.347236.core ------ [4/8] chunk ran 1 test (total:165.89s - test:165.87s) [fail] test.py::test_load[rdma-mirror2-split-read-cross] [default-linux-x86_64-relwithdebinfo] (164.47s) cloud/blockstore/tests/loadtest/local-mirror/test.py:321: in test_load return __run_test(test_case, use_rdma) cloud/blockstore/tests/loadtest/local-mirror/test.py:309: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk4/testing_out_stuff/test.py.test_load.rdma-mirror2-split-read-cross.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk4/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk4/testing_out_stuff/nbsd.349127.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk4/testing_out_stuff/nbsd.349127.backtrace.html Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror/test-results/py3test/chunk4/testing_out_stuff/nbsd.349127.core ------ FAIL: 5 - FAIL cloud/blockstore/tests/loadtest/local-mirror cloud/blockstore/tests/loadtest/local-mirror-lagging [size:large] nchunks:8 [tags: sb:cores=16, sb:logs_ttl=3, sb:ssd, sb:ttl=3, ya:fat, ya:force_sandbox] ------ [0/8] chunk ran 1 test (total:113.85s - test:113.79s) [fail] test.py::test_load[mirror2-fresh-device-migration-rdma] [default-linux-x86_64-relwithdebinfo] (112.37s) cloud/blockstore/tests/loadtest/local-mirror-lagging/test.py:370: in __run_test ret = run_test( cloud/blockstore/tests/python/lib/test_base.py:275: in run_test raise Exception( E Exception: Several (1) processes terminated prematurely. During handling of the above exception, another exception occurred: cloud/blockstore/tests/loadtest/local-mirror-lagging/test.py:396: in test_load return __run_test(test_case, use_rdma) cloud/blockstore/tests/loadtest/local-mirror-lagging/test.py:384: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk0/testing_out_stuff/test.py.test_load.mirror2-fresh-device-migration-rdma.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk0/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk0/testing_out_stuff/nbsd.355038.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk0/testing_out_stuff/nbsd.355038.backtrace.html Nbsd backtrace html2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk0/testing_out_stuff/nbsd.355038.backtrace.0.html Nbsd backtrace2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk0/testing_out_stuff/nbsd.355038.backtrace.0 Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd binary2: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk0/testing_out_stuff/nbsd.355038.core Nbsd core2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk0/testing_out_stuff/nbsd.355038.core.0 ------ [1/8] chunk ran 1 test (total:135.82s - test:135.66s) [fail] test.py::test_load[mirror2-migration-interconnect] [default-linux-x86_64-relwithdebinfo] (134.16s) cloud/blockstore/tests/loadtest/local-mirror-lagging/test.py:370: in __run_test ret = run_test( cloud/blockstore/tests/python/lib/test_base.py:275: in run_test raise Exception( E Exception: Several (1) processes terminated prematurely. During handling of the above exception, another exception occurred: cloud/blockstore/tests/loadtest/local-mirror-lagging/test.py:396: in test_load return __run_test(test_case, use_rdma) cloud/blockstore/tests/loadtest/local-mirror-lagging/test.py:384: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk1/testing_out_stuff/test.py.test_load.mirror2-migration-interconnect.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk1/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk1/testing_out_stuff/nbsd.355249.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk1/testing_out_stuff/nbsd.355249.backtrace.html Nbsd backtrace html2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk1/testing_out_stuff/nbsd.355249.backtrace.0.html Nbsd backtrace2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk1/testing_out_stuff/nbsd.355249.backtrace.0 Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd binary2: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk1/testing_out_stuff/nbsd.355249.core Nbsd core2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk1/testing_out_stuff/nbsd.355249.core.0 ------ [2/8] chunk ran 1 test (total:119.03s - test:118.93s) [fail] test.py::test_load[mirror2-migration-rdma] [default-linux-x86_64-relwithdebinfo] (117.51s) cloud/blockstore/tests/loadtest/local-mirror-lagging/test.py:370: in __run_test ret = run_test( cloud/blockstore/tests/python/lib/test_base.py:275: in run_test raise Exception( E Exception: Several (1) processes terminated prematurely. During handling of the above exception, another exception occurred: cloud/blockstore/tests/loadtest/local-mirror-lagging/test.py:396: in test_load return __run_test(test_case, use_rdma) cloud/blockstore/tests/loadtest/local-mirror-lagging/test.py:384: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk2/testing_out_stuff/test.py.test_load.mirror2-migration-rdma.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk2/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk2/testing_out_stuff/nbsd.357393.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk2/testing_out_stuff/nbsd.357393.backtrace.html Nbsd backtrace html2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk2/testing_out_stuff/nbsd.357393.backtrace.0.html Nbsd backtrace2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk2/testing_out_stuff/nbsd.357393.backtrace.0 Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd binary2: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk2/testing_out_stuff/nbsd.357393.core Nbsd core2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-mirror-lagging/test-results/py3test/chunk2/testing_out_stuff/nbsd.357393.core.0 ------ FAIL: 2 - GOOD, 3 - FAIL cloud/blockstore/tests/loadtest/local-mirror-lagging cloud/blockstore/tests/loadtest/local-nonrepl/dedicated [size:large] nchunks:4 [tags: sb:cores=16, sb:logs_ttl=3, sb:ssd, sb:ttl=3, ya:fat, ya:force_sandbox, ya:large_tests_on_single_slots] ------ [0/4] chunk ran 1 test (total:314.15s - setup:0.05s recipes:0.22s test:313.63s recipes:0.19s) [fail] test.py::test_load_rdma[migration-nbd] [default-linux-x86_64-relwithdebinfo] (312.25s) cloud/blockstore/tests/loadtest/local-nonrepl/test.py:347: in __run_test ret = run_test( cloud/blockstore/tests/python/lib/test_base.py:275: in run_test raise Exception( E Exception: Several (1) processes terminated prematurely. During handling of the above exception, another exception occurred: cloud/blockstore/tests/loadtest/local-nonrepl/test.py:390: in test_load_rdma return __run_test(test_case, backend=DISK_AGENT_BACKEND_AIO, use_rdma=True) cloud/blockstore/tests/loadtest/local-nonrepl/test.py:359: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk0/testing_out_stuff/test.py.test_load_rdma.migration-nbd.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk0/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk0/testing_out_stuff/nbsd.360138.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk0/testing_out_stuff/nbsd.360138.backtrace.html Nbsd backtrace html2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk0/testing_out_stuff/nbsd.360138.backtrace.0.html Nbsd backtrace2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk0/testing_out_stuff/nbsd.360138.backtrace.0 Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd binary2: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk0/testing_out_stuff/nbsd.360138.core Nbsd core2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk0/testing_out_stuff/nbsd.360138.core.0 ------ [1/4] chunk ran 1 test (total:296.19s - recipes:0.24s test:295.74s recipes:0.19s) [fail] test.py::test_load_rdma[migration] [default-linux-x86_64-relwithdebinfo] (294.17s) cloud/blockstore/tests/loadtest/local-nonrepl/test.py:347: in __run_test ret = run_test( cloud/blockstore/tests/python/lib/test_base.py:275: in run_test raise Exception( E Exception: Several (1) processes terminated prematurely. During handling of the above exception, another exception occurred: cloud/blockstore/tests/loadtest/local-nonrepl/test.py:390: in test_load_rdma return __run_test(test_case, backend=DISK_AGENT_BACKEND_AIO, use_rdma=True) cloud/blockstore/tests/loadtest/local-nonrepl/test.py:359: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk1/testing_out_stuff/test.py.test_load_rdma.migration.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk1/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk1/testing_out_stuff/nbsd.358969.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk1/testing_out_stuff/nbsd.358969.backtrace.html Nbsd backtrace html2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk1/testing_out_stuff/nbsd.358969.backtrace.0.html Nbsd backtrace2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk1/testing_out_stuff/nbsd.358969.backtrace.0 Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd binary2: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk1/testing_out_stuff/nbsd.358969.core Nbsd core2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/test-results/py3test/chunk1/testing_out_stuff/nbsd.358969.core.0 ------ FAIL: 2 - FAIL cloud/blockstore/tests/loadtest/local-nonrepl/dedicated cloud/blockstore/tests/loadtest/local-nonrepl/embedded [size:large] nchunks:4 [tags: sb:cores=16, sb:logs_ttl=3, sb:ssd, sb:ttl=3, ya:fat, ya:force_sandbox, ya:large_tests_on_single_slots] ------ [0/4] chunk ran 1 test (total:346.91s - recipes:0.20s test:346.50s recipes:0.19s) [fail] test.py::test_load_rdma[migration-nbd] [default-linux-x86_64-relwithdebinfo] (345.17s) cloud/blockstore/tests/loadtest/local-nonrepl/test.py:347: in __run_test ret = run_test( cloud/blockstore/tests/python/lib/test_base.py:275: in run_test raise Exception( E Exception: Several (1) processes terminated prematurely. During handling of the above exception, another exception occurred: cloud/blockstore/tests/loadtest/local-nonrepl/test.py:390: in test_load_rdma return __run_test(test_case, backend=DISK_AGENT_BACKEND_AIO, use_rdma=True) cloud/blockstore/tests/loadtest/local-nonrepl/test.py:359: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk0/testing_out_stuff/test.py.test_load_rdma.migration-nbd.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk0/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk0/testing_out_stuff/nbsd.360986.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk0/testing_out_stuff/nbsd.360986.backtrace.html Nbsd backtrace html2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk0/testing_out_stuff/nbsd.360844.backtrace.html Nbsd backtrace html3: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk0/testing_out_stuff/nbsd.360844.backtrace.0.html Nbsd backtrace2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk0/testing_out_stuff/nbsd.360844.backtrace Nbsd backtrace3: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk0/testing_out_stuff/nbsd.360844.backtrace.0 Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd binary2: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd binary3: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk0/testing_out_stuff/nbsd.360986.core Nbsd core2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk0/testing_out_stuff/nbsd.360844.core Nbsd core3: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk0/testing_out_stuff/nbsd.360844.core.0 ------ [1/4] chunk ran 1 test (total:343.10s - recipes:0.30s test:342.57s recipes:0.19s) [fail] test.py::test_load_rdma[migration] [default-linux-x86_64-relwithdebinfo] (341.14s) cloud/blockstore/tests/loadtest/local-nonrepl/test.py:347: in __run_test ret = run_test( cloud/blockstore/tests/python/lib/test_base.py:275: in run_test raise Exception( E Exception: Several (1) processes terminated prematurely. During handling of the above exception, another exception occurred: cloud/blockstore/tests/loadtest/local-nonrepl/test.py:390: in test_load_rdma return __run_test(test_case, backend=DISK_AGENT_BACKEND_AIO, use_rdma=True) cloud/blockstore/tests/loadtest/local-nonrepl/test.py:359: in __run_test nbs.stop() cloud/blockstore/tests/python/lib/nbs_runner.py:672: in stop self.__stop_daemon() cloud/blockstore/tests/python/lib/nbs_runner.py:667: in __stop_daemon super(LocalNbs, self).stop() cloud/storage/core/tools/common/python/daemon.py:215: in stop self.__terminate_process() cloud/storage/core/tools/common/python/daemon.py:111: in __terminate_process self.__verify_process() cloud/storage/core/tools/common/python/daemon.py:93: in __verify_process raise DaemonError("unexpected exit code", res) E cloud.storage.core.tools.common.python.daemon.DaemonError: Daemon failed with message: unexpected exit code. Process exit_code = -6. Log: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk1/testing_out_stuff/test.py.test_load_rdma.migration.log Logsdir: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk1/testing_out_stuff Nbsd backtrace: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk1/testing_out_stuff/nbsd.359724.backtrace Nbsd backtrace html: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk1/testing_out_stuff/nbsd.359724.backtrace.html Nbsd backtrace html2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk1/testing_out_stuff/nbsd.359600.backtrace.html Nbsd backtrace html3: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk1/testing_out_stuff/nbsd.359600.backtrace.0.html Nbsd backtrace2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk1/testing_out_stuff/nbsd.359600.backtrace Nbsd backtrace3: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk1/testing_out_stuff/nbsd.359600.backtrace.0 Nbsd binary: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd binary2: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd binary3: /home/github/tmp/out/cloud/blockstore/apps/server/nbsd Nbsd core: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk1/testing_out_stuff/nbsd.359724.core Nbsd core2: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk1/testing_out_stuff/nbsd.359600.core Nbsd core3: /home/github/tmp/out/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/chunk1/testing_out_stuff/nbsd.359600.core.0 ------ FAIL: 2 - FAIL cloud/blockstore/tests/loadtest/local-nonrepl/embedded cloud/blockstore/libs/storage/partition_nonrepl/ut [size:medium] nchunks:10 [tags: sb:MULTISLOT, sb:cores=8, sb:logs_ttl=3, sb:ssd, sb:ttl=3] ------ [0/10] chunk ran 1 test (total:2.23s - test:2.21s) [crashed] TMirrorPartitionTest::ShouldNotMigrateIfCantLockMigrationRange [default-linux-x86_64-relwithdebinfo] (0.00s) Test crashed (return code: -6) See logs for more info Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/partition_nonrepl/ut/test-results/unittest/chunk0/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/partition_nonrepl/ut/test-results/unittest/chunk0/testing_out_stuff/TMirrorPartitionTest.ShouldNotMigrateIfCantLockMigrationRange.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/partition_nonrepl/ut/test-results/unittest/chunk0/testing_out_stuff/TMirrorPartitionTest.ShouldNotMigrateIfCantLockMigrationRange.out ------ [1/10] chunk ran 1 test (total:2.83s - test:2.81s) [crashed] TMirrorPartitionTest::ShouldNotMigrateIfCantLockMigrationRangeDirectCopy [default-linux-x86_64-relwithdebinfo] (0.00s) Test crashed (return code: -6) See logs for more info Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/partition_nonrepl/ut/test-results/unittest/chunk1/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/partition_nonrepl/ut/test-results/unittest/chunk1/testing_out_stuff/TMirrorPartitionTest.ShouldNotMigrateIfCantLockMigrationRangeDirectCopy.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/partition_nonrepl/ut/test-results/unittest/chunk1/testing_out_stuff/TMirrorPartitionTest.ShouldNotMigrateIfCantLockMigrationRangeDirectCopy.out ------ FAIL: 2 - CRASHED cloud/blockstore/libs/storage/partition_nonrepl/ut cloud/blockstore/libs/storage/volume/ut [size:medium] nchunks:10 [tags: sb:MULTISLOT, sb:cores=8, sb:logs_ttl=3, sb:ssd, sb:ttl=3] ------ [0/10] chunk ran 1 test (total:3.43s - test:3.42s) [crashed] TVolumeTest::ShouldForwardRequestsToNonreplicatedPartitionDuringMigrationWithFatalDeviceErrors [default-linux-x86_64-relwithdebinfo] (0.00s) Test crashed (return code: -6) See logs for more info Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut/test-results/unittest/chunk0/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut/test-results/unittest/chunk0/testing_out_stuff/TVolumeTest.ShouldForwardRequestsToNonreplicatedPartitionDuringMigrationWithFatalDeviceErrors.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut/test-results/unittest/chunk0/testing_out_stuff/TVolumeTest.ShouldForwardRequestsToNonreplicatedPartitionDuringMigrationWithFatalDeviceErrors.out ------ FAIL: 1 - CRASHED cloud/blockstore/libs/storage/volume/ut cloud/blockstore/libs/storage/volume/ut_linked [size:medium] nchunks:10 [tags: sb:MULTISLOT, sb:cores=8, sb:logs_ttl=3, sb:ssd, sb:ttl=3] ------ [0/10] chunk ran 1 test (total:2.23s - test:2.21s) [crashed] TLinkedVolumeTest::ShouldManageFollowerLink [default-linux-x86_64-relwithdebinfo] (0.00s) Test crashed (return code: -6) See logs for more info Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk0/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk0/testing_out_stuff/TLinkedVolumeTest.ShouldManageFollowerLink.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk0/testing_out_stuff/TLinkedVolumeTest.ShouldManageFollowerLink.out ------ [1/10] chunk ran 1 test (total:1.53s - test:1.51s) [crashed] TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_MIRROR [default-linux-x86_64-relwithdebinfo] (0.00s) Test crashed (return code: -6) See logs for more info Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk1/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk1/testing_out_stuff/TLinkedVolumeTest.ShouldPrepareFollowerVolume_SSD_MIRROR.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk1/testing_out_stuff/TLinkedVolumeTest.ShouldPrepareFollowerVolume_SSD_MIRROR.out ------ [2/10] chunk ran 1 test (total:4.13s - test:4.12s) [crashed] TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_NRD [default-linux-x86_64-relwithdebinfo] (0.00s) Test crashed (return code: -6) See logs for more info Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk2/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk2/testing_out_stuff/TLinkedVolumeTest.ShouldPrepareFollowerVolume_SSD_NRD.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk2/testing_out_stuff/TLinkedVolumeTest.ShouldPrepareFollowerVolume_SSD_NRD.out ------ [3/10] chunk ran 1 test (total:4.74s - test:4.72s) [crashed] TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_NRD_WithCheckpointAfter [default-linux-x86_64-relwithdebinfo] (0.00s) Test crashed (return code: -6) See logs for more info Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk3/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk3/testing_out_stuff/TLinkedVolumeTest.ShouldPrepareFollowerVolume_SSD_NRD_WithCheckpointAfter.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk3/testing_out_stuff/TLinkedVolumeTest.ShouldPrepareFollowerVolume_SSD_NRD_WithCheckpointAfter.out ------ [4/10] chunk ran 1 test (total:4.13s - test:4.11s) [crashed] TLinkedVolumeTest::ShouldPrepareFollowerVolume_SSD_NRD_WithReboot [default-linux-x86_64-relwithdebinfo] (0.00s) Test crashed (return code: -6) See logs for more info Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk4/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk4/testing_out_stuff/TLinkedVolumeTest.ShouldPrepareFollowerVolume_SSD_NRD_WithReboot.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk4/testing_out_stuff/TLinkedVolumeTest.ShouldPrepareFollowerVolume_SSD_NRD_WithReboot.out ------ [5/10] chunk ran 1 test (total:5.34s - test:5.32s) [crashed] TLinkedVolumeTest::ShouldRetryPropagateFollowerLink [default-linux-x86_64-relwithdebinfo] (0.00s) Test crashed (return code: -6) See logs for more info Logsdir: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk5/testing_out_stuff Stderr: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk5/testing_out_stuff/TLinkedVolumeTest.ShouldRetryPropagateFollowerLink.err Stdout: /home/github/tmp/out/cloud/blockstore/libs/storage/volume/ut_linked/test-results/unittest/chunk5/testing_out_stuff/TLinkedVolumeTest.ShouldRetryPropagateFollowerLink.out ------ FAIL: 6 - CRASHED cloud/blockstore/libs/storage/volume/ut_linked Total 7 suites: 7 - FAIL Total 23 tests: 2 - GOOD 12 - FAIL 9 - CRASHED Cache efficiency ratio is 99.77% (26904 of 26965). Local: 17 (0.06%), dist: 0 (0.00%), by dynamic uids: 0 (0.00%), avoided: 26887 (99.71%) Dist cache download: count=0, size=0 bytes, speed=0.0 bytes/s Disk usage for tools/sdk 2.11 GiB Additional disk space consumed for build cache 0 bytes Critical path: [347180 ms] [TL] [rnd-1663232029020079502 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/loadtest/local-nonrepl/embedded/py3test [started: 0 (1759407970077), finished: 347180 (1759408317257)] [ 211 ms] [TA] [rnd-ktav0a6fjdd4e8us]: $(BUILD_ROOT)/cloud/blockstore/tests/loadtest/local-nonrepl/embedded/test-results/py3test/{meta.json ... results_accumulator.log} [started: 347193 (1759408317270), finished: 347404 (1759408317481)] Time from start: 706334.3989257812 ms, time elapsed by graph 347391 ms, time diff 358943.39892578125 ms. The longest 10 tasks: [347180 ms] [TL] [rnd-1663232029020079502 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/loadtest/local-nonrepl/embedded/py3test [started: 1759407970077, finished: 1759408317257] [343364 ms] [TL] [rnd-14617193948500142823 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/loadtest/local-nonrepl/embedded/py3test [started: 1759407924711, finished: 1759408268075] [314418 ms] [TL] [rnd-14490033454431754831 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/py3test [started: 1759407936085, finished: 1759408250503] [296459 ms] [TL] [rnd-13863671228964097581 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/loadtest/local-nonrepl/dedicated/py3test [started: 1759407911427, finished: 1759408207886] [272295 ms] [TL] [rnd-18150132824000672441 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/loadtest/local-mirror/py3test [started: 1759407791053, finished: 1759408063348] [167534 ms] [TL] [rnd-8070191919005414658 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/loadtest/local-mirror/py3test [started: 1759407796625, finished: 1759407964159] [166149 ms] [TL] [rnd-3383242748624705425 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/loadtest/local-mirror/py3test [started: 1759407639214, finished: 1759407805363] [161997 ms] [TL] [rnd-5645631076593315195 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/loadtest/local-mirror/py3test [started: 1759407635294, finished: 1759407797291] [161079 ms] [TL] [rnd-17182316562782115545 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/loadtest/local-mirror-lagging/py3test [started: 1759407635296, finished: 1759407796375] [158674 ms] [TL] [rnd-6287335111165740777 default-linux-x86_64 relwithdebinfo]: cloud/blockstore/tests/loadtest/local-mirror/py3test [started: 1759407635297, finished: 1759407793971] Total time by type: [2930428 ms] [TL] [count: 24, ave time 122101.17 msec] [ 47932 ms] [TM] [count: 30, ave time 1597.73 msec] [ 14962 ms] [prepare:get from local cache] [count: 17, ave time 880.12 msec] [ 2458 ms] [TA] [count: 7, ave time 351.14 msec] [ 589 ms] [prepare:clean] [count: 3, ave time 196.33 msec] [ 216 ms] [prepare:tools] [count: 14, ave time 15.43 msec] [ 161 ms] [prepare:AC] [count: 2, ave time 80.50 msec] Total tasks times: Total failed tasks time - 0 ms (0.00%) Total tests tasks time - 2980818 ms (100.00%) Total run tasks time - 2980818 ms Configure time - 12.7 s Statistics overhead 791 ms Info: Dump junit report to /home/github/tmp/junit.xml.2 Failed + echo 10