拝啓山田様
お久しぶりです。1度お会いしたきりなので先生は覚えていないでしょうが、私は覚えております。
山田先生は世の中のソフトウェアリカバリーの手段を調査し数十年（？）される中で、今でもReboot-Based Recoveyの概念を学術的に提唱しつづけておられます。
過去のご登壇されていた発表の内容は、それはもう素晴らしいものでした。
ですが、今回は山田先生に謝らければなりません。
↓ 再起動後 ↓
ストレージプールが無くなったとかいうエラーが出ていて
再起動したら復活するのはいかんでしょ？！？！？！？！？！？！？（大声）
ストレージプールが無くなるだけでなく、なぜか元に戻る怪現象。
今年は肝試しにチャレンジするつもりじゃなかったんだけど、どうしてこんなことになるかな～。
ごめんなさい、山田先生。私は再起動で治るような計算機を信用できないんです。ここは再起動でヨシ！としちゃイカンでしょ！！！
どうしてこうなってしまったのか、じっくり調査していきたいと思います。
ソフトを直すのか、買い替えなのか対策を打てれば御の字です。
現在の構成
今回エラーが出たNASについてですが、SynologyのOSが入ったNASにストレージ用にHDDが2台、それからHDDは交換前提と考慮してOS用にSSDを1台搭載しています。
bash-4.3# fdisk -l
Disk /dev/sda: 238.5 GiB, 256060514304 bytes, 500118192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x887d857e
Device Boot Start End Sectors Size Id Type
/dev/sda1 2048 4982527 4980480 2.4G fd Linux raid autodetect
/dev/sda2 4982528 9176831 4194304 2G fd Linux raid autodetect
/dev/sda3 9437184 499913375 490476192 233.9G fd Linux raid autodetect
Disk /dev/sdb: 10.9 TiB, 12000138625024 bytes, 23437770752 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 3488A54F-BCF5-42DB-B53B-F131BF556C05
Device Start End Sectors Size Type
/dev/sdb1 2048 4982527 4980480 2.4G Linux RAID
/dev/sdb2 4982528 9176831 4194304 2G Linux RAID
/dev/sdb5 9453280 7813830239 7804376960 3.6T Linux RAID
/dev/sdb6 7813846336 11720838239 3906991904 1.8T Linux RAID
/dev/sdb7 11720854336 15627846239 3906991904 1.8T Linux RAID
/dev/sdb8 15627862336 23437556799 7809694464 3.7T Linux RAID
Disk /dev/sdc: 12.8 TiB, 14000519643136 bytes, 27344764928 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 3689E169-ED17-4AB0-82A7-E447EC9DD52B
Device Start End Sectors Size Type
/dev/sdc1 2048 4982527 4980480 2.4G Linux RAID
/dev/sdc2 4982528 9176831 4194304 2G Linux RAID
/dev/sdc5 9453280 7813830239 7804376960 3.6T Linux RAID
/dev/sdc6 7813846336 11720838239 3906991904 1.8T Linux RAID
/dev/sdc7 11720854336 15627846239 3906991904 1.8T Linux RAID
/dev/sdc8 15627862336 23437556799 7809694464 3.7T Linux RAID
...（ややこしいので中略、後で続きを張ります）
このように3台の物理デバイスが接続されています。
パーティションがいくつかに分かれているのはＯＳ領域と、ストレージとしてのパーティションが分かれているためです。
なんなら、ストレージとしてのパーティションが4つぐらいに分割されていますが、
これはストレージプールとして1つとしてＮＡＳからは扱われているので図にまとめると下記のな１つのLinuxボリュームがあると考えてください。
図にまとめるとこんな感じです。
NASの機能を使って、随時HDDを足していっていたため、細切れになっているパーティションがいくらかある状態ですが、そこはご容赦いただければと思います。
すし on X: "これは、僕が片方だけのHDDを 容量アップするように取り替えていっている図 https://t.co/3NtQCrJvSm" / X
実際使ってる空き容量とかそのあたりもこんな感じです。
bash-4.3# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md0 2.3G 999M 1.2G 46% /
none 1.9G 0 1.9G 0% /dev
/tmp 1.9G 836K 1.9G 1% /tmp
/run 1.9G 3.3M 1.9G 1% /run
/dev/shm 1.9G 4.0K 1.9G 1% /dev/shm
none 4.0K 0 4.0K 0% /sys/fs/cgroup
cgmfs 100K 0 100K 0% /run/cgmanager/fs
/dev/vg1000/lv 11T 7.5T 3.0T 72% /volume1
shm 64M 0 64M 0% /volume1/@docker/containers/bde3d2b79705851efb7bfbec8f5a14685b4b9d0ee97d522f50e9256ee4dd374c/mounts/shm
shm 64M 0 64M 0% /volume1/@docker/containers/4ed7c3bcedfa3a395b37f46037c66372233384b1956aaf8797fea582e426496a/mounts/shm
bash-4.3#
bash-4.3# lvdisplay
--- Logical volume ---
LV Path /dev/vg1000/lv
LV Name lv
VG Name vg1000
LV UUID mBcNpR-kA8A-45jy-huLm-IEI7-KK2x-1GONx3
LV Write Access read/write
LV Creation host, time ,
LV Status available
# open 1
LV Size 10.91 TiB
Current LE 2859867
Segments 4
Allocation inherit
Read ahead sectors auto
- currently set to 4096
Block device 253:0
bash-4.3#
さっき中略したfdiskについてもこんな感じです。
bash-4.3# fdisk -l
...（さっきの続きから）...
Disk /dev/md0: 2.4 GiB, 2549940224 bytes, 4980352 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/md1: 2 GiB, 2147418112 bytes, 4194176 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
GPT PMBR size mismatch (102399 != 30965759) will be corrected by w(rite).
Disk /dev/synoboot: 14.8 GiB, 15854469120 bytes, 30965760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: C94E55EA-A4D2-4E78-9D73-46CBAE7A03EF
Device Start End Sectors Size Type
/dev/synoboot1 2048 32767 30720 15M EFI System
/dev/synoboot2 32768 94207 61440 30M Linux filesystem
/dev/synoboot3 94208 102366 8159 4M BIOS boot
Disk /dev/zram0: 1.1 GiB, 1210056704 bytes, 295424 sectors
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/zram1: 1.1 GiB, 1210056704 bytes, 295424 sectors
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/md4: 1.8 TiB, 2000378789888 bytes, 3906989824 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/md5: 3.7 TiB, 3998562516992 bytes, 7809692416 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/md2: 3.6 TiB, 3995839954944 bytes, 7804374912 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/md3: 1.8 TiB, 2000378789888 bytes, 3906989824 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/mapper/vg1000-lv: 10.9 TiB, 11995151597568 bytes, 23428030464 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
bash-4.3#
何が起きたのか
気が付いたのは9/7の23時のことでした、たまたまnasにログインするとまぁそれは過激な文章が画面上に出ていて、再起動したら直っていたというのが状況です。
その場ではHDDのエラーログが無かったことぐらいしか確認できませんでしたが、あらためてカーネルログなど漁っていきます。
HDDのエラーログを真っ先に見たのは、HDDの故障以外にも
こいつの何かしらの影響を考えてでしたが、今回はどうも違うようです。
2023-09-02T20:29:34+09:00 FStation kernel: [ 216.521506] md: md1: resync done.
2023-09-02T20:29:34+09:00 FStation kernel: [ 216.529229] md: md1: current auto_remap = 0
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.674156] scsi 0:0:0:0: Direct-Access SanDisk SD8SNAT256G1122 Z233 PQ: 0 ANSI: 5
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.674655] sd 0:0:0:0: [sda] 500118192 512-byte logical blocks: (256 GB/238 GiB)
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.674678] sd 0:0:0:0: [sda] Write Protect is off
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.674687] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.680802] sd 0:0:0:0: [sda] Attached SCSI disk
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.681815] ata2.00: supports DRM functions and may not be fully accessible
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.685778] scsi 1:0:0:0: Direct-Access WDC WD120EDAZ-11F3RA0 81.0 PQ: 0 ANSI: 5
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.688470] sd 1:0:0:0: [sdb] 23437770752 512-byte logical blocks: (12.0 TB/10.9 TiB)
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.688472] sd 1:0:0:0: [sdb] 4096-byte physical blocks
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.688488] sd 1:0:0:0: [sdb] Write Protect is off
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.688496] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.693380] scsi 2:0:0:0: Direct-Access WDC WD140EDGZ-11B1PA0 85.0 PQ: 0 ANSI: 5
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.697450] sd 2:0:0:0: [sdc] 27344764928 512-byte logical blocks: (14.0 TB/12.7 TiB)
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.697452] sd 2:0:0:0: [sdc] 4096-byte physical blocks
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.697468] sd 2:0:0:0: [sdc] Write Protect is off
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.697477] sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.760948] sd 2:0:0:0: [sdc] Attached SCSI disk
2023-09-07T23:17:23+09:00 FStation kernel: [ 3.770141] sd 1:0:0:0: [sdb] Attached SCSI disk
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.169970] md: invalid raid superblock magic on sda3
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.175298] md: sda3 does not have a valid v0.90 superblock, not importing!
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.217748] md: invalid raid superblock magic on sdb5
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.222977] md: sdb5 does not have a valid v0.90 superblock, not importing!
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.232424] md: invalid raid superblock magic on sdb6
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.237594] md: sdb6 does not have a valid v0.90 superblock, not importing!
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.246860] md: invalid raid superblock magic on sdb7
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.252093] md: sdb7 does not have a valid v0.90 superblock, not importing!
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.268573] md: invalid raid superblock magic on sdb8
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.273782] md: sdb8 does not have a valid v0.90 superblock, not importing!
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.317760] md: invalid raid superblock magic on sdc5
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.322958] md: sdc5 does not have a valid v0.90 superblock, not importing!
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.335250] md: invalid raid superblock magic on sdc6
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.340465] md: sdc6 does not have a valid v0.90 superblock, not importing!
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.348447] md: invalid raid superblock magic on sdc7
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.353731] md: sdc7 does not have a valid v0.90 superblock, not importing!
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.372931] md: invalid raid superblock magic on sdc8
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.378155] md: sdc8 does not have a valid v0.90 superblock, not importing!
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.378162] md: sda2 has different UUID to sda1
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.378165] md: sdb2 has different UUID to sda1
2023-09-07T23:17:23+09:00 FStation kernel: [ 4.378168] md: sdc2 has different UUID to sda1
2023-09-07T23:17:23+09:00 FStation kernel: [ 8.316157] scsi 8:0:0:0: Direct-Access Patriot Memory PMAP PQ: 0 ANSI: 6
2023-09-07T23:17:23+09:00 FStation kernel: [ 8.843270] sd 8:0:0:0: [synoboot] 30965760 512-byte logical blocks: (15.8 GB/14.7 GiB)
2023-09-07T23:17:23+09:00 FStation kernel: [ 8.844348] sd 8:0:0:0: [synoboot] Write Protect is off
2023-09-07T23:17:23+09:00 FStation kernel: [ 8.845404] sd 8:0:0:0: [synoboot] No Caching mode page found
2023-09-07T23:17:23+09:00 FStation kernel: [ 8.851313] sd 8:0:0:0: [synoboot] Assuming drive cache: write through
2023-09-07T23:17:23+09:00 FStation kernel: [ 8.861100] sd 8:0:0:0: [synoboot] No Caching mode page found
2023-09-07T23:17:23+09:00 FStation kernel: [ 8.866943] sd 8:0:0:0: [synoboot] Assuming drive cache: write through
2023-09-07T23:17:23+09:00 FStation kernel: [ 8.884445] sd 8:0:0:0: [synoboot] No Caching mode page found
2023-09-07T23:17:23+09:00 FStation kernel: [ 8.890341] sd 8:0:0:0: [synoboot] Assuming drive cache: write through
2023-09-07T23:17:23+09:00 FStation kernel: [ 8.897012] sd 8:0:0:0: [synoboot] Attached SCSI removable disk
2023-09-07T23:17:23+09:00 FStation kernel: [ 11.950709] sd 0:0:0:0: Attached scsi generic sg0 type 0
2023-09-07T23:17:23+09:00 FStation kernel: [ 11.950741] sd 1:0:0:0: Attached scsi generic sg1 type 0
2023-09-07T23:17:23+09:00 FStation kernel: [ 11.950772] sd 2:0:0:0: Attached scsi generic sg2 type 0
2023-09-07T23:17:23+09:00 FStation kernel: [ 11.950801] sd 8:0:0:0: Attached scsi generic sg3 type 0
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1345 ##### ACTION:add
2023-09-07T23:15:57+09:00 FStation hotplugd: DEVNAME:sda
2023-09-07T23:15:57+09:00 FStation hotplugd: DEVGUID:162075421401 0
2023-09-07T23:15:57+09:00 FStation hotplugd: DEVPATH:sda
2023-09-07T23:15:57+09:00 FStation hotplugd: SUBSYSTEM:block
2023-09-07T23:15:57+09:00 FStation hotplugd: PHYSDEVPATH:/devices/pci0000:00/0000:00:17.0/ata1/host0/target0:0:0/0:0:0:0
2023-09-07T23:15:57+09:00 FStation hotplugd: scemd_connector/scemd_connector.c:134 Fail to sendto() for scemd connector client.
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1444 ==== SATA disk [sda] hotswap [add] ====
2023-09-07T23:15:57+09:00 FStation hotplugd: disk/disk_config_single.c:234 apply /usr/syno/bin/DiskApmSet.sh 255 /dev/sda 1>/dev/null 2>&1
2023-09-07T23:15:57+09:00 FStation hotplugd: disk/disk_config_single.c:234 apply /usr/syno/bin/syno_disk_ctl --ncq-on /dev/sda 1>/dev/null 2>&1
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1465 ==== SATA disk [sda] Model: [SD8SNAT256G1122 ] ====
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1466 ==== SATA disk [sda] Serial number: [162075421401] ====
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1467 ==== SATA disk [sda] Firmware version: [Z2333000] ====
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1345 ##### ACTION:add
2023-09-07T23:15:57+09:00 FStation hotplugd: DEVNAME:sdb
2023-09-07T23:15:57+09:00 FStation hotplugd: DEVGUID:8CJS01UF 0
2023-09-07T23:15:57+09:00 FStation hotplugd: DEVPATH:sdb
2023-09-07T23:15:57+09:00 FStation hotplugd: SUBSYSTEM:block
2023-09-07T23:15:57+09:00 FStation hotplugd: PHYSDEVPATH:/devices/pci0000:00/0000:00:17.0/ata2/host1/target1:0:0/1:0:0:0
2023-09-07T23:15:57+09:00 FStation hotplugd: scemd_connector/scemd_connector.c:134 Fail to sendto() for scemd connector client.
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1444 ==== SATA disk [sdb] hotswap [add] ====
2023-09-07T23:15:57+09:00 FStation kernel: [ 35.164842] ata2.00: supports DRM functions and may not be fully accessible
2023-09-07T23:15:57+09:00 FStation kernel: [ 35.172228] ata2.00: supports DRM functions and may not be fully accessible
2023-09-07T23:15:57+09:00 FStation hotplugd: disk/disk_config_single.c:234 apply /usr/syno/bin/DiskApmSet.sh 255 /dev/sdb 1>/dev/null 2>&1
2023-09-07T23:15:57+09:00 FStation hotplugd: disk/disk_config_single.c:234 apply /usr/syno/bin/syno_disk_ctl --ncq-on /dev/sdb 1>/dev/null 2>&1
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1465 ==== SATA disk [sdb] Model: [WD120EDAZ-11F3RA0 ] ====
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1466 ==== SATA disk [sdb] Serial number: [8CJS01UF] ====
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1467 ==== SATA disk [sdb] Firmware version: [81.00A81] ====
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1345 ##### ACTION:add
2023-09-07T23:15:57+09:00 FStation hotplugd: DEVNAME:sdc
2023-09-07T23:15:57+09:00 FStation hotplugd: DEVGUID:Y6GXALGC 0
2023-09-07T23:15:57+09:00 FStation hotplugd: DEVPATH:sdc
2023-09-07T23:15:57+09:00 FStation hotplugd: SUBSYSTEM:block
2023-09-07T23:15:57+09:00 FStation hotplugd: PHYSDEVPATH:/devices/pci0000:00/0000:00:17.0/ata3/host2/target2:0:0/2:0:0:0
2023-09-07T23:15:57+09:00 FStation hotplugd: scemd_connector/scemd_connector.c:134 Fail to sendto() for scemd connector client.
2023-09-07T23:15:57+09:00 FStation hotplugd: hotplugd.c:1444 ==== SATA disk [sdc] hotswap [add] ====
2023-09-07T23:15:57+09:00 FStation kernel: [ 35.484400] ata3.00: supports DRM functions and may not be fully accessible
2023-09-07T23:15:57+09:00 FStation kernel: [ 35.490237] ata3.00: supports DRM functions and may not be fully accessible
2023-09-07T23:15:58+09:00 FStation hotplugd: disk/disk_config_single.c:234 apply /usr/syno/bin/DiskApmSet.sh 255 /dev/sdc 1>/dev/null 2>&1
2023-09-07T23:15:58+09:00 FStation hotplugd: disk/disk_config_single.c:234 apply /usr/syno/bin/syno_disk_ctl --ncq-on /dev/sdc 1>/dev/null 2>&1
2023-09-07T23:15:58+09:00 FStation hotplugd: hotplugd.c:1465 ==== SATA disk [sdc] Model: [WD140EDGZ-11B1PA0 ] ====
2023-09-07T23:15:58+09:00 FStation hotplugd: hotplugd.c:1466 ==== SATA disk [sdc] Serial number: [Y6GXALGC] ====
2023-09-07T23:15:58+09:00 FStation hotplugd: hotplugd.c:1467 ==== SATA disk [sdc] Firmware version: [85.00A85] ====
2023-09-07T23:16:47+09:00 FStation kernel: [ 85.258396] ata2.00: supports DRM functions and may not be fully accessible
2023-09-07T23:16:47+09:00 FStation kernel: [ 85.264651] ata2.00: supports DRM functions and may not be fully accessible
2023-09-07T23:16:47+09:00 FStation kernel: [ 85.338789] ata3.00: supports DRM functions and may not be fully accessible
2023-09-07T23:16:47+09:00 FStation kernel: [ 85.346006] ata3.00: supports DRM functions and may not be fully accessible
23時に再起動したときには「md: invalid raid superblock magic on sda3」とか出ていますが、これは昔からでていたログで特に問題はありません。（少なくとも今回の事象では）
ほかのログも見てみると、どうも9/5の 5:14ぐらいに書き込みができなくなっているログが出始めて9/7の23時に再起動するまでずっとエラーが出続けているログが見つかります。
これは普段では起きていないログなのでおそらく、この5:14に何かしらのトラブルが起きたものと思っています。
/var/log/messages
2023-09-04T20:28:38+09:00 FStation [173147.854629] init: tty main process ended, respawning
2023-09-04T20:28:47+09:00 FStation kernel: [173157.178434] FAT-fs (synoboot2): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
2023-09-04T20:28:47+09:00 FStation synoddsmd: utils.cpp:35 Fail to get synology account
2023-09-04T20:28:47+09:00 FStation synoddsmd: user.cpp:129 get account info fail [100]
2023-09-04T20:28:47+09:00 FStation synoddsmd: synoddsm-hostd.cpp:227 Fail to get DDSM licenses, errCode: 0x100
2023-09-04T20:28:48+09:00 FStation [173157.868358] init: tty main process (25102) terminated with status 1
2023-09-04T20:28:48+09:00 FStation [173157.868380] init: tty main process ended, respawning
.....
2023-09-05T05:14:57+09:00 FStation [204761.681357] init: tty main process (5939) terminated with status 1
2023-09-05T05:14:57+09:00 FStation [204761.681380] init: tty main process ended, respawning
2023-09-05T05:15:05+09:00 FStation kernel: [204769.826143] uart write failed
2023-09-05T05:15:06+09:00 FStation kernel: [204548.072049] [@72]
2023-09-05T05:15:07+09:00 FStation kernel: [204771.038304] uart write failed
2023-09-05T05:15:07+09:00 FStation kernel: [204549.282880] [@72]
2023-09-05T05:15:07+09:00 FStation [204771.695214] init: tty main process (5950) terminated with status 1
2023-09-05T05:15:07+09:00 FStation [204771.695238] init: tty main process ended, respawning
2023-09-05T05:15:08+09:00 FStation kernel: [204772.433770] uart write failed
2023-09-05T05:15:08+09:00 FStation kernel: [204551.676803] [@72]
2023-09-05T05:15:09+09:00 FStation kernel: [204773.646662] uart write failed
2023-09-05T05:15:09+09:00 FStation kernel: [204552.888402] [@72]
2023-09-05T05:15:11+09:00 FStation kernel: [204775.041561] uart write failed
2023-09-05T05:15:11+09:00 FStation kernel: [204553.281778] [@72]
2023-09-05T05:15:12+09:00 FStation kernel: [204554.283394] [@1000]
2023-09-05T05:15:12+09:00 FStation kernel: [204776.254951] uart write failed
2023-09-05T05:15:12+09:00 FStation kernel: [204554.493864] [@72]
2023-09-05T05:15:13+09:00 FStation synoaudiod: audiolib/json_util.cpp:61 Failed to fopen [/tmp/AudioStation/player.list.json], [No such file or directory]
2023-09-05T05:15:13+09:00 FStation kernel: [204555.207815] [@103]
2023-09-05T05:15:13+09:00 FStation kernel: [204555.496622] [@1000]
2023-09-05T05:15:13+09:00 FStation kernel: [204555.515651] [@103]
2023-09-05T05:15:14+09:00 FStation synoaudiod: audiolib/json_util.cpp:61 Failed to fopen [/tmp/AudioStation/player.list.json], [No such file or directory]
2023-09-05T05:15:15+09:00 FStation synoaudiod: audiolib/json_util.cpp:61 Failed to fopen [/tmp/AudioStation/player.list.json], [No such file or directory]
2023-09-05T05:15:16+09:00 FStation synoaudiod: audiolib/json_util.cpp:61 Failed to fopen [/tmp/AudioStation/player.list.json], [No such file or directory]
2023-09-05T05:15:17+09:00 FStation synoaudiod: SYSTEM: Last message 'audiolib/json_util.c' repeated 3 times, suppressed by syslog-ng on FStation
2023-09-05T05:15:17+09:00 FStation [204781.710584] init: tty main process (6213) terminated with status 1
2023-09-05T05:15:17+09:00 FStation [204781.710609] init: tty main process ended, respawning
2023-09-05T05:15:18+09:00 FStation synoaudiod: audiolib/json_util.cpp:61 Failed to fopen [/tmp/AudioStation/player.list.json], [No such file or directory]
2023-09-05T05:15:19+09:00 FStation synoaudiod: audiolib/json_util.cpp:61 Failed to fopen [/tmp/AudioStation/player.list.json], [No such file or directory]
2023-09-05T05:15:20+09:00 FStation synoaudiod: audiolib/json_util.cpp:61 Failed to fopen [/tmp/AudioStation/player.list.json], [No such file or directory]
2023-09-05T05:15:27+09:00 FStation synoaudiod: SYSTEM: Last message 'audiolib/json_util.c' repeated 9 times, suppressed by syslog-ng on FStation
2023-09-05T05:15:27+09:00 FStation [204791.725219] init: tty main process (6226) terminated with status 1
一つ目の疑問
とここで1つの疑問が浮かびます。/dev/ストレージプールが認識されていない状況で、ログは欠損していませんでした。
ということは、OSや/var/logは大丈夫で、/volumeにマウントされている、/dev/vg1000/lvだけロストしていたということでしょうか。
読んでいる人はどういうこと？と思うかもしれませんが、構成を再度確認にすると、
OSやログが保存されているストレージプールは/dev/md0で分かれています。
※さっきのまとめ画像の通りです。
/含め /var/logについては、SSDも含めて構成されているので、よっっっっっぽどのことが無い限り影響がありません。
/volumeは死んでいても、きちんとログは保存されておりました。
bash-4.3# mdadm --detail /dev/md0
/dev/md0:
Version : 0.90
Creation Time : Sat Jan 1 09:00:05 2000
Raid Level : raid1
Array Size : 2490176 (2.37 GiB 2.55 GB)
Used Dev Size : 2490176 (2.37 GiB 2.55 GB)
Raid Devices : 12
Total Devices : 3
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Sat Sep 9 15:00:35 2023
State : clean, degraded
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
UUID : 1ac6abb2:1b359f68:1842b64a:93cf1d09 (local to host FStation)
Events : 0.4815281
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 33 1 active sync /dev/sdc1
2 8 17 2 active sync /dev/sdb1
- 0 0 3 removed
- 0 0 4 removed
- 0 0 5 removed
- 0 0 6 removed
- 0 0 7 removed
- 0 0 8 removed
- 0 0 9 removed
- 0 0 10 removed
- 0 0 11 removed
bash-4.3#
2つ目の疑問：どうして再起動で治ったのか
わかりません。
作業をし始めたあと、/var/logのデータをすべてチェックして、該当の時間のエラーログを見つけられず。原因は不明です。
完全に不完全燃焼です。Synologyさんの動作ログいろいろ情報が足りてなくて難しい…。
というわけで状況証拠もそろわず、この状況だけから推察すると、
diskのエラーや、カーネルでのエラーが一切見つからなかったところから、おそらくボリュームが何らかのタイミングで刺さったか（しかしなぜカーネルエラーが残残ってない）。
あるいは、Synologyのパッケージ関係が止まって、壊れたように検知してしまったんでしょうか？
Storage Pool and Volume missing !!
とか海外では書かれているが、これはHWなのか、それともSWなのか…。
つまり、今回の事象では…
ということなのかもしれません。
山田先生、これからも尊敬しております。
