WARNING: FS3: 608: VMFS volume DATASTORE_LABEL/DATASTORE_UUID on naa.#################################:1 has been detected corrupted
VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x
If this has occurred:
1. Primary partition table may be overwritten:
partedUtil getptbl /vmfs/devices/disks/naa.################################
Error: The primary GPT table is corrupt, but the backup appears OK, so that will be used. Fix primary table ? diskPath (/dev/disks/naa.################################) diskSize (16777217280) AlternateLBA (1) LastUsableLBA (16777217246)
gpt
1044333 255 63 16777217280
1 2048 16777215999 AA31E02A400F11DB9590000C2911D1B8 vmfs 0
2. hexdump of the datastore shows that the vmfs header (including usually zeroed buffer blocks) is overwritten with random data:
hexdump -C /vmfs/devices/disks/naa.################################ | more
00000000 58 dd f3 79 2b 9e 6a 78 ef 97 16 ae f7 93 d8 9d |X..y+.jx........|
00000010 a7 ed 33 dc 6c d8 f6 ca 47 d9 24 bc 68 65 e8 95 |..3.l...G.$.he..|
00000020 10 53 07 0c 04 4b ae ee 48 4a d8 8d 31 2e 89 b2 |.S...K..HJ..1...|
00000030 62 a4 81 20 ec be 32 e2 d1 f2 85 29 d1 96 c6 39 |b.. ..2....)...9|
00000040 72 4a 43 e3 7c d2 4b 55 29 18 c2 26 49 28 a2 2a |rJC.|.KU)..&I(.*|
00000050 15 7f 61 cc 0c ae 8f 9a 27 f9 26 88 b0 a7 04 bb |..a.....'.&.....|
00000060 ee cf 4c 30 5f e2 bd 0e 3d b5 05 8c 4d d2 36 f2 |..L0_...=...M.6.|
00000070 07 9f 37 b5 9f 21 0d 48 bb 57 02 b5 be b0 b5 47 |..7..!.H.W.....G|
00000080 6c 52 72 1b 3a 62 3f eb 0a 26 92 4c 73 f4 d9 31 |lRr.:b?..&.Ls..1|
00000090 4c 6b cd 61 51 d5 3b ce 42 35 75 a7 0f 83 0c 39 |Lk.aQ.;.B5u....9|
000000a0 f2 a7 64 0a ee df 7c dc b9 e3 df 0e 5e d2 ae ce |..d...|.....^...|
000000b0 94 e3 80 90 be 51 4c 50 df 84 09 56 d6 be cc 32 |.....QLP...V...2|
000000c0 4a 3d 54 c7 00 9e 7c ed ee 4f 26 6f 8d 37 b2 46 |J=T...|..O&o.7.F|
000000d0 ef 6a 8e d0 66 f9 5a 18 15 e1 48 87 c2 a5 d9 24 |.j..f.Z...H....$|
000000e0 47 62 20 37 18 7d 47 95 0b bf bc 82 13 29 c2 93 |Gb 7.}G......)..|
000000f0 ec fa ad 62 08 fa bd 05 48 0f 26 6f 47 f9 1e d3 |...b....H.&oG...|
00000100 61 fe 52 86 52 c4 f2 b4 7e c1 ca fd 3d 03 f1 46 |a.R.R...~...=..F|
00000110 b4 ca 8d 3c 47 e7 5c f9 93 a5 ba d9 10 fa 69 61 |...<G.\.......ia|
00000120 94 ef 3c d2 7d 56 f9 ce f8 ab 4d 81 2a 0a 8b 30 |..<.}V....M.*..0|
00000130 0c 7f 06 06 65 af d6 84 90 42 34 99 88 5a dc 11 |....e....B4..Z..|
00000140 c8 3f fc f7 b4 7d 0b c3 e3 c5 20 f0 1c 12 4e 03 |.?...}.... ...N.|
00000150 99 47 e9 8b 43 b2 38 90 73 f6 28 2f 80 b7 0a 00 |.G..C.8.s.(/....|
00000160 8b 51 3a 97 ca 3e e3 fc 04 c5 f0 d7 27 a1 35 9e |.Q:..>......'.5.|
00000170 58 c5 20 91 d4 8e 96 2f 52 d8 fe b3 a6 65 62 2d |X. ..../R....eb-|
00000180 43 c8 0d bc 0f 74 eb 3e 98 b1 18 0f 8d e4 81 36 |C....t.>.......6|
00000190 80 fa 96 d4 9b 0e ab cd 43 ff af 4f 05 c8 61 e0 |........C..O..a.|
000001a0 cd 77 63 90 0b 0c 7e 38 3f e0 f7 24 f6 3c b8 d8 |.wc...~8?..$.<..|
000001b0 5b de 55 17 b6 38 4b c4 ba 1f d2 c9 f8 ae c3 15 |[.U..8K.........|
000001c0 70 9a ad 7d 73 26 7e 3b c7 e7 62 cb 01 88 96 49 |p..}s&~;..b....I|
000001d0 48 e6 d9 0b 2c 97 fe 5f ee 07 eb fd 51 74 46 3e |H...,.._....QtF>|
000001e0 a7 99 04 5e 72 9d b9 5d 5e 25 72 e0 94 32 99 a6 |...^r..]^%r..2..|
000001f0 04 bf 70 37 d6 77 d7 45 1d be 37 06 a2 3a bd d0 |..p7.w.E..7..:..|
00000200 8f 4d f8 74 4b 2a 96 dd c6 22 42 eb e8 da e6 24 |.M.tK*..."B....$|
00000210 ca e7 09 e3 3e 7d 9e 1e 1c 5f 50 e9 e4 d6 20 fe |....>}..._P... .|
00000220 fc 7d 61 23 54 64 4d 01 6b 20 2c d8 80 7f a3 41 |.}a#TdM.k ,....A|
00000230 76 97 6d 2d 46 a4 ed d4 95 bc 54 6c a2 9a aa e0 |v.m-F.....Tl....|
00000240 97 bc 0a 22 97 f0 36 dc c2 7a 1d 91 2c 3c ef a7 |..."..6..z..,<..|
00000250 0e b0 10 a2 6b 5b fa c5 e3 6c 0c 63 0b c2 2d 91 |....k[...l.c..-.|
00000260 60 c5 ea 3e d7 32 e3 f9 ec b3 c4 bd be f8 b8 79 |`..>.2.........y|
00000270 36 d8 54 87 0b 9d 45 fe a6 e5 d7 e8 23 80 3b b2 |6.T...E.....#.;.|
00000280 fd 6e 79 73 2e d4 64 29 bf fc 53 70 ff d5 d5 17 |.nys..d)..Sp....|
00000290 18 1e aa c1 37 6f 90 d0 3b 8a da b7 00 9c af 1b |....7o..;.......|
000002a0 93 ed 75 3b ee ff 38 1d 4b ca b7 64 b9 51 5e 7d |..u;..8.K..d.Q^}|
000002b0 22 b5 a5 fa c1 57 21 5e 67 4b fa bb 24 48 83 8e |"....W!^gK..$H..|
000002c0 68 ca b2 08 98 18 17 56 c4 61 e6 8b 1b 07 ea a8 |h......V.a......|
000002d0 71 30 1f 12 d9 67 b1 75 0f e9 84 8c ed af 44 2e |q0...g.u......D.|
000002e0 47 0b ee 04 b7 0a b8 71 73 16 19 d0 9a 24 47 be |G......qs....$G.|
000002f0 9e 1b 06 50 a6 34 52 e2 bf 10 2b 2c 95 9a d3 38 |...P.4R...+,...8|
00000300 7b 37 4c 50 a6 5d a1 da f8 bd b3 3c 49 1b 0f 96 |{7LP.].....<I...|
00000310 7f c9 90 af 46 00 05 10 19 3b 1a 99 9a 01 62 8d |....F....;....b.|
00000320 eb f4 d4 02 f5 67 0e e0 f2 1a 4e 4c 7c 89 fc fb |.....g....NL|...|
00000330 ff 29 f0 1f 30 98 a4 48 83 cc a0 74 e6 3b 65 c0 |.)..0..H...t.;e.|
3. The random data will typically have a repeating pattern, e.g. every 4 MB, as seen in the example below at offsets 000000c0, 004000c0, 008000c0.
ee 4f 26 6f 8d 37 b2 46
000000c0 4a 3d 54 c7 00 9e 7c ed ee 4f 26 6f 8d 37 b2 46 |J=T...|..O&o.7.F|
000000d0 ef 6a 8e d0 66 f9 5a 18 15 e1 48 87 c2 a5 d9 24 |.j..f.Z...H....$|
000000e0 47 62 20 37 18 7d 47 95 0b bf bc 82 13 29 c2 93 |Gb 7.}G......)..|
000000f0 ec fa ad 62 08 fa bd 05 48 0f 26 6f 47 f9 1e d3 |...b....H.&oG...|
00000100 61 fe 52 86 52 c4 f2 b4 7e c1 ca fd 3d 03 f1 46 |a.R.R...~...=..F|
00000110 b4 ca 8d 3c 47 e7 5c f9 93 a5 ba d9 10 fa 69 61 |...<G.\.......ia|
00000120 94 ef 3c d2 7d 56 f9 ce f8 ab 4d 81 2a 0a 8b 30 |..<.}V....M.*..0|
00000130 0c 7f 06 06 65 af d6 84 90 42 34 99 88 5a dc 11 |....e....B4..Z..|
00000140 c8 3f fc f7 b4 7d 0b c3 e3 c5 20 f0 1c 12 4e 03 |.?...}.... ...N.|
00000150 99 47 e9 8b 43 b2 38 90 73 f6 28 2f 80 b7 0a 00 |.G..C.8.s.(/....|
00000160 8b 51 3a 97 ca 3e e3 fc 04 c5 f0 d7 27 a1 35 9e |.Q:..>......'.5.|
00000170 58 c5 20 91 d4 8e 96 2f 52 d8 fe b3 a6 65 62 2d |X. ..../R....eb-|
00000180 43 c8 0d bc 0f 74 eb 3e 98 b1 18 0f 8d e4 81 36 |C....t.>.......6|
00000190 80 fa 96 d4 9b 0e ab cd 43 ff af 4f 05 c8 61 e0 |........C..O..a.|
000001a0 cd 77 63 90 0b 0c 7e 38 3f e0 f7 24 f6 3c b8 d8 |.wc...~8?..$.<..|
000001b0 5b de 55 17 b6 38 4b c4 ba 1f d2 c9 f8 ae c3 15 |[.U..8K.........|
000001c0 70 9a ad 7d 73 26 7e 3b c7 e7 62 cb 01 88 96 49 |p..}s&~;..b....I|
...
...
004000c0 4a 3d 54 c7 00 9e 7c ed ee 4f 26 6f 8d 37 b2 46 |J=T...|..O&o.7.F|
004000d0 ef 6a 8e d0 66 f9 5a 18 15 e1 48 87 c2 a5 d9 24 |.j..f.Z...H....$|
004000e0 47 62 20 37 18 7d 47 95 0b bf bc 82 13 29 c2 93 |Gb 7.}G......)..|
004000f0 ec fa ad 62 08 fa bd 05 48 0f 26 6f 47 f9 1e d3 |...b....H.&oG...|
00400100 61 fe 52 86 52 c4 f2 b4 7e c1 ca fd 3d 03 f1 46 |a.R.R...~...=..F|
00400110 b4 ca 8d 3c 47 e7 5c f9 93 a5 ba d9 10 fa 69 61 |...<G.\.......ia|
00400120 94 ef 3c d2 7d 56 f9 ce f8 ab 4d 81 2a 0a 8b 30 |..<.}V....M.*..0|
00400130 0c 7f 06 06 65 af d6 84 90 42 34 99 88 5a dc 11 |....e....B4..Z..|
00400140 c8 3f fc f7 b4 7d 0b c3 e3 c5 20 f0 1c 12 4e 03 |.?...}.... ...N.|
00400150 99 47 e9 8b 43 b2 38 90 73 f6 28 2f 80 b7 0a 00 |.G..C.8.s.(/....|
00400160 8b 51 3a 97 ca 3e e3 fc 04 c5 f0 d7 27 a1 35 9e |.Q:..>......'.5.|
00400170 58 c5 20 91 d4 8e 96 2f 52 d8 fe b3 a6 65 62 2d |X. ..../R....eb-|
00400180 43 c8 0d bc 0f 74 eb 3e 98 b1 18 0f 8d e4 81 36 |C....t.>.......6|
00400190 80 fa 96 d4 9b 0e ab cd 43 ff af 4f 05 c8 61 e0 |........C..O..a.|
004001a0 cd 77 63 90 0b 0c 7e 38 3f e0 f7 24 f6 3c b8 d8 |.wc...~8?..$.<..|
004001b0 5b de 55 17 b6 38 4b c4 ba 1f d2 c9 f8 ae c3 15 |[.U..8K.........|
004001c0 70 9a ad 7d 73 26 7e 3b c7 e7 62 cb 01 88 96 49 |p..}s&~;..b....I|
...
...
008000c0 4a 3d 54 c7 00 9e 7c ed ee 4f 26 6f 8d 37 b2 46 |J=T...|..O&o.7.F|
008000d0 ef 6a 8e d0 66 f9 5a 18 15 e1 48 87 c2 a5 d9 24 |.j..f.Z...H....$|
008000e0 47 62 20 37 18 7d 47 95 0b bf bc 82 13 29 c2 93 |Gb 7.}G......)..|
008000f0 ec fa ad 62 08 fa bd 05 48 0f 26 6f 47 f9 1e d3 |...b....H.&oG...|
00800100 61 fe 52 86 52 c4 f2 b4 7e c1 ca fd 3d 03 f1 46 |a.R.R...~...=..F|
00800110 b4 ca 8d 3c 47 e7 5c f9 93 a5 ba d9 10 fa 69 61 |...<G.\.......ia|
00800120 94 ef 3c d2 7d 56 f9 ce f8 ab 4d 81 2a 0a 8b 30 |..<.}V....M.*..0|
00800130 0c 7f 06 06 65 af d6 84 90 42 34 99 88 5a dc 11 |....e....B4..Z..|
00800140 c8 3f fc f7 b4 7d 0b c3 e3 c5 20 f0 1c 12 4e 03 |.?...}.... ...N.|
00800150 99 47 e9 8b 43 b2 38 90 73 f6 28 2f 80 b7 0a 00 |.G..C.8.s.(/....|
00800160 8b 51 3a 97 ca 3e e3 fc 04 c5 f0 d7 27 a1 35 9e |.Q:..>......'.5.|
00800170 58 c5 20 91 d4 8e 96 2f 52 d8 fe b3 a6 65 62 2d |X. ..../R....eb-|
00800180 43 c8 0d bc 0f 74 eb 3e 98 b1 18 0f 8d e4 81 36 |C....t.>.......6|
00800190 80 fa 96 d4 9b 0e ab cd 43 ff af 4f 05 c8 61 e0 |........C..O..a.|
008001a0 cd 77 63 90 0b 0c 7e 38 3f e0 f7 24 f6 3c b8 d8 |.wc...~8?..$.<..|
008001b0 5b de 55 17 b6 38 4b c4 ba 1f d2 c9 f8 ae c3 15 |[.U..8K.........|
008001c0 70 9a ad 7d 73 26 7e 3b c7 e7 62 cb 01 88 96 49 |p..}s&~;..b....I|
If the vmfs metadata is overwritten you will need to do one of the following: