INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: [Feature Request] File Integrity Verifier  (Read 1206 times)

sram16

  • Recent member
  • *
  • Posts: 24
[Feature Request] File Integrity Verifier
« on: May 23, 2019, 06:38:34 pm »

I know JRiver offers minor bad file detection through auto import, but it doesn't offer true detection of decoding errors with MD5 and CRC32 values (depending on the format). Foobar has a similar component that's really useful

https://www.foobar2000.org/components/view/foo_verifier
Logged

tij

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1563
Re: [Feature Request] File Integrity Verifier
« Reply #1 on: May 23, 2019, 08:22:01 pm »

If integrity of files (bit rot) is your worry use FreeNAS ... that’s what I did

it sits on OpenZFS file system that has build in error detection AND correction. BUT it is NOT replacement for good backup.

“Data integrity: All data includes a checksum of the data. When data is written, the checksum is calculated and written along with it. When that data is later read back, the checksum is calculated again. If the checksums do not match, a data error has been detected. ZFS will attempt to automatically correct errors when data redundancy is available.”
Logged
HTPC: Win11 Pro, MC: latest 31(64b), NV Driver: v425.31, CPU: i9-12900K, 32GB RAM, GeForce: 2080ti
Screen: LG 2016 E6
NAS: FreeNAS 11.1, SuperMicro SSG-5048R-E1CR36L, E5-1620v4, 64GB ECC RAM, 18xUltrastar He12-SAS3 drives, 2x240GB SSD (OS)

sram16

  • Recent member
  • *
  • Posts: 24
Re: [Feature Request] File Integrity Verifier
« Reply #2 on: May 31, 2019, 05:15:06 pm »

If integrity of files (bit rot) is your worry use FreeNAS ... that’s what I did

it sits on OpenZFS file system that has build in error detection AND correction. BUT it is NOT replacement for good backup.

“Data integrity: All data includes a checksum of the data. When data is written, the checksum is calculated and written along with it. When that data is later read back, the checksum is calculated again. If the checksums do not match, a data error has been detected. ZFS will attempt to automatically correct errors when data redundancy is available.”

Not talking about bitrot, more referring to files there were ripped inaccurately and have errors. Feels like a simple tools that could be under Right Click -> Library Tools -> Verify Integrity
Logged

blgentry

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 8014
Re: [Feature Request] File Integrity Verifier
« Reply #3 on: June 01, 2019, 08:10:16 am »

That plugin (from foobar) seems like it kind of does nothing.  As far as I can tell there's no way for it to detect "ripping errors".  That's got to be done at rip time; possibly by using the Accurate Rip database.  If you want perfect rips, use a ripping tool that has verification.

That tool appears to only check "integrity" for file formats that support CRCs (or MD5 checksums).  Which means that it will checksum (or CRC) the whole file and compare it to the stored value.  That will only detect if the file changes.  Which is "bitrot".  It won't detect an inaccurate rip.

Maybe I'm missing something.

Brian.
Logged

BryanC

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2663
Re: [Feature Request] File Integrity Verifier
« Reply #4 on: June 01, 2019, 08:23:16 am »

It simply attempts to identify corrupt audio files, whether they be from bad rips or bitrot. You can have a "bad rip" (i.e. not matching Accurate Rip) but still have a completely valid file. All that the foobar plugin does is attempt to decode the file completely and if it errors out, it marks it as corrupt. I think that MC may already do something similar with its ~bad database.
Logged

tij

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1563
Re: [Feature Request] File Integrity Verifier
« Reply #5 on: June 02, 2019, 04:30:20 am »

@blgentry is right ... most accurate way to detect if your rip is good is at ripping stage by comparing to database (i use Exact Audio Copy) ... your rip results are also uploaded to database

“bad rip” ... simply can mean you have different CD pressing ... i usually rerip several times these “bad rip” to see if i get same consistent results ... if i get different results every time i rerip, then i am screwed as CD is likely damaged (which i have a lot from my young days when preserving these things was not my primary concern ... i then choose rip that have most same results in hopes that its correct one)

attempting to verify file integrity through checksum after initial rip is detecting “bit rot” ... which begs another question - did your media got corrupted or file that stores checksum got corrupted? ... i guess MC can compare your checksum against database (even “bad rips” results are uploaded to database), but accessing those database might involve some fees that JRiver might not willing to shell out

again - if bit rot is your concern (it was mine ... but not for music - that i can buy again if needed ... it was for personal photos and videos, which are irreplacable) ... use ZFS file system (must stress that it is not substitude for backup) 

Logged
HTPC: Win11 Pro, MC: latest 31(64b), NV Driver: v425.31, CPU: i9-12900K, 32GB RAM, GeForce: 2080ti
Screen: LG 2016 E6
NAS: FreeNAS 11.1, SuperMicro SSG-5048R-E1CR36L, E5-1620v4, 64GB ECC RAM, 18xUltrastar He12-SAS3 drives, 2x240GB SSD (OS)

tij

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1563
Re: [Feature Request] File Integrity Verifier
« Reply #6 on: June 02, 2019, 04:40:57 am »

forgot to mention ... modern HDD have sophisicated way to detect and correct “bit rot” (you can check SMART stats of your ATA HDD to see this ... SAS HDD are not so forthcoming in that regard ... if SMART indicates that HDD did many corrections or is unable to recover data, then your HDD needs replacing before actual disaster strikes) ... so “bit rot” might be overated these days

does not stop me from beeing over paranoid and use ZFS lol
Logged
HTPC: Win11 Pro, MC: latest 31(64b), NV Driver: v425.31, CPU: i9-12900K, 32GB RAM, GeForce: 2080ti
Screen: LG 2016 E6
NAS: FreeNAS 11.1, SuperMicro SSG-5048R-E1CR36L, E5-1620v4, 64GB ECC RAM, 18xUltrastar He12-SAS3 drives, 2x240GB SSD (OS)

millst

  • Galactic Citizen
  • ****
  • Posts: 256
Re: [Feature Request] File Integrity Verifier
« Reply #7 on: June 02, 2019, 01:08:55 pm »

ZFS would also be beneficial for its self-healing ability, assuming you provide enough drives.
Logged
Pages: [1]   Go Up