INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: NEW: Audio CRC  (Read 13032 times)

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 41920
  • Shoes gone again!
NEW: Audio CRC
« on: April 30, 2021, 09:11:33 am »

Media 28.0.9 and newer add an "Audio CRC" checksum field that will be filled by the audio analysis.

This value is calculated for 16-bit and 24-bit stereo files.  The files will often need to be lossless (since a lossy format like MP3 is 64-bit).

The value we calculate matches common rippers like EAC (and saved in the *.log files of EAC).

I needed a way to check some files to see if they matched the rip, so thought adding this to MC made sense.
Logged
Matt Ashland, JRiver Media Center

BigCat

  • Regular Member
  • World Citizen
  • ***
  • Posts: 212
  • Believe...
Re: NEW: Audio CRC
« Reply #1 on: May 06, 2021, 08:01:02 pm »

Great news, Matt!
Logged

benjy

  • Recent member
  • *
  • Posts: 5
Re: NEW: Audio CRC
« Reply #2 on: May 10, 2021, 01:33:09 am »

Media 28.0.9 and newer add an "Audio CRC" checksum field that will be filled by the audio analysis.

Will there be a way to scan your library to check for corruption or changes? For example, if disk blocks went bad or if a file was somehow modified since the CRC was calculated?

In the event that one intentionally edits an audio file after the original CRC is calculated, will there be a way to update the checksum so that the file is not flagged during any library corruption check.

If one runs audio analysis on a file and the CRC changes from the last value, how will it be handled? Since filesystem / disk corruption occasionally happens, I hope that a new analysis flags the discrepancy for further action by the user instead of just silently updating the original value.

Having the ability to detect corruption and changes is very cool. Thanks!
Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 41920
  • Shoes gone again!
Re: NEW: Audio CRC
« Reply #3 on: May 10, 2021, 01:12:15 pm »

Maybe there should be another database field "CRC Changed" that is normally just false.

If something causes a CRC change, it could set it to true.

Then if you searched your library for any true values, you would find changes?

I'm the paranoid guy that invented a verify feature for Monkey's Audio so I routinely check my whole collection!
Logged
Matt Ashland, JRiver Media Center

Vocalpoint

  • Citizen of the Universe
  • *****
  • Posts: 2004
Re: NEW: Audio CRC
« Reply #4 on: May 10, 2021, 01:28:07 pm »

Maybe there should be another database field "CRC Changed" that is normally just false.

If something causes a CRC change, it could set it to true.

Then if you searched your library for any true values, you would find changes?

I'm the paranoid guy that invented a verify feature for Monkey's Audio so I routinely check my whole collection!

+1 I like this a lot!

But I think any CRC record check (or potentially a couple fields) should record date logic - especially a Modified Date so we can see exactly when a CRC was changed for troubleshooting purposes. Having the system simply state that the CRC has changed does not really help me determine when something has changed.

And while relying on the Windows Modified date should be good enough but quite frequently - but in many cases - it's not.

VP

Logged

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10710
Re: NEW: Audio CRC
« Reply #5 on: May 10, 2021, 02:01:07 pm »

MC is not going to continously monitor your files, it would only compute this during audio analysis - at which point it could notify you that something changed. Not sure a date field helps (or even the "CRC changed" field), since its not doing checks unless you ask it to check, and can immediately let you know then and there.

All that said, I view the CRC more generally useful to check rips or conversions, if you are worried about file corruption, the file system or the OS would probably be better equipped to monitor that on a regular basis, since verifying a file changed is faster if you don't also parse and decode the audio. My file server checks a portion of my files every night for integrity, and it gets through all files every few weeks that way.
Logged
~ nevcairiel
~ Author of LAV Filters

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 41920
  • Shoes gone again!
Re: NEW: Audio CRC
« Reply #6 on: May 10, 2021, 02:05:44 pm »

MC is not going to continously monitor your files, it would only compute this during audio analysis - at which point it could notify you that something changed. Not sure a date field helps (or even the "CRC changed" field), since its not doing checks unless you ask it to check, and can immediately let you know then and there.

All that said, I view the CRC more generally useful to check rips or conversions, if you are worried about file corruption, the file system or the OS would probably be better equipped to monitor that on a regular basis, since verifying a file changed is faster if you don't also parse and decode the audio.

If you choose a format like APE, you'll get CRC checking built in.  It takes around 6 seconds to check a CD, and the detection is bullet proof.
Logged
Matt Ashland, JRiver Media Center

Hendrik

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 10710
Re: NEW: Audio CRC
« Reply #7 on: May 10, 2021, 02:07:23 pm »

If you choose a format like APE, you'll get CRC checking built in.  It takes around 6 seconds to check a CD, and the detection is bullet proof.

And yet you don't want MC continously running checks on every music file in your library. So when does it verify it? In audio analysis.
Logged
~ nevcairiel
~ Author of LAV Filters

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 41920
  • Shoes gone again!
Re: NEW: Audio CRC
« Reply #8 on: May 10, 2021, 02:08:34 pm »

And yet you don't want MC continously running checks on every music file in your library. So when does it verify it? In audio analysis.

To Verify with Monkey's Audio you need to run the program.

It's one of the modes.

The check in Media Center is a lot heavier since it needs to decode all the data.
Logged
Matt Ashland, JRiver Media Center

Vocalpoint

  • Citizen of the Universe
  • *****
  • Posts: 2004
Re: NEW: Audio CRC
« Reply #9 on: May 10, 2021, 02:21:43 pm »

MC is not going to continously monitor your files, it would only compute this during audio analysis - at which point it could notify you that something changed. Not sure a date field helps (or even the "CRC changed" field), since its not doing checks unless you ask it to check, and can immediately let you know then and there.

Makes sense - and yes - if this is all manual in nature (like I need to run Audio Analysis to check the CRC) I probably would not bother with this at - with 100,000+ tracks - I would never have the time or patience to run Audio Anaylsis on the whole library.

And - I would never rerun Audio Analysis on a previously analyzed file. I do replace things with better copies from time to time but that would be a new incoming file.

Not exactly sure how this could work for large existing libraries. It would take me probably 24 hours to reanalyze everything AND I would be very annoyed if the file modified dates were altered due to a rescan as it would instantly cause my backups to need to take new copies of 100,000+ files.

Just as long as it's an option (and not automatically "On") ...

VP
Logged

dtc

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 3003
Re: NEW: Audio CRC
« Reply #10 on: May 11, 2021, 07:42:46 am »

Makes sense - and yes - if this is all manual in nature (like I need to run Audio Analysis to check the CRC)

Not exactly sure how this could work for large existing libraries. It would take me probably 24 hours to reanalyze everything AND I would be very annoyed if the file modified dates were altered due to a rescan as it would instantly cause my backups to need to take new copies of 100,000+ files.

If CRC were a separate option and did not require all the other Audio Analyze functions to run, then you could set it to not update tags and run it as needed. Having to rerun R128, HDCD, waveform just to do a CRC check seems unnecessary.  It really would be nice to be able to run these functions separately.  It seems that AA has taken on a bit of a bloat factor.
Logged

EnglishTiger

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 954
Re: NEW: Audio CRC
« Reply #11 on: May 11, 2021, 08:32:08 am »

Does this mean that MC's own Ripper will also be able to calculate a CRC and store it in the ripped file?
Logged
Win NUC - VENOEN 11Th NUC Mini PC Core i7 1165G7,Dual HDMI 2.0+Mini DP,Windows 11 Mini Desktop Computer,Thunderbolt 4.0,1 Lan, USB-C,Wifi,Bluetooth 5.0,32GB RAM Toshiba MQ04ABF100 ‎500Gb 5400 RPM ‎eSATA HD, Gigabyte GP-GSM2NE3512GNTD 1Tb NVMe SSD, Samsung 870 QVO 8 TB SATA 2.5 Inch SSD (MZ-77Q8T0) in Sabrent Ultra Slim USB 3.0 to 2.5-Inch SATA External Aluminium Hard Drive Enclosure (EC-UK30)

Apple 2020 Mac mini M1 Chip (8GB RAM, 512GB SSD)
Sabrent Thunderbolt 3 to Dual NVMe M.2 SSD Tool-Free Enclosure with Sabrent 2TB Rocket NVMe PCIe M.2 2280 High Performance SSD + Crucial P3 Plus 4TB M.2 PCIe

ET Skins & TrackInfo Plugins - https://englishtiger.uk/index.html

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 41920
  • Shoes gone again!
Re: NEW: Audio CRC
« Reply #12 on: May 11, 2021, 09:44:29 am »

Does this mean that MC's own Ripper will also be able to calculate a CRC and store it in the ripped file?

It's part of any audio analysis of a lossless file which ripping does perform.
Logged
Matt Ashland, JRiver Media Center

RD James

  • Citizen of the Universe
  • *****
  • Posts: 1871
Re: NEW: Audio CRC
« Reply #13 on: May 14, 2021, 11:04:38 am »

And yet you don't want MC continously running checks on every music file in your library. So when does it verify it? In audio analysis.
Perhaps there could be an option to check a file (or the album?) on playback, and notify the user/flag a file if there's a change.
Logged

newsposter

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 785
Re: NEW: Audio CRC
« Reply #14 on: May 14, 2021, 03:31:25 pm »

so, can this be used for my request from a few days ago for a "clean rip" library field or flag?
Logged

EnglishTiger

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 954
Re: NEW: Audio CRC
« Reply #15 on: June 21, 2021, 05:00:48 am »

so, can this be used for my request from a few days ago for a "clean rip" library field or flag?

Probably not because MC would have to store the CRC for every every track for every album to be able to compare your rips with those of others to tell if it was a "clean rip" or not.
Logged
Win NUC - VENOEN 11Th NUC Mini PC Core i7 1165G7,Dual HDMI 2.0+Mini DP,Windows 11 Mini Desktop Computer,Thunderbolt 4.0,1 Lan, USB-C,Wifi,Bluetooth 5.0,32GB RAM Toshiba MQ04ABF100 ‎500Gb 5400 RPM ‎eSATA HD, Gigabyte GP-GSM2NE3512GNTD 1Tb NVMe SSD, Samsung 870 QVO 8 TB SATA 2.5 Inch SSD (MZ-77Q8T0) in Sabrent Ultra Slim USB 3.0 to 2.5-Inch SATA External Aluminium Hard Drive Enclosure (EC-UK30)

Apple 2020 Mac mini M1 Chip (8GB RAM, 512GB SSD)
Sabrent Thunderbolt 3 to Dual NVMe M.2 SSD Tool-Free Enclosure with Sabrent 2TB Rocket NVMe PCIe M.2 2280 High Performance SSD + Crucial P3 Plus 4TB M.2 PCIe

ET Skins & TrackInfo Plugins - https://englishtiger.uk/index.html

flac.rules

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 1260
Re: NEW: Audio CRC
« Reply #16 on: June 21, 2021, 11:33:15 am »

This sounds excellent, I already use external software to CRC-check the audio on my files (regular CRCs doesn't work due to tag-changes). I really like a CRC-check-function. I forgot about the early upgrade this time around, but a CRC-checking function is worth it alone for me.
Logged

benjy

  • Recent member
  • *
  • Posts: 5
Re: NEW: Audio CRC
« Reply #17 on: July 04, 2021, 01:18:45 am »

Perhaps there could be an option to check a file (or the album?) on playback, and notify the user/flag a file if there's a change.

Agreed! CRC change notification at playback would be exceptionally useful to avoid embarrassing fails while DJing. I have personally experienced a situation where a known good song silently became corrupt, got queued during a gig, and gave a nasty surprise to the room part-way into the song.

In light of that, I'd rather receive the notification when a song gets added to any zone's play queue. That way, I'd most likely get the notification when either adding it to my preview zone or Playing Now, and have enough time to find a replacement without messing up the flow of the set. 
Logged

joelha

  • Junior Woodchuck
  • **
  • Posts: 96
Re: NEW: Audio CRC
« Reply #18 on: July 22, 2021, 10:44:13 pm »

Just to be clear, MC28 has a feature in which all the files which haven't been ripped on MC28 can still have their checksums tabulated in MC28?

Thanks.

Joel
Logged

AJSchmidt

  • Recent member
  • *
  • Posts: 18
Re: Audio CRC, Automatic-Verify before each play
« Reply #19 on: November 19, 2021, 02:03:03 pm »

Hello,
On 11/8/2021 I received an email from <mediacenter@listserver.jriver.com> with Subject: "A JRiver Offer For Black Friday".
An interesting paragraph appeared near the bottom of the page:

"Verify Your Audio Files"
"MC28 can write a CRC checksum to each audio file and use it to verify the integrity of your file each time it's played.  You can be certain that it's the same file, just as originally written."

That's the first I've read that MC has implemented the file integrity auto-verify feature, and it's a great idea! 
Unfortunately, the email didn't provide a link to find further details and instructions; and multiple searches turned up nothing. 

Please provide details for how to enable and configure this new CRC auto-check feature.

Thank you!
Logged

antenna

  • Galactic Citizen
  • ****
  • Posts: 409
Re: NEW: Audio CRC
« Reply #20 on: December 04, 2021, 12:45:40 pm »

Media 28.0.9 and newer add an "Audio CRC" checksum field that will be filled by the audio analysis....

Might this CRC also be used to help reduce duplicate uploads for Cloudplay?

Use case:  I've uploaded a playlist with, say, 20 tunes.   A couple three weeks later, I want to re-upload one of the tunes because, e.g.,  I found better cover art.   Unfortunately, instead of just the one tune being uploaded anew, many (sometimes most) of the tunes on the playlist are re-uploaded, even though they have not been changed (at least changed in a manner that may be significant to Cloudplay).

 
Logged
=========
Vinyl: Shure V15VxMR, Shure VN5MR stylus, VPI Scout turntable
Shellac: Shure M91, Shure N75-3 stylus,  Dual 1218 turntable

Apt Holman preamp (updated), Benchmark Media ADC-1, Benchmark Media DAC-1, Carver TFM-45 power amp (updated), Original Acoustic Research AR-9 speakers (LF surrounds replaced), Sennheiser HD590 headphones

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 71338
  • Where did I put my teeth?
Re: NEW: Audio CRC
« Reply #21 on: December 04, 2021, 06:47:24 pm »

Might this CRC also be used to help reduce duplicate uploads for Cloudplay?

Use case:  I've uploaded a playlist with, say, 20 tunes.   A couple three weeks later, I want to re-upload one of the tunes because, e.g.,  I found better cover art.   Unfortunately, instead of just the one tune being uploaded anew, many (sometimes most) of the tunes on the playlist are re-uploaded, even though they have not been changed (at least changed in a manner that may be significant to Cloudplay).
If the name of the playist doesn't change, only the changed file should be uploaded.
Logged

antenna

  • Galactic Citizen
  • ****
  • Posts: 409
Re: NEW: Audio CRC
« Reply #22 on: December 04, 2021, 06:58:42 pm »

If the name of the playist doesn't change, only the changed file should be uploaded.

Of course, that depends upon how "changed file" is defined.

But, let's say, if I use a tune in multiple playlists, and it is uploaded both times, the duplicate does not seem to be weeded out.

Bottom line, I don't have an issue one way or another.  I just wanted to bring your attention to a symptom I see.  If you're OK with it, I'll go away quietly.  :)

I just thought that Cloudplay may be having more storage requirements than needed because of the multiple uploads.  I'm trying to help, not criticize. 


Logged
=========
Vinyl: Shure V15VxMR, Shure VN5MR stylus, VPI Scout turntable
Shellac: Shure M91, Shure N75-3 stylus,  Dual 1218 turntable

Apt Holman preamp (updated), Benchmark Media ADC-1, Benchmark Media DAC-1, Carver TFM-45 power amp (updated), Original Acoustic Research AR-9 speakers (LF surrounds replaced), Sennheiser HD590 headphones

HTPC Videophile

  • World Citizen
  • ***
  • Posts: 105
Re: NEW: Audio CRC
« Reply #23 on: January 01, 2022, 04:06:29 am »

Is there a way to check the integrity of the video file. For example corrupt disks and bad sectors sometimes "eat away" chunks of file. How to keep a check on that if the stored video  file still maintains its integrity ?
Logged

Dawgincontrol

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 664
  • We have met the enemy and he is us.
Re: NEW: Audio CRC
« Reply #24 on: January 02, 2022, 09:14:56 am »

If the name of the playist doesn't change, only the changed file should be uploaded.

If you play the file, the metadata will change when Media Center updates the stats (i.e. Last Played).  Thus, it changes the file in the way Cloudplay sees it.  In other words, in Cloudplay's vision, the file has been changed just by playing it. 

You may not make any physical change, but Media Center will.  That's why you get so many reuploads (is that a word?) to Cloudplay.  It recognizes any metadata as a change.
Logged

ferrarabrainpan

  • World Citizen
  • ***
  • Posts: 154
Re: NEW: Audio CRC
« Reply #25 on: January 02, 2022, 09:35:20 am »

This value is calculated for 16-bit and 24-bit stereo files.  The files will often need to be lossless (since a lossy format like MP3 is 64-bit).

I have a handful of MP3 files in my 99% lossless library and when I view the details in MC28 the bit depth column shows them as 16-bit...?
Logged
"Let the great constellation of flickering ashes be heard..."   ~ Noel Scott Engel

username2013

  • Recent member
  • *
  • Posts: 7
Re: NEW: Audio CRC
« Reply #26 on: January 19, 2022, 03:55:13 pm »

Audio CRC is nice to calculate and compare it meets the original file but why don't you add par files to an album and if corruption occurs you will have the parity information to actually fix it back to the original bit for bit.  Using QuickPar is an awesome tool to check for integrity and if it's corrupt it will fix the files with the parity information back to it's original exactly.   http://www.quickpar.org.uk/
Check it out! This would be so cool to integrate into MC.  What do you think?

Steve
Logged
Pages: [1]   Go Up