INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Feature Req: Tag/Import files based on existing DB  (Read 1826 times)

Shady Bimmer

  • Regular Member
  • World Citizen
  • ***
  • Posts: 116
  • (too busy listening)
Feature Req: Tag/Import files based on existing DB
« on: July 01, 2004, 08:08:36 pm »
I know this one sounds odd but read through. . .

Summary
I'd be interested in a method to be able to match an entry in a local MC database to an unknown track.  The reason would be to be able to recover all MC fields in event of media file data loss.  Essentially I'm looking for something similar to matching an unknown track against YADB, only using the local database instead.

Details
Something I've noticed is that YADB does not store all MC fields but rather just a small subset common to most other public music databases, such as Artist, Album, Title, etc.

I spend a tremendous amount of time filling in many other fields and I am aware that many others do the same.  I currently have over 1000 "non-commercial" (let's leave it at that :) ) releases that have additional data including performance date, location,  media lineage, etc.

I store all my music in compressed lossless format (APE) unless I receive something in lossy which I keep in original format.  Every "work" I have is archived on data CD in the original format.  My original CDA Discs as well as these data discs serve as my backups, given that backing up 2 TB of data via any other method is not exactly inexpensive.

I'm using RAID so I can tolerate drive failures, but there are cases where data loss may not be preventable.  While I can restore the "works" from my CDs and recover basic information (hopefully) from YADB, a large amount of the additional data I had previously entered is useless and must be re-entered.  I *could* manually rename each file recovered exactly as the database thinks it should be, but this won't save much time.

Given that I have the MC database recovered (I do back this up regularly) it would be nice to be able to have MC recalculate a signature of an "unknown" file and match it to an entry in its local database (similar to what it does for YADB, only using local database instead) then offer to import this track exactly as the track it already has in its database (including renaming the file).  It could display the information (artist/album/title) for verification, and possibly ask for additional verification before replacing an existing file.  If MC finds a file missing (it shows with a red 'x' in tree listings) it may not need verification, but for sanity checks this might be a "good thing" anyway.  While doing this one-by-one could be very tedious, a list-box with the ability to check/uncheck files would be usefull.

Perhaps asking a bit much, but a feature that I think would be worthwhile.

Comments?


Logged

Jaguu

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1336
Re:Feature Req: Tag/Import files based on existing DB
« Reply #1 on: July 02, 2004, 01:55:33 am »
You can use "File>Export Playlists" to export all your tags to an *.html, *.mpl  or Ascii text file and then use Import to import selected entries.

There is also a plugin MCXML-Export if you need more sophisticated selections.
Logged

Shady Bimmer

  • Regular Member
  • World Citizen
  • ***
  • Posts: 116
  • (too busy listening)
Re:Feature Req: Tag/Import files based on existing DB
« Reply #2 on: July 06, 2004, 08:05:48 am »
Exporting / Importing the data isn't the problem.  It is matching the data to tracks.  

For example:
  • Rip a track from CD
  • Import into MC
  • Fill in lots of fields
  • Rename files from track info
Now say you lose that renamed track.  All the data is still in the database and I have the original CD (which may be CDA or Data for live shows in FLAC/SHN/APE).  I'm looking for a way to re-rip the track and have an automated way for MC to recognize and import this track in place of the lost file.

For a few tracks the file can be renamed by hand into the right location.  For a large number (thousands or tens of thousands) this isn't practical.

MC can lookup an unknown track by "signature" to fetch Artist/Album/Title in a central internet database.  I'm just looking for a way for it to do something similar using the local database.

Logged

GHammer

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1930
  • Stereotypes are a real timesaver!
Re:Feature Req: Tag/Import files based on existing DB
« Reply #3 on: July 08, 2004, 07:53:39 am »
Quote from: Jaguu on July 02, 2004, 01:55:33 am
You can use "File>Export Playlists" to export all your tags to an *.html, *.mpl  or Ascii text file and then use Import to import selected entries.

Where might that import function be? I know Import Media. But I have not found Import Tag Data, or whetever it might be called.

I too spent lots of time getting complete data for selections and would hate to lose it.
Logged

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re:Feature Req: Tag/Import files based on existing DB
« Reply #4 on: July 08, 2004, 11:12:39 am »
Quote
Summary
I'd be interested in a method to be able to match an entry in a local MC database to an unknown track.  The reason would be to be able to recover all MC fields in event of media file data loss.  Essentially I'm looking for something similar to matching an unknown track against YADB, only using the local database instead.

If i understand correctly, the issue is potential media loss.

How about mirroring data on to redundant drives ? that way in case of a loss, only data between backups is lost.

Its however not the cheapest. I guess cost depends on how much time you have to restore your library to the right state. At 2TBs, hoo boy, re-ripping's gonna take ages !!


Quote
I'm looking for a way to re-rip the track and have an automated way for MC to recognize and import this track in place of the lost file.

For a few tracks the file can be renamed by hand into the right location.  For a large number (thousands or tens of thousands) this isn't practical.

MC can lookup an unknown track by "signature" to fetch Artist/Album/Title in a central internet database.  I'm just looking for a way for it to do something similar using the local database.

This sounds like a good idea but i'm frightened by the amount of work required to re-rip "(thousands or tens of thousands)"

Again i wonder isnt mirroring easier than having to do all this or am i missing something here ?


In the meantime, i suppose its possible to write any information thats important to the file itself, so if the library gets corrupt then a simple import could handle it. Hopefully that information is not too large, resulting in the same info being duplicated in all files like it would in the case of lyrics/bios etc.

IN case of media loss, if you set the (backup) drives up correctly, MC will not even notice the difference. Except for stuff that got lost in between backups.

I like the idea of a checksum (for the data portion only) though just to keep tabs on file integrity across the database. HDs tend to fail gradually if not all of a sudden (in my experience anyway).


Quote
Where might that import function be? I know Import Media. But I have not found Import Tag Data, or whetever it might be called.

There isn't one. MC can export easily but tends to be quite limited in terms of import. Whether this is done by design, needs to be clarified.

Logged

GHammer

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1930
  • Stereotypes are a real timesaver!
Re:Feature Req: Tag/Import files based on existing DB
« Reply #5 on: July 10, 2004, 03:22:39 pm »
Quote from: hit_ny on July 08, 2004, 11:12:39 am
Quote
Summary
I'd be interested in a method to be able to match an entry in a local MC database to an unknown track.  The reason would be to be able to recover all MC fields in event of media file data loss.  Essentially I'm looking for something similar to matching an unknown track against YADB, only using the local database instead.

If i understand correctly, the issue is potential media loss.

How about mirroring data on to redundant drives ? that way in case of a loss, only data between backups is lost.

8 250 Gb drives to mirror to may be a bit pricy. In the case of a huge library like that, I'd go with RAID and a method to recreate the data in the event of partial loss.

Sounds like that is what the original post is looking for.
Logged

Shady Bimmer

  • Regular Member
  • World Citizen
  • ***
  • Posts: 116
  • (too busy listening)
Re:Feature Req: Tag/Import files based on existing DB
« Reply #6 on: July 10, 2004, 08:48:15 pm »
Yes - I think you understand what I'm looking for.

In another post I had pointed out that I have 2.5TB using RAID5 (6 200GB and 6 300GB drives)

Anyone that thinks any form of RAID is protection against data loss is seriously mistaken.  RAID (be it mirroring or parity-based protection on a striping scheme) is meant to protect against hardware failure.  It is also not an absolute - the higher the protection the greater the cost.  There are also many other methods by which data may be corrupted or removed/lost.

As I stated above, I already have CDA or CDR for all my media so the issue isn't losing media.  The issue is losing the library/media association.  All of the data is there, but there is currently no way to match a database entry to a media file (such as a music track) other than exact filepath.  If I re-rip tracks from CDA or CDR it is unlikely the track names will match those in the library (since the tracks had since been renamed).

MC already provides the ability to search a database to populate tags for an unknown track using a signature it calculates.  Currently this database is limited to YADB which stores limited information.  I think it would be worthwhile to do the same, only using the local MC database.

Currently there is a 'Library Tools -> Lookup Track info from YADB' function.  Using the same facilities already built into MC I'd like to see a 'Library Tools -> Lookup Track Info from Local DB'.  The reasoning being that the user may/likely have more information in the local database than YADB has.

This would of course require that MC store the signature in the Library - perhaps it does this already but I really don't know.  If not easy to add to 10, then 11 would be a prime opportunity to add this (while the new database is still being developed.)
Logged

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re:Feature Req: Tag/Import files based on existing DB
« Reply #7 on: July 11, 2004, 04:12:41 am »
Quote
The issue is losing the library/media association

OK how about periodically backing up the library then ?

make a batch file that backups the library every time you switch on the computer (or at any other regular period), and copy that backup to another drive/computer for more redundancy.

to do this, i wonder if its possible to do a library backup from the cmd-line using the mcc cmds. does any one know ?
Logged

Shady Bimmer

  • Regular Member
  • World Citizen
  • ***
  • Posts: 116
  • (too busy listening)
Re:Feature Req: Tag/Import files based on existing DB
« Reply #8 on: July 12, 2004, 09:42:32 am »
I think you are all missing something (Surely you aren't suggesting I backup 2.5+ Terabytes of data to other storage.  For the cost of a tape jukebox that can accomodate that I could purchase several more arrays.  I already have original CD media to serve as backups so why spend even more $$ for a third copy?)

As I said already - I DO backup my library regulary.

As I also said already - I DO have CD media (either Audio for commercial CDs or data for other works) for my library content.

Currently, as far as I know, the only association between the Media Center Library and the media files is filename.  If you lose your content and have to recover from CD the filenames will not match.  Renaming files one by one (or editing the MC library attributes) could take longer than re-inputing the data itself but without doing this the existing library would be useless.

Perhaps phrased this way:  What I am looking for is a more dynamic association between tracks and media center library.  MC can generate a signature from a track to identify that track from YADB.  If it could do this using the local database instead of YADB it would do exactly what I'm looking for.

This has nothing to do with protecting against data loss - I have that covered.  This has to do with significantly enhancing the ability to recover the tremendous effort some of us have spent correlating this data.  The association between the database and data is the key - if that is lost without a relatively easy method for recovery all the backups in the world will not help.  Using the signature that MC can already generate would help tremendously in the event these associations had to be recreated.


Logged

Matt

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 42389
  • Shoes gone again!
Re:Feature Req: Tag/Import files based on existing DB
« Reply #9 on: July 12, 2004, 09:45:25 am »
Quote from: Shady Bimmer on July 12, 2004, 09:42:32 am
Currently, as far as I know, the only association between the Media Center Library and the media files is filename.  If you lose your content and have to recover from CD the filenames will not match.  Renaming files one by one (or editing the MC library attributes) could take longer than re-inputing the data itself but without doing this the existing library would be useless.

MC 11 can fix broken links when you move files.  You just tell it the folders to scan and it works its magic.
Logged
Matt Ashland, JRiver Media Center

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re:Feature Req: Tag/Import files based on existing DB
« Reply #10 on: July 12, 2004, 09:59:38 am »
lets have a walkthrough

- library + content in sync, all is good.

- some dirs go bad or a whole partition goes bad, library still good.

- restore/re-rip from CD, all files re-created, filenames in the form track 01,02 etc and are untagged.

NOW..

MC is pointing to those recreated dirs, but cant find the files and says they dont exist.

Is there a way to tag those newly created  files given the data the library has ?
.....assuming of course you maintain the same dir structure.
Logged

Shady Bimmer

  • Regular Member
  • World Citizen
  • ***
  • Posts: 116
  • (too busy listening)
Re:Feature Req: Tag/Import files based on existing DB
« Reply #11 on: July 12, 2004, 12:40:59 pm »
Quote from: Matt on July 12, 2004, 09:45:25 am
MC 11 can fix broken links when you move files.  You just tell it the folders to scan and it works its magic.

Ah - Thanks Matt - this sounds like it would do exactly what I'm looking for.  

I haven't had a chance to try 11 yet.  What prompted this question in the first place was a read failure found while rebuilding an array after loss of an initial drive :(  The thought of having to recreate the associations (or re-enter all the data - take your pick) left me very uneasy.  Luckily I was able to put together a recovery utility that got past it but it is not an uncommon problem with Raid 5 and could recur again.

Hopefully now that I'm back up and running I'll be able to check out 11.
Logged
Pages: [1]   Go Up
« previous next »