INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Remove duplicates and replace with symbolic link?  (Read 611 times)

Sangie

  • Recent member
  • *
  • Posts: 43
Remove duplicates and replace with symbolic link?
« on: May 22, 2022, 10:03:04 pm »

I have been using Media Center for years on both Mac and Windows and Linux but one pet peeve I have is the inability to find duplicate tracks with similar track name and length. This would be the case on compilations and best of albums.

So basically it would appear in the library as the same file with same stats and play count and metadata would be the same it would just be listed under multiple albums. So basically one track could be on more than one album?

I could save a lot of space if this was possible as I use FLAC files with Media Center.

Also this might be possible and I don't know how to use the feature yet.
Logged

EnglishTiger

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 956
Re: Remove duplicates and replace with symbolic link?
« Reply #1 on: May 23, 2022, 08:04:48 am »

I have been using Media Center for years on both Mac and Windows and Linux but one pet peeve I have is the inability to find duplicate tracks with similar track name and length. This would be the case on compilations and best of albums.

So basically it would appear in the library as the same file with same stats and play count and metadata would be the same it would just be listed under multiple albums. So basically one track could be on more than one album?

I could save a lot of space if this was possible as I use FLAC files with Media Center.

Also this might be possible and I don't know how to use the feature yet.

One way of providing yourself with a list of those duplicates would be to set up a Smartlist for Media Type is Audio and in the Modify Results Section use:-
Only Duplicates of Artist, Duration, Name and then tell the smartlist to sort it on Artist, Name, Duration.

Caution: On a lot of Multi-Artist Compilation Albums sometimes the Duration is altered by the Compiler of the Compilation to get all the tracks they want on to each CD
Plus the Smartlist will only find those tracks where the Name Artist and Duration are Identical so it will not match "Our Lates Hit" with "Our Latest Hit [Edit] even if the Artist and  Duration are Identical.

And no you can't replace a physical track with a link to a duplicate of that track.

As a fan of Multiple Artist/Themed Compilations I have loads of Duplicated Tracks retaining them, even though they do occupy disc space, is the only way I know of preserving the Integrity of the Albums they appear on.

Don't forget SSD's are a lot cheaper theses days - certainly a lot cheaper than the hassle involved in removing them can cause.
Logged
Win NUC - VENOEN 11Th NUC Mini PC Core i7 1165G7,Dual HDMI 2.0+Mini DP,Windows 11 Mini Desktop Computer,Thunderbolt 4.0,1 Lan, USB-C,Wifi,Bluetooth 5.0,32GB RAM Toshiba MQ04ABF100 ‎500Gb 5400 RPM ‎eSATA HD, Gigabyte GP-GSM2NE3512GNTD 1Tb NVMe SSD, Samsung 870 QVO 8 TB SATA 2.5 Inch SSD (MZ-77Q8T0) in Sabrent Ultra Slim USB 3.0 to 2.5-Inch SATA External Aluminium Hard Drive Enclosure (EC-UK30)

Apple 2020 Mac mini M1 Chip (8GB RAM, 512GB SSD)
Sabrent Thunderbolt 3 to Dual NVMe M.2 SSD Tool-Free Enclosure with Sabrent 2TB Rocket NVMe PCIe M.2 2280 High Performance SSD + Crucial P3 Plus 4TB M.2 PCIe

ET Skins & TrackInfo Plugins - https://englishtiger.uk/index.html

zybex

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 2376
Re: Remove duplicates and replace with symbolic link?
« Reply #2 on: May 23, 2022, 11:15:25 am »

Note that if you have 2 identical tracks on different albums the files themselves are usually not identical due to the Tags saved in each of them. File 1 will have tags saying AlbumName=A and file 2 will have AlbumName=B. The CRC checksum for the files will be different even if the audio content is exactly the same.

If you create symlinks to have a single physical file (pointing file 2 to file 1), then when you change any tag in MC for file 1 the file will be updated will ALL tags of track A; so file 2 will now have the wrong tags for album B. MC will detect that file 2 was changed and will re-import it, overwriting your tags on Album B with wrong values.

As a workaround you can disable MC's feature to save tags to the files themselves. Then, only the DB data will matter but you won't have the files tagged properly for other player applications. DLNA players will also display the wrong info.

The only valid solution is to use a filesystem with block-level deduplication, such as ZFS. However as the british Tiger said above, storage is cheap these days.
Logged

blgentry

  • Regular Member
  • Citizen of the Universe
  • *****
  • Posts: 8009
Re: Remove duplicates and replace with symbolic link?
« Reply #3 on: May 23, 2022, 05:15:43 pm »

Finding duplicates is hard.  Because what humans think of as duplicates vary from human to human.  It also involves judgement in many cases.  I've developed several duplicate finder views in MC that I've used for hours at a time cleaning up other people's collections.  It gets deep pretty quickly.

I think it's useful to eliminate real duplicates from you collection.  Actual duplicates of the exact same album.  Maybe the album name or artist name is slightly different.  For example maybe you have an album by The Cure called Disintegration. But you have another album with Artist name:  Cure and Album name: Disintegration.  "Cure" and "The Cure" generally the same band, especially with that iconic album name.  They are duplicates; just different metadata.

On the other hand, maybe you have multiple editions of the exact same album.  Like "The Dark Side of The Moon" from the Columbia master and another "The Dark Side Of The Moon" from the Harvest master.  These albums are the same band and the same song, but the mastering makes them sound markedly different.

These are just a few examples of why finding duplicates is "hard".

Also, as everyone else said, storage is cheap and getting cheaper.  I no longer compress any kind of media.  BluRay rips come straight off of the disc with no transcoding.  Just the exact video and audio streams that are on the disc.  CDs get ripped to FLAC.  Because that keeps all the data and I'll never have to worry about not having it in the future.

Brian.
Logged
Pages: [1]   Go Up