INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1]   Go Down

Author Topic: Beyond a database of files  (Read 2673 times)

rick.ca

  • Citizen of the Universe
  • *****
  • Posts: 3729
Beyond a database of files
« on: September 15, 2011, 06:36:37 pm »

The reality is we are at the mercy of the file system...

But there's no reason why we should be. It's bad enough MC is so dependent on the arbitrary premise it's a database of files. Talk about abstraction—that's exactly what a file system is! It happens to be a very good one, convincing most there's actually something real and complete about a set of files. This in the face of a world where media can be anywhere, arranged in many ways and packaged in many forms. If MC doesn't at least loosen this arbitrary premise, it's going to become increasingly irrelevant. An obvious way of doing that without radically changing its very structure is to support the use of pseudo files. With that relatively simple change, we could create records representing media for which there is no "file," and, as discussed here, do the same for parts of files like chapters. You can call it another "abstraction layer" if you like, but the distinction is meaningless. I would call it the undoing of a completely unnecessary and confining abstraction.
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72439
  • Where did I put my teeth?
Re: MC is so dependent on the arbitrary premise it's a database of files
« Reply #1 on: September 15, 2011, 06:46:01 pm »

It's bad enough MC is so dependent on the arbitrary premise it's a database of files.
I don't think we've ever said that, but if we did, why is it a problem?
Logged

jmone

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 14463
  • I won! I won!
Re: MC is so dependent on the arbitrary premise it's a database of files
« Reply #2 on: September 15, 2011, 07:10:07 pm »

I think MC's abstraction model is actually very good and the recent discussion on "Deep Thought" has the potential of really expanding collection of meta data, lets look at the what it can do for the collection of meta data:

- OS Base Data:  Stuff presented by the OS such as Filename, size, date created, modified etc
- Explicit Meta Data:  Variety of tag based data for various formats such as Artist, Album, Track etc that are stored within the file itself
- Extracted Meta Data: MC collects information like Stream Formats, FPS, Bit Depth etc from examining the actual streams
- Collected Meta Data: MC can lookup meta data from 3rd party sources (YADB, Wikipedia, TV Guides etc)
- Inferred Meta Data: MC is working on the fuzzy logic that can assume and map to meta data values based on file name, directory names, (for Artist, Album, Season, Episode etc) and also Media Subtype based on Size, file type etc.

All of this meta data is then collected up in a normalised database format which completely abstracts the users from the confines of the souces and lets you manage the media on a meta data basis.  This approach makes great sense to me & I have to say I don't know of any other Media Manager that tries to drag in and normalise meta data from so many different possible elements.

Thanks
Nathan
Logged
JRiver CEO Elect

jmone

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 14463
  • I won! I won!
Re: MC is so dependent on the arbitrary premise it's a database of files
« Reply #3 on: September 15, 2011, 07:16:34 pm »

Don't get me wrong, this is not a completed task by any stretch of the imagination.  There is tons of additional meta data sources yet to be tapped correctly, though all have been touched on in various threads and many get added all the time:
- Better and wider meta data collection for online sources based on matching
- Wider extration of meta data from other media formats (eg MPLS, Chapters, advanced MKV features)
Logged
JRiver CEO Elect

DarkPenguin

  • Citizen of the Universe
  • *****
  • Posts: 1921
Re: MC is so dependent on the arbitrary premise it's a database of files
« Reply #4 on: September 15, 2011, 08:24:12 pm »

I think the problem here is that you think a file is a file.
Logged

Daydream

  • Citizen of the Universe
  • *****
  • Posts: 771
Re: MC is so dependent on the arbitrary premise it's a database of files
« Reply #5 on: September 15, 2011, 08:42:30 pm »

I don't think we've ever said that, but if we did, why is it a problem?

Not probably a problem per se; but, I think where this discussion is going is about a database of 'records', as in something far (further) detached from the file system. With an extremely flexible input. (files, dummy records, jump-points, references to references and I-don't-know what else).

I personally have a love-hate relationship with the idea. Here's why. For all purposes and intentions files are the 'real' thing (I saw your comment DarkPenguin, but for sanity's sake... :) ). What has happened (and it continues) in recent years with say, OSes from MS? What's on the Desktop? My Documents, My Pictures, My Music and so on. Libraries, etc. All good. Johnny average looks at them and always clicks on one of them to get his 'stuff'.
One ugly day the system breaks down. He is totally incapable to find his stuff anymore because he has no idea where the real thing is (and Win 7 is made of a lot of symlinks to complicate matters). Because of this we're getting close to 'the medium is the message', users get used to a very high level of abstraction, and nobody knows what makes the real thing tick. We all end up a bunch of idiots, with 5 people on the planet knowing what's what. Those 5 people disappear when the next Elenin comet actually hits Earth on a tangent, and all of the survivors are doomed to the depth of irreversible idiocy. That's why I hate complicated abstractions, because it makes people dumb in the name of increasing the comfort zone.

Why I like it is of course the lighting fast search. It's faster to search-as-you-type then to pick a shortcut/file/etc after navigating a tree-style organization.
Logged

Daydream

  • Citizen of the Universe
  • *****
  • Posts: 771
Re: MC is so dependent on the arbitrary premise it's a database of files
« Reply #6 on: September 15, 2011, 08:49:33 pm »

Don't get me wrong, this is not a completed task by any stretch of the imagination. 

But it's a good start. My take would be to increase the use of the current abstraction layer by augmenting with whatever else is (already) available (mkv features or whatever). As in a horizontal expansion. If you do a vertical one - an in increasing the complexity to deal in a new way (also semi-proprietary in the sense that it doesn't have a counterpart anywhere else), like we're discussing on the other thread, that doesn't sit well with me for the reasons in the post above. But like I said that's just me. Maybe something will come out of all these heads colliding :).
Logged

rick.ca

  • Citizen of the Universe
  • *****
  • Posts: 3729
Beyond a database of files
« Reply #7 on: September 15, 2011, 11:28:30 pm »

I don't think we've ever said that, but if we did, why is it a problem?

I think I made it clear why it's a problem—in the post you've quoted, in the topic that was originally posted to, and many times over the years. For example...

Sometimes I wonder if MC really needs to be so tightly designed around the existence of a media file. It seems to me there are many uses for "information" records that are not associated with any file. They would be associated with other file-based records based on common metadata. If such a thing were "allowed," you would be able to create a parent record for a series—properly associated with episode records/files by designating it season 0, episode 0. Also, I'm a movie information collector, not a movie media collector. The population of movies I'm interested in are those I've seen, and those I may want to see. Most of those don't exist as media I currently own. So I have to "trick" MC into creating these records by creating dummy files, and then being careful not to allow MC to scan them and find out they're dummies. :P

...and more recently...

I do hope to see a more general capability not yet mentioned here. That's that ability of MC to maintain records for media that does not exist in the file system. I currently use dummy files to record information about movies I've seen but do not own, and series. I could do the same for movies I wish to see or purchase, but prefer to download and tag a trailer for those. I believe it's important this capability be built into MC so these things can be done in a more natural an intuitive manner. That would also allow such records to make a seamless transition to representing a real file when the media is acquired.

This also has an important role in the case of a currently running series. As mentioned, some way of recording information about the series itself is necessary. But information about upcoming episodes is usually available for at least a few weeks in advance. It doesn't make sense to restrict information to just the episodes for which files are available. I generally watch and delete episodes. That doesn't mean I wouldn't appreciate having information about what I've watched. I might also be more interested in seeing information about what's being downloaded right now or coming next week than anything else.

Even if not automatic, this capability—along with such a meta data system—would still be fairly efficient at obtaining such information. You would just create a pseudo record (e.g., with [Name]="Some New Movie Release" and maybe [Year]="2011") and update it with your chosen meta data sources. It should also be possible to import a list of such items, and then update them all. An import facility should be able to import a list, not adding a pseudo record for anything already existing in the library. You'd then be able to do things like import the IMDb Top 250, and create records for all the movies in that list that you don't own. 8)
Logged

darichman

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1362
Re: MC is so dependent on the arbitrary premise it's a database of files
« Reply #8 on: September 15, 2011, 11:52:57 pm »

I would be interested in a setup whereby MC could maintain a database of 'records' to which 'files' could be assigned.
I have no understanding of the implications this might have on performance or searches etc.

  • 'Records' could be anything essentially. Titles (movies, series etc). People (artists, all the contributors to a song or title). Assets (digital record of things I own in the home maybe).
  • 'Records' could be tagged just as any file is now
  • 'Files' can be assigned to as many records as I choose
  • 'Records' can be assigned to other records eg a movie 'Title' might be assigned to the various 'People' contributing.
  • There might naturally be more than one person contributing, so files or records can be assigned to more than one record of the same type

I create an entry [Titanic]
I assign it to the Record type {Title}
I tag my new Titanic title with a bunch of things... year, country, coverart etc
I rip Titanic from my (legally acquired) Blu-Ray and import it.
"Do you want to assign this movie to any existing Records?" MC asks me.
"Yes!..." I exclaim, and I select Titanic (if I hadn't already created a record maybe MC could prompt me now to create a new one)
MC now knows the file is a movie (because I assigned it to a movie title and not an album, say) and inherits all the appropriate info from the title record
I might subsequently create a record [James Cameron].
I assign it to the {People} record type.
I might take my [Titanic] title record and assign it to my new [James Cameron] people record.
If MC is really clever it might even ask me to define the relationship (I might say 'Director')
I've just purchased the Titanic soundtrack now.
I rip my audio files with MC and import them.
I can assign them to the [Titanic] title record as I did above with the movie.
MC might ask me to define the relationship (I might specify soundtrack, if it was movie posters or wallpapers I could say so)
Similarly I might add them to the [James Horner] people record

This way we can build a database where everything is connected by information and relationships (not just by files and their tags). We could build databases of people, film/tv titles, books, assets with meaningful relationships between them. A filter could be used - "Show all records" or "Show records with files only"

Practically this need not mean creating a million separate records manually, but records could be generated from existing tags etc.

How all of this might fit into the current tagging system I'd leave up to the developers.
Logged

rick.ca

  • Citizen of the Universe
  • *****
  • Posts: 3729
Re: Beyond a database of files
« Reply #9 on: September 16, 2011, 01:37:58 am »

Quote
I would be interested in a setup whereby MC could maintain a database of 'records' to which 'files' could be assigned.

Although I'm advocating a less restrictive use for records, I think they need to be confined to media. Not only is this likely essential for maintaining performance, the completely flexible way of assigning relationships you describe would be overwhelmingly complicated for both the application and the user. Even an application built on a relational database requires some definition to what relationships are to be maintained. The application is then built around that structure to provide something usable and efficient to the user.

Personal Video Database provides a simple illustration of the idea. It's a relational database application based primarily on the relationship of video media to the people associated with that media. It's as much a database of videos and the people associated with them as it is a database of people and the videos they're associated with. Even though it's a relational database, it doesn't allow users to define their own relations. If that were possible, the application wouldn't know what to do with them. It might allow a custom "career" to people (so you can add hairdressers and key grips to your database), but the only reason this would work they would be handled in the same systematic manner as other existing categories.

I don't think it necessary or advisable for MC to be or behave like a relational database. There's nothing wrong with it being focused on media—and not being able to handle a focus on people (or anything else) with equal capability. It is, after all, a media manager. There are other ways to add relational-like functionality that are more appropriate to that reality. They're far from a complete solution, but relational fields are an example of this. And we've discussed in the past ways in which supplementary relational information might be handled. Such information (e.g., biographical information about people) could be saved in records completely separate from the media records, associated with category values, and presented on demand. This might be so when you hover on a person's name (e.g., an actor in a movie record), their photo and biography are shown in a popup.

But all this is way beyond the simple need this topic is about. The need for records for media that doesn't exist is going to become critical once we have an Integrated Automatic Meta Data System. Once meta data for any media can be easily obtained, many users are going to want it for things like keeping records of movies they've seen but do not own, movies they're considering seeing or buying, TV series, upcoming TV episodes, etc.
Logged

jmone

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 14463
  • I won! I won!
Re: Beyond a database of files
« Reply #10 on: September 16, 2011, 04:02:13 am »

3rd time I'll agree with Rick (I'll stop after this I promise). 

There is NO good reason why we should not be able to make entries into the database that either (Sorry for the double negative!):
- Don't point to a file at all (eg why should I have to make 300 fake files to be able to catalogue my Physical Disc collection) or;
- Point to the same file (eg the MPLS workaround to get multiple playback options on a single file)

Logged
JRiver CEO Elect
Pages: [1]   Go Up