INTERACT FORUM

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 [2]   Go Down

Author Topic: YARB -- Relational Database Annual Convention  (Read 25878 times)

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re: YARB -- Relational Database Annual Convention
« Reply #50 on: April 18, 2009, 12:54:52 am »

Right, let's recap...

We all agree there needs to be some sort of parent-child, container construct or entity that is able to hold subordinate [Fields]. These entities are to be created maually.

When tagging, in a file list view or AW, all a user would do is select from a dropdown  if a similar record of that entity exists otherwise enter in a name for it, just like with any [Field] currently.

If one was not interested in any of this it would be for all intents & purposes invisible. No entity definition created, no entities :)

I'm good with points 1,2,3 & 4 of darichman's post above.

It's taken some time for this all to sink in and at the end of it all find, the only new idea being suggested is the container or entity concept. This proposal gives the ability to specialise existing [Fields] and easily apply existing meta-data to new records.

Is this all that's required 'data wise' to get MC on a good footing with other movie managers ?

PS - The term relational is misleading in this discussion as there is absolutely nothing relational or even hierachical, in the conventional sense, being suggested here.
Logged

rick.ca

  • Citizen of the Universe
  • *****
  • Posts: 3729
Re: YARB -- Relational Database Annual Convention
« Reply #51 on: April 18, 2009, 03:48:09 am »

Quote
Right, let's recap...

Funny, it seems I've been providing you with recap after recap. ::)

Quote
The term relational is misleading in this discussion as there is absolutely nothing relational or even hierachical, in the conventional sense, being suggested here.

I'm pretty sure I've respected both the normal meaning of the word "relational," and its particular meaning as a mathematical concept.

relational: relating to, using, or being a method of organizing data in a database so that it is perceived by the user as a set of tables.

relational database: database in which all data are represented in tabular form. The description of a particular entity is provided by the set of its attribute values, stored as one row or record of the table, called a tuple. Similar items from different records can appear in a table column. The relational approach supports queries that involve several tables by providing automatic links across tables.

My proposal is relational. Darichman's proposal is relational, but quite different than mine. (If you're wondering why, please read my previous post.) Neither are fully relational in the context of how the same things would be done in a relational database. What are you talking about, if not the perception of two tables (the existing media records plus the new entity) related by common elements?

Quote
Is this all that's required 'data wise' to get MC on a good footing with other movie managers ?

No. The movie manager I use is built on a relational database. It's far more effective in relating movies to the people associated with them, and relating people to the movies they are associated with (as "filmographies"). What I've proposed, however, is appropriate for a media manager that is not trying to be a full-blown movie manager.
Logged

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re: YARB -- Relational Database Annual Convention
« Reply #52 on: April 18, 2009, 09:04:01 am »

relational database: database in which all data are represented in tabular form. The description of a particular entity is provided by the set of its attribute values, stored as one row or record of the table, called a tuple. Similar items from different records can appear in a table column. The relational approach supports queries that involve several tables by providing automatic links across tables.

Thats the bit i mean, there is no linking of tables in darichman's proposal. That to me is the crucial point in the term 'relational'. Tables linked = related to. Just the presence of tables does not imply relational at all. For all we know they might already be linked to some extent under the hood but i'm speaking purely in user space.

Darichman's proposal introduces the concept of user tables that are defined by custom entites, whilst currently the only table in existance is the file table.


Darichman's proposal is relational, but quite different than mine.

Reading through your posts, repeatedly now, you talk about links existing when a new record is added. Yes & no. It exists when you import but you still have to assign it as you do with any existing [Field] currently. The big difference NOW of course is you are assigning a lot more meta-data in one fell swoop than just a [Field] ie the goal of this mission  ;D

You talk about a People record, fine, select the appropiate person for the new file and you are set. Agreed it does not need to be more work that that :)

Othewise i don't see much difference to what you & Darichman have said.


What are you talking about, if not the perception of two tables (the existing media records plus the new entity) related by common elements?
Thing is I don't see that sort of relation in your descriptions. Because the user tables are not linked, at all. They exist totally by themselves. Yes they might share a few [Fields] between but otherwise nothing  more.

The reason is there is no concept of keys here at all. You can't query table 'A' using a value that is common between it and say table 'B' (ie a key) and ask get me another attribute from table B. There is no way to join tables. You have to query each table directly to get the desired info.

Do you see what i mean ?


No. The movie manager I use is built on a relational database. It's far more effective in relating movies to the people associated with them, and relating people to the movies they are associated with (as "filmographies"). What I've proposed, however, is appropriate for a media manager that is not trying to be a full-blown movie manager.

I'd ask why its far more effective even if darichman's proposal is implemented but think maybe its better he answer this one :)
Logged

rick.ca

  • Citizen of the Universe
  • *****
  • Posts: 3729
Re: YARB -- Relational Database Annual Convention
« Reply #53 on: April 18, 2009, 02:56:59 pm »

Quote
Do you see what i mean ?

All I see is intellectual dishonesty and an unwillingness to allow others to share ideas in a productive and positive manner. EOD.  ::)
Logged

gappie

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 4583
Re: YARB -- Relational Database Annual Convention
« Reply #54 on: April 18, 2009, 03:07:18 pm »

All I see is intellectual dishonesty and an unwillingness to allow others to share ideas in a productive and positive manner. EOD.  ::)

its interesting to see how you comment on people that just disagree on parts and want to discuss the ideas you present in a productive and postive manner.

I'm sorry, but none of these statements are justified based on the ideas presented. Perhaps you mean you don't want these additional features/improvements, you fear they would be implemented poorly—so as to reduce, instead of increase flexibility—and you're sure everyone else agrees with you.


just be open to other ideas and people who do take the time to read all you typed and disagree, could make the discussion much more interesting.

 :)
gab
Logged

JimH

  • Administrator
  • Citizen of the Universe
  • *****
  • Posts: 72544
  • Where did I put my teeth?
Re: YARB -- Relational Database Annual Convention
« Reply #55 on: April 18, 2009, 04:06:24 pm »

Logged

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re: YARB -- Relational Database Annual Convention
« Reply #56 on: April 18, 2009, 05:00:35 pm »

What we need now is more examples, in addition to the 3 already suggested in Darichman's post.

Once ppl get an idea of what is possible it might help clariify the potential here.

In the music world, you could have [Artist] bios. A band of several members, could be defined as list fields for it. Eah of those members could also be [Artists] on their own.

The other question i have is assuming the system is in place how much work would be required to make use of it ?

How do we tranform [Fields] to [Entities], just a Move/Copy Field or more  ?
Logged

darichman

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1363
Re: YARB -- Relational Database Annual Convention
« Reply #57 on: April 19, 2009, 12:18:43 am »

Rick, hit_ny I think we're getting caught up in terminology and nomenclature again... And, I don't think what you guys are suggesting is very different at all functionally - the confusion is all tied up in what we each understand of the nomenclature. It's definitely clear to me that what I thought was relational may not, in fact, be relational in the strict sense.  So, as one or two others have suggested... let's throw around some illustrative examples - it's clear there are a few ways all of this might be done developmentally, but in the end it's the user functionality we're interested in. If we can agree on a functional model, then and only then maybe J River could offer much in the way of feasibility :)

Also, I would suggest that any implementation should require as little work for the user as possible (ie be as transparent as possible). Both Rick and hit_ny have suggested that new "entities" be created manually. Does this need to be the case? What if simply entering a value for an existing field (let's say [Artist]) worked as creating an entity as we've defined above. This "Artist" entity could then be tagged with fields assigned specifically to the [Artist] parent field.

Here are three examples of what I'd like to see. One each for Audio, Video and Images (Oh My!)
I'm going to use "Parent Field" and "Child Field" terminology (because that's what makes sense you me) but you could easily substitute "entity" for "Parent Field" I think. Note that some of the fields I mention below are custom fields. My proposal for assigning Child Fields to Parent Fields (something which need only be done once) can be seen here: http://yabb.jriver.com/interact/index.php?topic=51456.msg351225#msg351225

1. Artists
The Parent Field is [Artist]
Child fields might include [Nationality] [Bio] [Formed] [Disbanded] [Discography] [Primary Genre] [Primary Styles] [Primary Moods] etc

So, once child fields have been assigned to the Parent Field as described above...
1. I import a file for tagging.
2. "Queen" could be added to the [Artist] field just as it is in the current version. Doing so would automatically create "Queen" as a taggable entity. It is an "Artist" which now exists in the database, just as a file does now.
3. Child fields [Nationality] etc could be filled in for "Queen" as per below
4. If I import a new file and tag [Artist] as "Queen", the file will inherit all of the Child Field values assigned above. Perhaps this could be accompanied by a "Would you like to assign this file to the Artist "Queen" and inherit all associated metadata?"
5. If I select a file already assigned to Queen with the artist metadata already filled out, and change the value of of a child field - [Nationality] for example - the user might receive the message: "Are you sure you which to change the Artist "Queen"? Note this change will affect all files assigned to the Artist "Queen"

This is the interface the user might see in the Tagging AW

>   Artist: Queen
            Bio: … (extended field)
            Nationality: UK
            Primary Genre: Pop/Rock
            Primary Styles: Glam-Rock; Hard-Rock; Album-Rock etc
            Members (a list field): Bon Scott; Malcolm Young
            Formed: 1971 in London, England
            Disbanded: 1995
>   Album: A Night at the Opera
            Album Genre: Pop/Rock
            Album Year: 1975
            Album Styles: Prog-Rock/ Art Rock etc
            Album Moods: Witty; Theatrical; Elaborate etc
            Label: Elektra
>   Rest of the fields etc etc

Note that all of the fields listed above exist/can be created in the current MC - the key is that the child fields (tabbed) are all common to and inherited based on a singular Parent Field. I know some people get iffy about affecting files which aren't currently selected - but we're talking about a shift here from "changing the files" to "changing the artist to which the files belong" - ie any fields you assign as child fields of "Artist" should have the same values for all files belonging to that artist.

2. Movies
The Parent Field is [Movie] <- I use this as a custom field for the movie name
Child fields might include [Country] [Director] [Actors] .. pretty much all the new video metadata fields which have been added recently.

Movies as a media concept will generally have fewer files, but the idea might be useful when we're also including associated media like posters, trailers, special features etc. Having a "Movie" existing as its own entity in the database can act as a mechanism to tie together all media related to that movie. This seems much tidier and intuitive than having a whole bunch of disparate media files which exist separately in the database - MC has no real way of knowing they share a common element. Remember - we're talking here about building a movie database and then assigning files to that movie...

>  Movie: The Crow
            Year: 1994
            Director: Alex Proyas
            Country: USA
            Actors: Brandon Lee; ...
            Genre: Action; Fantasy
            Subgenre: Action Thriller, Superhero Film, Tech Noir
            Studio: Miramax
            MPAA: Rated R for a great amount of strong violence and language, and for drug use and some sexuality.
            Synopsis: Based on the graphic novel by James O'Barr... (ext. field)
            Review: (ext. field)
            Themes: Righting the Wronged, Reincarnation, Dangerous Friends, Supernatural Romance
            Tones: Goth, Stylized, Gritty, Menacing, Bittersweet, Elegiac, Dreamlike
            Flags: Graphic Violence, Sexual Situations, Nudity, Adult Language
            Keywords: revenge, crow [bird], murder, martial-arts, reincarnation, family-member, multiple-murder
>   Rest of the fields etc etc

3. People
Just a simple one here to illustrate how this might be useful in photos
The Parent Field is [People]
Child fields might include [Association] [Birthday]
I'll define association as their loose relation to the user - ie values might include: Friends, Family, Colleagues, Friends of Friends

So, once child fields have been assigned to the Parent Field as described above...
1. I import a photo for tagging.
2. "John Smith" and "Anna Claire" could be added to the [People] field, along with any other people who may be in the photo, just as it is in the current version. Doing so would automatically create "John Smith" and "Anna Claire" as taggable people entities. They are "People" who exist in the database and may be tagged just as files are now. I'm going to say that John Smith is my brother (family) and Anna Claire is a friend.
3. Child fields eg. [Association] could be filled in for "John Smith" and "Anna Claire" as per below
4. If I import a new file and tag [People] as "John Smith", MC will know that John Smith is family - I need not enter that value again. Once again this could be accompanied by a "Would you like to assign this file to the Person "John Smith" and inherit all associated metadata?" This prompt could perhaps be optional.

Once again, the interface the user might see in the Tagging AW

>   Album: Melbourne Trip
>   People: John Smith
            Association: Family
            Birthday: 4/5/1982
>   Rest of the fields etc etc

-------------

I think the advantage of these approaches are as follows:
1. Less repetitive work for the user - adding files to an existing entry in the database is much easier than re-entering the same values for the same fields each time I add new files.
2. Logical grouping of common concepts - in effect, we can easily group fields as they relate to common entities like [Artist] or [Movie]. Adding a file to a [Movie] immediately tells MC (and the user) a lot about the file.
3. Less ambiguity about some fields - using this approach, the user can easily tell the difference between tagging the year of a file, or the year an album was released.
4. Intuitive and useful artist/album/movie summaries might be easily created based on this info.
5. The entire concept is completely optional to the user... if they never assign child fields to parent fields - nothing changes and they continue to use MC as they have been.

Quote
The other question i have is assuming the system is in place how much work would be required to make use of it?

It depends on whether you're talking about the user or the developer here. In relation (excuse the pun) to the user, hopefully I've come close to answering some of that above. From a development standpoint - while those of us around here with programming & database knowledge around here (ie not me - and I will emphasise that!) might be able to offer some ideas, in the end only J River can answer that one. Matt/Jim... do any of the suggestions made so far seem feasible?

I think the aim of this thread is to state what individual users would like to see in terms of functionality, establish whether there is a wider interest for proposals made... and then based on that establish dialogue about whether J River thinks this model might work.

It took me a while to write this - sorry I haven't posted for a while....
Logged

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re: YARB -- Relational Database Annual Convention
« Reply #58 on: April 19, 2009, 04:37:55 am »

Will use the term parent-child fields from now on as it
- is accurate
- is less intimidating
- keeps this proposal feasible as I believe it does not impose a fundamental change to the existing library structure.

Custom fields brought MC to where it is today, parent-child fields will take it much more further :)

The roots for this proposal i think lie in the way MC can be used as shown here. Some of the suggestions ought to be on MC's home page, you won't get a better summary than this.

Also, I would suggest that any implementation should require as little work for the user as possible (ie be as transparent as possible). Both Rick and hit_ny have suggested that new "entities" be created manually. Does this need to be the case? What if simply entering a value for an existing field (let's say [Artist]) worked as creating an entity as we've defined above. This "Artist" entity could then be tagged with fields assigned specifically to the [Artist] parent field.

It assumes that the user has already designated  [Artist] as a parent field or had it done for them. If so then yes, it should work exactly as you described.

This takes us to the next question as to how much work is required for the user to avail of this new functionality. By that i mean, you are on build 'x' which is like any other currently and then one fine day, out of the blue, you see this...

Quote
NEW: MC now supports Parent - Child fields

Now correct if I'm wrong but i think you visualise the following :
- All existing fields suddenly acquire the ability to add child fields and  that every new [Field] created hence, automatically also has this ability. There will be one more button in the 'Manage Library Fields' dialog with the title 'Add child fields'

- All that is required is for the user to specify (if desired) which child fields and then the definition is complete. This way its completely transparent to those that won't need this ability. Life goes on as normal for them.

Quite acceptable :)


I think the advantage of these approaches are as follows:
1. Less repetitive work for the user - adding files to an existing entry in the database is much easier than re-entering the same values for the same fields each time I add new files.
2. Logical grouping of common concepts - in effect, we can easily group fields as they relate to common entities like [Artist] or [Movie]. Adding a file to a [Movie] immediately tells MC (and the user) a lot about the file.
3. Less ambiguity about some fields - using this approach, the user can easily tell the difference between tagging the year of a file, or the year an album was released.
4. Intuitive and useful artist/album/movie summaries might be easily created based on this info.
5. The entire concept is completely optional to the user... if they never assign child fields to parent fields - nothing changes and they continue to use MC as they have been.

Thats a very good summary :)

Here are some thoughts i've had regarding performance and any impact this proposal might have. I'll use King as an anecdote here. He had in excess of 250k audio files. He also wrote a cpl of plugins to populate the Bio & Lyrics fields. So lots of text was added to the library. This would have been circa v10 or v11. He soon found out the library became sluggish and had to break up the library into chunks of 50k files each to improve usability.

Now with v12, a lot of speed improvements were made, MC could handle all the extra text and it was no longer necessary to partition the library.

So, now in the v13 era, not seeing too many problems with the additional text, even upto 200k files.

The other point is that when they introduced nested fields, there was a performance hit as it had not been optimised as yet. But this hit only affected those users that wanted to use nested fields. In came v13 and the handling speeded up.

All this is to say there might be a slight hit in performance intially to those that want to make use of this feature but over time it will diminish as they OWN the libray and can tweak it up as they always do :)

PS: Here is  one more example with classical music, about as meta-data rich as music gets, Imagine how much simpler your post here will become !!
Logged

darichman

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1363
Re: YARB -- Relational Database Annual Convention
« Reply #59 on: April 19, 2009, 05:57:41 am »

Now correct if I'm wrong but i think you visualise the following :
- All existing fields suddenly acquire the ability to add child fields and  that every new [Field] created hence, automatically also has this ability. There will be one more button in the 'Manage Library Fields' dialog with the title 'Add child fields'
- All that is required is for the user to specify (if desired) which child fields and then the definition is complete. This way its completely transparent to those that won't need this ability. Life goes on as normal for them.

That's pretty much exactly what I imagined :)

PS: Here is  one more example with classical music, about as meta-data rich as music gets, Imagine how much simpler your post here will become !!

It's funny you mentioned that - it was tagging classical music a few years ago which got me interested in all this. "Composers" and "Works" are two fields which could benefit readily in this area. Both may appear across disparate files belonging to various albums, artists, performers, conductors etc... Simply having a discrete and tangible [Work] entity to which you could add new files as you collect them would save hours of time for the classical music collector, in terms of inheriting metadata for the fields (which could now be "Child Fields") described in that thread.

Some other problems from the board this approach might solve:

1. Allowing various files, including posters, trailers, special features ets to be assigned to a [Movie] parent field might help solve this: http://yabb.jriver.com/interact/index.php?topic=49820.0
In fact, if J River implemented this, many doors might open up as far as interface goes... Consider being able to click on a movie thumbnail in theatre view, have a roller appear with "Play Film" "Watch Trailer" "View Posters" "Hear Soundtrack" ... Once users have added various media files to the [Movie] parent field, this could all occur with very little effort. All you need is a simple field to tell what type of media a file is (Film, Trailer, Soundtrack etc) and then have the user assign the file to a movie.

2. Some people have expressed an interest in creating records where files do not exist. Perhaps this could be done through taggable Parent Fields. So... a user could create an [Artist] or a [Movie], but have no files assigned to that artist or movie - so you could still build a movie database for example, without necessarily having files for that movie... This functionality could rival software like Collectorz and cataloguing software. You could use MC to catalog your books with a [Book] parent field and tag the book with fields like [Read] (read or unread) and add coverart as per point 3 below. So the user can catalog a collection of pretty much anything without needing digital files on the computer.

3. The issue of Artist pictures comes up again and again. How about adding the ability to add [Image File] as a child field to any parent field? This way [Artist], [Album], [Movie], [Book] or anything the user might imagine could easily be assigned art separately. We would certainly still have file coverart as it exists now, but any parent field could have its own art as well, if the user chooses to use it. Users could browse Artists by an artist picture, movies by posters etc when using thumbnails for viewschemes etc. Consider the ability to add image files to the [Media Subtype] field - users could have their own icons, in effect, to browse through the various media types in their library. Let's face it, the fanned/stacked thumbnail approach of every file thumbnail in a category isn't very useful...

Thoughts?
Logged

darichman

  • MC Beta Team
  • Citizen of the Universe
  • *****
  • Posts: 1363
Re: YARB -- Relational Database Annual Convention
« Reply #60 on: April 19, 2009, 06:26:53 am »

Custom fields brought MC to where it is today, parent-child fields will take it much more further :)
The roots for this proposal i think lie in the way MC can be used as shown here. Some of the suggestions ought to be on MC's home page, you won't get a better summary than this.

Haha, I'd forgotten about that thread. And yes, I think we're on the same page. I did plan to update that and write a guide (with pictures and maybe video) to show how you can use MC to organise all the media types mentioned in that thread (I do have more now :P)

Most people use MC for audio, and many for video and photos. What I think MC could work towards is a shift from this "Media Format" paradigm to a "Media Concept" paradigm. So what on Earth do I mean by that? Simply that files may belong to multiple forms of media defined variously in a library as... Music, Movie, Television, Book, Game, Art (and some others). If the above proposal were implemented, each of these could be tagged as parent fields and I'd have the perfect player/organisation tool/cataloguer and, if theatre view continues to improve, HTPC solution.

To illustrate...

1. The concept of "Music" exists both as a Media Format and a Media Concept. As a media format, it is simply audio files... belonging to albums and soundtracks etc. As a media concept, "music" might include albums, music videos (which could not by definition belong to music as a media format) and any images associated with the album or its artists. It's all music.
2. Similarly, for movies. "Movie" as a media format is video files. I use "Movie" as a media concept and include all the other types of media associated with the movie (trailers, posters, scripts, special features)

So to give an example... A movie soundtrack exists in my library both as "Movie" and "Music". That's what makes sense to me. If I want to limit to actual audio or video files, I can use the [Media Type] field, but that's not how my library is structured.

I was hoping for a more user-friendly way of tying in those media categories from the software end, however, and believe the ideas in the last several posts here could help achieve this.

For what it's worth, I still use the [Type 1] and [Type 2] fields mentioned in that thread. They are list-type fields and not as limiting as [Media Type] and [Media Subtype] :)
Logged

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re: YARB -- Relational Database Annual Convention
« Reply #61 on: April 19, 2009, 02:09:20 pm »

3. The issue of Artist pictures comes up again and again. How about adding the ability to add [Image File] as a child field to any parent field? This way [Artist], [Album], [Movie], [Book] or anything the user might imagine could easily be assigned art separately. We would certainly still have file coverart as it exists now, but any parent field could have its own art as well, if the user chooses to use it. Users could browse Artists by an artist picture, movies by posters etc when using thumbnails for viewschemes etc. Consider the ability to add image files to the [Media Subtype] field - users could have their own icons, in effect, to browse through the various media types in their library. Let's face it, the fanned/stacked thumbnail approach of every file thumbnail in a category isn't very useful...

Thoughts?


Mentioned here

Within you will find an interesting post

How do we manage multiple images? With cover art and other meta-data, there is one (or maybe a few) correct answer. But with artist images, there are many "correct" answers.  At 700,000+ artists, that could be quite a bit of data, both in terms of storage and bandwidth.

This explains possible reluctance on allowing multiple images per album or artist.

It also begs the question of how YADB will deal with parent-child fields and reconcile user submissions. Possible deal breaker here ?

Think of Bios, if there is just a character different it overwrites whats already submitted by someone else :(

Allowing users to correct data is a wonderful and, at the same time, scary thought. I'd like to hook a wiki for metadata, but that's a big project with uncertain payback.
Logged

hit_ny

  • Citizen of the Universe
  • *****
  • Posts: 3310
  • nothing more to say...
Re: YARB -- Relational Database Annual Convention
« Reply #62 on: September 03, 2009, 10:21:40 am »

Always interesting to see how relationships are implemented in other software.

Check out how IDimager does it.
Logged
Pages: 1 [2]   Go Up