Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[JimH]

A couple of people mentioned this.

Marty3d

* Relational db (I've mentioned motivation for this so many times, I won't go there again)

leezer3

4. Finally, the pet wish of many of us- A relational database. As you're adding more ways to display & use metadata, it becomes more and more important to have some degree of relativity, however basic.

[gappie]

i have no problem with a relational database if the datebase will be as flexiable as the database is now. if the relations are user controlable and the user can make its own tables. and i guess that it should also be easy to understand for new users..

 
gab

[rpalmer68]

i have no problem with a relational database if the datebase will be as flexiable as the database is now. if the relations are user controlable and the user can make its own tables. and i guess that it should also be easy to understand for new users..

 
gab

And it needs to be designed so it can be multi-user.  For me the multi-user part is more important than the relational side as I'm yet to see where I actually need a relational DB but need multi-user now.  Of course once it's there I'm sure I'll use it

R

[GHammer]

Can you explain to me why I want a relational database?
In easy to understand terms.

[hit_ny]

THE person to consult on this is darichman, he has tenaciously posted on this topic at least for the last 3 years.

The YARB thread i think is definitive in terms of his requirements defintion.

YARB -- Relational Database Annual Convention

And the term relational is a misnomer, there is nothing relational in the suggestion wrt to the user. All it suggests is parent-child fields.

[gappie]

THE person to consult on this is darichman, he has tenaciously posted on this topic at least for the last 3 years.

The YARB thread i think is definitive in terms of his requirements defintion.

YARB -- Relational Database Annual Convention

And the term relational is a misnomer, there is nothing relational in the suggestion wrt to the user. All it suggests is parent-child fields.

i dont think its a misnomer, the parent child fields is an other option that has potential. i guess at this moment there is already a few fields that works that way. calculated fields, but still. one is 'album gain'. change the replay gain for one song and see how the album gain is changed for all the songs of that album. the 'album gain' is the child, and the album is the parent (not really, and that makes it more complex, its a combination of album and album artist (auto)).

GHammer: one example a relational database could do, or the parent child field: a lot of times people asked for an album rating field. based on the average of the song rating, or someting a user can set him self. in the case of the relational database the album rating has to be set one time and it shows up for all songs. the parent child variant is in the end the same because the field are set by mc in every song when it changes for one song. (the same problem with parent child field as for replaygain is around the corner, i would not want all my albums called 'live ' have the same rating, in principle this problem should not be there in a relational database.)

the possebilities are big, as shown in the yarb thread, but the complexity introduced should not be ignored and will have its price, and i am worried about that a bit..

 
gab

[hit_ny]

i dont think its a misnomer,

I meant in the sense that MC is wrt to the user nothing more than a fancy shopping list.

In  a relational dB the user can specify links or relations between entities. In MC the only entity is the file. There is no way to specify a link between files automatically. You can perform operations manually via the GUI, ie setting say the Album field or other for a bunch of files.

You cannot use a key from one file to get information about another etc.  I think this is a key requirement to use the term relational. Relational implies links or relations between entities.

Now it could well turn out that under-the-hood things are linked up and indirectly DO work in this manner but this functionality cannot be customised specifically by the user. The user is unable to specify what links to what, expressions work only on per file level.

The only user link is that of the GUI with the file.

change the replay gain for one song and see how the album gain is changed for all the songs of that album.

This is only an exception rather than the general case.

a lot of times people asked for an album rating field. based on the average of the song rating, or someting a user can set him self. in the case of the relational database the album rating has to be set one time and it shows up for all songs.

I would love to have this kind of functionality, but it would need a grouping ability i think to work. In the sense you would define a group, in this case the [Album] field and then perform operations on all files in that group for many [Albums].

I'm not sure how parent-child could be used in this case.

the possebilities are big, as shown in the yarb thread, but the complexity introduced should not be ignored and will have its price, and i am worried about that a bit..

This would depend on how open-ended the implementation is. But as darichman has spec'd it out it does not seem to be more problematic that nested fields are currently. Three years ago i would have agreed with you as the library was then not as optimised to the point it is now. My collection has grown during this time but the machine i use is still the same and MC has defnitely become faster in response

I pointed out a potential problem with YADB tho i'm not sure how much of a showstopper that will be. People currently use different naming conventions for the same file and it can handle this already.

[gappie]

I meant in the sense that MC is wrt to the user nothing more than a fancy shopping list.

i agree, also with the rest of your comment to that item.  

This is only an exception rather than the general case.

i also agree, just wanted to point out that there was already something like that.

I would love to have this kind of functionality, but it would need a grouping ability i think to work. In the sense you would define a group, in this case the [Album] field and then perform operations on all files in that group for many [Albums].

I'm not sure how parent-child could be used in this case.

i think this is what the patent child is all about. when it is just a wizzard you have to run manually its great, but does not do any right to what people want when asking for an relational database. when i get the artist bios filled in i would expect that when i change a typo in it, it will change for all the fields under the same parent.

This would depend on how open-ended the implementation is. But as darichman has spec'd it out it does not seem to be more problematic that nested fields are currently. Three years ago i would have agreed with you as the library was then not as optimised to the point it is now. My collection has grown during this time but the machine i use is still the same and MC has defnitely become faster in response

I pointed out a potential problem with YADB tho i'm not sure how much of a showstopper that will be. People currently use different naming conventions for the same file and it can handle this already.

i use a lot of nested fields, i think they are still problematic.  
yes, the database now is working great. and to be sure to get this over, im not opposing the idea, just worried about some black clouds at the horizon.

 
gab

edit.. with all this fancy quoting i deleted part of my answer .. retyped a shorter version. 

[Marty3d]

Here's a quick list of why using a relational database could be of use for the end user:

• Fields that are unique for an artist, ie Origin, Age, Genre, Pictures associated to an artist etc. would be set only once. When you buy a new album by that artist and import it into MC, all those fields are automatically populated. This means less work.
• Album art is connected to an album, not all files within that album
• Play statistics. Right now, you can only see when a song is LAST played (two fields are filled out, Play count and Last played). That means you cannot see stuff like: 1. "Did I play this song during the last Christmas party?" 2. Yearly/Monthly/Weekly top lists (Top 100 songs 2006 for example)

[gappie]

Here's a quick list of why using a relational database could be of use for the end user:

• Fields that are unique for an artist, ie Origin, Age, Genre, Pictures associated to an artist etc. would be set only once. When you buy a new album by that artist and import it into MC, all those fields are automatically populated. This means less work.
• Album art is connected to an album, not all files within that album
• Play statistics. Right now, you can only see when a song is LAST played (two fields are filled out, Play count and Last played). That means you cannot see stuff like: 1. "Did I play this song during the last Christmas party?" 2. Yearly/Monthly/Weekly top lists (Top 100 songs 2006 for example)

now .. i would love something like your 3rd item. and that is something that really calls for a relational database.

 
gab

[hit_ny]

i also agree, just wanted to point out that there was already something like that.

Yeah, thx for that it goes to show that this functionality could be extended to be user configurable.

i think this is what the patent child is all about. when it is just a wizzard you have to run manually its great, but does not do any right to what people want when asking for an relational database. when i get the artist bios filled in i would expect that when i change a typo in it, it will change for all the fields under the same parent.

Let me explain parent-child in another way for those that still have not understood darichmans proposal.

All he's asking for is to be able to create user records.

To go from custom [Field] to custom [Record]

For a record like [Actor] to be able to hold child fields like [Bio], [Filmography], etc. But now [Actor] is a record instead of just a field and can contain other embedded fields in it.

Thats it.

When you import a movie that has the same actor then all you would do is select the actors name from the pulldown and assign it just like you do with [Artist] fields currently. The difference is that in addition to just the [Actor] name you would get the child fields along as well. End result is less repetitive work for media that has complex meta-data like movies or even classical music.

So you can see there is nothing relational about this suggestion at all and daresay its pretentious

i use a lot of nested fields, i think they are still problematic. 

Ah, i think you should start a thread about it then and give your suggestions.

I've used nested fields for record labels with audio and did not notice much problems.

now .. i would love something like your 3rd item. and that is something that really calls for a relational database.

Simple way to do this is to convert [Last Played] into a list like field that contains the time& date of say the last ten times played.

[darichman]

I would still like to see this.

Rather than posting again here, I'll point to a few posts which I think summarise potential endpoints well:

Functional Outcomes
Practical Examples
Potential Implementation

To briefly summarise potential advantages

1. Less repetitive work for the user - adding files to an existing entry in the database is much easier than re-entering the same values for the same fields each time I add new files.
2. This could solve the common request for Artist Coverart etc... each parent field could have an [Image] field as a child field.
3. Logical grouping of common concepts - in effect, we can easily group fields as they relate to common entities like [Artist] or [Movie]. Adding a file to a [Movie] immediately tells MC (and the user) a lot about the file.
4. Less ambiguity about some fields - using this approach, the user can easily tell the difference between tagging the year of a file, or the year an album was released.
5. Intuitive and useful artist/album/movie summaries might be easily created based on this info.
6. The entire concept is completely optional to the user... if they never assign child fields to parent fields - nothing changes and they continue to use MC as they have been.
7. Calculations across fields

[hit_ny]

i dont think its a misnomer, the parent child fields is an other option that has potential. i guess at this moment there is already a few fields that works that way. calculated fields, but still. one is 'album gain'. change the replay gain for one song and see how the album gain is changed for all the songs of that album. the 'album gain' is the child, and the album is the parent (not really, and that makes it more complex, its a combination of album and album artist (auto)).

yes and no. Logically they appear similar but functionally they are different.

The Album Gain example performs some calculations before assigning values to album gain. Currently its a hard-wired expression that works across files inside MC and is done during AA. I would like in the future to have the ability to define what grouping so it could be done automatically with expressions. Because it would allow expressions to automatically work 'across-files' in addition to the 'within-files-only' way right now.

So i think its different to parent-child or super-fields which is only a way to assign a previously entered, fixed value (rather bunch of values) to another. There is no calculation or additional processing on top nor is there any requirement to do so.

[KingSparta]

I Have No Idea What A "Relational Database" Is

Is Not A "Database" Is A "Database"

What You Do With The "Database" After You Have A Database Is More Of A Program Function.

[dcwebman]

Unless I missed someone saying it, the item a relationship database could do that people have been asking for years is the ability to assign the exact same song to multiple albums. The answer was always that hard drive space is cheap but why waste the space if all you have to do is point one song to multiple albums? That gives more room to add other songs.

[leezer3]

Couple of thoughts-
A user submitted YADB isn't really feasible on a relational scale, simply because of the scope for variation. It could possibly be an option to partner with a data source provider if you wanted to implement a DB lookup for relational actor/ artist biographies etc though.


Parent/ child fields aren't bad, but they're nowhere near good enough. Here's an example for you-
I have a lot (C. 2,000) of audiobooks. The basic fields I've used at present are these (Plenty of others, but these are irrelevant at present):
* Author
* Narrator
* Studio (For the audiobook, may also contain publisher)
* Publisher (For the print edition)
A good example for this is Stephen Fry- He is present in both the author and narrator fields. Adding a biography child field would have to be done in both the author and narrator fields, and any updates would have to be manually synced across. This issue also holds for publishers etc.
On the other hand within a relational DB, all that needs to be done (I'm well aware I'm over simplifying the technicalities here) is to establish a link between the base biography field for Stephen Fry, and anywhere that he is used

Another issue that would be addressed by a relational DB, would be the ability to add a rating for anything. At present, there's no easy way to apply an overall rating to an artist etc, other than either adding an identical field to all of their files, or a smartlist average. Neither of these is anywhere near perfect, and requires manual updating at best.

Cheers

-Leezer-

[hit_ny]

A good example for this is Stephen Fry- He is present in both the author and narrator fields. Adding a biography child field would have to be done in both the author and narrator fields, and any updates would have to be manually synced across. This issue also holds for publishers etc.

By manually are you saying one has to do more than just assign Fry to another file ?

On the other hand within a relational DB, all that needs to be done (I'm well aware I'm over simplifying the technicalities here) is to establish a link between the base biography field for Stephen Fry, and anywhere that he is used

You are correct that the Bio field would need to be added twice but its a one off unless the versatile Fry appears in a third role

Chances are under the hood this is exactly what happens. Library backups are half the size in 13 compared to v12 so big time pooling happening.

Course the challenge then becomes how do you do the GUI ?

Another issue that would be addressed by a relational DB, would be the ability to add a rating for anything. At present, there's no easy way to apply an overall rating to an artist etc, other than either adding an identical field to all of their files, or a smartlist average. Neither of these is anywhere near perfect, and requires manual updating at best.

Yep

Unless I missed someone saying it, the item a relationship database could do that people have been asking for years is the ability to assign the exact same song to multiple albums. The answer was always that hard drive space is cheap but why waste the space if all you have to do is point one song to multiple albums? That gives more room to add other songs.

I'm not sure how it could be done because there is no suggestion to have a one to many relationship with files. The 1:1 connection will still be retained between the actual files and the list view.

[leezer3]

By manually are you saying one has to do more than just assign Fry to another file ?

I'm referring to the major issue with parent/ child fields here which a relational structure would avoid-
Basically, there would be two distinct copies of the data, one the child of the Author field, and one the child of the Narrator field
Updating one in a parent/ child structure would not update both.

You are correct that the Bio field would need to be added twice but its a one off unless the versatile Fry appears in a third role Smiley

Chances are under the hood this is exactly what happens. Library backups are half the size in 13 compared to v12 so big time pooling happening.

Course the challenge then becomes how do you do the GUI ?

Stephen Fry was just an example really
If things go down this road, I'll need to link his biography to the Actor field, and probably a couple of others as well. An extreme example I admit, but these are the best way to break things  

The GUI is perfectly simple when you think about it though. The first thing that needs to be done is to lose much of the current concept of a database field & its contents (Not literally, but I can't think of a better way to put this)-
1. There need to be three distinct types of fields (Again, field isn't really the correct term here)-
* 'Unique per file' fields. These would consist of data such as title, track number etc. (Brown)
* 'Containers'. These fields are simply a container for global fields. (Yellow)
* 'Global' fields. These would be fields which are common between more than one file. They should be able to have child fields added, which can be expanded by default or on a per file basis (White)

It would be necessary for a global field to be a member of any number of containers, all these really are is lists
In this example Stephen Fry is a member of both the Authors and Narrators containers, but only his role as a narrator is relevant, hence it is only here that he is displayed.

2. A small button needs to be added to the tag window allowing us to select which Containers our example file possesses (These could be there by default if you prefer). When we've established that it possesses the Narrator and Author containers, these should be editable as any list based field is at present- Select which global field(s) to display from within them.
3. Our file now posesses the Narrator and Author containers, and we've selected the appropriate global fields from within them. The child data for these global fields now populates the file information, so that the presented tags for our file would now look something like this:
Name: Harry Potter
Track: Track #1
Author: JK Rowling
Narrator: Stephen Fry
Narrator Biography: ...........

Obviously, if we don't want his biography displayed in this particular instance, all it needs is a simple right-click option to hide this child field

(I apolgise for the rambling nature of this, it's harder to explain on paper than it is to build, honest)

-Leezer-

[hit_ny]

No its good what you've said. I think it adds more flexibilty without any sacrifices.

i use a lot of nested fields, i think they are still problematic.

There is one point i can offer in support of parent-child here

Parent child as i understand it and if darichman agrees, only allows one generation of child. Whereas there is no limit with how many generations one can use with nested fields.

Now whether limiting child to just one generation is a good idea or not from a user perspective, is yet to be decided. But at the implementation level it would keep things quick without too much complexity.

[GHammer]

Hmmm.

I see lots of development with little payback.

One thing that comes up over and over is how you could (note the could) assign coverart/images to the same artist. Ok, I'll bite. How many of the same images do you have? Not real practicle for movies I'd think. For a band, maybe. For books, ok, got me there.
A bio would be more useful as a repetitive field.

What fields do you see as most important?

[gappie]

There is one point i can offer in support of parent-child here

Parent child as i understand it and if darichman agrees, only allows one generation of child. Whereas there is no limit with how many generations one can use with nested fields.

Now whether limiting child to just one generation is a good idea or not from a user perspective, is yet to be decided. But at the implementation level it would keep things quick without too much complexity.

yes.. i thought the same, on the otherhand, a relational database could make the nested fields more or less redundant... 

what i would like to know is: is mc really thinking about the possebilities of a relational db or a child parent one? all options have pros and cons, and it depends so much on the implementation, that i think the discussion about which is to preferr is not so interesting as the one what should be the end result. i mean.. what is the goal. and i think the answer for that is fairly clear when reading this thread and a few previous ones.. which tool to use to get there is the next step.

 
gab

[leezer3]

Important fields which I'd be using in a relational manner (I use these in Artist, Director, Narrator and Author at present, but would like to extend these further):
* Biography.
* Notes. (Different purpose to biography, may contain comments on style etc)
* Portrait.
* Rating- Important for all of these I've currently got a kludged smartlist, but the ability to assign a unique rating to anything be exceedingly nice; For example:
List all audiobooks with an author rating of 3 or above, and a narrator rating of 4. This basically gives the opportunity to revist things, and also a good way to instantly sort new stuff into what it's likely to be like. An album rating which isn't a kludge would also be most welcome.

I'm also looking to be able to do things like adding a DVD cover to a TV series, and then having individual thumbnails for each file. The theatre view viewscheme for this would work something like this:
1. Root level- List of genres. Basically as it is now.
2. Second level- List of overarching series within a genre. Again relatively as it is now- Thumbnail displayed would be generated from the series images (DVD covers).
3. Third level- List of induvidual series. Thumbnail for each series is the series image (DVD cover for individual series).
3. Final level- Episode thumbnails.
Equally, the second level could be eliminated (Or alphabetical for that matter), but I prefer to keep remakes etc. within a series structure.


If you want me to be honest, a relational DB is one of those things which you're either going to use or not. I have a combined total of over a thousand authors and narrators which if a relational DB was implemented I'd be doing major work on, simply as all of the data is of use to me one way or another
The core programming concepts themselves are also relatively simple (Although I'll freely admit that implementing them into something like MC may require very major work; All I really am is a dabbler at best!), all it really requires to do this is an appropriate way to query the database and provide the data returned in an intelligent way
Something like this anyway- An author is stored in the file's unique record. The DB engine then queries the author's container and locates the author named. The data for the author is then presented transparently within the file's tags and is editable. Any changes are written back to the global author.
Most of the work would IMHO go into sorting the GUI and presentation so that things seem relatively transparent, rather than pure backend work.

Cheers

-Leezer-

[GHammer]

I think the entire project would depend upon the existing MC database.

If it were easy (relative) to extend/change to a relational database, if all the code that uses the database were reusable, then I'd be in favor.

If those things were not true, then it wouldn't matter because I suspect that a rewrite of MC is not gonna happen.

For my limited needs, the stock tags suffice. For audio. Video and images, well... But those tags would be served fine from the existing database.

[darichman]

I was a little reticent to ask for something that would be a) too hard for the user to pick up b) too difficult to implement, which is why we came up with the Parent/Child field concept, which isn't a big jump from where we are now. It's simplistic, but still more functional than the current "file-centric" model.

Put simply, I think MC could benefit greatly from the ability to establish "Containers" (with their own attributes, art etc). Files and other containers could then be assigned to a container and inherit any associated information. Call them containers, parent fields, elements, or whatever... the naming isn't all that important, just the functional outcome for the user.

Parent/ child fields aren't bad, but they're nowhere near good enough. Here's an example for you-
I have a lot (C. 2,000) of audiobooks. The basic fields I've used at present are these (Plenty of others, but these are irrelevant at present):
* Author
* Narrator
* Studio (For the audiobook, may also contain publisher)
* Publisher (For the print edition)
A good example for this is Stephen Fry- He is present in both the author and narrator fields. Adding a biography child field would have to be done in both the author and narrator fields, and any updates would have to be manually synced across. This issue also holds for publishers etc.
On the other hand within a relational DB, all that needs to be done (I'm well aware I'm over simplifying the technicalities here) is to establish a link between the base biography field for Stephen Fry, and anywhere that he is used

I'm not attached to this idea - it's just the best I could come up with that didn't stray too far from current implementation. Would it work if child fields could belong to more than one parent field concurrently? The program would then need to know that Stephen Fry the author and Stephen Fry the narrator is the same person.

What if like fields could be grouped together in a container of sorts? Like fields would then share similar child fields.

Container: Work.... [Album] [Film] [Show] [Classical Work] [Art Collection] [Book]

Container: Credits... [Artist] [Composer] [Soloist] [Band/Orchestra] [Conductor] [Lyricist] [Actors] [Director] [Producers] [Screenwriter] [Studios] [Author] [Publisher]... all are really "Credits" entities... the actual entries ("Stephen Fry" for eg) could be stored in any of the Credit fields. The Stephen Fry entry could then be edited (bio etc) from any of the fields, as he is effectively the same person in the database (just in different "roles")

I know I just jump from one idea to the next sometimes... just brainstorming 

[hit_ny]

Haha, going from parent-child or embedding to multiple inheritance

Container as a concept does not exist in MC, this appears to be more flexible as leezer pointed out but unless Matt says something we will not know how feasible this idea is. As its implies links 'across' files whereas currently eveything is a highly tuned shopping list or line items with no knowledge of other line items. Adding links to other items will slow things down depending on how the user specifies those links. This is why MC currently can handle as many items as it does and can scale.

So to reuse code, it means keeping the same line item approach and not introducing links between items. Nested fields & list fields are  possible as its just operating on the same line item. One uses a ';' to delimit, the other a '\' but the GUI handles the display and gives the illusion of 'container' for nested fields.

How does one implement parent-child with a simple list ?

It would mean the parent would have to somehow contain a copy to the child items. Limiting it to just one generation makes it easier than having a relational or flat linked model.

I think how fast nested fields currently work or will in the future will give an indication of how feasible parent-child will be. Nested fields is parent-child with just ONE field vertically.

Parent child is doing the same thing but horizontally ie adding more fields.

[JONCAT]

Unless I missed someone saying it, the item a relationship database could do that people have been asking for years is the ability to assign the exact same song to multiple albums. The answer was always that hard drive space is cheap but why waste the space if all you have to do is point one song to multiple albums? That gives more room to add other songs.

Various pressings/masterings exist in excess in my archives...this occurs especially with "collectible" material.

DC

[raldo]

I did a text search on "slow" in the previous posts. No one seems to mention this when talking about relational databases.

If you want "fast" when dealing with relational databases, you definitively cannot use any of the off the shelf client/server software out there. You'd basically have to reinvent your database and the access API.

[JimH]

Not to throw cold water on this or anything, but...

We like the MC database.  It's fast and reliable and tuned for media.

We may extend it in the future in ways that are useful to you, but replacing it isn't even a remote possibility.

I don't think many people appreciate how solid and smart it is.

[rick.ca]

Not to throw cold water on this or anything, but...

I don't take as "cold water," but necessary clarification—one which encourages me to rejoin what has been a draining conversation...

We may extend it in the future in ways that are useful to you, but replacing it isn't even a remote possibility.

I've never doubted this is where you stand, or considered questioning the wisdom of this. And it was with this in mind that I attempted to describe a concept which might be feasible as an "extension" to the current architecture. (I did so in a series of posts, but this one might be a reasonable starting point for anyone who is interested.) To sum it up, it's simply involves adding a new kind of record that would contain whatever information a user wanted to record about specific values used in their media record categories. The record, for example, could be for a name ("value") of an artist ("category") to which the user could add things like birth date, biography, etc. Media records would be associated by virtue of a particular "value" in a particular "category." This would be enough to "link" these "additional information" records so: (1) they appear in a second view when the corresponding value is selected in a media record; (2) the additional information could be incorporated into the media record via expression columns.

I think the preceding paragraph might have been much less painful had I been permitted to use the word "relational." The definition of this word I believe is most useful for this conversation is: relating to, using, or being a method of organizing data in a database so that it is perceived by the user as a set of tables. This permits the use of the word in describing database behaviour or results that might be desirable for users (e.g., "I would like to be able to record an artist bio without having to copy it to each media record associated with that artist.") It doesn't say or imply anything about database architecture.

So, MC is never going to be built on a relational database engine. That means there will be many "relational" things it will not be able to do. But it already handles some relations—even more efficiently than a relational database would. So it's not difficult to imagine it could be extended do more. I think users are being asked what sort of relational functionality we would like to see, so the developers can consider what might be done. If we were able to do so, maybe they could provide some feedback about what is feasible from a technical point-of-view.

[JimH]

Rick,
I appreciate your thoughtfullness, in this and previous posts.

Jim

[jack wallstreet]

I don't think many people appreciate how solid and smart it is.

I suspect a lot of people who use MC feel that MC has an exceptionally fast and stable database.  We probably just don't think about it until we are reminded.

[Gl3nn]

The speed and performance of the existing database is, IMO, exceptional.  That being said, I'd love to see functionality like album and artist ratings made easily available, however it's done.

[raldo]

their media record categories. The record, for example, could be for a name ("value") of an artist ("category") to which the user could add things like birth date, biography, etc. Media records would be associated by virtue of a particular "value" in a particular "category." This would be enough to "link" these "additional information" records so:

Allow for a new media type, say, metadata. This mediatype does not require a 1-1 relationship with disk data.

You could then use smart expressions (as opposed to sql select) to tie, for example, videos together with people metadata.

[JONCAT]

I have relations with my database ALL night long....we play tag together among other things  

It's quite functional, flexible, and if I'm not in the mood, I just restore to an earlier version!

dc

[rick.ca]

Allow for a new media type, say, metadata. This mediatype does not require a 1-1 relationship with disk data. You could then use smart expressions (as opposed to sql select) to tie, for example, videos together with people metadata.

Exactly. You seem to understand I'm not suggesting how this would be implemented "under the hood." I think it's safe to assume, especially considering the obvious brilliance of this development team, that it's at least "feasible." One reason for assuming so is it's not going to have any impact on performance—unless it is used. In this sense, I imagine it's impact might be similar to the current effect of using expressions in a view. If they are many and/or complex, they will slow things down. I don't think anyone objects to the existence of this feature because this is so.

If such a thing were to be implemented, there would obviously be many interface and behaviour design issues to be considered. I think the best approach would be to keep it simple, generic and intuitive. If users can understand exactly what it does, and have complete flexibility as to where it's used, they are more likely to appreciate it's utility and accept it's limitations. So, yes, I see this simply as the ability to create a separate table unrelated to media in which meta data about category items may be stored.

The idea is probably easier to convey by example. Imagine this feature exists, but I'm not yet using it. I now decide I want to record the bio of one of my favourite actors, Billy Bob Thornton. I create a view scheme—a new type that facilitates this feature. In the view configuration, I add a mandatory "key" or index field for which I must specify a "relation" which is a category in the media record—in this case, Actor. I suppose it would be appropriate for this special field to be named "Actor." I then add a Biography field. Having defined my new view, I would then simply enter "Billy Bob Thornton" as an actor and add his biography.

And now for the interface design issues... I imagine being able to do some or all of the following:

• Incorporate Biography into any other view using an expression, which might look like =Related(Actor, Biography).
• Use a split view whereby when I select the Actor "Billy Bob Thornton" in a media view, his related record would be displayed in the other view.
• Switch to the "additional data view" using the context menu for (i.e., right-clicking on) "Billy Bob Thornton"
• See a pop-up Biography on mouse-over of any occurrence of "Billy Bob Thornton" in the media record Actor category.

Now here's another design consideration, and my real reason for using Billy Bob as an example. He's also a musician. So do we need the ability to relate these additional information records to multiple categories (e.g., so Billy Bob is associated with his bio whether he appears as an actor or a musician)? Judging from this infamous interview, Billy Bob would take your head off if he knew we not respecting his split personality. But that's kind of rare—so it would be nice to be able to specify multiple categories, although I wonder if that makes things too complicated on the technical side. If not, I would then wonder if we might have the ultimate—the ability to specify a relation using an expression!

[Matt]

If you create a field "Artist Biography" and put a long biography of Aerosmith into 1000 files, Media Center will only store the data on disk and in memory once.

So it seems like the only mechanism that's really needed is, per field, to be able to say:

• Store this field per file
• Store this field per artist
• Store this field per album
• Store this field per camera
• etc.

When a value is changed, it would be changed for all others in the relationship.  This would be a relatively straight-forward addition to the database.

However, there are a few tricky issues.

We don't keep a name and a unique user-manageable ID for each field (which I would argue is right).  But this means if you set a relationship of "Store per album" and had ten Greatest Hits albums, it could get a little messy on those ten albums.  We already have a runtime album analyzer that generates album IDs by looking at file paths and artist names, so it would help in this case.  I'm not sure if the same problem would apply to relationships of other fields.

Also, if you import a _new_ Aerosmith file with a different biography in the tag, should it inherit the existing data, overwrite the existing data for all the files in the relationship, or just be different?

Finally, what should happen if you edit an existing field to say 'Store per artist'?  I suppose it might leave the data alone and only modify everything in a relationship at change time, but it's a gray area.

[leezer3]

We don't keep a name and a unique user-manageable ID for each field (which I would argue is right).  But this means if you set a relationship of "Store per album" and had ten Greatest Hits albums, it could get a little messy on those ten albums.  We already have a runtime album analyzer that generates album IDs by looking at file paths and artist names, so it would help in this case.  I'm not sure if the same problem would apply to relationships of other fields.

User managable I agree with (Plenty of potential for breakage, with little benefit), but can I ask why not an internal ID?
Personally here, I'd be generating any ID from an album hash similar to what YADB is using- This way the different greatest hits albums would be kept separate. I'd assume you could store this hash in a non-visible internal tag, and query it as necessary
As we've just proved, a file centric model whilst flexible and powerful has it's drawbacks!

Also, if you import a _new_ Aerosmith file with a different biography in the tag, should it inherit the existing data, overwrite the existing data for all the files in the relationship, or just be different?

This needs a small confirmation dialog- Simple and practical Either on import, or probably better on the first tag edit a small popup-
Biography for #File# conflicts with the already existing biography for Aerosmith. Would you like to 1. Inherit existing data? 2. Overwrite existing data? 3. Assign unique biography?
A checkbox to always do this should probably be provided, either somewhere in the options or the dialog itself.

Finally, what should happen if you edit an existing field to say 'Store per artist'?  I suppose it might leave the data alone and only modify everything in a relationship at change time, but it's a gray area.

Again a small confirmation dialog- 1. Populate from existing data. 2. Create this as a blank field, leaving the original data intact.

Cheers

-Leezer-

[rick.ca]

I may not understand exactly what you're getting at, Matt, but I wonder if the "tricky issues," at least in part, are not the result of looking at my suggestion from a media-centric point of view. While this is natural given the existing design, I don't believe it's the most intuitive way of looking at it. I'm quite sure I would normally look at it the other way around. I would want to create a separate record for Aerosmith (in which I would record the biography), and then associate that with the media record category Artist. This is important because I'm not likely to, and I don't want to "add" these fields to media records (i.e., as is implied by "Store this field per..."). I want to add them as if they were a separate table. For example, if I wanted to add a bunch of artist bios, it would be much more intuitive for me to ignore media tracks and enter such information to (what appears to be) an actor table.

Yes, I realize the need for the "index" field to be unique is a limitation. But the same limitation is inherent in the current design—it just doesn't matter. To be fair, it should be recognized one would have a choice in this circumstance: record information for these non-unique albums on a per file basis (i.e., as you would now), or make the album names unique (e.g., "Greatest Hits [Artist]") and use the new feature. BTW, just to be clear, the creation of a relation doesn't mean there would automatically be an "additional information record" for every value in the category. I imagine being able to create an Album relation, but that would include a record for "Greatest Hits" only if I created one. This issue would be greatly reduced if it were possible to define a relation with multiple categories. So, for example, it probably doesn't make sense to try to create an Album relation because Album is not unique—and duplications can be much less obvious than "Greatest Hits". But an Artist-Album relation would work fine.

We already have a runtime album analyzer that generates album IDs by looking at file paths and artist names, so it would help in this case.

Perhaps that would be helpful if the objective were to provide some additional functionality that would be completely transparent to the user. I don't think that's necessary or desirable. Let the user define the relationship, and accept responsibly for the resulting behaviour and limitations. I think that would be easier for users to understand, and easier for you to make sure the basic functionality is efficient and consistent.

Also, if you import a _new_ Aerosmith file with a different biography in the tag, should it inherit the existing data, overwrite the existing data for all the files in the relationship, or just be different?

Now that's a good question—I didn't consider the possibility. Maybe the only feasible solution, initially at least, is to simply not allow any possibility of saving these "additional information" fields in tags. If there were a bio tag in files which fills a bio library field, these would be completely separate from a user-created "additional information" bio. Since the answer to this question now is "they would just be different," the addition of the related bio field can only be an improvement. In any case, I think the sensible solution in most cases would be to maintain such information in the library, and forgo the tags.

Finally, what should happen if you edit an existing field to say 'Store per artist'?  I suppose it might leave the data alone and only modify everything in a relationship at change time, but it's a gray area.

Again, I'm not sure I understand your point of view, but I think this is why I'm wary of the media-centric view. I prefer to create a separate table item, say "Aerosmith" as Artist (plus the additional information—like a bio) which will be associated with all my Aerosmith tracks by virtue of the fact Artist = "Aerosmith." In other words, I would save the bio in the "additional information" view, and it would appear in the media view if there were an expression column for it. Rather than a 'Store per artist' command, I would appreciate a 'Create/change additional information for this artist' context command that would jump to that record in the "additional information" view.

[Matt]

I would want to create a separate record for Aerosmith (in which I would record the biography), and then associate that with the media record category Artist. This is important because I'm not likely to, and I don't want to "add" these fields to media records (i.e., as is implied by "Store this field per..."). I want to add them as if they were a separate table. For example, if I wanted to add a bunch of artist bios, it would be much more intuitive for me to ignore media tracks and enter such information to (what appears to be) an actor table.

So in a way, you're saying you want to create a blob of information.  Maybe that blob is a biography, maybe it's actor information.

Then you want to link that blob to files either manually or by some rule like "artist is aerosmith" or "actor contains billy bob thorton"?

[leezer3]

Bingo, precisely  (Although I'm not the person you're quoting)
My twiddly diagrams were an attempt to visualise that, not sure how helpful they were though!

Just to re-emphasise, we really need a method of filtering what is displayed though. Bouncing back to my earlier example of Stephen Fry, where he is present in an actor role, there is various audiobook related information, which doesn't need to be displayed. Obviously, I could create a second 'blob', but I'd much rather be able to link 'blobs' to blobs, and filter etc. dynamically.

Cheers

-Leezer-

[rick.ca]

So in a way, you're saying you want to create a blob of information.  Maybe that blob is a biography, maybe it's actor information.
Then you want to link that blob to files either manually or by some rule like "artist is aerosmith" or "actor contains billy bob thorton"?

I am the one you quoted, and I don't know if "bingo" is the appropriate response.

The term "blob" scares me, because I believe it actually means something in connection with relational databases. I prefer to call it a table, related by a common value in a specific media record category and that in the table's "key" column. I'm not suggesting how that might work—just that from a user perspective the "link" would be made by the values being equal. Having established this link, the user could add whatever additional fields necessary (bio, birth date, shoe size, etc).

Also, I'm not just looking for something that will facilitate the storing of common information in a media file database. In most cases, the related table I'm talking about would be of some significance to me all by itself. For example, a table of artist biographical information, a table of photo album date/travel/geographical information, a table of music style descriptions, etc. I would want to be able to browse and maintain these tables in their own views.

When a value is changed, it would be changed for all others in the relationship.  This would be a relatively straight-forward addition to the database.

After my previous, rather long-winded response, it occurred to me I hadn't commented on what may the most important part of your message. This sounds very encouraging, but might also imply a limit to what you believe is feasible—as an "extension" to the current database/program. Not knowing the "mechanism" you refer to, it's difficult for me to imagine whether it might be adapted to provide the functionality I'm describing. I'm also not sure whether our ideas about this are fundamentally different, or just different perspectives of the same thing. I suspect the answer to that is somewhere in between.

I'm trying to stick with my non-technical definition of "relational" (a method of organizing data in a database so that it is perceived by the user as a set of tables) but I suspect my suggestion actually involves the "additional information" being saved in what is "physically" a separate table. Do you think that's true? If so, is it feasible? I realize there must be performance issues, but it seems to me those will cut both ways. First, it's optional—it won't have any impact unless used. Considering the nature of "additional information," it may be a way to separate seldom used information from the main media record database so it doesn't have to be involved in all processes. I know I don't understand all the technical issues. A premise of my idea, I suppose, is the notion information like an artist's bio is surely more efficient to "look up" when needed. If it's not like that, maybe it should be saved in the usual manner (e.g., so it's always readily available in media record views).

[newsposter]

Blobs aren't all that scary.  Unless you try to embed them in database tables.

A proper blob implementation (as Oracle learned to the pain and suffering of a load of users back in the mid 1990s) can be thought of as meta-meta-data.

User-field points to blob-meta-data points to external file.

Note that this is much the same way that music and cover art 'data' is referenced.  We now allow things like RTF, PDF, XLS, ODF, and TXT files to be attached.  The definition of these blobs also passes along certain characteristics to the skinning API to make it very easy to 'play' (display) that text and graphics data in a device/screen independant way.

[rick.ca]

Blobs aren't all that scary.  Unless you try to embed them in database tables.

Or think of the 1958 movie ("an alien life form consumes everything in its path as it grows and grows").

After reading the Wikipedia entry for blob, I thought perhaps it is a fitting term for what I've awkwardly been calling an "additional information record." Especially if it is to be handled as such in a table separate from the main media file table. All that matters to the latter is the relationship itself—that there is additional information in the blob. The additional information could be anything, and it doesn't matter to the normal operation of the database what that information is. It is, of course, made up of user-defined fields, but these are of no consequence to a media file view unless referenced by an expression.

If the "blob table" idea is feasible, it will be important to recognize any field currently part of the media file database, with the exception of the media file path, could be handled handled as part of a blob. Calling it a blob emphasizes the fact that, for many fields, this would not be a good idea. Any field regularly used for searching and sorting media records should not be in a blob. So I might create an artist blob for biography, date of birth, country of origin, and date of death. That might be fine, but if I like to sort or group my media by country, perhaps it should be part of the media record instead.

This idea is becoming clearer to me (the concept—not whether or not it will work). Now all I need is for darichman to show up to explain parent vs. child blobs, and whether blobs are containers or things needing containment. 

[newsposter]

Calling database 'piles' like that came from Oracle.  Binary Large OBjects.

When I say that Oracle learned on the back of it's users that embedded blobs are not a good idea I mean it.  Oracle 8 and it's initial implementation of blobs just about brought the company down.  Of course, when they fixed it Larry Ellison became a basillionaire, but it was a near thing for about a year.

Database records are supposed to be small and granular.  Blobs are anything but.

'Inventing' a decent, redundant, recoverable repository for blobs might be a good idea.  It might already have been done in the many content management systems out there.  But lets keep the hugeness external, not internal to the database.

[Matt]

Any field regularly used for searching and sorting media records should not be in a blob. So I might create an artist blob for biography, date of birth, country of origin, and date of death. That might be fine, but if I like to sort or group my media by country, perhaps it should be part of the media record instead.

That sort by country bit is an argument in favor of the first approach.

If you make fields "Biography", "Date of Birth", "Country of Origin" and "Date of Death" and set them to be stored per artist it would be easy to use the fields in searches, for sorting, etc.

Since you can make a view that groups by artist, you would effectively be able to look at the table of "Biography" values just by using such a view.

I like that this approach fits nicely with the existing architecture and doesn't introduce any performance penalty in normal usage.  (sets of relational fields would be a little slower, but sets are not too common)

[darichman]

I think whatever happens here should definitely be done in baby steps. As Jim has pointed out the database is streamlined and very efficient in its current incarnation. It would be foolish to disregard all that and sweep it all aside for something new and untested. That doesn't rule out extending things and testing the boundaries  

Matt, you've mentioned the possibility of simple inheritance. I think this is a really good idea. You've also mentioned some potential limitations or areas which may be ambiguous in its implementation and use...

We don't keep a name and a unique user-manageable ID for each field (which I would argue is right).  But this means if you set a relationship of "Store per album" and had ten Greatest Hits albums, it could get a little messy on those ten albums.  We already have a runtime album analyzer that generates album IDs by looking at file paths and artist names, so it would help in this case.  I'm not sure if the same problem would apply to relationships of other fields.

The "Greatest Hits" issue has always presented a problem - an album pane will not list separate entities for files with the same values populated for a field. There's nothing wrong with this, it's just how the program works, without separate "entries" for albums, as it were. This pops up with movies too... same release titles or re-releases which keep the same name. I think it's just something the user needs to be aware of (or even add a suffix at the end to differentiate, which is what I try to do). Hopefully the number of situations where this does arise is pretty small?

As far as how this affects inheritance... a dialogue box which confirms the changes might be useful: "Would you like to assign this file (these files) to the album Greatest Hits ([Album Artist])?" Album artist could be used to differentiate, if there is no separate Album ID to query? A "Compare Fields" dialogue would be really useful (remember aTagger?)

Also, if you import a _new_ Aerosmith file with a different biography in the tag, should it inherit the existing data, overwrite the existing data for all the files in the relationship, or just be different?

This is how I envisaged how things might work (forget about parent/child fields if you wish and just pretend Nationality etc is inherited based on artist)

1. Artists
The Parent Field is [Artist]
Child fields might include [Nationality] [Bio] [Formed] [Disbanded] [Discography] [Primary Genre] [Primary Styles] [Primary Moods] etc

So, once child fields have been assigned to the Parent Field as described above...
1. I import a file for tagging.
2. "Queen" could be added to the [Artist] field just as it is in the current version. Doing so would automatically create "Queen" as a taggable entity. It is an "Artist" which now exists in the database, just as a file does now.
3. Child fields [Nationality] etc could be filled in for "Queen" as per below
4. If I import a new file and tag [Artist] as "Queen", the file will inherit all of the Child Field values assigned above. Perhaps this could be accompanied by a "Would you like to assign this file to the Artist "Queen" and inherit all associated metadata?"
5. If I select a file already assigned to Queen with the artist metadata already filled out, and change the value of of a child field - [Nationality] for example - the user might receive the message: "Are you sure you which to change the Artist "Queen"? Note this change will affect all files assigned to the Artist "Queen"

This is the interface the user might see in the Tagging AW

>   Artist: Queen
            Bio: … (extended field)
            Nationality: UK
            Primary Genre: Pop/Rock
            Primary Styles: Glam-Rock; Hard-Rock; Album-Rock etc
            Members (a list field): Bon Scott; Malcolm Young
            Formed: 1971 in London, England
            Disbanded: 1995
>   Album: A Night at the Opera
            Album Genre: Pop/Rock
            Album Year: 1975
            Album Styles: Prog-Rock/ Art Rock etc
            Album Moods: Witty; Theatrical; Elaborate etc
            Label: Elektra
>   Rest of the fields etc etc

Note that all of the fields listed above exist/can be created in the current MC - the key is that the child fields (tabbed) are all common to and inherited based on a singular Parent Field. I know some people get iffy about affecting files which aren't currently selected - but we're talking about a shift here from "changing the files" to "changing the artist to which the files belong" - ie any fields you assign as child fields of "Artist" should have the same values for all files belonging to that artist.

Matt, does the whole parent/child field make any sense from a development standpoint? Could it be implemented in such a way that it is only invoked when files are tagged or when database values are changed? So we're not talking so much about how the data is stored, just when it is stored.... I'm not sure if I'm explaining that very well - If MC knows that when I tag "Queen" or "Aerosmith" files that it needs to make the corresponding changes to relevant fields in*all* "Queen" or "Aerosmith" files, we haven't changed anything about how MC is actually storing the data (we've only told it to "tag these files too!"). This wouldn't cause any speed problems in normal views etc because the fields are stored as they are currently... the only time performance will take a hit is at the time of tagging (MC needs to query for all files with that [Artist])...

The other foreseeable issue here is list fields... simple inheritance like this won't allow me to tag "Tom Cruise" in a file with [Actor] = Tom Cruise;Nicole Kidman unless there was another layer to this.

Blobs might be cool. But again, baby steps... I don't know anything about software developing - I really want to be clear about that - I just know what I'd like to see functionally. Whatever way you guys can come up with to do that (if at all) I'm happy with... Development knows the software better than we do - what works and what will completely break the architecture of the program. The last thing any of us want is for MC to crash and burn because of big changes which came about at the insistence of demanding users. And I'm sure I belong in that category Thanks for listening

[rick.ca]

I like that this approach fits nicely with the existing architecture and doesn't introduce any performance penalty in normal usage.

Okay. If I understand you correctly, there is no advantage to saving anything outside the current database structure. It could all be handled by the "Store this field per [category]" mechanism you mentioned. There would be the "tricky issues" you mentioned, but I assume these are things you believe could be overcome.

Since you can make a view that groups by artist, you would effectively be able to look at the table of "Biography" values just by using such a view.

I see in Customize View I can select View as "[category] details." I've never tried that before, but I suppose it will give me something like the "blob table" view I was looking for. I'll have to study that more, and try to visualize how that might work. It would be nice if there were a "panes and [category] details" view available. I'm also not sure how this will work when the subject category is a list field (e.g., actors, directors). It appears that in a "[category] details" view, Name shows the unique values existing in [category]. I never noticed that before!

Understandably, I cannot "rename" anything in these views. Would I be able to use the "Store this field per [category]" command?

Edit: Hmmm. For list values, I think not—that would require a relational database! 

[hit_ny]

If you make fields "Biography", "Date of Birth", "Country of Origin" and "Date of Death" and set them to be stored per artist it would be easy to use the fields in searches, for sorting, etc.

If i understood what Matt is saying then taking darichman's example with Queen

"Biography", "Date of Birth", "Country of Origin" and "Date of Death" would become child fields for the store 'by' or 'per' [Field] which is the parent field. In this case we choose to store by [Artist] which is the parent field.

But when adding a new Queen album if the user indicates the artist is queen, do the child fields automatically carry over ?

If they are blank one would expect this to be the case, if one of them is filled then the user is presented with a dialog to decide whether to
- inherit existing library data for that child field or
- keep seperate or
- overwrite existing data in the library for that child field

Functionally i think it would meet the aim of darichman's proposal ie A way to automatically assign several child fields by specifying just a parent field.

I guess what I would like to be able to do is create a group, with it's own characteristics, and allow membership of files to that group so that they would then inherit those characteristics.

Files could naturally belong to more than one group (Album, Artist etc)
It would be a move to a true information database, rather than just a file database.

However they will limited to belonging to just one group here unless Matt allows to store 'per' more [Fields].

I also expect there would be a way  to tell which [Fields] have been chosen for the parent, via the Action Window say.

What we need to do more is stress test this idea with more use-cases to understand better the limitations (if any) with it. This is purely from a user perspective as Matt already indicated that this approach is a feasible one

I like that this approach fits nicely with the existing architecture and doesn't introduce any performance penalty in normal usage.  (sets of relational fields would be a little slower, but sets are not too common)

[JimH]

'Inventing' a decent, redundant, recoverable repository for blobs might be a good idea.  It might already have been done in the many content management systems out there.  But lets keep the hugeness external, not internal to the database.

How about one that mirrored the blobs to your private account on an Internet server?

[rick.ca]

How about one that mirrored the blobs to your private account on an Internet server?

OT: Why is there never a moderator around when you need one?