Topic: Otto Import (Read 24931 times)

JimH · « **on:** May 10, 2006, 11:11:07 am »

OK, it's Auto Import, right? You've been asking. We're thinking.

The problems:
1. Anything running in the background steals CPU cycles.
2. Some people won't want it.
3. Interface issues can lead to complexity.

So what if we did something like this:

Message:
Media Center can automatically watch folders (directories) and import certain filetypes from them. Begin by choosing "Add Folder".

Buttons:
[Add folder button] (User chooses media type and filetypes, similar to what we have now)
[Ignore folder button] (Same dialog)

[ ] MP3
[ ] WAV
[ ] WMV
[ ] JPG
[ ] GIF
[ ] MPG
[ ] MP4
[ ] ETC

Audio
Directory: C:\music
Currently importing: MP3, WMV, APE
Auto: Yes

Images
Directory or Folder: C:\images
Currently importing: JPG, BMP
Auto: No

Directory or Folder: C:\movies
Currently importing: JPG, BMP
Auto: Yes

Video
Directory or Folder: C:\video
Currently importing: MP4, MPG
Auto: No

Don't import C:\music\cover art

Mr ChriZ · « **Reply #1 on:** May 10, 2006, 11:16:53 am »

Sounds good to me

Matt · « **Reply #2 on:** May 10, 2006, 11:21:51 am »

There are two parts:

1) Reconcile disk and database: this runs in a low priority background thread and takes a few seconds on a modern machine

2) Check for changes while running: this takes no cycles due to how Windows allows this.

#1 runs on startup of Media Center and then anytime #2 finds an external change. Nothing runs if MC isn't running.

As Jim mentioned, we're still brainstorming about configuration. Really, there's no need for an Import tool. In options, tell MC what folders you use and the rest is transparent. The hitch is that people are used to the old model.

Thanks for any suggestions.

jgreen · « **Reply #3 on:** May 10, 2006, 11:31:42 am »

How about a pref for how frenetic Otto is when doing his background thing? Say:

How Often Should Otto Check for New Stuff?
--Constantly
--Frequently
--Occassionally

Also, will Otto do away with the macro that forces you to look at a"Recently Imported" playlist even if you're working on something else and don't care? Some sort of daily dialogue box/smartlist might be enough.

Lastly: Even though Otto's sure to be our new best friend, will we still have shell integration/right click importation? Dead handy, IMO.

Deivit · « **Reply #4 on:** May 10, 2006, 12:02:41 pm »

I'm worried about how Otto will react with large databases.

If I can still force the manual import either via shell integration or the current "import media" feature, Otto is OK to me, and it's probably a nice addition for many users.

LonWar · « **Reply #5 on:** May 10, 2006, 12:14:32 pm »

Quote from: Deivit on May 10, 2006, 12:02:41 pm

I'm worried about how Otto will react with large databases.

If I can still force the manual import either via shell integration or the current "import media" feature, the Otto is OK to me, and it's probably a nice addition for many users.

I have a fairly large DB, How long from when I paste a file in the folder, to when MC will pick it up.
If this isn't almost instantly, then we would still need the import function.

Also when anything is auto imported, it should go into a imported Smartlist.
\

Thanks,

hit_ny · « **Reply #6 on:** May 10, 2006, 12:40:26 pm »

Pls allow the user to select whether they want to use this function or not.

I very much prefer to do this manually and Import with most of its checkboxes switched off works great for me.

No surprises, and helps keep confusion to a minimum when library sizes are large. I know 100% that nothing got imported unless i said so.

enervation · « **Reply #7 on:** May 10, 2006, 12:53:37 pm »

Sounds very good to me - presumably MC will immediately do an auto import on startup?

I'm looking forward to this now

IanG · « **Reply #8 on:** May 10, 2006, 01:05:39 pm »

Looks good! I'd like to see the interval between updates being user definable, in minutes, with "Never" as an option. Also an "Import now" option to duplicate the current system. Has importing documents been left out deliberately? It would be nice if arbitrary file types could be defined, as per KingSparta's plugin.

Ian G

Matt · « **Reply #9 on:** May 10, 2006, 01:49:02 pm »

Quote from: imjustagamer on May 10, 2006, 12:14:32 pm

I have a fairly large DB, How long from when I paste a file in the folder, to when MC will pick it up.

Seconds. And as I mentioned earlier, there's no CPU cost for this monitoring.

Once a change occurs, there'll be a little background activity as MC reconciles the disk and library.

With a 10,000 file library, this activity will last less than 10 seconds.

LonWar · « **Reply #10 on:** May 10, 2006, 02:00:00 pm »

Quote from: Matt on May 10, 2006, 01:49:02 pm

Seconds. And as I mentioned earlier, there's no CPU cost for this monitoring.

Once a change occurs, there'll be a little background activity as MC reconciles the disk and library.

With a 10,000 file library, this activity will last less than 10 seconds.

Sounds Great Matt.... So, when can we have a look at this? Is this part of the secret closed MC12 Alpha/Beta

Deivit · « **Reply #11 on:** May 10, 2006, 02:17:48 pm »

Quote from: Matt on May 10, 2006, 01:49:02 pm

Seconds. And as I mentioned earlier, there's no CPU cost for this monitoring.

Once a change occurs, there'll be a little background activity as MC reconciles the disk and library.

With a 10,000 file library, this activity will last less than 10 seconds.

So, with a 305,000 file library it would be... 5 minutes? $:-\$

jgreen · « **Reply #12 on:** May 10, 2006, 02:21:03 pm »

It all sounds great Matt, just that there are times when a ten-second pause during playback might be objectionable. I leave MC open all day, even if I'm not playing music, just so I can get at a track quicker when I need it--oh, and so other programs can to humble themselves by comparison.

Thanks to your work, MC sleeps very lightly, so I keep it open even when I've pushed taskmanager to the firewall with processing. I don't want Otto burping and farting when I'm using the computer for something I (the guy who pays the bills around here) might view as more important.

I agree Otto can take a lot of the pain out of file management. But even Copernic (not that that's your model here) allows you to suspend the auto-import.

Jaguu · « **Reply #13 on:** May 10, 2006, 02:34:10 pm »

Yes, I would miss a documents folder too.

Better would be to define arbitrary folders and select what kind of type they should be, like:

Add Folder: D:\Media\Image
Select Type: Image [Audio|Video|Docs|TV|Movies etc.]
Define Naming Rules: [for Folders and Files]

This could even be integrated in the view scheme definition panel.

Add View Scheme Name: Artworld
Add Folder: D:\Media\Artworld
Select Type: Image
Define Naming Rules: [for Folders and Files]

Andy sub scheme would inherit the information or allow to overwrite.

enervation · « **Reply #14 on:** May 10, 2006, 03:06:56 pm »

Presumably, as MC will know exactly what has been added, the process will be quicker than a full import as it's done currently? I have a 5k library, and it takes about 10 seconds to go through, but would I be correct in thinking that this would be a lot shorter with auto import?

danrien · « **Reply #15 on:** May 10, 2006, 05:51:39 pm »

what would happen when MC grabs cover art off the net? i know that can at least freeze MC's display while its grabbing images (or searching for them), so could that affect performance?

GHammer · « **Reply #16 on:** May 10, 2006, 06:33:48 pm »

If I can define types and locations to watch, I'm happy with it.
I use that capability in other apps and it has always been a handy thing.

Tolga · « **Reply #17 on:** May 10, 2006, 07:04:11 pm »

Sounds cool and it is probably a very desired function. Certainly a very good reason to update to MC12.

Two questions:

1) Would it be smart enough to keep track of external changes without needing to go over all watched folders. For example, if you move a file, rename a directory, or change the tags of a file with an external tool (i.e. explorer), will it be able to update only relevant changes without looking to the rest of the files.

2) will it be able to watch subdirectories of a watched directory automatically?

scthom · « **Reply #18 on:** May 10, 2006, 08:43:45 pm »

Quote from: hit_ny on May 10, 2006, 12:40:26 pm

Pls allow the user to select whether they want to use this function or not.

I very much prefer to do this manually and Import with most of its checkboxes switched off works great for me.

No surprises, and helps keep confusion to a minimum when library sizes are large. I know 100% that nothing got imported unless i said so.

I agree.

sapnho · « **Reply #19 on:** May 10, 2006, 11:14:06 pm »

The key for me is: Will it work on a network? All my audio files are stored on a server.

Pink Waters · « **Reply #20 on:** May 11, 2006, 12:57:11 am »

Sounds good to me too

park · « **Reply #21 on:** May 11, 2006, 01:39:59 am »

Quote from: Tolga on May 10, 2006, 07:04:11 pm

Sounds cool and it is probably a very desired function. Certainly a very good reason to update to MC12.

Two questions:

1) Would it be smart enough to keep track of external changes without needing to go over all watched folders. For example, if you move a file, rename a directory, or change the tags of a file with an external tool (i.e. explorer), will it be able to update only relevant changes without looking to the rest of the files.

2) will it be able to watch subdirectories of a watched directory automatically?

That's the $10,000 question for me too. I like the way that Adobe Premiere and After Effects can relink missing files without changing any of the properties (edits, effects etc.) of those files within the projects.
If MC could detect changes to the location of files and reapply library tag info to them, it would come a monumental step closer to being able to tidy up after beginners (and even normal users) who forget to do stuff within MC.
The "fix broken links" feature at the moment ends up deleting library information when those files disappear, so I dont use it.

Link · « **Reply #22 on:** May 11, 2006, 10:39:57 am »

Here is my problem: I often rip/copy music to my PC, and place it on my Music drive.... Where it would be added by auto import.

However, I then edit the folder names, add subfolders, etc, to conform to my arrangement scheme.

Issues:
1)There would be orphans from the first import
2) If I changed folder names, then moved the folders into another subfolder (soundtracks for example), would they be imported 2 more times?

Matt · « **Reply #23 on:** May 11, 2006, 10:53:37 am »

Quote from: Link on May 11, 2006, 10:39:57 am

Issues:
1)There would be orphans from the first import
2) If I changed folder names, then moved the folders into another subfolder (soundtracks for example), would they be imported 2 more times?

It will reconnect broken links. (so there's no double-import or orphans) One important feature is that you can rearrange your folders using an external program and MC will stay nicely in step.

The MC 11.1 import tool will actually do this already. (assuming you didn't turn it off in Import options) The only hitch is that you have to run it by hand.

Matt · « **Reply #24 on:** May 11, 2006, 10:56:08 am »

Quote from: park on May 11, 2006, 01:39:59 am

That's the $10,000 question for me too. I like the way that Adobe Premiere and After Effects can relink missing files without changing any of the properties (edits, effects etc.) of those files within the projects.
If MC could detect changes to the location of files and reapply library tag info to them, it would come a monumental step closer to being able to tidy up after beginners (and even normal users) who forget to do stuff within MC.
The "fix broken links" feature at the moment ends up deleting library information when those files disappear, so I dont use it.

Actually, fix broken links will reconnect to a new file assuming it can find the file. Just be sure to tell MC about the places you keep your media and it'll actually fix, not remove.

Marty3d · « **Reply #25 on:** May 11, 2006, 05:06:58 pm »

Quote from: Matt on May 11, 2006, 10:56:08 am

Actually, fix broken links will reconnect to a new file assuming it can find the file. Just be sure to tell MC about the places you keep your media and it'll actually fix, not remove.

Please tell us a bit more on this behaviour. I'd gladly see this Otto if it ment no more hassling with MC's internal file manager (my last try moving a couple of files to a subfolder created one folder with about 100 subfolders put in one another, like folder 1 > folder 2 >folder 3 etc).
Can I for example move Season One of 'Monty pythons Flying Circus' to a new subfolder named 'Season 1' in Explorer and MC reconnects the files?

Matt · « **Reply #26 on:** May 11, 2006, 05:42:54 pm »

Quote from: Marty3d on May 11, 2006, 05:06:58 pm

Please tell us a bit more on this behaviour. I'd gladly see this Otto if it ment no more hassling with MC's internal file manager (my last try moving a couple of files to a subfolder created one folder with about 100 subfolders put in one another, like folder 1 > folder 2 >folder 3 etc).
Can I for example move Season One of 'Monty pythons Flying Circus' to a new subfolder named 'Season 1' in Explorer and MC reconnects the files?

This should work today as long as the old and the new location (or a parent folder like D:\Data) are in the import dialog. I was recently testing it with audio files and it worked well. I think it should also work with images and video.

Marty3d · « **Reply #27 on:** May 11, 2006, 06:26:54 pm »

Great! I'll try next time!

Tolga · « **Reply #28 on:** May 11, 2006, 07:31:32 pm »

Quote from: Matt on May 11, 2006, 05:42:54 pm

This should work today as long as the old and the new location (or a parent folder like D:\Data) are in the import dialog. I was recently testing it with audio files and it worked well. I think it should also work with images and video.

If there are tags in MC that are not stored on the file, would MC update the new location but keep the tags that are only stored on MC side. How does MC determine the sameness of the files? From tags? unique id in the tags?, media content? I just try to understand how safe this proccess is, because if you end up linking to an incorrect file, the tags that are stored on MC and on the file will get inconsistent and there is no mechanism to review this proccess.

benn600 · « **Reply #29 on:** June 22, 2006, 07:03:26 am »

I've been asking for this feature and I see this thread is kinda' old, but I want to explain my situation.

I really don't care how long it takes to update, as long as the updates get made. I have a ~6 computer household where each family member has their own user account. The library for each user will be pointing to the same folder on the server (the 1st, main 500GB drive). I am not going to manually log on to each user's account (four) and click import. Plus, two of the other 3 users have very little computer knowledge and therefore, I must make everything as simple as possible for them. This is one reason why MC is the only program that I found that works for me because everything else is too complicated. But the point is that as long as the database could be updated eventually, say in an hour running very low priority, that would be reasonable. I know I listen to music all the time and MC practically stays open.

Off topic: I was copying my FLAC library from my 1st backup to my big 500GB drive for my second backup and I suddenly got the error, "Cannot copy file ** because the directory is corrupt and unreadable." I am a little sensitive because I bought 3 WD 500GB drives (all w/ same data) and one stopped working in the first week. Does this mean the souce is corrupt or the desination? (ignore this unless you don't want to)

JONCAT · « **Reply #30 on:** June 23, 2006, 11:28:21 am »

What about remote files on a server that is on or off depending on the time of day. I have a Video Server that I turn off most of the time to save energy. Lately, I have Imported files and mistakenly had the fix broken links or remove missing files options checked.

Is is possible, now, to fix/repair external changes and remove broken links for cetain locations or file types? I want to be able to work on my audio database w/o removing all the video fiels. How would an auto-import handle this? I tried to exlcude search locations and I can't figure out if there is a way to currently import/repair/remove files w/o creating a second library. It would be nice if Auto-Import developement could incorporate this by allowing for certain file types or locations to be ignored and allow a user to cleaning up the database, thus allowing on master database with audio,video, docs, xls, etc.

DC

zirum · « **Reply #31 on:** July 13, 2006, 08:50:20 am »

To the above issue:
How about creating a checkbox for "network storage" or etc, witch will not perform update/removals if it is unable to browse the specified root folder or something similar.

Second suggestion:
Maybe the autoimport could use a temporary table, where no album and extra imports are added. Just the file with the txt tags. A lightweight import if you understand what i am thinking of.
I don't know why the import scans several times thru the db, but mayby the full import could be optional, possibly even with a seperate scheduler.

KingSparta · « **Reply #32 on:** July 13, 2006, 11:54:38 am »

Quote

I don't know why the import scans several times thru the db, but mayby the full import could be optional, possibly even with a seperate scheduler.

It is doing many different tasks.

Finding New Files, Validating Links, Creating Thumbs, Correcting Broken Links, making sure it does not import a file two or more times. etc.....

enervation · « **Reply #33 on:** July 13, 2006, 11:58:58 am »

A little addition to this... it would also be nice if you could an auto-analyze-audio function so that auto-imported music is immediately and automatically analyzed (for replay gain purposes).

drosoph · « **Reply #34 on:** August 08, 2006, 04:26:53 pm »

Heck, since we're begging for stuff ... Why not give us the options to Auto-Analyze Music, Lookup Cover Art, Rotate Pics (using the EXIF data and current orientation), and SEND TO EXTERNAL for videos (to have them transcoded to DIVX on import)

Just dreaming ... how about we just get this in the next build ... just the automated portion of import ? Please ?

JimH · « **Reply #35 on:** August 08, 2006, 04:31:17 pm »

Auto import is a feature of MC12, which you should see in open beta in the next several days.

slipknot · « **Reply #36 on:** August 08, 2006, 04:47:49 pm »

Quote from: Matt on May 10, 2006, 11:21:51 am

As Jim mentioned, we're still brainstorming about configuration. Really, there's no need for an Import tool. In options, tell MC what folders you use and the rest is transparent. The hitch is that people are used to the old model.

I think there is a need for importing "on demand" and NOT automatically (which is not to say that some may want auto-importing).

Virtually NOTHING I import has proper tagging. And I don't want to import unless I have time to properly tag them right then and there. It sounds way too easy to lose things that were imported and need tagging attention when it's automatic.

(oh yea, and I still program using vi...)

KingSparta · « **Reply #37 on:** August 08, 2006, 05:20:08 pm »

Quote

I think there is a need for importing "on demand" and NOT automatically

Remember that commercial about spaghetti sauce "it's In there"

Listener · « **Reply #38 on:** August 08, 2006, 06:46:26 pm »

Quote from: jgreen on May 10, 2006, 11:31:42 am

How about a pref for how frenetic Otto is when doing his background thing? Say:

How Often Should Otto Check for New Stuff?
--Constantly
--Frequently
--Occassionally

Also, will Otto do away with the macro that forces you to look at a"Recently Imported" playlist even if you're working on something else and don't care? Some sort of daily dialogue box/smartlist might be enough.

Lastly: Even though Otto's sure to be our new best friend, will we still have shell integration/right click importation? Dead handy, IMO.

I think I missed this thread when it appeared. Perhaps the subject didn't ring a bell for me.

I'm glad to read that you are thinking along these lines. I've tried out a number of tag editors and players before settling on MC 11. The import model NEVER works the way I want it to. I'm always fighting the s/w to get it to drop old files, recognize changes and add new folders.

Importing and updating Library or Tags in MC 11 works better than in most programs I've used. However, I want the sort of model described in Jim's first post. That fits the way I think about the task:

I specify a list of folders containing media files. MC scans those folders, looking for added files or deleted files and for changes to existing files (using file creation and modification dates.)

When MC otto-imports a file, it can store the file's creation/modification tags in the library database. It can use those stored dates in future folder scans. If MC finds that the modification date for a file recorded in the folder directory is newer than the date stored in the library, MC should read the file and collect the tag values. It should provide a way to resolve conflicts between old and new values so that the user can repair unwanted changes.

MC could collect changes to the library database in a "recently scanned and updated" view. This would allow the user to reverse any changes that were not what he wanted.

You may need both a background folder scanning strategy and a command the user can execute to cause an immediate scan after he has made external changes.

If a folder is not currently available because it is on a device that is not present, just skip it. Don't change those library records at all. That would work for me and make sense to me.

Bill

adamsp70 · « **Reply #39 on:** August 09, 2006, 10:31:31 am »

In a client-server MC setup:

If Otto picks up new files and updates the server library - will the client MCs automatically update their libraries too?

Alex B · « **Reply #40 on:** August 09, 2006, 10:47:58 am »

A client never updates its database automatically. On a reconnect a client downloads a new copy of the database (with filenames reformatted as server addresses). The only way to update a client database is to reconnect.

adamsp70 · « **Reply #41 on:** August 09, 2006, 10:50:40 am »

I understand that's the way it currently is - so i guess i'm making a request.

If Otto makes library-updates automatic, it's all lost (for me) if i then i have to go round reconnecting my clients.

Hope that makes sense....

INTERACT FORUM

Author Topic: Otto Import (Read 24931 times)

JONCAT