Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[marko]

1st off, for the record, I am not accusing MC.
It does need to be eliminated from enquiries though. 99.99% of all my jpg work is via MC. 100% of my jpg tagging is via MC.

Back in August, I had a scary moment when I deleted my entire picture catalogue.

After recovering my files, I felt the need to validate their integrity and went in search of software to perform the task. I found this little program called "Bad Peggy"...

It searches for damaged JPEG files and will validate and verify pictures for truncation and other blemishes which sometimes occur when such files are getting downloaded or restored from bad backups.

It's quite useful for example to check if your precious photo library is still all intact (mine unexpectedly wasn't) and it's fast, comfortable to use and reliable in the analysis it does. Runs under Windows XP, 2000 and 98SE. Please notice that you might have to download the .NET Framework 2.0 if it hasn't been installed on your system so far.

It returned many, many corrupt jpg files. I painstakingly opened each one in AcdSee and re-saved the file over the top of itself without making any changes.
When I was done, I re-scanned with Bad Peggy and there were no corrupt files present.

Fast forward four months.
Problems with the flickr uploader alerted me to potential problems, so, as promised, I got Bad Peggy to re-scan my photos folders, and the result is 151 corrupt files from 8552.


I see there is a tell-tale "~!~MCTEMP~!~.jpg" file in the mix there!
All of these "corrupt" files appear to be normal. They open just fine with no visible artefacts and their metadata is both readable and writeable.
None of these files will upload to flickr.
The flickr uploadr rejects them prior to upload, while the MC uploader attempts to upload them, fails on the first one and aborts the entire upload job.

I'll clean these up (again) and then make a concerted effort to create some corrupt jpgs.
I wonder if anyone else is curious enough to let Bad Peggy give their photos the once over?

Could it be possible that some of my numerous (virtually every MC session) playingnow induced crashes are causing this, if the correct mix of variables happens to be present at the time of the crash?

-marko.

[gappie]

im giving it a shot... first ill backup my pics...
you will hear... also had a crash nightmare 2 years ago..

lets see

 
gab

[gappie]

guess im not much of a help  ... scanned 11500 pics. two had corruption. one 'premature end of datasegment' and one '2768 extraenous bytes before marker $E9'

the last one i could open in fax image viewer, paint, internet explorer, mc.... but not in ultimate paint.
the first one also opened in everything and even in ultimate paint under protest... resaving it with ultimate paint repaired the file..
i dont have flikr so that part i can not test.

 
gab

[JimH]

Thanks, gab.  Could you send one to tom at jriver? 

[gappie]

done...

[GHammer]

Is there any lossless way to resave a jpeg?

[Matt]

Is anyone able to take an image that doesn't have extra bytes, do something in Media Center with it (like tag), and then see extra bytes?

We sort of need that before we can be much help.  Thanks.

[Magic_Randy]

I ran the scan using Bad Peggy. No problems were identified.
Note: my sample size is small, only 1988 pictures.

[DarkPenguin]

Is there any lossless way to resave a jpeg?

As a jpeg?  No.

[Matt]

As a jpeg?  No.

But you can reheader / tag the file and keep the image data unchanged.

[cncb]

I ran into problems a little while ago when I was trying out Photoshop Elements.  Most of my images would not import into the PSE organizer.  My guess is that it has to do with keywords and the XMP/dc tag.  When you remove a keyword it appears that MC just blanks out the last subject[] entry instead of removing it entirely.  I ended up pulling all my photos into the Vista Photo Gallery, adding a temporary keyword tag there and then removing it.  After it "rewrote" all the tags/files I was able to import all of them into PSE fine.

While we are on the subject, I also noticed the MJMD tags added to all photos.  Are these really necessary (I removed most of these when trying to solve my problem above)?

[Matt]

While we are on the subject, I also noticed the MJMD tags added to all photos.  Are these really necessary (I removed most of these when trying to solve my problem above)?

Yes.  It's the only tag type that supports all fields.  They nicely follow the JPEG AppID metadata system, so they will not cause problems.

They can be removed using Media Center's "Remove Tags".

[GHammer]

But you can reheader / tag the file and keep the image data unchanged.

Lossless rotate then rotate back to original position?

[GHammer]

Is anyone able to take an image that doesn't have extra bytes, do something in Media Center with it (like tag), and then see extra bytes?

Tagged a few with .91 and Bad Peggy shows no errors.

[leezer3]

Run it over my 64,000 odd files, nothing out of the ordinary found, other than a couple of corrupt files, which is understandable after 7 years of collecting
A few files that it thinks aren't jpegs (Very old), but nothing interesting really.

FWIW, all I've ever done with these in MC is basic tagging and importing, all the main work goes into the accompanying video/ HTML files

-Leezer-

[marko]

I ran into problems a little while ago when I was trying out Photoshop Elements.  Most of my images would not import into the PSE organizer.  My guess is that it has to do with keywords and the XMP/dc tag.  When you remove a keyword it appears that MC just blanks out the last subject[] entry instead of removing it entirely.  I ended up pulling all my photos into the Vista Photo Gallery, adding a temporary keyword tag there and then removing it.  After it "rewrote" all the tags/files I was able to import all of them into PSE fine.

Definitely keyword related...

Take the three files from the flickr thread... [download]

I put these three files into a folder on their own.
Scan with bad peggy. All three have problems.

Load up the photos in Windows Live Gallery (which I assume is very similar to the Vista Gallery?)
Add a new keyword tag to the three files.
Re-scan with bad peggy. All three are good and the flickr uploadr is happy with them.

Browse to the photos using MC, (I have not imported them) add a new keyword tag to the three files.
Re-scan with bad peggy. All three report errors.

I don't know the whys and wherefores, I'm just reporting what I'm seeing, and the test above shouldn't be too hard to reproduce.
I hope we get something conclusive from it, because right now, I feel like I've got a big target on my back!!

-marko.

[park]

I don't know the whys and wherefores, I'm just reporting what I'm seeing, and the test above shouldn't be too hard to reproduce.
I hope we get something conclusive from it, because right now, I feel like I've got a big target on my back!!

-marko.

I am having the same flickr problems as you marko. I'll do the bad peggy test as soon as I have the time.

Bri

[jimn]

I am having the same flickr problems as you marko. I'll do the bad peggy test as soon as I have the time.

Bri

Should be fixed in the next build.
JimN

[marko]

Thanks Jim, it appears to be fixed, though I'm a little confused...

In four months, I accrued 151 faulty files.

If the problem could be replicated by simply adding a keyword, surely I, and everyone else, should have had a raft of 'corrupt' files?
That's not the case, which suggests to me there's more to it than adding a keyword.

Files that are already corrupted can be fixed by simply adding a keyword using another application such as AcdSee or windows live gallery.
Once fixed, it would appear that adding/removing keywords with MC does not produce the corruption.

What adds to my confusion, is that adding a keyword to a corrupt file using MC, does not repair the image, which leaves me wondering if the problem really has been nailed?

My head hurts...
Does any of that make sense?

-marko.

[JimH]

This problem is fixed in build 93.  Other programs may have trouble reading the tag on some JPG files that have been tagged more than once in MC.  However, no information is lost, and JPG files are not corrupted.  Only the header of the file was an issue.

[jimn]

My head hurts...
Does any of that make sense?

-marko.

To expand on what Jim said; the file was labeled "corrupted" because the xmp segment length was shorter then the space to the next segment. No data was corrupted.

On files it affected, any change to a tag would trigger the problem, not just keywords (also, the blank subject fields was not the cause the problem). The problem occurs when the existing xmp segment; with the same data; happens to be larger than the one MC generates when writing out changed tags, so it wasn't a problem for all jpeg files. Also, not all programs/websites are "fussy" about this particular discrepancy, as flickr and BadPeggy is.

The fix was simply to set the xmp segment size correctly on writing (updating) the jpeg file.

Hope this clarifies.
JimN

[cncb]

The fix was simply to set the xmp segment size correctly on writing (updating) the jpeg file.

Will this only update the xmp segment size when changing a tag that is found in the xmp segment or will changing any tag do it?  I'm curious if I can just apply a "Genre" to all my photos to have them "fixed".  Thanks.

[jimn]

Will this only update the xmp segment size when changing a tag that is found in the xmp segment or will changing any tag do it?  I'm curious if I can just apply a "Genre" to all my photos to have them "fixed".  Thanks.

Just doing "Update Tags from Library" should fix the file. HOWEVER, is another fix for the next build because some files still doesn't get fixed in the current build.

So wait for the next build.
JimN

[DarkPenguin]

I am having the same flickr problems as you marko. I'll do the bad peggy test as soon as I have the time.

Bri

The next build fixes this.  (Tested with your three files.)  Be sure to update tags before uploading.

[marko]

Just doing "Update Tags from Library" should fix the file. HOWEVER, is another fix for the next build because some files still doesn't get fixed in the current build.

So wait for the next build.
JimN

Fix confirmed.
Thanks for listening.

-marko.