Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Album Art Downloader XUI (Read 2066389 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

Album Art Downloader XUI

Reply #1750
I cannot say for sure that the crash occurred after the scan was completed.

In that case, there are a couple of other diagnostic checks that might be worth trying. One is to use file path pattern matching instead of ID3 tags. This will test to see if the problem is with reading tags from the files, including embedded image tags. It will also be a lot faster to run, particularly if you can use a pattern with an absolute root and ending in a \. (meaning that only the folder name is checked, no files within it), for example: S:\Music\%artist%\%album%\.

The other thing would be to change the "Specify path to find images" to something like invalid.name or anything that you know won't ever match. That will test to see if the problem is with loading image files to read their resolution.

Alex


I am not sure what you want me to do.

Search for audio files is currently set to: F:\MP3\Min musik\Artister
Would you like me to search for just one album (like F:\MP3\Min musik\Artister\10,000 Maniacs\In My Tribe (1987), for example)?

The "Specify path to find images" is set to: %preset%.%extension%|Cover%preset%.%extension%
Would you like me to set that to Invalid.name?

Shall I do this in one go or start with only the Search for audio files option and then (with your suggested Search for audio files option) change the Specify path to find images option?

EDIT: PS. The scan is up to some 5,800 albums now after about 45 minutes and uses some 200 MB of RAM and only 2% of CPU.

Album Art Downloader XUI

Reply #1751
Would anything like that be possible or do you, or someone else, know of a tool that already can do this?

That sort of thing is really out of scope for AAD, I'm afraid. It's a library and tag management sort of task, for which you need library and tag management software. I don't use embedded art myself, so I can't really recommend any software to do that from personal experience. In general, foobar2000 and MP3Tag would be my go-to tools, but I don't know if they have any consistency checking built in. Perhaps someone else can offer suggestions?

I am not sure what you want me to do.

I'd like you to try searching by path pattern, rather than ID3 tag. To do this, under Options in the file browser, select the option "Use file path pattern matching", and in the box put "F:\MP3\Min musik\Artister\%artist%\%album% (*)\." including the . but not the " marks. (I'm using your example to guess your naming scheme). Still do the search for all the albums though, I'm pretty sure that searching for just one wouldn't cause any problems!

Also, yes, please try setting "Specify path to find images" to Invalid.name.

If you make both changes at once, and it doesn't stall, then try putting the "Specify path to find images" back to normal again. Or, if you are feeling more optimistic, you could make just the path pattern change and only try the path to find images change if the first one still stalled, whichever you prefer :-)

Alex

Album Art Downloader XUI

Reply #1752
After 6,585 albums into the complete scan I hit stop and everything is normal so far. 227 MB of RAM is used and 0% of CPU.

In the scan that completed I had the hits sorted by Dimensions (smallest first) and noticed that the first entry was a rather large "Found: folder.jpg" of about 1500x1500, which looked strange since the other hits all had an Artwork status of "Found: [filename]<0>. I am afraid I cannot remember which file that was, but I do remember checking it out with Windows Explorer and found nothing extraordinary about that file or the album.


Album Art Downloader XUI

Reply #1754
OK, I am back. Thanks for the reply!

That sort of thing is really out of scope for AAD, I'm afraid. It's a library and tag management sort of task, for which you need library and tag management software. I don't use embedded art myself, so I can't really recommend any software to do that from personal experience. In general, foobar2000 and MP3Tag would be my go-to tools, but I don't know if they have any consistency checking built in. Perhaps someone else can offer suggestions?

OK, I understand.

MP3Tag cannot do this for multiple albums. Perhaps there is a foobar plugin I do not know about.

I'd like you to try searching by path pattern, rather than ID3 tag. To do this, under Options in the file browser, select the option "Use file path pattern matching", and in the box put "F:\MP3\Min musik\Artister\%artist%\%album% (*)\." including the . but not the " marks. (I'm using your example to guess your naming scheme). Still do the search for all the albums though, I'm pretty sure that searching for just one wouldn't cause any problems!

Also, yes, please try setting "Specify path to find images" to Invalid.name.

If you make both changes at once, and it doesn't stall, then try putting the "Specify path to find images" back to normal again. Or, if you are feeling more optimistic, you could make just the path pattern change and only try the path to find images change if the first one still stalled, whichever you prefer :-)

I did both changes recommended and the search completed very fast and found no artwork. I then put the path to find images back and did another scan. This time artwork was found. No crashes or performance issues whatsoever.

I should mention that the path pattern does not always correctly reflect the artist or album name in the tags.

Album Art Downloader XUI

Reply #1755
I did both changes recommended and the search completed very fast and found no artwork. I then put the path to find images back and did another scan. This time artwork was found. No crashes or performance issues whatsoever.

I should mention that the path pattern does not always correctly reflect the artist or album name in the tags.
OK, so it's not a problem with simply having too many albums in memory, or showing too many in the list, as using the path pattern would produce roughly the same set of results. Given that it still works fine with searching for images too, I guess there must be some issue with the tag reading code. I'll investigate a bit and see if I can spot any problems.

The path pattern's accuracy depends entirely on the accuracy of your folder naming, and consistency with the pattern. With the pattern I gave you, it would be tripped up by album names with brackets in them (as it assumes that the brackets contain the date), or album names without the date in brackets at the end. Of course the pattern can be made more complex to cope with these variations, but given the restrictions on characters allowed in folder names, reading the tags will always be more accurate (unless you have badly tagged files!).

Alex

Album Art Downloader XUI

Reply #1756
OK, I did a scan the normal way again. The PC stalled at the end when all 7,454 albums were scanned and AAD reported Searching Thumbs.db. AAD then used up all available memory (total usage 99%) but hardly any CPU. This went on for a couple of minutes when the C was unresponsive. Then the message Done appeared and I was able to control the PC again but with performance issues. Almost all available memory was still used by AAD (some 5 or 6 GB). This went on for a couple of minutes and then memory usage for AAD was back at normal (some 135 MB). No crash this time.

I know it is easy to fix the paths (tags are in order, paths not) and I realise that this can be useful. The downside of that for me would be that it will brake the listening statistics system I use since a couple of years (and it will trigger massive backup operations via LAN and via ftp over the Internet).

Album Art Downloader XUI

Reply #1757

Thanks for the details. Have you already tried the scan using tags, but with the path to find images set to invalid?

It's interesting that the last progress report is scanning Thumbs.db - it's probably a coincidence, but could you do a quick search for Thumbs.db and just make sure there isn't a gigabyte sized Thumbs.db file or anything screwy like that going on?

After Done appears, I think the performance issues would be due to .net garbage collecting all that memory. That's not a problem, it's what it should be doing. The question really is why it used up all that memory to start with! If you have the time, could you try this experimental version: AlbumArtDownloaderXUI-0.40.experimental.zip?

It has a couple of tweaks to the various artists detection - firstly, it moves some stuff that it does into background priority which might help prevent unresponsiveness, and secondly, it reports on what it is doing. The status bar should say "Detecting Various Artists albums..." when it starts, then "Resolving Various Artists album... " for each album that it's detected as being Various Artists and processing. If it shows either of those while it is unresponsive and using excessive memory, then I'll know what needs fixing.

I can understand you not wanting to change your file paths, and you shouldn't have to - it should work properly using tags too. That is the default setting, after all, and was chosen as such for accuracy over speed.

Alex

Album Art Downloader XUI

Reply #1758
Windows Explorer did not find any Thumbs.db. It is a hidden file and I do not think I have another file browser installed (but you could perhaps recommend one).

I will try the experimental version with search by tags and Invalid.name. Could you just tell me if the Cover Paradise script in this version is the enhanced script that finds albums with the same name as the artist,

Album Art Downloader XUI

Reply #1759

I use Windows Explorer, but I have Show Hidden Files and Folders turned on, and Hide protected operating system files turned off (View/Folder Options, View tab). I'm not sure which of those covers Thumbs.db, maybe both!

The cover-paradies script in the zip file is the very latest unreleased version, which does the accurate search then falls back on the old inaccurate one, and also marks booklet images as such. You can of course keep using the old script if you prefer, for the experiment the scripts are not involved at all anyway.

Thanks,

Alex

Album Art Downloader XUI

Reply #1760
OK, that sounds good. I want to get some more artwork while scanning. That was why I asked about the script. (The version is 0.39.1.0, not 0.40 as the zip file indicates, but I guess it is the correct version and not an old one).

I have all files visible in Windows.

Album Art Downloader XUI

Reply #1761
The version is 0.39.1.0, not 0.40 as the zip file indicates [...] I have all files visible in Windows.

That's not right, it should be 0.40. I've tried downloading it myself, and it looks like 0.40 to me...

Thumbs.db is automatically created by Windows Explorer in folders with images in them, and is marked with the hidden and system attributes. I'm not sure what else to tell you, but if AAD was scanning it, it must exist!

Album Art Downloader XUI

Reply #1762
OK, I have the 0.40 version now. I do not know what happened. I downloaded the zip file and copied the contents to the AAD program folder. Probably some user error...

I do see the Thumbs.db files in Window Explorer. It is just that if I search for Thumbs.db or *.db Windows Explorer shows no hits.

Album Art Downloader XUI

Reply #1763
Here's the script: qobuz.boo. If you could give it a try, I'd like to include it in 0.40 tomorrow.

Seems to work ok.

P.S. Chartstats seems to freeze after one result (the source on the right just shows 1, progress bar freezed showing one "bar" in the left side and a stop link) for a long time, eventually it stops. Also the one result shown in the left side list is "unknown" type (I've "fetch image when unknown" set) and getting the real image takes really long time, but in the end it gets the image, though the size is only 100x84px. But when I open the image webpage (from the "i" button), the webpage and the image loads very fast. Also the size of the image on the webpage is 300x254px. Where did the 100x84px come from? (I don't use this script but I noticed this behavior while testing the qobuz script) I was searching for "radiohead - king of limbs". This seems to happen with every different search, really slow and the image is ~100x84px while the webpage has a bigger one.

And here's a new script idea:

hmvdigital - maybe you should replace the "HMV Canada" script with this? I searched for "Vesa-Matti Loiri - 4 + 20" and the "Canada" script didn't find anything. This "digital" site on the other hand has it.. in 800px (or 500px). This source seems to use the same image source as 7digital (and some others IIRC). Just remember that some of the images might not be found in 800px, only 500px! Unfortunately I don't have an example for this now (I've seen this couple of times in the past with 7digital, and since the image source seems to be the same, it's likely to happen with hmvdigital also).

While you're at it, make the 7digital too fall back on 500px in case 800px image is not found.  BTW. 7digital is doing some "essential work" on the site ATM (site is up and down intermittently).. might break script *wink wink*.

AND, it might be wise to change the 7digital to the "google site" search type since, as you might have noticed, 7digital has additional 15 sites for different countries (check the links at the bottom of the page) which might have slightly different results per country. E.g. for a Finnish band "Egotrippi" the main site has 11 albums & 2 singles while the Finnish 7digital site has the same 11 albums but 6 singles.

The current script doesn't find the correct e.g. "Egotrippi - Asfaltin Pinta" cover image, even though I've set max results to 50 (search gives 24 results). But when I do search with google, I get the correct album/single result from 7digital Ireland (1st google result) & Spain sites (4th result).. for some reason no Finnish site in the search though (maybe in the next google search result pages).. but still, better result than the current script. (check the end of the post about doing more than 10 results per page search in google)


And a maybe-not-so-great-but-worth-checking-at-least:

GigaCrate (use JavaScript) - Here's an example search. Just take the album URL from the search results page (http://www.gigacrate.com/Albums/25853) and change it to find "large" image (295px) (http://www.gigacrate.com/images/album_art/25853Large.jpg) and/or "XL" image (500px) (http://www.gigacrate.com/images/album_art/25853XL.jpg).

Still 4 sites to go (had 6 but two were changed/restricted to only small images). Maybe it's better to drop these slowly and not all together.. easier to test and discuss here.


BTW. Did you have some scripts that use google search and therefore the scripts are limited to only 10 results? I vaguely remember couple of discussion/notes about it here. Or was it with google image search maxing to 20? At least you could use this for the eMusic and future scripts.  Well, here's a workaround; "Google Instant" has been turned on automatically now for everyone using google (AFAIK) and it disables the "num=X" switch which could be used to show X number of search results per page. Workaround is to use "as_qdr=all" switch with the "num" switch. Example search with 46 results per page.

And if you want to use old google image search (without JavaScript), just use the "gbv=1" switch.

Album Art Downloader XUI

Reply #1764
I will try the experimental version with search by tags and Invalid.name.

AAD reported Done when the computer stalled for a couple of minutes this time with AAD using up all available memory.

Looking forward to the final .40 release. It looks promising! (Let me know if you would like input now on the experimental version.)

Album Art Downloader XUI

Reply #1765
[Chartstats problems]
I just tried a search for radiohead / king of limbs here and it worked fine, but you say it was of Unknown *type*? It's supposed to be of unknown size, but they should all be of type Front coming from chartstats. If it wasn't, then it can't have come from the Chartstats script. I've never seen the 100x84 image either - do you think it might be a flood protection thing? Weird, anyway.

[other scripts]
I think the point of the HMV Canada source was to get Canada specific releases, if I remember right. If it's the same underlying source as 7digital, though, is there any need for it too? 7digital currently falls back on 350x350 if the 800x800 one isn't found. If you know any examples where 800 isn't available, but 500 is, that would be helpful. I don't think I should make any changes to 7digital until we see the outcome of the "essential work", though - where do you see that posted? I had a look at their blog, but it wasn't mentioned.

I'm not convinced by doing hybrid google searches unless absolutely necessary, though. I'd rather not over-rely on any single site; one of the benefits of having lots of sources is that when one goes down, others can still be used - but if everything routes through google, and that stops working, then they all go down. Not that I'm suggesting google will actually stop working, but it might well change in such a way as to break the script, either through design changes or deliberately.

GigaCrate might be interesting - I get the impression that it specialises in music for DJ's, which is not an area I've ever looked at before. If it's under-served by the existing scripts, it could be worth adding. I might leave it until after 0.40 though as I really wanted to get that out this weekend. Also, I think you're right that it's better to concentrate on one at a time, until we're happy it's working fine before moving on to the next.

if you want to use old google image search (without JavaScript), just use the "gbv=1" switch.
That's what the google script currently uses. Boo doesn't execute javascript, which makes using the new google image search really tricky. If really necessary, more results could be obtained by requesting further search pages, but 20 seemed like enough that it wasn't worth making two requests for every search.

AAD reported Done when the computer stalled for a couple of minutes this time with AAD using up all available memory.

Looking forward to the final .40 release. It looks promising! (Let me know if you would like input now on the experimental version.)
OK, I'm going to try rewriting the Various Artists detection to optimise for memory efficiency (at the cost of some speed), and give you another experimental version to try. You are welcome to give feedback on the other features going into 0.40 that you've had a sneak preview of - I know not everything you requested has made it in, though!

Alex

Album Art Downloader XUI

Reply #1766
AAD reported Done when the computer stalled for a couple of minutes this time with AAD using up all available memory.

OK, here's another experimental version, this time with a memory-usage optimised algorithm for finding various artists: AlbumArtDownloaderXUI-0.40.experimental.1.zip. I've versioned it as 0.40.0.1, but that's just to make it easy for you to check you've got the right version running - when released it will just be 0.40.

If it still gives the memory-gobbling stalling behaviour, then I've guessed wrong and it's something else that is causing that. Which would be irritating.

Alex

Album Art Downloader XUI

Reply #1767
Re: The experimental version.

I am sure that you are aware of the fact that Inside is called Inlay in that version and that if you change the type to Inside the image groups with Inlay. Inlay is also missing in the general options and in the type changing drop down menu.

Darktown calls what is usually an Inside Inlay, so it would be correct more often to set Inlay images from Darktown to Inside. Cover Paradise has a type called Digipack, which should get its own type or sort under Booklet.

When pulling down the Group or Sort by drop down menu the legend ("Group by", for example) moves. (This may be intentional.)

Now that we can group by web page (Thank you!) an option to download all images from one group with just one mouse click (and the additional clicks required to make unique file names) could be useful.

The experimental version came with many scripts and I cannot sort them in the right hand list (to get the most essential scripts on top). Some indication of when all scripts are finished would be useful (maybe a progressbar like the download one).

What does the Sort by Area option do?

Thank you for all your hard work!

Album Art Downloader XUI

Reply #1768
I've just had a quick look inside GigaCrate, and it wasn't as hard to parse as their use of Javascript led me to first believe. I think this can be squeezed into the 0.40 release, if you'd like to give it a try: gigacrate.boo

RE: Inlay/Inside. This is not an intentional change. The term "Inlay" was renamed as "Inside" a long time ago, to reflect the fact that not all of the images from the inside of the case could rightly be termed Inlay. Inlay was too specific. Giving it a quick look now and I see that the group header is wrongly showing Inlay instead of Inside, but that's not something new in 0.40 is it? Have you spotted anywhere else that "Inlay" appears in the UI?

Could you point me at an example album with a Digipack cover, that should be assigned the Booklet type?

I'm aware that the Group and sort drop-downs resize the first time they are accessed - that's not new, but it's much more noticeable now. I think I might just have "Page" in there rather than "Web Page" so it isn't so much wider than the other entries.

An option to download all the albums in a group will not be part of 0.40. I'll consider it for future versions, but it's not going in this one, sorry.

I wouldn't generally recommend having every single script installed, just delete the boo files for the ones you aren't using! There isn't a global progress bar for all scripts, but as a handy indicator the "Stop All" link at the top right of the sources list will disappear once they've all finished.

Sort by Area sorts by (width * height). When the images aren't perfectly square, this can give a more reliable measure of which results are the 'biggest', at the cost of a potentially confusing sort order. As an extreme example, under the Size sort, the order might be 1000x800, 900x900, 800x900. Area sort would have: 900x900, 1000x800, 800x900. It was requested quite a while ago.

Alex

Album Art Downloader XUI

Reply #1769
I just tried a search for radiohead / king of limbs here and it worked fine, but you say it was of Unknown *type*? It's supposed to be of unknown size, but they should all be of type Front coming from chartstats. If it wasn't, then it can't have come from the Chartstats script. I've never seen the 100x84 image either - do you think it might be a flood protection thing? Weird, anyway.

Sorry, I meant that it's unknown size. And "front" type. But, it freezes like I described and it shows the 100x84px image after a while.. and after the image is fetched, the script is still freezed in the right scripts panel for a while and then it stops. Something is not right here. Dunno whether it is a flood protection or not. I'm not using that script like I wrote so I'm not so concerned, just reported this anomaly.

I think the point of the HMV Canada source was to get Canada specific releases, if I remember right. If it's the same underlying source as 7digital, though, is there any need for it too?

Ok. Just add HMVdigital script then? I think that it would be good to have alternatives/failbacks (I mean HMVdigital to 7digital). For example, today 7digital was intermittently down.. plus Coverlandia was down earlier today, they might have changed the site (I couldn't tell because I haven't visited the site for a while).

7digital currently falls back on 350x350px if the 800x800 one isn't found. If you know any examples where 800 isn't available, but 500 is, that would be helpful.

I don't have any right now unfortunately.. but would it be a big trouble to check for the 500px before falling back to 350 if 800px is not found first? Do we really need an example? I'm 100% sure that there are those cases where there's no 800px but 500px is available. I remember seeing it happen at least twice in the 7digital webpage (before you made the script) and once in AAD.

I don't think I should make any changes to 7digital until we see the outcome of the "essential work", though - where do you see that posted? I had a look at their blog, but it wasn't mentioned.

Just tried to load any page earlier today, sometimes there was a temporary "we are doing some essential work with the site" page, sometimes pages didn't load at all and sometimes the correct page loaded.

I'm not convinced by doing hybrid google searches unless absolutely necessary, though. I'd rather not over-rely on any single site; one of the benefits of having lots of sources is that when one goes down, others can still be used - but if everything routes through google, and that stops working, then they all go down. Not that I'm suggesting google will actually stop working, but it might well change in such a way as to break the script, either through design changes or deliberately.

I agree with you, not everything should use the same google search style. If you don't feel confident of changing the current script, what if you put an alternative 7digital script with the method I described? A "7digital International" script if you like.

GigaCrate might be interesting ... I might leave it until after 0.40 though as I really wanted to get that out this weekend. Also, I think you're right that it's better to concentrate on one at a time, until we're happy it's working fine before moving on to the next.

Ok, just put it to your notes or whatever.. I deleted it and I might forget it.

Let's test the HMVdigital (if you make it) first.. and perhaps the 7digital and "7digital International" if you change and/or make it. The rest of the 4 other sites I'm holding off for now are IMHO really great, either big big images and/or great quality. Stay tuned folks.. or just you Alex.

Album Art Downloader XUI

Reply #1770
Thank you for the reply!

RE: Inlay/Inside. This is not an intentional change. The term "Inlay" was renamed as "Inside" a long time ago, to reflect the fact that not all of the images from the inside of the case could rightly be termed Inlay. Inlay was too specific. Giving it a quick look now and I see that the group header is wrongly showing Inlay instead of Inside, but that's not something new in 0.40 is it? Have you spotted anywhere else that "Inlay" appears in the UI?


Oh, I thought you were going to add another type named Inlay.

The group header was Inside, not Inlay, before. This version saves the Inside images as Inlay.jpg, whereas the earlier versions savved them as Inside.jpg.

Could you point me at an example album with a Digipack cover, that should be assigned the Booklet type?

http://cover-paradies.to/?Module=ViewEntry&ID=392256 It actually should be a category of its own.

I'm aware that the Group and sort drop-downs resize the first time they are accessed - that's not new, but it's much more noticeable now. I think I might just have "Page" in there rather than "Web Page" so it isn't so much wider than the other entries.

Or you could just realign according to the width of the actually selected option (not the width of the widest option in the drop down menu). This is of course just cosmetics, but it looked a bit odd to me.

An option to download all the albums in a group will not be part of 0.40. I'll consider it for future versions, but it's not going in this one, sorry.

I'll wait for that and an option to automatically number duplicates (without the overwrite dialog). Then a one click download of all images for one album would be possible!

There isn't a global progress bar for all scripts, but as a handy indicator the "Stop All" link at the top right of the sources list will disappear once they've all finished.

Aha, I had not noticed that.

Sort by Area sorts by (width * height). When the images aren't perfectly square, this can give a more reliable measure of which results are the 'biggest', at the cost of a potentially confusing sort order. As an extreme example, under the Size sort, the order might be 1000x800, 900x900, 800x900. Area sort would have: 900x900, 1000x800, 800x900. It was requested quite a while ago.

Aha, I thought it might have something to do with the regional Amazon stores or the Canada discussion you just had with Akkurat (i.e. area=region of the world). Thanks for explaining.

Album Art Downloader XUI

Reply #1771
Hi again,

I just wanted to report that a scan with my normal setting seems to have finished without me even noticing it or seeing any memory hogging. Whatever you did seems to have worked. Thanks!


 

Album Art Downloader XUI

Reply #1773
would it be a big trouble to check for the 500px before falling back to 350 if 800px is not found first? Do we really need an example?
Technically, no, but I'm not entirely comfortable releasing an updated script without testing that it actually works. Here's an updated version of it for you to try, but if I don't find a 500x500 and a 350x350 image to try it with before I release 0.40, I'm going to back out the changes: 7digital.boo.

The group header was Inside, not Inlay, before. This version saves the Inside images as Inlay.jpg, whereas the earlier versions savved them as Inside.jpg.
Well that's weird. I can't think why it would have changed, but it clearly has. I think I've fixed it now.

[Digipack] actually should be a category of its own.
I don't think I want to add another category for this. From what I can tell, a digipack should really be scanned as separate images for front, back and inside. It isn't really a booklet, so I'm just going to leave it as Unknown for the moment.

a scan with my normal setting seems to have finished without me even noticing it or seeing any memory hogging.
Hooray! I'm glad that's sorted out now, thanks for helping me find and test it.

Alex

Album Art Downloader XUI

Reply #1774
Technically, no, but I'm not entirely comfortable releasing an updated script without testing that it actually works. Here's an updated version of it for you to try, but if I don't find a 500x500 and a 350x350 image to try it with before I release 0.40, I'm going to back out the changes: 7digital.boo.

That was quick, I just started to search with artist names and found an example. If you try to load the 800px image (http://cdn.7static.com/static/img/sleeveart/00/011/867/0001186742_800.jpg), 7digital shows a default "not found" image (http://cdn.7static.com/static/img/sleeveart/00/000/000/0000000016_350.jpg) which is 350px. You should detect and discard this default image in the script.