I think that this problem might be a mix of two things.
1. The cache might be to small or there is something wrong with it, making backgrounds download unnecessary
2. The time of transitions and trigger for new backgrounds is off in some cases. If the cache works, I would like to see the faster background switching used in the main menu for browsing new Series, Artists and Movies. The slow transition makes it hanging faaaaar behind most of the time. And that gives the user a wrong impression of the selected items.
I suggest that you use the Themes engine to handle the backgrounds, and use the short transition effect for items Media Sub type iten browsing. Like browsing TV Shows, movies and artists. And you can keep the slower slide show for items selected and not changing. Like the slideshow of different backgrounds within one single TV Show. Example: Browsing Movies or TV shows gives you fast transition upon selection, and entering an item will change the images with a slower frequency and slower transition effect, because the fast switching is not needed, and the background loading can use the time needed.