xevidroid

@Retro808 @Jason Carr Ok, after a few weeks of testing I can say with 100% certainty that I've fixed the problem, and I'm pretty sure I know what it was. As some of you have suggested, the culprit was Google Drive: I originally dismissed that option, because I had a pretty good idea of what the internal file structure of my data folder was and it hadn't changed much for months, as far as I could see; I always had those xml files with the [conflict] tag, but since they hadn't been causing any issues for months or even years, I figured Launchbox was simply ignoring them (because they had the wrong name, why read "windows[conflict].xml" when "windows.xml" is already there as expected). As it turns out (and I don't know if this is something new Jason introduced with 9.10), Launchbox is now VERY confused by the presence of those files. First, the problems were only there for platforms with conflict files in the data folder. As for the duplication issue, it appears LB was reading the custom fields from the xml array in the conflict files AS WELL as in the regular file, but when closing the program, it was probably writing back the fields to one file only, thus completely messing it up and multiplying the lines. I guess the read happened from one file or the other at random, which caused the shifting number of duplications. As for the videos disappearing, I guess that when adding videos LB was committing the changes to the wrong file, or not committing them at all due to the conflict being there. These two are only my guesses based on outside observation, what's 100% sure is that by going back to an older backup of the data files without the duplication, and then deleting the conflict files, the issue went completely away and hasn't returned. So, my issue has been fixed. What I suggest is either discourage using Google Drive as the main directory for Launchbox (at this moment in time, the Google Drive symbol appears on the description page for LB on the website), or if possible writing the xml parsing code in a way that simply ignores duplicated platform files, or specifically platform files with "[conflict]" in the name, as at this moment in time LB seems to basically merge the two data structures together at launch and then go crazy when figuring out what or where to commit changes. I hope this proves useful and avoids the same happens to other users in the future (as it was pretty dishearting for me).

@Retro808 I'll try that and report back.

Okay so, I had already tried my backup from 9.8 and it didn't have the duplication bug, but it did have the video deletion bug. I tried a fresh install of the 9.10 version and then adding the same games that were having problems in the same way, but the bug didn't happen. I then tried adding my whole data folder to the new install and the bug wasn't there, at least that I could see. I added the whole images folder and the bug wasn't there. Same goes for my videos and music folders. Basically there has to be something that has gone wrong in my current folder, and I don't know what. I could simply do a fresh install etc, but A) would this work? @Jason Carr which folders do I need to move to the new install to have a perfect 1:1 copy of the old database + media? B) This doesn't solve the issues with my current bug, because if it happens again, we're back to square one. Wouldn't it be better to solve the problem once and for all by finding the culprit? I mean, I've had my library basically vandalized, videos disappearing, changes not saving, games disappearing, information duplicating..all in one go. This is pretty destructive, it really should be investigated..

@Jason Carr The thing is, I seriously doubt my system is the culprit. If I move the folder to another PC (manually, through a USB stick) I see the same bug. I see the same problem whether I load from an old HDD or a fast SSD. I don't have any problems with any other software installed, everything runs smooth..So I either have a very specific Launchbox killer virus, or it's something else.. Google Drive COULD have messed things up, it's totally possible. I don't know what precisely could have gone wrong, but I do see how syncing 11k files through a garbage app like "Backup and Sync" could cause errors. The problem is though, I'm not using Drive anymore, the folder is on my desktop and I'm only moving it through USB to test things out. Basically Google Drive could have messed up some file or dependency and started the problem, but it's not the thing that actively causes issues at runtime. My programmer instinct tells me there is a deeper problem with how the read/write of the XML is carried out, but as the source is not on Github I can't check for myself. As I said multiple times, if you don't trust my system, or you don't trust my word or whatever, the only solution I see is I send you my setup and you open it in a sandboxed VM or whatever and check it out for yourself. Either this, or at the very least I'd like some suggestions on how to solve the issues once and for all, because after 60$ and hundreds of hours put into designing my own box art covers for each of those 650 games I really don't want to give up Launchbox like this, and starting over would be insane, especially if I know the bug could still happen at random in the future.

@DOS76 Yes I error checked my drive and that's not the issue. @Jason Carr Okay, the videos disappearing might be caused by anything else (Windows? I've seen it happen before), but I've reported a big number of anomalies that are clearly related to the inner workings of Launchbox and could not be caused by something else in my system. For example, every time I open or close the program, my custom fields get duplicated a random number of times, like in the picture I posted. If I stop there and read the XML file, the fields have been duplicated on a "database" level too. It's not a read error or a virus, it's literally the program saving a wrong value on its database. The same goes for the entry for videos disappearing, the URL gets deleted from the XML even though the video is still physically present in the folder. As I said, I can send you the zipped folder with and/or without the bug and you can check for yourself that it's not something in my system in particular. My best guess would be read and write errors when parsing or updating the XMLs, due to something unusual in my particular library, maybe some unusually heavy media resources, maybe some funky encoding in some text somewhere. The program clearly struggles to load my 650 games, as I noticed that sometimes when I open it, it says I have 649 and some game is not shown, but it's back there if I restart. I'm not trying to insinuate it's bad programming or someone's fault, bugs happen and I love Launchbox, I wouldn't have paid for the full version or posted here if I didn't care Just to be clear, my setup doesn't include any ROM nor any personal information, just images, videos and a list of games I have played or I legally own. I think it wouldn't be a problem to share it.

Today I was quickly browsing my Launchbox collection to check if anything else was out of place, and I noticed something alarming. SOME (but not all) of the videos that had been disappearing from the game details, had been physically deleted on my disk. This means that some of the stuff I put into Launchbox, somewhere, could at any time completely vanish and for those videos I didn't even have a backup. This issues NEED to be investigated and corrected, just because it's happening to one person (that we know of), it doesn't mean it couldn't start to affect others. I've seen people with libraries of thousand of games, people who have put even more time than I have into completing their setup..These bugs are putting hundreds of hours of work into danger! Please get in touch so that I can send you all the necessary information! @Jason Carr @DOS76

One last test: I went back to my backup from version 9.8, the only backup I still have around to my knowledge. The duplication bug is NOT there, I'm 99% on this because I tried many times and the information hasn't ever duplicated once. I also tested the videos, and adding a video to a game that did lose it on 9.10, resulted in it losing the video on 9.8 too. The bug seems to be present, but unrelated to the duplication one. I also tried to go on and update the 9.8 folder directly to 9.10 (didn't go through 9.9 like I did originally), but that by itself didn't trigger the bug, at least for now, so the answer might reside in the difference between my May 2019 and August 2019 folders. The video kept disappearing after the update to 9.10, at first I thought it was because I was putting videos in the wrong platform folder under Videos (not that it should matter too much, I thought), but it turns out that even by changing folder the video still disappears. It also seems to happen to specific platforms: Android games always lose their videos, but weren't affected at all by the duplication glitch. Windows games never lose their videos but get duplicated. Android games that had videos PRIOR to all of this starting to happen, seem unaffected by the video disappearing bug, as is any other game that already had a video. Might this be related to an encoding error in these two specific platform xml files? The XML is definitely valid, but it may be a Launchbox internal error and not a syntax one.

I've tested the laptop. The duplication didn't seem to happen at first. Then I tried adding a video to a game that DIDN'T show the duplicating behaviour on the desktop, but DID delete the video every single time I tested. The moment I tried and restarted, the video disappeared and the other games started duplicating. I don't know if this means that the duplication is triggered by adding the videos, but what I do know is that when I noticed it had started happening, I was in the middle of video adding spree, and I had added something like 50 videos in a short amount of time...so they may be related. It's VERY weird. I can't think of any other test to do myself, I'll wait for an answer here. On request I can share my entire folder, in the version with or without the bug already in place. I also have a backup from a few months ago, I don't know if it can help. One thing I could try is installing the program in an empty folder (maybe even an older version), then moving the data (xmls, images and videos). Still, I'll wait for a response.

Have you got any very large videos or images? I can see that causing issues.

I'm currently investigating the issue. I was lucky enough to find out that my copy of the folder on the other PC was still unaffected by the bug. I made a backup and then moved a copy to the PC where I initially found the issue. The first time I open Launchbox, everything's fine and the games only have 1 entry. After closing and reopening Launchbox, the data had been duplicated. Closing and reopening again didn't duplicate it a second time though. I'm trying different combinations of opening and closing to see when the bug triggers. I'm also trying to find out if the videos disappearing is part of the same bug, or is something completely unrelated. I'm also going to find out if the issue affects the laptop too, or only happens if the computer that is opening the program is the desktop. Findings: If I set a video path for some games, it gets immediately removed after one restart of the program, EVEN when the duplication bug doesn't happen. Some other games retain the videos, even after restarting when the duplication does happen. Only ONE platform seems to be affected by the duplication glitch (currently it's windows, I've yet to see it happen for other xmls). The video deletion bug seems to happen at random just like the duplication one. Sometimes you reload and the video is there, sometimes it is not. The "duplication" happens with no discernible rules to the amount of times the items are duplicated. Sometimes it stays the same, sometimes it jumps from 2 to 7, or 8, sometimes it goes down to 2 or 3. I've never seen it go back to 1. Some games (even NOT in the Windows platform) lose the video instantly, after one restart, and never get it back. Some other games I add the video and when I restart, sometimes it's there, sometimes it's not. For games that never get it back, the video is not found in the xml. Some games that have it disappearing and coming back, retain it in the xml file. The duplication of the custom fields CAN be seen in the xmls. It changes every time it changes in the program, differently from the videos. In short, it's a big big mess. As it stands the program is unusable, because any change I make is in constant danger of being ruined or deleted. I can't add videos because they vanish, I can't add games because I've seen some vanish too, I can't edit information because I simply can't trust the xmls and I don't know if I'll have to roll back to my old backup. I don't know if I'm alone in this and if the program uses a separate database from the xmls and that's what got corrupted for me and simply resetting that will fix everything..I simply don't know. Please get in touch and let's solve this together, because it's really a dealbreaker and if this can happen to random customers, it's a big, big issue. If anything, we'll find out if Google Drive was the initial cause of the issue (maybe some file was lost in synchronization and now the whole installation is busted), and installation inside cloud folders could get discouraged from now on.

Something very funky has been happening to my Launchbox. I've provided a screenshot of one of the occasions where custom fields are being duplicate, this seems to affect only some games and only some fields, and to get worse over time. I spent 3 hours setting the game video for these games, but the video source keeps getting reset, despite the games being present in the respective folders. I've also lost my last few added games, but I really can't say for sure if something else is missing. This has started happening today, after turned on my PC. Last time I had used launchbox it was on my other PC, and the two folders are synced through Google Drive. The problem can't be Drive though, because despite the xml files sometimes conflicting, the issue is in one xml file only, the one in use. For example if I open the custom field entry for 60 seconds, I can see (at the bottom of the Windows.xml file) 8 different entries of the game's id + a repeated field. This is very frustrating and potentially distructive to my setup which I put hundreds of hours into. I really can't keep using the program until it gets fixed because I risk destroying my collection and further corrupting my xmls...and I really don't have any way to check if or what is getting screwed up or deleted, which really drives me insane )= has this ever been reported before? I can share my ENTIRE setup with the devs on request, please contact me if needed and I'll share the whole installation folder.

I don't have problems, currently. I uninstalled the plugin some time ago, but the other user was reporting that currently it's working correctly for him.

I was working on a fan android app, but if an official one does come, mine will be made unnecessary..I'm happy about that =D I wanted one anyway, I don't care who makes it, I'm happy I was able to make the cogs start spinning in that direction!

The problem is continuity..the idea that a third party plugin could break makes me kinda not want to put the effort in to put my whole movie catalogue into Launchbox. I hope for official support someday..

This is also particularly a problem with active plugins that scrape information from other websites or services (here for example from Fanart)...the moment changes are made to that external API, you're out of luck. I really hope we eventually get official support for importing other types of media, I would be so happy to move all of my BoardGameGeek, Lybrarything and Movie Collection databases to Launchbox. It would be so cool to just have a "Games" section and then some space for all of your other collections.

This doesn't appear to work anymore, when I select Scrape Movie nothing happens at all. Am I the only one to get this?

I have quite a few reasons to prefer Launchbox to stuff like calibre or LibraryThing (which I both tried). A) the UI is quite underwhelming and that's a good part of the appeal for me B) I already have a lifetime license for Launchbox and I'm invested in making it better C) I'd like to have all my stuff in a single application, not 3 or 4 D) Launchbox seems sustainable long term, being DRM free, easily to backup and plugin friendly. This is a big thing for a collection app that I could go on to use for years and years E) calibre is too much ebook focused and my collection is 99% paper books. LB on the other hand is unintrusive and unopinionated Anyway if the code from calibre is available somewhere that would certainly be useful to make a port of LB.

Something kinda like this: But for books, so that we could make a new "platform" for them and download the information by putting in the title or the ISBN, kinda like Librarything allows you to do. I would love to have all of my collections in the same app (movies, books, games). I don't know how much of the existing code for these plugins could be repurposed, but aside from the API call to a book database website the work shouldn't be impossible.

There are a few options to add movies to Launchbox, and for music we now have a separate app altogether. Is anyone interested in making a book scraping plugin? If not, what would it take to make one? I'm not new to programming but i've never made a plugin for Launchbox.