KDE Almost Lost All Of Their Git Repositories

There was almost "The Great KDE Disaster Of 2013" when the KDE project almost lost all of their 1,500+ Git repositories.

Last week the virtual machine running git.kde.org was taken down for security updates. When the host came back up, which handles the 1,500+ Git repositories for the project, there was EXT4 file-system corruption. The KDE back-up system mirrors the Git trees of this server, but it turns out that these mirrors were also pulling in the corrupted Git repositories.

The mirroring process wasn't checking to see whether the original source was corrupted, so the problem was replicating and affecting all of the KDE Git repositories. Fortunately, in the end, a clean server mirror was found that was uncorrupted and could recover the master KDE Git virtual machine. "With git.kde.org and every other anongit completely dead, whereas we should have had four or five complete copies of the KDE Git repositories, this new server alone retained a pristine copy of all 1500 of them."

For more details on this near disaster for the KDE project, see this developer blog post about the situation by Jeff Mitchell.