Skip to main content
aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2019-09-03Merge branch 'stable-4.9' into stable-4.10stable-4.10Matthias Sohn2-2/+1
* stable-4.9: Bazel: Update bazlets to the latest master revision Bazel: Remove FileTreeIteratorWithTimeControl from BUILD file Change-Id: I8bb74aacc1c010cfed9a0c8acffb520724f0e395 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03Merge branch 'stable-4.9' into stable-4.10Matthias Sohn5-271/+165
* stable-4.9: BatchRefUpdate: repro racy atomic update, and fix it Delete unused FileTreeIteratorWithTimeControl Fix RacyGitTests#testRacyGitDetection Change RacyGitTests to create a racy git situation in a stable way Silence API warnings Change-Id: Id5bf44645655fca40ad22bb1f1ad20a7c2e8f6db Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03Bazel: Update bazlets to the latest master revisionstable-4.9David Pursehouse1-1/+1
The latest revision includes various fixes to allow the build to work with recent versions of Bazel. Change-Id: I72c100b99762010946d9b2784286af560bbdf185 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-09-03Bazel: Remove FileTreeIteratorWithTimeControl from BUILD fileDavid Pursehouse1-1/+0
FileTreeIteratorWithTimeControl was deleted in a024759, but was not removed from the BUILD file, thus causing the bazel build to fail. Change-Id: I892c0ffcac947298d0d6009374ee2c5d9afefb66 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> (cherry picked from commit e54fde8616f53a6fab5201db4a0c7e67dd0145c5)
2019-09-03BatchRefUpdate: repro racy atomic update, and fix itHan-Wen Nienhuys2-60/+87
PackedBatchRefUpdate was creating a new packed-refs list that was potentially unsorted. This would be papered over when the list was read back from disk in parsePackedRef, which detects unsorted ref lists on reading, and sorts them. However, the BatchRefUpdate also installed the new (unsorted) list in-memory in RefDirectory#packedRefs. With the timestamp granularity code committed to stable-5.1, we can more often accurately decide that the packed-refs file is clean, and will return the erroneous unsorted data more often. Unluckily timed delays also cause the file to be clean, hence this problem was exacerbated under load. The symptom is that refs added by a BatchRefUpdate would stop being visible directly after they were added. In particular, the Gerrit integration tests uses BatchRefUpdate in its setup for creating the Admin group, and then tries to read it out directly afterward. The tests recreates one failure case. A better approach would be to revise RefList.Builder, so it detects out-of-order lists and automatically sorts them. Fixes https://bugs.eclipse.org/bugs/show_bug.cgi?id=548716 and https://bugs.chromium.org/p/gerrit/issues/detail?id=11373. Bug: 548716 Change-Id: I613c8059964513ce2370543620725b540b3cb6d1 Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03Delete unused FileTreeIteratorWithTimeControlMatthias Sohn2-177/+0
The only usage of this test iterator was removed in df637928d. Hence delete this iterator and associated test. Change-Id: I47710133ec3edc675c21db210960c024982668c6 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> (cherry picked from commit a024759cf5bf1cd6b9beb4f790d484943761a7e1)
2019-09-03Fix RacyGitTests#testRacyGitDetectionMatthias Sohn1-11/+26
This test case assumed file system timestamp resolution of 1 second. On filesystems with a finer resolution this test fails since the index entry is only smudged if the file index entry's lastModified and the lastModified of the git index itself are within the same filesystem timer tick. Fix this by ensuring that these timestamps are identical which should work for any filesystem timer resolution. Bug: 548188 Change-Id: Id84d59e1cfeb48fa008f8f27f2f892c4f73985de Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Christian Halstrick <christian.halstrick@sap.com> (cherry picked from commit 1159f9dd7c80a53c2509cd75d997a6afed37f9a6)
2019-09-03Change RacyGitTests to create a racy git situation in a stable wayMasaya Suzuki1-25/+24
By using File#setLastModified, we can create a racy git situation stably. Tested with --runs_per_test=100 Bug: 526111 Change-Id: I60b3632d353e19f335668325aa603640be423f58 Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> (cherry picked from commit df637928d2ef4b9ee06af7e37344c7848f870ce4)
2019-09-03Silence API warningsMatthias Sohn1-8/+51
Change-Id: I27fd62de51ca0eedcc7e2e256487bda1e18bce8a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-18Merge branch 'stable-4.9' into stable-4.10Matthias Sohn6-27/+80
* stable-4.9: Prepare 4.9.11-SNAPSHOT builds JGit v4.9.10.201904181027-r Prepare 4.7.10-SNAPSHOT builds JGit v4.7.9.201904161809-r Prepare 4.5.8-SNAPSHOT builds JGit v4.5.7.201904151645-r Remember the cause for invalidating a packfile Fix API problem filters Fix pack files scan when filesnapshot isn't modified Change-Id: I94c9e6300b5f77b185ec6661fa76f0c74a5fc2a9 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-18Prepare 4.9.11-SNAPSHOT buildsMatthias Sohn56-333/+333
Change-Id: I99137bc4958167773d2fc8b1d48fbb508af52be5 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-18JGit v4.9.10.201904181027-rv4.9.10.201904181027-rMatthias Sohn56-59/+59
Change-Id: I3034d1ad263fdfb23347304f866309354930fd19 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-18Merge branch 'stable-4.8' into stable-4.9Matthias Sohn6-28/+81
* stable-4.8: Prepare 4.7.10-SNAPSHOT builds JGit v4.7.9.201904161809-r Prepare 4.5.8-SNAPSHOT builds JGit v4.5.7.201904151645-r Remember the cause for invalidating a packfile Fix API problem filters Fix pack files scan when filesnapshot isn't modified Change-Id: Idaa789e699f1ef568ea957184d0641355d9e3181
2019-04-16Merge branch 'stable-4.7' into stable-4.8stable-4.8Matthias Sohn5-28/+82
* stable-4.7: Prepare 4.7.10-SNAPSHOT builds JGit v4.7.9.201904161809-r Prepare 4.5.8-SNAPSHOT builds JGit v4.5.7.201904151645-r Remember the cause for invalidating a packfile Fix API problem filters Fix pack files scan when filesnapshot isn't modified Change-Id: I7984f68833f2d615399296e53cb9a64e5b4ca6ed Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-16Prepare 4.7.10-SNAPSHOT buildsstable-4.7Matthias Sohn56-321/+321
Change-Id: I662ee441521c1370a31f274cc6d001723ad0c528 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-16JGit v4.7.9.201904161809-rv4.7.9.201904161809-rMatthias Sohn56-59/+59
Change-Id: Id6e379c3ce52c83455e37cbc3d77db0e188fdb85 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-16Merge branch 'stable-4.6' into stable-4.7Matthias Sohn5-29/+83
* stable-4.6: Prepare 4.5.8-SNAPSHOT builds JGit v4.5.7.201904151645-r Remember the cause for invalidating a packfile Fix API problem filters Fix pack files scan when filesnapshot isn't modified Change-Id: I6ea51dde6608a4163d681aa1ebf710f06da44b21 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-16Merge branch 'stable-4.5' into stable-4.6stable-4.6Matthias Sohn5-44/+82
* stable-4.5: Prepare 4.5.8-SNAPSHOT builds JGit v4.5.7.201904151645-r Remember the cause for invalidating a packfile Fix API problem filters Fix pack files scan when filesnapshot isn't modified Change-Id: I0b4eaa521ebdea83ab18c05915d691c07a575a7d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-15Prepare 4.5.8-SNAPSHOT buildsstable-4.5Matthias Sohn56-302/+302
Change-Id: I70628cb8fcad0a60598dc937abbed63606a78599 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-15JGit v4.5.7.201904151645-rv4.5.7.201904151645-rMatthias Sohn56-59/+59
Change-Id: I3e32cf13f5cb99d8e570412d80d941740399c07d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-11Remember the cause for invalidating a packfileLuca Milanesio5-23/+76
Keep track of the original cause for a packfile invalidation. It is needed for the sysadmin to understand if there is a real underlying filesystem problem and repository corruption or if it is simply a consequence of a concurrency of Git operations (e.g. repack or GC). Change-Id: I06ddda9ec847844ec31616ab6d17f153a5a34e33 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-10Fix API problem filtersMatthias Sohn2-24/+1
Change-Id: I96e0ddc34251348ec4877c9d94b045eb1c53e758 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-09Fix pack files scan when filesnapshot isn't modifiedLuca Milanesio2-6/+54
Do not reload packfiles when their associated filesnapshot is not modified on disk compared to the one currently stored in memory. Fix the regression introduced by fef78212 which, in conjunction with core.trustfolderstats = false, caused any lookup of objects inside the packlist to loop forever when the object was not found in the pack list. Bug: 546190 Change-Id: I38d752ebe47cefc3299740aeba319a2641f19391 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-21Merge branch 'stable-4.9' into stable-4.10Matthias Sohn2-2/+22
* stable-4.9: Fix GC to delete empty fanout directories after repacking Change-Id: Ibdbfe08eb290286fa738010bad1c604e857885cd Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-20Fix GC to delete empty fanout directories after repackingMatthias Sohn2-2/+22
The prune method did not delete empty fanout directories when loose objects moved to a new pack file but only when loose unreferenced objects were pruned. Change-Id: Ia068f4914c54d9cf9f40b75e8ea50759402b5000 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13Merge branch 'stable-4.9' into stable-4.10Matthias Sohn6-13/+97
* stable-4.9: Prepare 4.9.10-SNAPSHOT builds JGit v4.9.9.201903122025-r Prepare 4.7.9-SNAPSHOT builds JGit v4.7.8.201903121755-r Prepare 4.5.7-SNAPSHOT builds JGit v4.5.6.201903121547-r Check for packfile validity and fd before reading Move throw of PackInvalidException outside the catch Use FileSnapshot to get lastModified on PackFile Include size when comparing FileSnapshot Do not reuse packfiles when changed on filesystem Silence API warnings for new API introduced for fixes Change-Id: Ifcd8ea6a2b59f6f8e9d896d8407cfcaef0a9375d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13Prepare 4.9.10-SNAPSHOT buildsMatthias Sohn56-333/+333
Change-Id: I2bdbe822ee623c5f15a26d092136d8602cef06ac Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13JGit v4.9.9.201903122025-rv4.9.9.201903122025-rMatthias Sohn56-59/+59
Change-Id: I807d24461b3557eee48294bfaa652b2a13d78113 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13Merge branch 'stable-4.8' into stable-4.9Matthias Sohn6-13/+97
* stable-4.8: Prepare 4.7.9-SNAPSHOT builds JGit v4.7.8.201903121755-r Prepare 4.5.7-SNAPSHOT builds JGit v4.5.6.201903121547-r Check for packfile validity and fd before reading Move throw of PackInvalidException outside the catch Use FileSnapshot to get lastModified on PackFile Include size when comparing FileSnapshot Do not reuse packfiles when changed on filesystem Silence API warnings for new API introduced for fixes Change-Id: I9a47153831f8eb10d3cd91b4157cf45385e5b13a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13Merge branch 'stable-4.7' into stable-4.8Matthias Sohn6-13/+97
* stable-4.7: Prepare 4.7.9-SNAPSHOT builds JGit v4.7.8.201903121755-r Prepare 4.5.7-SNAPSHOT builds JGit v4.5.6.201903121547-r Check for packfile validity and fd before reading Move throw of PackInvalidException outside the catch Use FileSnapshot to get lastModified on PackFile Include size when comparing FileSnapshot Do not reuse packfiles when changed on filesystem Silence API warnings for new API introduced for fixes Change-Id: I1136eaa1590b225ceaf16ba7552af0374253a74e Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Prepare 4.7.9-SNAPSHOT buildsMatthias Sohn56-321/+321
Change-Id: I1602a6b811fd9f95e284153b560d8d9f855fae9f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12JGit v4.7.8.201903121755-rv4.7.8.201903121755-rMatthias Sohn56-59/+59
Change-Id: Ia6dbde615671ed09789326aacad1655e7a7a59ad Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Merge branch 'stable-4.6' into stable-4.7Matthias Sohn6-13/+97
* stable-4.6: Prepare 4.5.7-SNAPSHOT builds JGit v4.5.6.201903121547-r Check for packfile validity and fd before reading Move throw of PackInvalidException outside the catch Use FileSnapshot to get lastModified on PackFile Include size when comparing FileSnapshot Do not reuse packfiles when changed on filesystem Silence API warnings for new API introduced for fixes Change-Id: I3d1544d034783fe0fa1385dfe9b03ad8e9247c63 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Merge branch 'stable-4.5' into stable-4.6Matthias Sohn6-13/+113
* stable-4.5: Prepare 4.5.7-SNAPSHOT builds JGit v4.5.6.201903121547-r Check for packfile validity and fd before reading Move throw of PackInvalidException outside the catch Use FileSnapshot to get lastModified on PackFile Include size when comparing FileSnapshot Do not reuse packfiles when changed on filesystem Silence API warnings for new API introduced for fixes Change-Id: I029e1797447e6729de68bd89d4d69b324dbb3f5f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Prepare 4.5.7-SNAPSHOT buildsMatthias Sohn56-302/+302
Change-Id: I5c275c542e12746c3d8ecf8462791969f9e89e12 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12JGit v4.5.6.201903121547-rv4.5.6.201903121547-rMatthias Sohn56-59/+59
Change-Id: I5a071ed10e1ac1ab28f992d45cde335c12556a80 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Check for packfile validity and fd before readingLuca Milanesio1-0/+8
When reading from a packfile, make sure that is valid and has a non-null file-descriptor. Because of concurrency between a thread invalidating a packfile and another trying to read it, the read() may result into a NPE that won't be able to be automatically recovered. Throwing a PackInvalidException would instead cause the packlist to be refreshed and the read to eventually succeed. Bug: 544199 Change-Id: I27788b3db759d93ec3212de35c0094ecaafc2434 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12Move throw of PackInvalidException outside the catchLuca Milanesio1-2/+3
When a packfile is invalid, throw an exception explicitly outside any catch scope, so that is not accidentally caught by the generic catch-all cause, which would set the packfile as valid again. Flagging an invalid packfile as valid again would have dangerous consequences such as the corruption of the in-memory packlist. Bug: 544199 Change-Id: If7a3188a68d7985776b509d636d5ddf432bec798 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12Use FileSnapshot to get lastModified on PackFileLuca Milanesio1-1/+1
Do not redundantly call File.lastModified() for extracting the timestamp of the PackFile but rather use consistently the FileSnapshot which reads all file attributes in a single bulk call. Change-Id: I932675ae4fe56dcd3833dac249816f097303bb09 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Include size when comparing FileSnapshotLuca Milanesio5-9/+107
Due to finite filesystem timestamp resolution the last modified timestamp of files cannot detect file changes which happened in the immediate past (less than one filesystem timer tick ago). Read and consider file size also, so that differing file size can help to more accurately detect file changes without reading the file content. Use bulk read to avoid multiple stat calls to retrieve file attributes. Change-Id: I974288fff78ac78c52245d9218b5639603f67a46 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Do not reuse packfiles when changed on filesystemLuca Milanesio2-2/+15
The pack reload mechanism from the filesystem works only by name and does not check the actual last modified date of the packfile. This lead to concurrency issues where multiple threads were loading and removing from each other list of packfiles when one of those was failing the checksum. Rely on FileSnapshot rather than directly checking lastModified timestamp so that more checks can be performed. Bug: 544199 Change-Id: I173328f29d9914007fd5eae3b4c07296ab292390 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12Silence API warnings for new API introduced for fixesMatthias Sohn1-0/+14
Change-Id: I3ea7ff2efd33ca6c780afaef9010cec82780d7fa Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-11Merge branch 'stable-4.9' into stable-4.10Matthias Sohn1-26/+35
* stable-4.9: Reduce contention on PackFile.idx() function. Change-Id: I277e53aa752c8ffb8560de710d27ecb58871ec02 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-11Reduce contention on PackFile.idx() function.Juergen Denner1-26/+35
In case of concurrent pack file access, threads may wait on the idx() function even for already open files. This happens especially with a slow file system. Performance numbers are listed in the bug report. Bug: 543739 Change-Id: Iff328d347fa65ae07ecce3267d44184161248978 Signed-off-by: Juergen Denner <j.denner@sap.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-06Merge branch 'stable-4.9' into stable-4.10David Pursehouse7-11/+74
* stable-4.9: Fix error log message in ObjectDirectory.handlePackError() Properly format pack checksums in PackFile.idx() Cancel gc if thread was interrupted PackFile: report correct message for checksum mismatch ObjectDirectory: Clean up logging Bazel: Stop using native.git_repository ObjectDirectory: extra logging on packfile exceptions Change-Id: I0847251eb010616a705e0b91df4bdebc225fa95d Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-03-06Fix error log message in ObjectDirectory.handlePackError()Matthias Sohn2-3/+3
Change-Id: I154f392ad025c4b642eb1123d375a0afaa853885 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> (cherry picked from commit 997d785418d55dce5a1188fdb95e6d2b4ab0bde5)
2019-03-06Properly format pack checksums in PackFile.idx()Matthias Sohn1-2/+3
Change-Id: Id805850dbe9a3d633168f3056e06ddeafd86f961 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> (cherry picked from commit a33e4dc58a87daf34072b82643aba0fd4456c165)
2019-03-06Cancel gc if thread was interruptedMatthias Sohn2-1/+53
see https://groups.google.com/d/msg/repo-discuss/oDB2rl3doDc/tFEh5Xt0CAAJ Change-Id: Ia6d4631c64e065d8b9b09e0b45e7a9ea8ac3f41d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> (cherry picked from commit 882fed0d96c533513c43ae77aaff9cc07b94012c)
2019-03-06PackFile: report correct message for checksum mismatchLuca Milanesio2-6/+7
When the packfile checksum does not match the expected one report the correct checksum error instead of reporting that the number of objects is incorrect. Change-Id: I040f36dacc4152ae05453e7acbf8dfccceb46e0d Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> (cherry picked from commit 436c99ce5946f31f06b8704b1fd33136f39dc814)
2019-03-06ObjectDirectory: Clean up loggingDavid Pursehouse3-2/+7
Externalize the message and log the pack file with absolute path. Change-Id: I019052dfae8fd96ab67da08b3287d699287004cb Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> (cherry picked from commit 9665d86ba1dd2937ca26f6aba63bb16aa277f888)

Back to the top