Skip to main content
aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2019-09-03Bazel: Update bazlets to the latest master revisionstable-4.9David Pursehouse1-1/+1
The latest revision includes various fixes to allow the build to work with recent versions of Bazel. Change-Id: I72c100b99762010946d9b2784286af560bbdf185 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
2019-09-03Bazel: Remove FileTreeIteratorWithTimeControl from BUILD fileDavid Pursehouse1-1/+0
FileTreeIteratorWithTimeControl was deleted in a024759, but was not removed from the BUILD file, thus causing the bazel build to fail. Change-Id: I892c0ffcac947298d0d6009374ee2c5d9afefb66 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> (cherry picked from commit e54fde8616f53a6fab5201db4a0c7e67dd0145c5)
2019-09-03BatchRefUpdate: repro racy atomic update, and fix itHan-Wen Nienhuys2-60/+87
PackedBatchRefUpdate was creating a new packed-refs list that was potentially unsorted. This would be papered over when the list was read back from disk in parsePackedRef, which detects unsorted ref lists on reading, and sorts them. However, the BatchRefUpdate also installed the new (unsorted) list in-memory in RefDirectory#packedRefs. With the timestamp granularity code committed to stable-5.1, we can more often accurately decide that the packed-refs file is clean, and will return the erroneous unsorted data more often. Unluckily timed delays also cause the file to be clean, hence this problem was exacerbated under load. The symptom is that refs added by a BatchRefUpdate would stop being visible directly after they were added. In particular, the Gerrit integration tests uses BatchRefUpdate in its setup for creating the Admin group, and then tries to read it out directly afterward. The tests recreates one failure case. A better approach would be to revise RefList.Builder, so it detects out-of-order lists and automatically sorts them. Fixes https://bugs.eclipse.org/bugs/show_bug.cgi?id=548716 and https://bugs.chromium.org/p/gerrit/issues/detail?id=11373. Bug: 548716 Change-Id: I613c8059964513ce2370543620725b540b3cb6d1 Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-09-03Delete unused FileTreeIteratorWithTimeControlMatthias Sohn2-177/+0
The only usage of this test iterator was removed in df637928d. Hence delete this iterator and associated test. Change-Id: I47710133ec3edc675c21db210960c024982668c6 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> (cherry picked from commit a024759cf5bf1cd6b9beb4f790d484943761a7e1)
2019-09-03Fix RacyGitTests#testRacyGitDetectionMatthias Sohn1-11/+26
This test case assumed file system timestamp resolution of 1 second. On filesystems with a finer resolution this test fails since the index entry is only smudged if the file index entry's lastModified and the lastModified of the git index itself are within the same filesystem timer tick. Fix this by ensuring that these timestamps are identical which should work for any filesystem timer resolution. Bug: 548188 Change-Id: Id84d59e1cfeb48fa008f8f27f2f892c4f73985de Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> Signed-off-by: Christian Halstrick <christian.halstrick@sap.com> (cherry picked from commit 1159f9dd7c80a53c2509cd75d997a6afed37f9a6)
2019-09-03Change RacyGitTests to create a racy git situation in a stable wayMasaya Suzuki1-25/+24
By using File#setLastModified, we can create a racy git situation stably. Tested with --runs_per_test=100 Bug: 526111 Change-Id: I60b3632d353e19f335668325aa603640be423f58 Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> (cherry picked from commit df637928d2ef4b9ee06af7e37344c7848f870ce4)
2019-09-03Silence API warningsMatthias Sohn1-8/+51
Change-Id: I27fd62de51ca0eedcc7e2e256487bda1e18bce8a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-18Prepare 4.9.11-SNAPSHOT buildsMatthias Sohn56-333/+333
Change-Id: I99137bc4958167773d2fc8b1d48fbb508af52be5 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-18JGit v4.9.10.201904181027-rv4.9.10.201904181027-rMatthias Sohn56-59/+59
Change-Id: I3034d1ad263fdfb23347304f866309354930fd19 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-18Merge branch 'stable-4.8' into stable-4.9Matthias Sohn6-28/+81
* stable-4.8: Prepare 4.7.10-SNAPSHOT builds JGit v4.7.9.201904161809-r Prepare 4.5.8-SNAPSHOT builds JGit v4.5.7.201904151645-r Remember the cause for invalidating a packfile Fix API problem filters Fix pack files scan when filesnapshot isn't modified Change-Id: Idaa789e699f1ef568ea957184d0641355d9e3181
2019-04-16Merge branch 'stable-4.7' into stable-4.8stable-4.8Matthias Sohn5-28/+82
* stable-4.7: Prepare 4.7.10-SNAPSHOT builds JGit v4.7.9.201904161809-r Prepare 4.5.8-SNAPSHOT builds JGit v4.5.7.201904151645-r Remember the cause for invalidating a packfile Fix API problem filters Fix pack files scan when filesnapshot isn't modified Change-Id: I7984f68833f2d615399296e53cb9a64e5b4ca6ed Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-16Prepare 4.7.10-SNAPSHOT buildsstable-4.7Matthias Sohn56-321/+321
Change-Id: I662ee441521c1370a31f274cc6d001723ad0c528 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-16JGit v4.7.9.201904161809-rv4.7.9.201904161809-rMatthias Sohn56-59/+59
Change-Id: Id6e379c3ce52c83455e37cbc3d77db0e188fdb85 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-16Merge branch 'stable-4.6' into stable-4.7Matthias Sohn5-29/+83
* stable-4.6: Prepare 4.5.8-SNAPSHOT builds JGit v4.5.7.201904151645-r Remember the cause for invalidating a packfile Fix API problem filters Fix pack files scan when filesnapshot isn't modified Change-Id: I6ea51dde6608a4163d681aa1ebf710f06da44b21 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-16Merge branch 'stable-4.5' into stable-4.6stable-4.6Matthias Sohn5-44/+82
* stable-4.5: Prepare 4.5.8-SNAPSHOT builds JGit v4.5.7.201904151645-r Remember the cause for invalidating a packfile Fix API problem filters Fix pack files scan when filesnapshot isn't modified Change-Id: I0b4eaa521ebdea83ab18c05915d691c07a575a7d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-15Prepare 4.5.8-SNAPSHOT buildsstable-4.5Matthias Sohn56-302/+302
Change-Id: I70628cb8fcad0a60598dc937abbed63606a78599 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-15JGit v4.5.7.201904151645-rv4.5.7.201904151645-rMatthias Sohn56-59/+59
Change-Id: I3e32cf13f5cb99d8e570412d80d941740399c07d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-11Remember the cause for invalidating a packfileLuca Milanesio5-23/+76
Keep track of the original cause for a packfile invalidation. It is needed for the sysadmin to understand if there is a real underlying filesystem problem and repository corruption or if it is simply a consequence of a concurrency of Git operations (e.g. repack or GC). Change-Id: I06ddda9ec847844ec31616ab6d17f153a5a34e33 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-10Fix API problem filtersMatthias Sohn2-24/+1
Change-Id: I96e0ddc34251348ec4877c9d94b045eb1c53e758 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-04-09Fix pack files scan when filesnapshot isn't modifiedLuca Milanesio2-6/+54
Do not reload packfiles when their associated filesnapshot is not modified on disk compared to the one currently stored in memory. Fix the regression introduced by fef78212 which, in conjunction with core.trustfolderstats = false, caused any lookup of objects inside the packlist to loop forever when the object was not found in the pack list. Bug: 546190 Change-Id: I38d752ebe47cefc3299740aeba319a2641f19391 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-20Fix GC to delete empty fanout directories after repackingMatthias Sohn2-2/+22
The prune method did not delete empty fanout directories when loose objects moved to a new pack file but only when loose unreferenced objects were pruned. Change-Id: Ia068f4914c54d9cf9f40b75e8ea50759402b5000 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13Prepare 4.9.10-SNAPSHOT buildsMatthias Sohn56-333/+333
Change-Id: I2bdbe822ee623c5f15a26d092136d8602cef06ac Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13JGit v4.9.9.201903122025-rv4.9.9.201903122025-rMatthias Sohn56-59/+59
Change-Id: I807d24461b3557eee48294bfaa652b2a13d78113 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13Merge branch 'stable-4.8' into stable-4.9Matthias Sohn6-13/+97
* stable-4.8: Prepare 4.7.9-SNAPSHOT builds JGit v4.7.8.201903121755-r Prepare 4.5.7-SNAPSHOT builds JGit v4.5.6.201903121547-r Check for packfile validity and fd before reading Move throw of PackInvalidException outside the catch Use FileSnapshot to get lastModified on PackFile Include size when comparing FileSnapshot Do not reuse packfiles when changed on filesystem Silence API warnings for new API introduced for fixes Change-Id: I9a47153831f8eb10d3cd91b4157cf45385e5b13a Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-13Merge branch 'stable-4.7' into stable-4.8Matthias Sohn6-13/+97
* stable-4.7: Prepare 4.7.9-SNAPSHOT builds JGit v4.7.8.201903121755-r Prepare 4.5.7-SNAPSHOT builds JGit v4.5.6.201903121547-r Check for packfile validity and fd before reading Move throw of PackInvalidException outside the catch Use FileSnapshot to get lastModified on PackFile Include size when comparing FileSnapshot Do not reuse packfiles when changed on filesystem Silence API warnings for new API introduced for fixes Change-Id: I1136eaa1590b225ceaf16ba7552af0374253a74e Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Prepare 4.7.9-SNAPSHOT buildsMatthias Sohn56-321/+321
Change-Id: I1602a6b811fd9f95e284153b560d8d9f855fae9f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12JGit v4.7.8.201903121755-rv4.7.8.201903121755-rMatthias Sohn56-59/+59
Change-Id: Ia6dbde615671ed09789326aacad1655e7a7a59ad Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Merge branch 'stable-4.6' into stable-4.7Matthias Sohn6-13/+97
* stable-4.6: Prepare 4.5.7-SNAPSHOT builds JGit v4.5.6.201903121547-r Check for packfile validity and fd before reading Move throw of PackInvalidException outside the catch Use FileSnapshot to get lastModified on PackFile Include size when comparing FileSnapshot Do not reuse packfiles when changed on filesystem Silence API warnings for new API introduced for fixes Change-Id: I3d1544d034783fe0fa1385dfe9b03ad8e9247c63 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Merge branch 'stable-4.5' into stable-4.6Matthias Sohn6-13/+113
* stable-4.5: Prepare 4.5.7-SNAPSHOT builds JGit v4.5.6.201903121547-r Check for packfile validity and fd before reading Move throw of PackInvalidException outside the catch Use FileSnapshot to get lastModified on PackFile Include size when comparing FileSnapshot Do not reuse packfiles when changed on filesystem Silence API warnings for new API introduced for fixes Change-Id: I029e1797447e6729de68bd89d4d69b324dbb3f5f Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Prepare 4.5.7-SNAPSHOT buildsMatthias Sohn56-302/+302
Change-Id: I5c275c542e12746c3d8ecf8462791969f9e89e12 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12JGit v4.5.6.201903121547-rv4.5.6.201903121547-rMatthias Sohn56-59/+59
Change-Id: I5a071ed10e1ac1ab28f992d45cde335c12556a80 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Check for packfile validity and fd before readingLuca Milanesio1-0/+8
When reading from a packfile, make sure that is valid and has a non-null file-descriptor. Because of concurrency between a thread invalidating a packfile and another trying to read it, the read() may result into a NPE that won't be able to be automatically recovered. Throwing a PackInvalidException would instead cause the packlist to be refreshed and the read to eventually succeed. Bug: 544199 Change-Id: I27788b3db759d93ec3212de35c0094ecaafc2434 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12Move throw of PackInvalidException outside the catchLuca Milanesio1-2/+3
When a packfile is invalid, throw an exception explicitly outside any catch scope, so that is not accidentally caught by the generic catch-all cause, which would set the packfile as valid again. Flagging an invalid packfile as valid again would have dangerous consequences such as the corruption of the in-memory packlist. Bug: 544199 Change-Id: If7a3188a68d7985776b509d636d5ddf432bec798 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12Use FileSnapshot to get lastModified on PackFileLuca Milanesio1-1/+1
Do not redundantly call File.lastModified() for extracting the timestamp of the PackFile but rather use consistently the FileSnapshot which reads all file attributes in a single bulk call. Change-Id: I932675ae4fe56dcd3833dac249816f097303bb09 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Include size when comparing FileSnapshotLuca Milanesio5-9/+107
Due to finite filesystem timestamp resolution the last modified timestamp of files cannot detect file changes which happened in the immediate past (less than one filesystem timer tick ago). Read and consider file size also, so that differing file size can help to more accurately detect file changes without reading the file content. Use bulk read to avoid multiple stat calls to retrieve file attributes. Change-Id: I974288fff78ac78c52245d9218b5639603f67a46 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-12Do not reuse packfiles when changed on filesystemLuca Milanesio2-2/+15
The pack reload mechanism from the filesystem works only by name and does not check the actual last modified date of the packfile. This lead to concurrency issues where multiple threads were loading and removing from each other list of packfiles when one of those was failing the checksum. Rely on FileSnapshot rather than directly checking lastModified timestamp so that more checks can be performed. Bug: 544199 Change-Id: I173328f29d9914007fd5eae3b4c07296ab292390 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com>
2019-03-12Silence API warnings for new API introduced for fixesMatthias Sohn1-0/+14
Change-Id: I3ea7ff2efd33ca6c780afaef9010cec82780d7fa Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-11Reduce contention on PackFile.idx() function.Juergen Denner1-26/+35
In case of concurrent pack file access, threads may wait on the idx() function even for already open files. This happens especially with a slow file system. Performance numbers are listed in the bug report. Bug: 543739 Change-Id: Iff328d347fa65ae07ecce3267d44184161248978 Signed-off-by: Juergen Denner <j.denner@sap.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2019-03-06Fix error log message in ObjectDirectory.handlePackError()Matthias Sohn2-3/+3
Change-Id: I154f392ad025c4b642eb1123d375a0afaa853885 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> (cherry picked from commit 997d785418d55dce5a1188fdb95e6d2b4ab0bde5)
2019-03-06Properly format pack checksums in PackFile.idx()Matthias Sohn1-2/+3
Change-Id: Id805850dbe9a3d633168f3056e06ddeafd86f961 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> (cherry picked from commit a33e4dc58a87daf34072b82643aba0fd4456c165)
2019-03-06Cancel gc if thread was interruptedMatthias Sohn2-1/+53
see https://groups.google.com/d/msg/repo-discuss/oDB2rl3doDc/tFEh5Xt0CAAJ Change-Id: Ia6d4631c64e065d8b9b09e0b45e7a9ea8ac3f41d Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> (cherry picked from commit 882fed0d96c533513c43ae77aaff9cc07b94012c)
2019-03-06PackFile: report correct message for checksum mismatchLuca Milanesio2-6/+7
When the packfile checksum does not match the expected one report the correct checksum error instead of reporting that the number of objects is incorrect. Change-Id: I040f36dacc4152ae05453e7acbf8dfccceb46e0d Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com> (cherry picked from commit 436c99ce5946f31f06b8704b1fd33136f39dc814)
2019-03-06ObjectDirectory: Clean up loggingDavid Pursehouse3-2/+7
Externalize the message and log the pack file with absolute path. Change-Id: I019052dfae8fd96ab67da08b3287d699287004cb Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> (cherry picked from commit 9665d86ba1dd2937ca26f6aba63bb16aa277f888)
2019-03-06Bazel: Stop using native.git_repositoryDavid Pursehouse1-1/+3
The native.git_repository method doesn't work in the latest version of bazel, and causes the build to fail with: type 'struct' has no method git_repository() Change-Id: Id6a57369b681c0afe811e9e3740b141fb7fb4653 Signed-off-by: David Pursehouse <david.pursehouse@gmail.com> (cherry picked from commit ec5fc57b791081fa073fc5fd91286347238f8f7c)
2019-03-06ObjectDirectory: extra logging on packfile exceptionsLuca Milanesio1-0/+2
Display extra logging, including the exception with the associated stacktrace, whenever a packFile can't be read and thus removed from the packlist. Change-Id: I97a4e31dc427bfcc0baae438dcbe2dcd4704b824 Signed-off-by: Luca Milanesio <luca.milanesio@gmail.com> (cherry picked from commit 962babc4b27ffd90058fe7734f17ed1c4e77d958)
2018-12-24Prepare 4.9.9-SNAPSHOT buildsMatthias Sohn56-333/+333
Change-Id: I0f7238ce8f79063a410c85701020ffce2b1234cf Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24JGit v4.9.8.201812241815-rv4.9.8.201812241815-rMatthias Sohn56-59/+59
Change-Id: Ic75e4611cce396b186680bc962f0c73b2ee5fff1 Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24UploadPack: Test filtering by AdvertiseRefsHook in stateless transportsMasaya Suzuki3-3/+94
AdvertiseRefsHook is used to limit the visibility of the refs in Gerrit. If this hook is not called, then all refs are treated as visible, causing the server to serve commits reachable from branches the client should not be able to access, if asked to via a request naming a guessed object id. Until 3a529361a76e8267467071e0b13ebb36b97d8fb2 (Call AdvertiseRefsHook before validating wants, 2018-12-18), UploadPack would invoke this hook at ref advertisement time but not during negotiation and when serving a pack file. Add a test to avoid regressing. Stateful bidirectional transports were not affected, so the test uses HTTP. [jn: split out when backporting the fix to stable-4.5. The test passes as long as v4.9.0.201710071750-r~169 (fetch: Accept any SHA-1 on lhs of refspec, 2017-06-04) is cherry picked along with it.] Change-Id: I8c017107336adc7cb4c826985779676bf043e648 Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24Merge branch 'stable-4.8' into stable-4.9Matthias Sohn2-14/+10
* stable-4.8: Prepare 4.7.8-SNAPSHOT builds JGit v4.7.7.201812240805-r Fix feature versions imported by feature org.eclipse.jgit.pgm Prepare 4.5.6-SNAPSHOT builds JGit v4.5.5.201812240535-r Call AdvertiseRefsHook before validating wants Change-Id: Ib44e314a68bca2349b45f4937257aa1298c8d74b Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
2018-12-24Merge branch 'stable-4.7' into stable-4.8Matthias Sohn1-13/+9
* stable-4.7: Prepare 4.7.8-SNAPSHOT builds JGit v4.7.7.201812240805-r Fix feature versions imported by feature org.eclipse.jgit.pgm Prepare 4.5.6-SNAPSHOT builds JGit v4.5.5.201812240535-r Call AdvertiseRefsHook before validating wants Change-Id: I68a21067705b580b40840f8039001ff1e5273c15 Signed-off-by: Jonathan Nieder <jrn@google.com> Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>

Back to the top