utils-system/darakian.ddh
1
0
Fork 0
mirror of https://github.com/darakian/ddh.git synced 2026-09-10 07:06:21 -04:00
Code Issues Actions Packages Projects Releases Wiki Activity
427 commits 16 branches 3 tags 403 KiB
Commit graph

427 commits

This branch
This branch
All branches
Author SHA1 Message Date
Jon Moroney 89b2e8d439
Merge pull request #35 from darakian/factor-out-fileinfo 
Factor out fileinfo to its own file
 2020-08-02 15:41:03 -07:00
Jon Moroney cb44786864 Factor out fileinfo to its own file 
Move the file info struct to its own file. lib.rs is getting a bit big
 2020-08-02 15:38:29 -07:00
Jon Moroney 7035276acd
Merge pull request #33 from darakian/impl-serialize-for-FileInfo 
Remove serde-derive
 2020-08-02 14:56:34 -07:00
Jon Moroney c147bac7b5 Remove serde-derive 
This PR implements serialize for Fileinfo and removes serde derive.
This speeds up compile time.
 2020-08-02 14:35:11 -07:00
Jon Moroney 3636ebf424
Merge pull request #34 from darakian/skip-hash-unwrap 
Skip unrapping hashes in comparison
 2020-08-02 14:33:47 -07:00
Jon Moroney 806821e6cc Skip unrapping hashes in comparison 
Rust will inspect the contents of an option in equality checking
Skipping the unwrap step resolves thread panics where a zero length file
will have no hash
 2020-08-02 14:30:38 -07:00
Jon Moroney 4e7c67c926
Merge pull request #32 from darakian/use-candidate-name-for-results 
Use get_candidate_name() for results output
 2020-07-13 09:15:28 -07:00
Jon Moroney c68d7640d5 Use get_candidate_name() for results output 2020-07-13 09:11:32 -07:00
Jon Moroney ec63f1d9a7
Merge pull request #31 from darakian/bump-version-to-0.11.1 
update version number and remove unused import
 2020-07-12 11:17:29 -07:00
Jon Moroney 2eed388ff9 update version number and remove unused import 2020-07-12 11:08:49 -07:00
Jon Moroney 2992e6c0fd
Merge pull request #28 from darakian/debug-thread-file-panics 
Debug thread file panics
 2020-07-09 17:11:51 -07:00
Jon Moroney dddf30caa1
Merge pull request #29 from darakian/only-use-gh-action-on-pr 
Restrict GH actions to PRs
 2020-07-09 17:10:19 -07:00
Jon Moroney 92f2f62350 Remove redundant type conversion 
and
Use get filetype directly

Use fs::symlink_metadata rather than fs::metadata. Checking for symlinks always fails in the latter
Correct behavior is to not follow symlinks. Following symlinks will come later as an opt in feature.
 2020-07-09 17:09:14 -07:00
Jon Moroney 0e7ffbe699 Restrict GH actions to PRs 2020-07-09 17:07:35 -07:00
Jon Moroney 5360ce1477
Merge pull request #26 from darakian/add-discombobulate 
Add chunk iter
 2020-07-06 18:24:04 -07:00
Jon Moroney 927d1bab3d Add chunk iter 
This PR adds a new struct which breaks a file into chunks
of a given size.
 2020-07-06 18:20:54 -07:00
Jon Moroney 4c8a32d73a
Merge pull request #25 from darakian/remove-appveyor-travis 
Remove travis-ci and appveyor in favor of github actions
 2020-07-02 13:01:52 -07:00
Jon Moroney 3fcbe2a475 Remove travis-ci and appveyor in favor of github actions 
Travis and appveyor are now redundant
 2020-07-02 11:43:22 -07:00
Jon Moroney 720de9111c
Merge pull request #24 from darakian/update-rayon 
Bump rayon
 2020-06-20 16:03:37 -07:00
Jon Moroney 98684c83f5 Bump rayon 
Update rayon to the 1.3.x release branch
 2020-06-20 16:01:40 -07:00
Jon Moroney 1630cb9dd0 Fixup toml 2020-06-17 10:12:12 -07:00
Jon Moroney 031a5783aa Merge branch 'master' of github.com:darakian/ddh 2020-06-17 10:08:45 -07:00
Jon Moroney d340ccac47 Move to LGPL 3.0 license and bump version to ensure a clear point of change 2020-05-03 16:55:13 -07:00
Jon Moroney 3e49650335
Merge pull request #20 from bernaferrari/patch-1 
add syntax highlighting in README
 2020-03-17 09:11:06 -07:00
Bernardo Ferrari 6e44732dda
add syntax highlighting in README 2020-03-16 22:31:04 -03:00
Jon Moroney 1d4f4922ba Bump version for new api signature 2020-02-22 17:36:34 -08:00
Jon Moroney 35b5e2e801
Merge pull request #19 from demurgos/generic-deduplicate 
Make `deduplicate_dirs` generic
 2020-02-21 10:13:57 -08:00
Charles Samborski 34a80ec233
Make deduplicate_dirs generic 
This commit updates the signature of `deduplicate_dirs` to allow passing any type compatible with `Path` instead of requiring `&str`.

This change aligns the API with Rust's standard library, which uses `AsRef<Path>` for its FS functions.

This is an extra feature and should not break any existing code since `&str` implements `AsRef<Path>`.
 2020-02-21 18:16:16 +01:00
Jon Moroney fb1dc95010
Merge pull request #18 from darakian/document-public-methods 
Document public methods
 2020-02-18 10:33:42 -08:00
Jon Moroney 052d35fc2a Add some public docs and examples. Bump version so that they show up on docs.rs/ddh 
Also include a fixed github action workflow
 2020-02-18 10:28:03 -08:00
Jon Moroney a55235e573 Bump version to reflect the removal of bufreader. Minor performance improvement should be seen. 2019-12-22 10:52:32 -10:00
Jon Moroney 0048d5009b
Merge pull request #17 from darakian/remove-bufreader 
Remove bufreader
 2019-12-22 10:51:30 -10:00
Jon Moroney 1fe72df294 16KB seems to give better performance than 64KB 
Number of system calls is lower than on master, but performance is erratic.
I need a better system to test this on to determin if this is really a good change.`

Upon further analysis syscall count is roughly half compared to the master branch.
Total runtime is still erratic but is inline with master while sometimes being faster
and sometimes slower. The FS read ahead behavior is probably to blame/thank there, but
¯\_(ツ)_/¯

Given the reduction in syscalls I'm going to call this a win. In the event of a multidisk
system or a system with a more performant FS I expect this change to improve performance.
 2019-12-22 10:39:56 -10:00
Jon Moroney 701ef480b2 Basic removal with arbitrary 64KB block choice 2019-12-22 10:38:54 -10:00
Jon Moroney 2c7ee5e83d Remove deserialize derive 2019-12-21 21:17:53 -10:00
Jon Moroney 0e31253de3 Remove deserialize derive 2019-09-28 20:28:55 -07:00
Jon Moroney f615ab85ee Fix panic issue around unreadable files. 2019-09-25 10:38:02 -07:00
Jon Moroney 6fea9633da Match assignment for dir metadata 2019-09-25 10:10:48 -07:00
Jon Moroney e88f902613 Oneline fileinfo construction 2019-09-25 09:49:21 -07:00
Jon Moroney 1313a60d8a Move rather than copy file paths. Compiler may have optimized this away. Not sure. 2019-09-24 15:51:03 -07:00
Jon Moroney a1cd60b0eb Be explicit about types in hashmap for extra readability 2019-09-24 15:02:46 -07:00
Jon Moroney e00b391be3 Bump version and update README for readability 2019-09-16 14:57:05 -07:00
Jon Moroney 69760139a7 Bump version and update README for readability 2019-09-16 14:56:43 -07:00
Jon Moroney d1adf21427 Commit to nohash-hasher 2019-09-16 14:54:10 -07:00
Jon Moroney 0c0c9ff4dc Merge branch 'master' of github.com:darakian/ddh 2019-09-13 16:20:07 -07:00
Jon Moroney 87abc41b51 Testing nohash hasher 2019-09-13 16:20:02 -07:00
Jon Moroney 3c0c2ba064 Bump version so crates.io can see the readme 2019-08-31 12:12:46 -07:00
Jon Moroney 1593cae52c Merge branch 'master' of github.com:darakian/ddh 2019-08-28 15:42:14 -07:00
Jon Moroney 135ec8b1e9 Remove old dedupe code 2019-08-28 15:42:06 -07:00
Jon Moroney f52570b8b2
Merge pull request #16 from darakian/add-package-metadata 
Add package metadata
 2019-08-28 13:32:33 -07:00