orhun
9adfe8bc1f
readme: Update README about installation from AUR ( #101 )
2020-10-12 20:56:20 +02:00
Patrick Sier
fa549f2fed
Remove hash fragment from scraped URLs ( #100 )
...
* refactor: remove hash fragment from scraped urls
* refactor: handle fragments in download paths
* test: split url_to_path tests
2020-10-12 16:17:50 +02:00
orhun
d9c17d9ec2
fix: Use CARGO_BIN_EXE instead of hardcoded path for tests ( #98 )
2020-10-12 13:43:10 +02:00
CohenArthur
5705175132
readme: Update README with latest additions ( #97 )
...
The README lacked the latest features added to suckit, such as random delays between downloads or user authentication
2020-10-12 10:14:46 +02:00
Patrick Sier
f3f19fd1e8
Add support for HTTP basic auth args ( #96 )
...
* feat: add arg for HTTP basic auth
* feat: chunk space-separated auth args
* refactor: update auth hashmap, host handling
Require host for auth HashMap, defaulting to the origin URL's host value
if not provided otherwise.
* refactor: auth help string, constant updates
* test: add auth functional tests
2020-10-12 09:17:44 +02:00
Jaslo Ziska
794d8ddd53
Add dry run flag ( #95 )
...
Add --dry-run flag to stop files from being saved, as mentioned in
https://github.com/Skallwar/suckit/issues/94
2020-10-03 22:36:35 +02:00
Esteban Blanc
ffec62254d
cargo: Add caret requirements ( #93 )
2020-08-31 16:52:40 +02:00
Esteban Blanc
16dc20519d
misc: Add info for crates.io ( #92 )
2020-08-16 17:22:05 +02:00
Esteban Blanc
e65560beb4
ci: Check before build, build needs lint ( #90 )
2020-07-02 15:56:18 +02:00
Esteban Blanc
809bdb5e0e
ci: Fix steps names ( #89 )
2020-07-02 15:46:23 +02:00
Jakub Szymański
8e6a5ebbb1
Add error handling for persistent errors ( #88 )
...
* Add error handling for persistent errors
* lint file and add init for new parameter
* Update src/args.rs
Co-authored-by: CohenArthur <arthur.cohen@epita.fr>
* misc: Run cargo fmt
Co-authored-by: CohenArthur <arthur.cohen@epita.fr>
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
2020-07-02 15:24:42 +02:00
Rahul Sharma
397315b5f3
Add support for including excluding pages based on regex patterns ( #83 )
2020-05-17 14:21:03 +02:00
Atul Bhosale
8a311c2526
Add cargo fmt to github-actions config ( #85 )
2020-05-14 21:48:52 +02:00
Atul Bhosale
ae20a2713e
Format code using 'cargo fmt' ( #84 )
2020-05-13 21:02:14 +02:00
CohenArthur
c5bed43ee5
aciinema: Add new public recording ( #80 )
2020-05-08 19:19:59 +02:00
Rahul Sharma
16a5ed9cf7
Feature: Custom user agent as a CLI argument ( #76 )
...
* Allow passing user agent via cli args
* user-agent: Short and long CLI arguments
Co-authored-by: creativcoder <rahul.sharma@atherenergy.com>
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
2020-05-06 19:29:39 +02:00
SpyrosRoum
c857a2c9dd
Feature: Add unlimited download depth ( #73 )
...
* Add -1 for depth to act as unlimited download and default for depth
* Use static instead of magic number for infinite depth
2020-05-06 17:53:32 +02:00
SpyrosRoum
e11ed48b46
feature: Add random ( #66 )
...
* Add a random delay of 2 - 7 seconds between downloads
* Add command line args for delay and move delay to method
* Ignore .idea folder
* Update test for scraper to work with new low and high arguments
* Add a comment
* Remove unnecessary `&` for method call to self
* Rename `low` and `high` to `--delay` and `--random-range`
* Test different values for new arguments
* rename variable for consistency
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
* change help for clarity purposes
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
* Some whitespace
Co-authored-by: CohenArthur <arthur.cohen@epita.fr>
* Some whitespace
Co-authored-by: CohenArthur <arthur.cohen@epita.fr>
* rename variable for consistency
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
* Fix variable names
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
Co-authored-by: CohenArthur <arthur.cohen@epita.fr>
2020-05-06 14:44:04 +02:00
CohenArthur
1d122621ae
readme: Add cargo install --git instead of cloning ( #65 )
...
* Update README.md
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
* readme: Add `cargo install --git` instead of cloning
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
2020-05-05 20:52:14 +02:00
CohenArthur
f9f6bc6c47
speed_regression: Change from 5 retries to 10 ( #61 )
...
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
2020-05-05 17:31:02 +02:00
CohenArthur
dc57581baa
readme: Add installation instructions ( #62 )
2020-05-05 17:28:53 +02:00
Esteban Blanc
8dc6945c3c
Update README.md
2020-05-05 14:33:28 +02:00
CohenArthur
6e93171185
Better help for CLI arguments ( #56 )
...
* args: Add 'help' section
* benchmark: Add more data
* Update src/args.rs
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
* Update src/args.rs
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
Co-authored-by: Esteban Blanc <estblcsk@gmail.com>
2020-05-01 13:48:39 +02:00
Esteban Blanc
5233aecbc1
doc: Add documentation ( #54 )
...
* doc: Add documentation part 1
* doc: More doc
* doc: src/args.rs
Co-authored-by: CohenArthur <arthur.cohen@epita.fr>
* doc: src/downloader.rs
Co-authored-by: CohenArthur <arthur.cohen@epita.fr>
* doc: src/url_helper.rs
Co-authored-by: CohenArthur <arthur.cohen@epita.fr>
* doc: src/url_helper.rs
Co-authored-by: CohenArthur <arthur.cohen@epita.fr>
Co-authored-by: CohenArthur <arthur.cohen@epita.fr>
2020-04-30 16:25:53 +02:00
Esteban Blanc
fa28eb13ea
Refactor whole project ( #53 )
...
* refactor: Move `Response` into its own module
* coding-style: Use field shorthand
Use `tries` instead of `tries: tries`
* cleanup: Refactor whole project part 1
* cleanup: Refactor whole project part 2
Co-authored-by: cohenarthur <arthur.cohen@epita.fr>
2020-04-30 13:31:11 +02:00
Esteban Blanc
fa32b55c20
Update README.md
2020-04-30 00:07:31 +02:00
CohenArthur
f685b2284b
Add asciinema embed link ( #47 )
...
* readme: Add asciinema
* readme: Embed asciinema
* readme: Add script for asciinema
* readme: Embed instead of script
2020-04-30 00:04:51 +02:00
Esteban Blanc
e318cfba59
Merge pull request #46 from Skallwar/update_licenses
...
Update Licenses
2020-04-29 23:56:44 +02:00
Esteban Blanc
5f2819ea59
Create LICENSE-MIT
2020-04-29 23:53:17 +02:00
Esteban Blanc
520a278f3f
Rename LICENSE to LICENSE-APACHE
2020-04-29 23:51:14 +02:00
Esteban Blanc
24cff8a3d0
Update README.md
2020-04-29 23:49:02 +02:00
Esteban Blanc
8b4bed5832
Merge pull request #44 from Skallwar/add-logo-readme
...
Add Suckit Logo
2020-04-29 22:51:22 +02:00
Esteban Blanc
1264745379
Merge branch 'master' into add-logo-readme
2020-04-29 22:50:33 +02:00
cohenarthur
1ec12c4042
logo: Uncrop logo
2020-04-29 23:21:45 +02:00
cohenarthur
469170e334
logo: Crop image
2020-04-29 23:21:06 +02:00
cohenarthur
840da51851
Merge branch 'master' into add-logo-readme
2020-04-29 23:16:15 +02:00
cohenarthur
7ebbc7a8ab
logo: Add suckit logo
2020-04-29 23:14:45 +02:00
CohenArthur
c247a60824
Merge pull request #45 from Skallwar/clippy_fmt
...
misc: Run clippy and rustfmt
2020-04-29 22:37:33 +02:00
Esteban Blanc
bc76cd3c9a
Merge branch 'master' into clippy_fmt
2020-04-29 22:30:05 +02:00
Esteban Blanc
c2cc1ba34c
misc: Remove dead code
2020-04-29 22:29:24 +02:00
Esteban Blanc
79c9a5d4bb
misc: Run clippy and rustfmt
2020-04-29 22:24:26 +02:00
Esteban Blanc
88a5f6cd9f
Merge pull request #43 from Skallwar/prettify-readme
...
Prettify readme
2020-04-29 20:27:55 +02:00
cohenarthur
781befd3e0
Merge branch 'prettify-readme' of github.com:skallwar/suckit into prettify-readme
2020-04-29 20:37:53 +02:00
cohenarthur
8f6c32b632
readme: Switch from "|" separator to ", "
2020-04-29 20:36:50 +02:00
cohenarthur
6e0ac9527d
readme: Add contributing links
2020-04-29 20:33:08 +02:00
cohenarthur
a69ff75645
readme: Close unclosed backtick
2020-04-29 20:26:51 +02:00
cohenarthur
a3b535ff85
readme: Escape pipe separators for short/long options
2020-04-29 20:26:05 +02:00
cohenarthur
be1b6f07a6
readme: Add options and example
2020-04-29 20:25:06 +02:00
CohenArthur
0107f6559f
Merge branch 'master' into prettify-readme
2020-04-29 19:34:34 +02:00
Esteban Blanc
70edc12515
Update README.md
2020-04-29 19:18:33 +02:00
