1
0

Updated changelog and readme.

This commit is contained in:
Alexander Andreev 2020-11-18 23:50:58 +04:00
parent 7754a90313
commit 86b6278657
2 changed files with 27 additions and 7 deletions

View File

@ -1,6 +1,26 @@
# Changelog # Changelog
## 0.3 - 2020-09-09 ## 0.4.0 - 2020-11-18
### Added
- For 2ch.hk check for if a file is a sticker was added;
- Encoding for `!op.txt` file was explicitly set to `utf-8`;
- Handling of HTTP errors was added so now program won't crash if file doesn't
exist or not accessible for any other reason;
- To a scraper was added matching of hashes of two files that happen to share
same name and size, but hash reported by an imageboard is not the same as of
a file. It results in excessive downloading and hash calculations. Hopefully,
that only the case for 2ch.hk.
### Changed
- FileInfo class is now a frozen dataclass for memory efficiency.
### Fixed
- Found that arguments for match function that matches for `image.ext` pattern
were mixed up in places all over the parsers;
- Also for 2ch.hk checking for if `sub` and `com` was changed to `subject` and
`comment`.
## 0.3.0 - 2020-09-09
### Added ### Added
- Parser for lolifox.cc. - Parser for lolifox.cc.

View File

@ -1,8 +1,8 @@
This is a tool for scraping files from imageboards' threads. This is a tool for scraping files from imageboards' threads.
It extracts the files from a JSON version of a thread. And then downloads 'em It extracts the files from a JSON representation of a thread. And then downloads
in a specified output directory or if it isn't specified then creates following 'em in a specified output directory or if it isn't specified then creates
directory hierarchy in a working directory: following directory hierarchy in a working directory:
<imageboard name> <imageboard name>
|-<board name> |-<board name>
@ -25,8 +25,8 @@ separately. E.g. `4chan b 1100500`.
`-o`, `--output-dir` -- output directory where all files will be dumped to. `-o`, `--output-dir` -- output directory where all files will be dumped to.
`--no-op` -- by default OP's post will be saved in a `!op.txt` file. This flag `--no-op` -- by default OP's post will be saved in a `!op.txt` file. This flag
disables this behaviour. I desided to put an `!` in a name so this file will be disables this behaviour. An exclamation mark `!` in a name is for so this file
on the top in a directory listing. will be on the top of a directory listing.
`-v`, `--version` prints the version of the program, and `-h`, `--help` prints `-v`, `--version` prints the version of the program, and `-h`, `--help` prints
help for a program. help for a program.
@ -37,4 +37,4 @@ help for a program.
- [lainchan.org](https://lainchan.org) since 0.1.0 - [lainchan.org](https://lainchan.org) since 0.1.0
- [2ch.hk](https://2ch.hk) since 0.1.0 - [2ch.hk](https://2ch.hk) since 0.1.0
- [8kun.top](https://8kun.top) since 0.2.2 - [8kun.top](https://8kun.top) since 0.2.2
- [lolifox.cc](https://lolifox.cc) since 0.3 - [lolifox.cc](https://lolifox.cc) since 0.3.0