Arav/ScrapTheChan
1
0
Fork 0
Code Releases 8 Activity

8 Releases 8 Tags

RSS Feed
  • 0.5.1 43909c2b29
    Compare

    ScrapTheChan ver. 0.5.1 Pre-Release

    Arav released this 2021-05-04 04:05:22 +04:00 | 0 commits to master since this release

    Added

    • Message when a file cannot be retrieved.

    Fixed

    • Removed excessive hash comparison when files has same name;
    • A string forgotten to set to be a f-string, so now it displays a reason of why
      thread wasn't found.
    Downloads
  • 0.5.0 245e33f40d
    Compare

    ScrapTheChan ver. 0.5.0 Pre-Release

    Arav released this 2021-05-03 02:47:21 +04:00 | 5 commits to master since this release

    Added

    • Now program makes use of skip_posts argument. Use CLI option -S <number>
      or --skip-posts <number> to set how much posts you want to skip.

    Changed

    • Better, minified messages;
    • Fixed inheritance of Scraper's subclasses and its sane rewrite that led to
      future easy extension with way less repeating.
    • Added a general class TinyboardLikeParser that implements post parser for
      all imageboards based on it or the ones that have identical JSON API. From now
      on all such generalisation classes will end with *LikeParser;
    • Changed file_base_url for 8kun.top.

    Removed

    • Support for Lolifox, since it's gon
    Downloads
  • 0.4.0 986fdbe7a7
    Compare

    ScrapTheChan ver. 0.4.0 Pre-Release

    Arav released this 2020-11-19 01:34:33 +04:00 | 26 commits to master since this release

    Added

    • For 2ch.hk check for if a file is a sticker was added;
    • Encoding for !op.txt file was explicitly set to utf-8;
    • Handling of connection errors was added so now program won't crash if file
      doesn't exist or not accessible for any other reason and if any damaged files
      was created then they will be removed;
    • Added 3 retries if file was damaged during downloading;
    • To a scraper was added matching of hashes of two files that happen to share
      same name and size, but hash reported by an imageboard is not the same as of
      a file. It results in excessive downloading and hash calculations. Hopefully,
      that only the case for 2ch.hk.

    Changed

    • FileInfo class is now a frozen dataclass for memory efficiency.

    Fixed

    • Found that arguments for match function that matches for image.ext pattern
      were mixed up in places all over the parsers;
    • Also for 2ch.hk checking for if sub and com was changed to subject and
      comment.
    Downloads
  • 0.3.0 2043fc277f
    Compare

    ScrapTheChan ver. 0.3.0 Pre-Release

    Arav released this 2020-09-09 04:42:04 +04:00 | 38 commits to master since this release

    • Added lolifox.cc.
    • Removed BasicScraper.
    • Fixed User-Agent.
    Downloads
  • 0.2.2 7825b53121
    Compare

    ScrapTheChan ver. 0.2.2 Pre-Release

    Arav released this 2020-07-20 04:32:30 +04:00 | 40 commits to master since this release

    Added 8kun.top support.

    Downloads
  • 0.2.1 f21ff0aff5
    Compare

    ScrapTheChan ver. 0.2.1 Pre-Release

    Arav released this 2020-07-20 02:55:54 +04:00 | 49 commits to master since this release

    Now text output is more informational.

    Downloads
  • 0.2.0 22961611da
    Compare

    ScrapTheChan ver. 0.2.0 Pre-Release

    Arav released this 2020-07-18 04:44:37 +04:00 | 52 commits to master since this release

    Now scraper is threaded and hella fast. So program became actually useful.

    Downloads
  • 0.1.0 0ed1e896ca
    Compare

    ScrapTheChan ver. 0.1.0 Pre-Release

    Arav released this 2020-07-08 23:13:32 +04:00 | 56 commits to master since this release

    Downloads