A tool for scraping files from imageboards’ threads.

Go to file

Alexander "Arav" Andreev 0287d3a132 Turned a string into f-string.		2021-05-04 03:55:32 +04:00
scrapthechan	Turned a string into f-string.	2021-05-04 03:55:32 +04:00
.gitignore	Initial commit with all the files.	2020-07-08 22:53:39 +04:00
CHANGELOG.md	Updated CHANGELOG with version 0.5.0.	2021-05-03 02:44:19 +04:00
COPYING	Initial commit with all the files.	2020-07-08 22:53:39 +04:00
Makefile	Makefile updated to version 0.5.0.	2021-05-03 02:44:37 +04:00
README.md	README updated. lolifox.cc removed. Option --skip-posts added.	2021-05-03 02:45:41 +04:00
setup.cfg	Lolifox removed. Development Status classifier is changed to Alpha. Python 3.7 classifier left to represent oldest supported version.	2021-05-03 02:43:49 +04:00
setup.py	Initial commit with all the files.	2020-07-08 22:53:39 +04:00

README.md

This is a tool for scraping files from imageboards' threads.

It extracts the files from a JSON representation of a thread. And then downloads 'em in a specified output directory or if it isn't specified then creates following directory hierarchy in a working directory:

<imageboard name>
|-<board name>
  |-<thread>
    |-[!op.txt]
    |-...
  |-...

Usage

scrapthechan [OPTIONS] (<url> | <imageboard> <board> <thread>)

<url> -- URL of a thread.

<imageboard> <board> <thread> -- imageboard name, board name and thread ID separately. E.g. 4chan b 1100500.

-o, --output-dir -- output directory where all files will be dumped to.

-N, --no-op -- by default OP's post will be saved in a !op.txt file. This flag disables this behaviour. An exclamation mark ! in a name is for so this file will be on the top of a directory listing.

-S <num>, --skip-posts <num> -- skip given number of posts.

-v, --version prints the version of the program.

-h, --help prints help for a program.

Supported imageboards

4chan.org since 0.1.0
lainchan.org since 0.1.0
2ch.hk since 0.1.0
8kun.top since 0.2.2

TODO

Sane rewrite of a program;
Thread watcher.