Arav/ScrapTheChan
1
0
Fork 0
Code Releases 8 Activity
A tool for scraping files from imageboards’ threads.
26 Commits 1 Branch 8 Tags 118 KiB
Python 98%
Makefile 2%
bb47b50c5f
Go to file
Alexander "Arav" Andreev bb47b50c5f _is_file_ok now is _check_file and modified to be more efficient. Also added check for if files happened to share same name and size, but IB said wrong hash.
2020-11-18 23:47:26 +04:00
scrapthechan _is_file_ok now is _check_file and modified to be more efficient. Also added check for if files happened to share same name and size, but IB said wrong hash. 2020-11-18 23:47:26 +04:00
.gitignore Initial commit with all the files. 2020-07-08 22:53:39 +04:00
CHANGELOG.md Added support for lolifox.cc. Fixed User-Agent usage, so it applied correctly everywhere now. 2020-09-09 04:34:41 +04:00
COPYING Initial commit with all the files. 2020-07-08 22:53:39 +04:00
Makefile No right to fuck up! Shit... Forgot third part of a version. 2020-09-09 04:39:33 +04:00
README.md Added support for lolifox.cc. Fixed User-Agent usage, so it applied correctly everywhere now. 2020-09-09 04:34:41 +04:00
setup.cfg Added support for lolifox.cc. Fixed User-Agent usage, so it applied correctly everywhere now. 2020-09-09 04:34:41 +04:00
setup.py Initial commit with all the files. 2020-07-08 22:53:39 +04:00

README.md

This is a tool for scraping files from imageboards' threads.

It extracts the files from a JSON version of a thread. And then downloads 'em in a specified output directory or if it isn't specified then creates following directory hierarchy in a working directory:

<imageboard name>
|-<board name>
  |-<thread>
    |-[!op.txt]
    |-...
  |-...

Usage

scrapthechan [OPTIONS] (<url> | <imageboard> <board> <thread>)

<url> -- URL of a thread.

<imageboard> <board> <thread> -- imageboard name, board name and thread ID separately. E.g. 4chan b 1100500.

-o, --output-dir -- output directory where all files will be dumped to.

--no-op -- by default OP's post will be saved in a !op.txt file. This flag disables this behaviour. I desided to put an ! in a name so this file will be on the top in a directory listing.

-v, --version prints the version of the program, and -h, --help prints help for a program.

Supported imageboards