Commit graph

62 commits

Author SHA1 Message Date
Yetangitu
349d3ba164 refresh_libgen: update version number to 0.7.0 2021-07-01 20:07:57 +00:00
Yetangitu
da8476a78c refresh_libgen: add streaming filters; filter out updated_edited and description_edited
Streaming filters (which are inserted into a pipe) make it possible to import the database without using a temporary file. This saves both time as well as storage space.

This change also filters out the updated_edited and description_edited tables and the related trigger from the database dump since these are not used in the *books* applications. This saves about 60% of database storage space compared to the full dump.
2021-07-01 19:09:40 +00:00
Yetangitu
097314f558 Remove leftover debug output 2021-06-02 13:15:27 +00:00
Yetangitu
3b9a8c73e4 Add 0.7.2 release file 2021-06-01 14:44:44 +00:00
Yetangitu
9e55d72896 Update README, update help() in update_libgen 2021-06-01 14:39:51 +00:00
Yetangitu
391b0189fb update_libgen: add -c option to integrat classify into the update process
config file: add these fields (values for demonstration purpose):
```
classify_xml="/home/username/Project/libgen_classify/xml"
classify_csv="/home/username/Project/libgen_classify/csv"
classify_sql="/home/username/Project/libgen_classify/sql"
classify_fields="ddc,lcc,nlm,fast,title,author"
classify_tor_ports="9100,9102,9104,9106,9108"
```
These fields are used by update_libgen to configure classify and import_metadata

All programs: the -@ switch now takes a mandatory parameter: -@ TORPORT
2021-06-01 14:12:09 +00:00
Yetangitu
b5af8887b3 Add import_metadata and import_metadata.sh to release file 2021-05-28 12:24:26 +00:00
Yetangitu
103b02a347 Replace accidentally uploaded older version of import_metadata.sh with current version 2021-05-21 22:07:51 +00:00
Yetangitu
5908d71308 More README futzing 2021-05-21 22:00:47 +00:00
Yetangitu
bfd7d9b60f typo 2021-05-21 21:45:46 +00:00
Yetangitu
f3d4345c61 Add import_metadata (python) and import_metadata.sh (bash) scripts
These two scripts both serve the same purpose in that they can take the data
produced by *classify* to update the database and/or produce SQL to do so at a
later moment. The python version is much faster at parsing CSV data and
updating the database, for the rest the scripts are functionally identical.
2021-05-21 21:18:04 +00:00
Yetangitu
4c7b585cb3 Updated README and added new version of 0.7.1 release file 2021-05-16 15:39:44 +00:00
Yetangitu
e528293f23 Update classify help 2021-05-16 15:37:43 +00:00
Yetangitu
b9849dee0f Modify help 2021-05-16 14:42:27 +00:00
Yetangitu
cf540a3b95 Add (currently unused) authors request 2021-05-16 14:33:53 +00:00
Yetangitu
bbc4371183 Update version (0.4.1) 2021-05-16 14:25:06 +00:00
Yetangitu
7da142f921 Remove SQL generation, add XML save option, implement multiple identifier search, add CSV generation, remove filename generation 2021-05-16 14:23:57 +00:00
Yetangitu
2bd34ab5c4 update classify version/release 2021-05-16 11:13:35 +00:00
Yetangitu
8f0ec5bd65 Add CDF output (-C md5), fix newlines in DDC/LCC 2021-05-16 11:12:05 +00:00
Yetangitu
7e2dccbc6c Add logging 2021-05-16 11:11:43 +00:00
Yetangitu
f2eebd826e New release file 0.7.1 2021-05-12 16:32:41 +00:00
Yetangitu
71683899b2 Remove outdated info about password options, these are no longer supported (use .my.cnf instead) 2021-05-12 15:28:19 +00:00
Yetangitu
4c06982500 Add classify 2021-05-12 15:23:59 +00:00
Yetangitu
787378a32e tm: shellcheck'ed 2021-05-12 15:16:05 +00:00
Yetangitu
dafb6ce6d1 update_libgen: refactoring, split out common code into library, shellcheck'ed 2021-05-12 15:15:36 +00:00
Yetangitu
7874a8d35c refresh_libgen: refactoring, split out common code into library, shellcheck'ed 2021-05-12 15:14:59 +00:00
Yetangitu
eba03f2669 books: refactoring, split out common code into library, shellcheck'ed 2021-05-12 15:14:03 +00:00
Yetangitu
e1ae2cdb00 books_functions: split out common code into library which can be sourced by other *books* tools 2021-05-12 15:13:12 +00:00
Yetangitu
ce5c75a0fb classify: OCLC classifier helper script
This script uses the OCLC classifier to find DDC (Dewey Decimal Code) and LCC (Library of Congress Code) for publications, it optionally produces SQL to update libgen/libgen diction databases.
2021-05-12 15:11:11 +00:00
Yetangitu
c0b00bf7dd enable use_deep_path, this created leading directories (db_prefix/language/topic/author/series for db=libgen, db_prefix/language/author/series for db=libgen_fiction) 2021-05-08 19:19:23 +00:00
Yetangitu
e8040ab9d3 Update README 2021-05-07 12:15:06 +00:00
Yetangitu
506d8ef80b gen.lib.rus.ec -> libgen.rs 2021-05-07 06:38:19 +00:00
Yetangitu
af7b702ae5 books 0.7.0: add IPFS download, refactor direct download
update_libgen 0.6.1: update API server
2021-05-06 14:03:42 +00:00
Yetangitu
7481ef1c69 refactor downloading code 2020-08-16 00:25:55 +00:00
Yetangitu
22f1499085 Bash is not C - use -gt instead of >
Add v0.6.4 release file
2020-08-14 23:21:19 +00:00
Yetangitu
d76c7bb4f5 add v0.6.3 to releases 2020-08-14 08:24:00 +00:00
Yetangitu
b2e5dd860c trim overly long filenames to 255 characters by ellipsizing in the middle 2020-08-14 08:22:24 +00:00
Yetangitu
69e8cd616b add 0.6.2 release file 2020-08-13 19:44:07 +00:00
Yetangitu
3523d99876 fix failure when downloading multitple files
remove some debugging code
2020-08-13 19:41:22 +00:00
Yetangitu
8b9c9e00e5 Add v0.6.1 tarball 2020-08-05 23:56:36 +00:00
Yetangitu
8353ca1af2 Correct markdown errors 2020-08-05 23:40:28 +00:00
Yetangitu
e7e96b17ed fix logic error in list views which caused fake failed torrent downloads on cancelling views without selecting any publications 2020-08-05 23:34:27 +00:00
Yetangitu
17190415ac Minor documentation changes 2020-08-05 12:25:07 +00:00
Yetangitu
1b69894e7d Add releases, screenshots 2020-08-05 12:19:47 +00:00
Yetangitu
c08bd57a65 Minor documentation updates 2020-08-05 12:16:36 +00:00
Yetangitu
5105afa81f Minor documentation updates 2020-08-05 12:14:09 +00:00
Yetangitu
7e562f142c Minor documentation changes 2020-08-05 11:56:46 +00:00
Yetangitu
153f5c0d52 Minor documentation changes 2020-08-05 11:46:35 +00:00
Yetangitu
9beb717faa Minor documentation and help text changes 2020-08-05 11:43:35 +00:00
Yetangitu
863fab7783 use is_true for use_torrent 2020-08-05 11:31:52 +00:00