Upload to nyaa.si, anidex & TokyoTosho [done] (#1633)
* Update upload.go * Update helpers.go * Update template_test.go * Update upload_multiple.jet.html * change variable names * wrong copypaste * change variable name * ditto * Fix travis * Update main.css * Update upload.go * Update upload.jet.html * Update main.css * Update upload.go * More compact form * CSS changes to go along * Update main.css * Update upload.jet.html * Slightly lower bottom margin * Update upload_multiple.jet.html * tomorrow color adjustements * small css adjustements * Update upload.go * Update default_config.yml * Update default_config.yml * Update structs.go * Update user.go * Update structs.go * Update upload.go * Update default_config.yml * Update upload_multiple.jet.html * Update upload.go * Update upload.jet.html * Update template_test.go * this one is optional for anidex * Update default_config.yml * Possible improvement As mentionned in my comment, it seems that having a checkbox already checked and disabled for logged in user + having the input text for apikey is a bit tedious. Moreover, asking to check and fill the input text to upload in anidex when you are not logged is doing things twice. If you don't want to upload to anidex, don't fill the input. So this commit push a new behaviour: * If you are logged in and you have filled your anidex/nyaasi api key : the form should only show the checkbox asking if you want to upload to nyaasi. * If you're not logged in or you don't have filled your api key in your profile settings: you only show the input text for the api key and not the checkbox. If someone wants to upload to nyaasi/anidex, he will just need to fill the input. * This adds back the support of anonymous upload when logged in or not. This works simply by checking if the user wants to upload as anon (when he checks the checkbox "upload as anonymous") or by checking the emptyness of apikey. + reverts the condition statements since it was needed to have apikey empty * Forgot to save this file * This commit adds the go routines for each upload service (anidex, nyaasi, tosho). New controller and url to check the upload status (/upload/status/:id). If you add ?json at the end, it outputs a json format of the multi upload status. To prevent memory overload, we only keep in memory the multiupload status for 5 minutes. Moved multipleform struct from templates to upload utils. * fix form display on classic theme * This commit adds: * Javascript refreshing of the upload status * Localization of the pages * Fixed some bugs This works, only need to get torrent file now. * Added a new function to get the Path to the torrent file. * Added a function to check that a torrent file exists and if not generate it on multiupload * Enabled file upload through http post request to anidex * Fixed bugs * Modified the behaviour to generate torrent preivously. It is a synchrone function now. So we need to make it asynchronous in download.go Now torrents file are always generated at upload time in a background processus with the correct trackers (needed ones + the ones provided by the user). I made the anidex tracker as a needed one to allow multiupload even if the user hasn't included in his torrent file/magnet url. * Moving deadtrackers list to default config Support torrent file creation when upload without scraping * Fix possible bug where no one is seeding a torrent yet (first time upload) Add primary tracker in torrent file * Fixing issue with boolean pointer for private torrent * fixing index out of range for announcelist filtering Fixing anidex tracker not found * Fixing lang id error * Anidex upload works * Fix for multiupload with magnet * This adds TokyoTosho support. Since TokyoTosho API only support upload by URL (and not torrent file upload). We need to make the download url available without ddos protection. Also TokyoTosho doesn't need a category select. Since it does have limited option, we can automatically convert our categories to the tokyotosho one. * Removing the folder tosho and put categories.go directly in upload package. Added category conversion for nyaasi too. Added tests on category conversion. * This should add nyaasi support. Is not tested though. * Updated defaut config with new nyaa.si behaviour? * Forgot to commit this file
Cette révision appartient à :
Parent
418dc81e19
révision
d5c2c6b676
|
@ -54,6 +54,23 @@ scraper:
|
|||
max_stat_scraping_frequency: 5
|
||||
# MaxStatScrapingFrequencyUnknown : On-demand stat scraping for unknown torrents can only be called every X minutes
|
||||
max_stat_scraping_frequency_unknown: 2
|
||||
# Default config for upload
|
||||
upload:
|
||||
# The following token/accounts are used when an user wants to upload to another website
|
||||
# but doesn't have his own API token.
|
||||
anidex_api_token:
|
||||
# TokyoTosho : Be aware that file upload doesn't work with the host (only url upload)
|
||||
# so you need to enable users to download without ddos protection
|
||||
tokyot_api_token:
|
||||
# Nyaa.Si : we need a username and a password for the account upload, please use a difficult one!
|
||||
nyaasi_username:
|
||||
nyaasi_password:
|
||||
# UploadsDisabled : Disable uploads for everyone except below
|
||||
uploads_disabled: false
|
||||
# AdminsAreStillAllowedTo : Enable admin torrent upload even if UploadsDisabled is true
|
||||
admins_are_still_allowed_to: true
|
||||
# TrustedUsersAreStillAllowedTo : Enable trusted users torrent upload even if UploadsDisabled is true
|
||||
trusted_users_are_still_allowed_to: true
|
||||
# Config by default for the cache
|
||||
cache:
|
||||
dialect: nop
|
||||
|
@ -113,12 +130,6 @@ torrents:
|
|||
storage_link: /download/%s
|
||||
# TorrentCacheLink : Url of torrent site cache
|
||||
cache_link: #http://anicache.com/torrent/%s.torrent
|
||||
# UploadsDisabled : Disable uploads for everyone except below
|
||||
uploads_disabled: false
|
||||
# AdminsAreStillAllowedTo : Enable admin torrent upload even if UploadsDisabled is true
|
||||
admins_are_still_allowed_to: true
|
||||
# TrustedUsersAreStillAllowedTo : Enable trusted users torrent upload even if UploadsDisabled is true
|
||||
trusted_users_are_still_allowed_to: true
|
||||
trackers:
|
||||
# Trackers : Default trackers supported
|
||||
default:
|
||||
|
@ -138,6 +149,22 @@ torrents:
|
|||
# NeededTrackers : Array indexes of Trackers for needed tracker in a torrent file
|
||||
needed:
|
||||
- 0
|
||||
- 12
|
||||
dead:
|
||||
- ://open.nyaatorrents.info:6544
|
||||
- ://tracker.openbittorrent.com:80
|
||||
- ://tracker.publicbt.com:80
|
||||
- ://stats.anisource.net:2710
|
||||
- ://exodus.desync.com
|
||||
- ://open.demonii.com:1337
|
||||
- ://tracker.istole.it:80
|
||||
- ://tracker.ccc.de:80
|
||||
- ://bt2.careland.com.cn:6969
|
||||
- ://announce.torrentsmd.com:8080
|
||||
- ://open.demonii.com:1337
|
||||
- ://tracker.btcake.com
|
||||
- ://tracker.prq.to
|
||||
- ://bt.rghost.net
|
||||
# TorrentOrder : Default sorting field for torrents
|
||||
order: date
|
||||
# TorrentSort : Default sorting order for torrents
|
||||
|
|
|
@ -107,55 +107,57 @@ type I18nConfig struct {
|
|||
|
||||
// ScrapeConfig : Config struct for Scraping
|
||||
type ScrapeConfig struct {
|
||||
URL string `json:"scrape_url" yaml:"url,omitempty"`
|
||||
Name string `json:"name" yaml:"name,omitempty"`
|
||||
IntervalSeconds int64 `json:"interval" yaml:"interval,omitempty"`
|
||||
URL string `json:"scrape_url" yaml:"url,omitempty"`
|
||||
Name string `json:"name" yaml:"name,omitempty"`
|
||||
IntervalSeconds int64 `json:"interval" yaml:"interval,omitempty"`
|
||||
}
|
||||
|
||||
// ScraperConfig : Config struct for Scraper
|
||||
type ScraperConfig struct {
|
||||
Addr string `json:"bind" yaml:"addr,omitempty"`
|
||||
NumWorkers int `json:"workers" yaml:"workers,omitempty"`
|
||||
IntervalSeconds int64 `json:"default_interval" yaml:"default_interval,omitempty"`
|
||||
Trackers []ScrapeConfig `json:"trackers" yaml:"trackers,omitempty"`
|
||||
StatScrapingFrequency float64 `json:"stat_scraping_frequency" yaml:"stat_scraping_frequency,omitempty"`
|
||||
StatScrapingFrequencyUnknown float64 `json:"stat_scraping_frequency_unknown" yaml:"stat_scraping_frequency_unknown,omitempty"`
|
||||
MaxStatScrapingFrequency float64 `json:"max_stat_scraping_frequency" yaml:"max_stat_scraping_frequency,omitempty"`
|
||||
MaxStatScrapingFrequencyUnknown float64 `json:"max_stat_scraping_frequency_unknown" yaml:"max_stat_scraping_frequency_unknown,omitempty"`
|
||||
Addr string `json:"bind" yaml:"addr,omitempty"`
|
||||
NumWorkers int `json:"workers" yaml:"workers,omitempty"`
|
||||
IntervalSeconds int64 `json:"default_interval" yaml:"default_interval,omitempty"`
|
||||
Trackers []ScrapeConfig `json:"trackers" yaml:"trackers,omitempty"`
|
||||
StatScrapingFrequency float64 `json:"stat_scraping_frequency" yaml:"stat_scraping_frequency,omitempty"`
|
||||
StatScrapingFrequencyUnknown float64 `json:"stat_scraping_frequency_unknown" yaml:"stat_scraping_frequency_unknown,omitempty"`
|
||||
MaxStatScrapingFrequency float64 `json:"max_stat_scraping_frequency" yaml:"max_stat_scraping_frequency,omitempty"`
|
||||
MaxStatScrapingFrequencyUnknown float64 `json:"max_stat_scraping_frequency_unknown" yaml:"max_stat_scraping_frequency_unknown,omitempty"`
|
||||
}
|
||||
|
||||
// TrackersConfig ; Config struct for Trackers
|
||||
type TrackersConfig struct {
|
||||
Default ArrayString `yaml:"default,flow,omitempty"`
|
||||
NeededTrackers []int `yaml:"needed,flow,omitempty"`
|
||||
DeadTrackers ArrayString `yaml:"dead,flow,omitempty"`
|
||||
}
|
||||
|
||||
// TorrentsConfig : Config struct for Torrents
|
||||
type TorrentsConfig struct {
|
||||
Status []bool `yaml:"status,omitempty,omitempty"`
|
||||
SukebeiCategories map[string]string `yaml:"sukebei_categories,omitempty"`
|
||||
CleanCategories map[string]string `yaml:"clean_categories,omitempty"`
|
||||
EnglishOnlyCategories ArrayString `yaml:"english_only_categories,omitempty"`
|
||||
NonEnglishOnlyCategories ArrayString `yaml:"non_english_only_categories,omitempty"`
|
||||
AdditionalLanguages ArrayString `yaml:"additional_languages,omitempty"`
|
||||
FileStorage string `yaml:"filestorage,omitempty"`
|
||||
StorageLink string `yaml:"storage_link,omitempty"`
|
||||
CacheLink string `yaml:"cache_link,omitempty"`
|
||||
Trackers TrackersConfig `yaml:"trackers,flow,omitempty"`
|
||||
Order string `yaml:"order,omitempty"`
|
||||
Sort string `yaml:"sort,omitempty"`
|
||||
Tags Tags `yaml:"tags,flow,omitempty"`
|
||||
GenerationClientPort int `yaml:"generation_client_port,flow,omitempty"`
|
||||
Status []bool `yaml:"status,omitempty,omitempty"`
|
||||
SukebeiCategories map[string]string `yaml:"sukebei_categories,omitempty"`
|
||||
CleanCategories map[string]string `yaml:"clean_categories,omitempty"`
|
||||
EnglishOnlyCategories ArrayString `yaml:"english_only_categories,omitempty"`
|
||||
NonEnglishOnlyCategories ArrayString `yaml:"non_english_only_categories,omitempty"`
|
||||
AdditionalLanguages ArrayString `yaml:"additional_languages,omitempty"`
|
||||
FileStorage string `yaml:"filestorage,omitempty"`
|
||||
StorageLink string `yaml:"storage_link,omitempty"`
|
||||
CacheLink string `yaml:"cache_link,omitempty"`
|
||||
Trackers TrackersConfig `yaml:"trackers,flow,omitempty"`
|
||||
Order string `yaml:"order,omitempty"`
|
||||
Sort string `yaml:"sort,omitempty"`
|
||||
Tags Tags `yaml:"tags,flow,omitempty"`
|
||||
GenerationClientPort int `yaml:"generation_client_port,flow,omitempty"`
|
||||
}
|
||||
|
||||
// UploadConfig : Config struct for uploading torrents
|
||||
type UploadConfig struct {
|
||||
DefaultAnidexToken string `yaml:"anidex_api_token,omitempty"`
|
||||
DefaultNyaasiToken string `yaml:"nyaasi_api_token,omitempty"`
|
||||
DefaultTokyoTToken string `yaml:"tokyot_api_token,omitempty"`
|
||||
UploadsDisabled bool `yaml:"uploads_disabled,omitempty"`
|
||||
AdminsAreStillAllowedTo bool `yaml:"admins_are_still_allowed_to,omitempty"`
|
||||
TrustedUsersAreStillAllowedTo bool `yaml:"trusted_users_are_still_allowed_to,omitempty"`
|
||||
DefaultAnidexToken string `yaml:"anidex_api_token,omitempty"`
|
||||
DefaultNyaasiUsername string `yaml:"nyaasi_api_username,omitempty"`
|
||||
DefaultNyaasiPassword string `yaml:"nyaasi_api_password,omitempty"`
|
||||
DefaultTokyoTToken string `yaml:"tokyot_api_token,omitempty"`
|
||||
UploadsDisabled bool `yaml:"uploads_disabled,omitempty"`
|
||||
AdminsAreStillAllowedTo bool `yaml:"admins_are_still_allowed_to,omitempty"`
|
||||
TrustedUsersAreStillAllowedTo bool `yaml:"trusted_users_are_still_allowed_to,omitempty"`
|
||||
}
|
||||
|
||||
// UsersConfig : Config struct for Users
|
||||
|
@ -224,9 +226,9 @@ type ModelsConfig struct {
|
|||
}
|
||||
|
||||
type DefaultThemeConfig struct {
|
||||
Theme string `yaml:"theme,omitempty"`
|
||||
Dark string `yaml:"dark,omitempty"`
|
||||
Forced string `yaml:"forced,omitempty"`
|
||||
Theme string `yaml:"theme,omitempty"`
|
||||
Dark string `yaml:"dark,omitempty"`
|
||||
Forced string `yaml:"forced,omitempty"`
|
||||
}
|
||||
|
||||
// SearchConfig : Config struct for search
|
||||
|
@ -274,3 +276,14 @@ func (ty TagTypes) Get(tagType string) TagType {
|
|||
}
|
||||
return TagType{}
|
||||
}
|
||||
|
||||
// GetDefault returns the first tracker from the needed ones
|
||||
func (tc TrackersConfig) GetDefault() string {
|
||||
if len(tc.NeededTrackers) > 0 {
|
||||
return tc.Default[tc.NeededTrackers[0]]
|
||||
}
|
||||
if len(tc.Default) > 0 {
|
||||
return tc.Default[0]
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
|
|
@ -31,16 +31,16 @@ func DownloadTorrent(c *gin.Context) {
|
|||
templates.Render(c, "errors/torrent_file_missing.jet.html", variables)
|
||||
return
|
||||
}
|
||||
|
||||
|
||||
if c.Query("js_query") != "" {
|
||||
exists := true
|
||||
generating := false
|
||||
|
||||
|
||||
if len(config.Get().Torrents.FileStorage) == 0 {
|
||||
exists = false
|
||||
} else {
|
||||
Openfile, err := os.Open(fmt.Sprintf("%s%c%s.torrent", config.Get().Torrents.FileStorage, os.PathSeparator, hash))
|
||||
defer Openfile.Close()
|
||||
defer Openfile.Close()
|
||||
if err != nil {
|
||||
exists = false
|
||||
generating = true
|
||||
|
@ -52,14 +52,11 @@ func DownloadTorrent(c *gin.Context) {
|
|||
trackers = torrent.GetTrackersArray()
|
||||
}
|
||||
magnet := format.InfoHashToMagnet(strings.TrimSpace(torrent.Hash), torrent.Name, trackers...)
|
||||
if upload.GenerateTorrent(magnet) != nil {
|
||||
//Error during the generation
|
||||
generating = false
|
||||
}
|
||||
go upload.GenerateTorrent(magnet)
|
||||
}
|
||||
}
|
||||
c.JSON(200, gin.H{ // Better to use gin for that, less code
|
||||
"exists": exists,
|
||||
"exists": exists,
|
||||
"generating": generating,
|
||||
})
|
||||
return
|
||||
|
@ -82,7 +79,7 @@ func DownloadTorrent(c *gin.Context) {
|
|||
}
|
||||
|
||||
//Check if file exists and open
|
||||
Openfile, err := os.Open(fmt.Sprintf("%s%c%s.torrent", config.Get().Torrents.FileStorage, os.PathSeparator, hash))
|
||||
Openfile, err := os.Open(torrent.GetPath())
|
||||
if err != nil {
|
||||
//File not found, send 404
|
||||
variables := templates.Commonvariables(c)
|
||||
|
@ -94,9 +91,7 @@ func DownloadTorrent(c *gin.Context) {
|
|||
}
|
||||
magnet := format.InfoHashToMagnet(strings.TrimSpace(torrent.Hash), torrent.Name, trackers...)
|
||||
variables.Set("magnet", magnet)
|
||||
if upload.GenerateTorrent(magnet) != nil {
|
||||
messages.AddError("errors", "Could not generate torrent file")
|
||||
}
|
||||
go upload.GenerateTorrent(magnet)
|
||||
templates.Render(c, "errors/torrent_file_missing.jet.html", variables)
|
||||
return
|
||||
}
|
||||
|
|
|
@ -4,4 +4,5 @@ import "github.com/NyaaPantsu/nyaa/controllers/router"
|
|||
|
||||
func init() {
|
||||
router.Get().Any("/upload", UploadHandler)
|
||||
router.Get().Any("/upload/status/:id", multiUploadStatus)
|
||||
}
|
||||
|
|
|
@ -2,6 +2,7 @@ package uploadController
|
|||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
|
||||
|
@ -9,13 +10,14 @@ import (
|
|||
"github.com/NyaaPantsu/nyaa/models"
|
||||
"github.com/NyaaPantsu/nyaa/models/torrents"
|
||||
"github.com/NyaaPantsu/nyaa/templates"
|
||||
"github.com/NyaaPantsu/nyaa/utils/cache"
|
||||
"github.com/NyaaPantsu/nyaa/utils/captcha"
|
||||
"github.com/NyaaPantsu/nyaa/utils/log"
|
||||
msg "github.com/NyaaPantsu/nyaa/utils/messages"
|
||||
"github.com/NyaaPantsu/nyaa/utils/publicSettings"
|
||||
"github.com/NyaaPantsu/nyaa/utils/upload"
|
||||
"github.com/NyaaPantsu/nyaa/utils/validator/torrent"
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/NyaaPantsu/nyaa/utils/log"
|
||||
)
|
||||
|
||||
// UploadHandler : Main Controller for uploading a torrent
|
||||
|
@ -67,10 +69,53 @@ func UploadPostHandler(c *gin.Context) {
|
|||
messages.AddError("errors", err.Error())
|
||||
}
|
||||
|
||||
AnidexUpload := false
|
||||
NyaaSiUpload := false
|
||||
TokyoToshoUpload := false
|
||||
|
||||
if c.PostForm("anidex_api") != "" || c.PostForm("anidex_upload") == "true" {
|
||||
AnidexUpload = true
|
||||
}
|
||||
if c.PostForm("nyaasi_api") != "" || c.PostForm("nyaasi_upload") == "true" {
|
||||
NyaaSiUpload = true
|
||||
}
|
||||
if c.PostForm("tokyot_api") != "" || c.PostForm("tokyot_upload") == "true" {
|
||||
TokyoToshoUpload = true
|
||||
}
|
||||
|
||||
if !messages.HasErrors() {
|
||||
// add to db and redirect
|
||||
// add to db
|
||||
torrent, err := torrents.Create(user, &uploadForm)
|
||||
log.CheckErrorWithMessage(err, "ERROR_TORRENT_CREATED: Error while creating entry in db")
|
||||
|
||||
if AnidexUpload || NyaaSiUpload || TokyoToshoUpload {
|
||||
go func(anidexApiKey string, anidexFormCategory string, anidexFormLang string, nyaasiUsername string, nyaasiPassword string, toshoApiKey string) {
|
||||
err := upload.GotFile(torrent)
|
||||
if err != nil {
|
||||
log.CheckError(err)
|
||||
return
|
||||
}
|
||||
// User wants to upload to other websites too
|
||||
if AnidexUpload {
|
||||
go upload.ToAnidex(torrent, anidexApiKey, anidexFormCategory, anidexFormLang)
|
||||
}
|
||||
|
||||
if NyaaSiUpload {
|
||||
go upload.ToNyaasi(nyaasiUsername, nyaasiPassword, torrent)
|
||||
}
|
||||
|
||||
if TokyoToshoUpload {
|
||||
go upload.ToTTosho(toshoApiKey, torrent)
|
||||
}
|
||||
}(c.PostForm("anidex_api"), c.PostForm("anidex_form_category"), c.PostForm("anidex_form_lang"), c.PostForm("nyaasi_username"), c.PostForm("nyaasi_password"), c.PostForm("tokyot_api"))
|
||||
// After that, we redirect to the page for upload status
|
||||
url := fmt.Sprintf("/upload/status/%d", torrent.ID)
|
||||
c.Redirect(302, url)
|
||||
return
|
||||
}
|
||||
|
||||
// We don't need it to be synchronous since the generation can be left in a background process
|
||||
go upload.GotFile(torrent)
|
||||
url := "/view/" + strconv.FormatUint(uint64(torrent.ID), 10)
|
||||
c.Redirect(302, url+"?success")
|
||||
}
|
||||
|
@ -88,3 +133,27 @@ func UploadGetHandler(c *gin.Context) {
|
|||
}
|
||||
templates.Form(c, "site/torrents/upload.jet.html", uploadForm)
|
||||
}
|
||||
|
||||
// multiUploadStatus : controller to show the multi upload status
|
||||
func multiUploadStatus(c *gin.Context) {
|
||||
id := c.Param("id")
|
||||
if id == "" {
|
||||
c.AbortWithStatus(http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
if found, ok := cache.C.Get("tstatus_" + id); ok {
|
||||
uploadMultiple := found.(upload.MultipleForm)
|
||||
// if ?json we print the json format
|
||||
if _, ok = c.GetQuery("json"); ok {
|
||||
c.JSON(http.StatusOK, uploadMultiple)
|
||||
return
|
||||
}
|
||||
// else we send the upload multiple form (support of manual F5)
|
||||
variables := templates.Commonvariables(c)
|
||||
variables.Set("UploadMultiple", uploadMultiple)
|
||||
templates.Render(c, "site/torrents/upload_multiple.jet.html", variables)
|
||||
} else {
|
||||
// here it means the upload status is already flushed from memory
|
||||
c.AbortWithStatus(http.StatusNotFound)
|
||||
}
|
||||
}
|
||||
|
|
|
@ -5,6 +5,7 @@ import (
|
|||
"errors"
|
||||
"fmt"
|
||||
"html/template"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strconv"
|
||||
|
@ -165,11 +166,22 @@ func (t *Torrent) IsDeleted() bool {
|
|||
return t.DeletedAt != nil
|
||||
}
|
||||
|
||||
// IsAnon : Return if a torrent is displayed as anon
|
||||
// Be aware, it doesn't mean that the owner is anonymous!
|
||||
func (t *Torrent) IsAnon() bool {
|
||||
return t.Hidden || t.UploaderID == 0
|
||||
}
|
||||
|
||||
// GetDescriptiveTags : Return the descriptive tags
|
||||
func (t *Torrent) GetDescriptiveTags() string {
|
||||
return t.AcceptedTags
|
||||
}
|
||||
|
||||
// GetPath : Helpers to get the path to the torrent file
|
||||
func (t *Torrent) GetPath() string {
|
||||
return fmt.Sprintf("%s%c%s.torrent", config.Get().Torrents.FileStorage, os.PathSeparator, t.Hash)
|
||||
}
|
||||
|
||||
// AddToESIndex : Adds a torrent to Elastic Search
|
||||
func (t Torrent) AddToESIndex(client *elastic.Client) error {
|
||||
ctx := context.Background()
|
||||
|
@ -223,7 +235,7 @@ func (t *Torrent) ParseTrackers(trackers []string) {
|
|||
}
|
||||
}
|
||||
trackers = tempTrackers
|
||||
|
||||
|
||||
v["tr"] = trackers
|
||||
t.Trackers = v.Encode()
|
||||
}
|
||||
|
@ -348,23 +360,14 @@ func (t *Torrent) ToJSON() TorrentJSON {
|
|||
} else if t.OldUploader != "" {
|
||||
uploader = t.OldUploader
|
||||
}
|
||||
torrentlink := ""
|
||||
if len(config.Get().Torrents.CacheLink) > 0 { // Only use torrent cache if set, don't check id since better to have all .torrent
|
||||
if config.IsSukebei() {
|
||||
torrentlink = "" // torrent cache doesn't have sukebei torrents
|
||||
} else {
|
||||
torrentlink = fmt.Sprintf(config.Get().Torrents.CacheLink, t.Hash)
|
||||
}
|
||||
} else if len(config.Get().Torrents.StorageLink) > 0 { // Only use own .torrent if storage set
|
||||
torrentlink = fmt.Sprintf(config.Get().Torrents.StorageLink, t.Hash)
|
||||
}
|
||||
|
||||
scrape := Scrape{}
|
||||
if t.Scrape != nil {
|
||||
scrape = *t.Scrape
|
||||
}
|
||||
|
||||
|
||||
statsObsolete := []bool{false, false}
|
||||
|
||||
|
||||
if scrape.LastScrape.IsZero() || (scrape.Seeders == 0 && scrape.Leechers == 0 && scrape.Completed == 0) {
|
||||
statsObsolete[0] = true
|
||||
//The displayed stats are obsolete, S/D/L will show "Unknown"
|
||||
|
@ -373,7 +376,7 @@ func (t *Torrent) ToJSON() TorrentJSON {
|
|||
statsObsolete[1] = true
|
||||
//The stats need to be refreshed, either because they are valid and older than one month (not that reliable) OR if they are unknown but have been scraped 1h (or more) ago
|
||||
}
|
||||
|
||||
|
||||
t.ParseLanguages()
|
||||
res := TorrentJSON{
|
||||
ID: t.ID,
|
||||
|
@ -393,7 +396,7 @@ func (t *Torrent) ToJSON() TorrentJSON {
|
|||
WebsiteLink: sanitize.Safe(t.WebsiteLink),
|
||||
Languages: t.Languages,
|
||||
Magnet: template.URL(magnet),
|
||||
TorrentLink: sanitize.Safe(torrentlink),
|
||||
TorrentLink: sanitize.Safe(t.Download()),
|
||||
Leechers: scrape.Leechers,
|
||||
Seeders: scrape.Seeders,
|
||||
Completed: scrape.Completed,
|
||||
|
@ -520,11 +523,25 @@ func (t *Torrent) DeleteTags() {
|
|||
}
|
||||
}
|
||||
|
||||
func contains(s []string, e string) bool {
|
||||
for _, a := range s {
|
||||
if a == e {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
// Download generate a download link for a torrent
|
||||
func (t *Torrent) Download() (torrentlink string) {
|
||||
if len(config.Get().Torrents.CacheLink) > 0 { // Only use torrent cache if set, don't check id since better to have all .torrent
|
||||
if !config.IsSukebei() { // torrent cache doesn't have sukebei torrents
|
||||
torrentlink = fmt.Sprintf(config.Get().Torrents.CacheLink, t.Hash)
|
||||
}
|
||||
return
|
||||
}
|
||||
if len(config.Get().Torrents.StorageLink) > 0 { // Only use own .torrent if storage set
|
||||
torrentlink = fmt.Sprintf(config.Get().Torrents.StorageLink, t.Hash)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
func contains(s []string, e string) bool {
|
||||
for _, a := range s {
|
||||
if a == e {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
|
|
@ -70,6 +70,7 @@ a:hover, .h-nav a:hover, .h-nav:active a:hover {
|
|||
.upload-form-table .checkbox-container+input {
|
||||
width: 385px;1
|
||||
}
|
||||
|
||||
.upload-form-table .table-checkboxes {
|
||||
padding: 3px 0!important;
|
||||
width: 100%!important;
|
||||
|
@ -677,7 +678,7 @@ span.tag {
|
|||
}
|
||||
|
||||
.upload-form-table {
|
||||
width: auto;
|
||||
width: 542px;
|
||||
}
|
||||
.upload-form-table .table-input-label label::after {
|
||||
content: ':';
|
||||
|
|
|
@ -334,7 +334,11 @@ span.tag {
|
|||
|
||||
.upload-form-table .checkbox-container {
|
||||
border-color: #0c0d0e;
|
||||
background: #333438;
|
||||
background: #28282b;
|
||||
}
|
||||
|
||||
#anidex-upload-info > div input[type="text"], #anidex-upload-info > div select {
|
||||
background-color: #28282b;
|
||||
}
|
||||
|
||||
.torrent-info-row .tr-se span, .torrent-info-row .tr-le span, .torrent-info-row .tr-dl span {
|
||||
|
|
|
@ -1,8 +1,8 @@
|
|||
{{ extends "layouts/index_admin" }}
|
||||
{{block title()}}{{ T("moderation_guidelines") }}{{end}}
|
||||
{{ block content_body()}}
|
||||
<div class="results box">
|
||||
<h1>{{ T("moderation_guidelines") }}</h1>
|
||||
<img src="https://www.themarysue.com/wp-content/uploads/2016/09/harassment-guide.png">
|
||||
</div>
|
||||
{{end}}
|
||||
{{ extends "layouts/index_admin" }}
|
||||
{{block title()}}{{ T("moderation_guidelines") }}{{end}}
|
||||
{{ block content_body()}}
|
||||
<div class="results box">
|
||||
<h1>{{ T("moderation_guidelines") }}</h1>
|
||||
<img src="https://www.themarysue.com/wp-content/uploads/2016/09/harassment-guide.png">
|
||||
</div>
|
||||
{{end}}
|
||||
|
|
|
@ -66,6 +66,90 @@
|
|||
<td class="table-input-label"><label for="desc">{{ T("torrent_description")}}</label></td>
|
||||
<td class="table-input markdown-container"><textarea name="desc" id="desc" class="form-input up-input" style="height: 10rem;">{{Form.Description}}</textarea></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td class="table-input-label">{{ T("upload_to") }} Anidex, Nyaa.si, TokyoTosho:</td>
|
||||
<td class="table-input">
|
||||
<details id="anidex-upload-info">
|
||||
<summary class="form-input">{{ T("upload_to_other") }}</summary>
|
||||
<div class="form-input">
|
||||
<p>{{ T("upload_to") }} <b>Anidex</b>:</p>
|
||||
<div>
|
||||
<div class="checkbox-container"><input type="checkbox" value="true" name="anidex_upload" id="anidex_upload" class="form-torrent" {{if User.AnidexAPIToken != ""}}checked{{end}} /></div>
|
||||
<input name="anidex_api" id="anidex_api" placeholder="Anidex API token" class="form-input" type="text" value="{{User.AnidexAPIToken}}" {{if User.AnidexAPIToken != ""}}disabled{{end}}/>
|
||||
<br/>
|
||||
<select class="form-input" name="anidex_form_category">
|
||||
<option>{{ T("choose_category") }}</option>
|
||||
<option value="1" style="background-color: #9f9;">Anime - Sub</option>
|
||||
<option value="2" style="background-color: #9c9;">Anime - Raw</option>
|
||||
<option value="3" style="background-color: #3f3;">Anime - Dub</option>
|
||||
<option value="4" style="background-color: #9ff;">LA - Sub</option>
|
||||
<option value="5" style="background-color: #9cc;">LA - Raw</option>
|
||||
<option value="6" style="background-color: #ff6;">Light Novel</option>
|
||||
<option value="7" style="background-color: #f9f;">Manga - Translated</option>
|
||||
<option value="8" style="background-color: #c9c;">Manga - Raw</option>
|
||||
<option value="9" style="background-color: #f99;">Audio - Lossy</option>
|
||||
<option value="10" style="background-color: #c99;">Audio - Lossless</option>
|
||||
<option value="11" style="background-color: #f66;">Video</option>
|
||||
<option value="12" style="background-color: #ff9;">Games</option>
|
||||
<option value="13" style="background-color: #cc9;">Applications</option>
|
||||
<option value="14" style="background-color: #99c;">Pictures</option>
|
||||
{{if Sukebei()}}<option value="15" style="background-color: #99f;">Adult Video</option>{{end}}
|
||||
<option value="16" style="background-color: #999;">Other</option>
|
||||
</select>
|
||||
<select class="form-input" name="anidex_form_lang">
|
||||
<option>{{ T("choose_language") }}</option>
|
||||
<option value="1">English</option>
|
||||
<option value="2">Japanese</option>
|
||||
<option value="3">Polish</option>
|
||||
<option value="4">Serbo-Croatian</option>
|
||||
<option value="5">Dutch</option>
|
||||
<option value="6">Italian</option>
|
||||
<option value="7">Russian</option>
|
||||
<option value="8">German</option>
|
||||
<option value="9">Hungarian</option>
|
||||
<option value="10">French</option>
|
||||
<option value="11">Finnish</option>
|
||||
<option value="12">Vietnamese</option>
|
||||
<option value="13">Greek</option>
|
||||
<option value="14">Bulgarian</option>
|
||||
<option value="15">Spanish</option>
|
||||
<option value="16">Portuguese (Brasil)</option>
|
||||
<option value="17">Portuguese (Portugal)</option>
|
||||
<option value="18">Swedish</option>
|
||||
<option value="19">Arabic</option>
|
||||
<option value="20">Danish</option>
|
||||
<option value="21">Chinese (Simplified)</option>
|
||||
<option value="22">Bengali</option>
|
||||
<option value="23">Romanian</option>
|
||||
<option value="24">Czech</option>
|
||||
<option value="25">Mongolian</option>
|
||||
<option value="26">Turkish</option>
|
||||
<option value="27">Indonesian</option>
|
||||
<option value="28">Korean</option>
|
||||
<option value="29">Spanish (LATAM)</option>
|
||||
<option value="30">Persian</option>
|
||||
<option value="31">Malaysian</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<p>{{ T("upload_to") }} <b>Nyaa.si</b>:</p>
|
||||
<div>
|
||||
<div class="checkbox-container"><input type="checkbox" value="true" name="nyaasi_upload" id="nyaasi_upload" class="form-torrent"/></div>
|
||||
<input name="nyaasi_username" id="nyaasi_username" placeholder="Nyaa.si Username" class="form-input up-input" type="text" value="" />
|
||||
<br/>
|
||||
<input name="nyaasi_password" id="nyaasi_password" placeholder="Nyaa.si Password" class="form-input up-input" type="password" value="" />
|
||||
</div>
|
||||
|
||||
<p>{{ T("upload_to") }} <b>TokyoTosho</b>:</p>
|
||||
<div>
|
||||
<div class="checkbox-container"><input type="checkbox" value="true" name="tokyot_upload" id="tokyot_upload" {{if User.TokyoTAPIToken != ""}}checked{{end}} class="form-torrent"/></div>
|
||||
<input name="tokyot_api" id="tokyot_api" placeholder="TokyoTosho API token" class="form-input up-input" type="text" value="{{User.TokyoTAPIToken }}" {{if User.AnidexAPIToken != ""}}disabled{{end}}/>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</details>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td class="table-input-label">{{ T("torrent_tags")}}:</td>
|
||||
<td class="table-input">
|
||||
|
|
|
@ -2,20 +2,30 @@
|
|||
{{block title()}}{{ T("upload")}}{{end}}
|
||||
{{block content_body()}}
|
||||
<div style="text-align: left;" class="box">
|
||||
<h1 style="text-align:center;margin-top; 4px;">Upload status</h1>
|
||||
<h1 style="text-align:center;margin-top: 4px;">{{ T("upload_status") }}</h1>
|
||||
<table class="multiple-upload">
|
||||
<tbody>
|
||||
<tr><td colspan="4"><h3>Nyaa Pantsu upload status</h3></td></tr>
|
||||
<tr class="upload-status"><td class="upload-site-name">Nyaa Pantsu</td><td>Status:</td><td class="icon-finished finished"></td><td class="upload-progress finished">Finished</td></tr>
|
||||
<tr><td colspan="4" class="uploaded-url"><input type="text" class="form-input" placeholder="http://" value="http://nyaa.pantsu.cat/view/4556456" disabled></td></tr>
|
||||
<tr><td colspan="4"><h3>Nyaa Pantsu {{ T("upload_status") }}</h3></td></tr>
|
||||
<tr class="upload-status"><td class="upload-site-name">Nyaa Pantsu</td><td>{{ T("status") }}:</td><td class="icon-finished finished"></td><td class="upload-progress finished">{{ T("finished") }}</td></tr>
|
||||
<tr><td colspan="4" class="uploaded-url"><a href="/view/{{UploadMultiple.PantsuID}}" target="_blank"><input type="text" class="form-input" value="{{if Sukebei()}}{{Config.WebAddress.Sukebei}}{{else}}{{Config.WebAddress.Nyaa}}{{end}}/view/{{UploadMultiple.PantsuID}}" disabled></a></td></tr>
|
||||
|
||||
<tr><td colspan="4"><h3>Nyaa.si upload status</h3></td></tr>
|
||||
<tr class="upload-status"><td class="upload-site-name">Nyaa.si</td><td>Status:</td><td class="icon-pending pending"></td><td class="upload-progress pending">Pending</td></tr>
|
||||
<tr><td colspan="4" class="uploaded-url"><input type="text" class="form-input" placeholder="Uploading..." disabled></td></tr>
|
||||
{{ if UploadMultiple.Nyaasi.Status != 0 }}
|
||||
<tr><td colspan="4"><h3>Nyaa.si {{ T("upload_status") }}</h3></td></tr>
|
||||
<tr class="upload-status nyaasi"><td class="upload-site-name">Nyaa.si</td><td>{{ T("status") }}:</td><td class="statusicon icon-{{ if UploadMultiple.Nyaasi.Status == 1}}pending pending{{else if UploadMultiple.Nyaasi.Status == 2}}error error{{else}}finished finished{{end}}"></td><td class="upload-progress {{ if UploadMultiple.Nyaasi.Status == 1}}pending{{else if UploadMultiple.Nyaasi.Status == 2}}error{{else}}finished{{end}}">{{ if UploadMultiple.Nyaasi.Status == 1}}{{ T("pending") }}{{else if UploadMultiple.Nyaasi.Status == 2}}{{ T("error") }}{{else}}{{ T("finished") }}{{end}}</td></tr>
|
||||
<tr><td colspan="4" class="uploaded-url"><input type="text"id="nyaasiMess" class="form-input" value="{{UploadMultiple.Nyaasi.Message}}" disabled></td></tr>
|
||||
{{end}}
|
||||
|
||||
<tr><td colspan="4"><h3>Anidex upload status</h3></td></tr>
|
||||
<tr class="upload-status"><td class="upload-site-name">Anidex</td><td>Status:</td><td class="icon-error error"></td><td class="upload-progress error">Error</td></tr>
|
||||
<tr><td colspan="4" class="uploaded-url"><input type="text" class="form-input" placeholder="http://" value="Error: XXXX" disabled></td></tr>
|
||||
{{ if UploadMultiple.Anidex.Status != 0 }}
|
||||
<tr><td colspan="4"><h3>Anidex {{ T("upload_status") }}</h3></td></tr>
|
||||
<tr class="upload-status anidex"><td class="upload-site-name">Anidex</td><td>{{ T("status") }}:</td><td class="statusicon icon-{{ if UploadMultiple.Anidex.Status == 1}}pending pending{{else if UploadMultiple.Anidex.Status == 2}}error error{{else}}finished finished{{end}}"></td><td class="upload-progress {{ if UploadMultiple.Anidex.Status == 1}}pending{{else if UploadMultiple.Anidex.Status == 2}}error{{else}}finished{{end}}">{{ if UploadMultiple.Anidex.Status == 1}}{{ T("pending") }}{{else if UploadMultiple.Anidex.Status == 2}}{{ T("error") }}{{else}}{{ T("finished") }}{{end}}</td></tr>
|
||||
<tr><td colspan="4" class="uploaded-url"><input type="text" id="anidexMess" class="form-input" value="{{UploadMultiple.Anidex.Message}}" disabled></td></tr>
|
||||
{{end}}
|
||||
|
||||
{{ if UploadMultiple.TTosho.Status != 0 }}
|
||||
<tr><td colspan="4"><h3>Tokyo Tosho {{ T("upload_status") }}</h3></td></tr>
|
||||
<tr class="upload-status ttosho"><td class="upload-site-name">Tokyo Tosho</td><td>{{ T("status") }}:</td><td class="statusicon icon-{{ if UploadMultiple.TTosho.Status == 1}}pending pending{{else if UploadMultiple.TTosho.Status == 2}}error error{{else}}finished finished{{end}}"></td><td class="upload-progress {{ if UploadMultiple.TTosho.Status == 1}}pending{{else if UploadMultiple.TTosho.Status == 2}}error{{else}}finished{{end}}">{{ if UploadMultiple.TTosho.Status == 1}}{{ T("pending") }}{{else if UploadMultiple.TTosho.Status == 2}}{{ T("error") }}{{else}}{{ T("finished") }}{{end}}</td></tr>
|
||||
<tr><td colspan="4" class="uploaded-url"><input type="text" id="ttoshoMess" class="form-input" value="{{UploadMultiple.TTosho.Message}}" disabled></td></tr>
|
||||
{{end}}
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
@ -24,4 +34,38 @@
|
|||
{{ block footer_js()}}
|
||||
<script type="text/javascript" src="/js/translation.js?v={{ Config.Version}}{{ Config.Build }}"></script>
|
||||
<script type="text/javascript" src="/js/upload.js?v={{ Config.Version}}{{ Config.Build }}"></script>
|
||||
<script type="text/javascript">
|
||||
var multiUploadStatus = ["", "pending", "error", "finished"]
|
||||
T.Add("pending", "{{ T("pending") }}")
|
||||
T.Add("error", "{{ T("error") }}")
|
||||
T.Add("finished", "{{ T("finished") }}")
|
||||
function checkStatus() {
|
||||
Query.Get(window.location.href+"?json", function(data) {
|
||||
count = 3
|
||||
if (data.anidex.status > 0) {
|
||||
document.getElementById("anidexMess").value = data.anidex.message
|
||||
document.querySelector("tr.upload-status.anidex .statusicon").className = "statusicon icon-"+multiUploadStatus[data.anidex.status]+" "+multiUploadStatus[data.anidex.status]
|
||||
document.querySelector("tr.upload-status.anidex .upload-progress").className = "upload-progress "+multiUploadStatus[data.anidex.status]
|
||||
document.querySelector("tr.upload-status.anidex .upload-progress").textContent = T.r(multiUploadStatus[data.anidex.status])
|
||||
}
|
||||
if (data.anidex.status != 1) count--
|
||||
if (data.nyaasi.status > 0) {
|
||||
document.getElementById("nyaasiMess").value = data.nyaasi.message
|
||||
document.querySelector("tr.upload-status.nyaasi .statusicon").className = "statusicon icon-"+multiUploadStatus[data.nyaasi.status]+" "+multiUploadStatus[data.nyaasi.status]
|
||||
document.querySelector("tr.upload-status.nyaasi .upload-progress").className = "upload-progress "+multiUploadStatus[data.nyaasi.status]
|
||||
document.querySelector("tr.upload-status.nyaasi .upload-progress").textContent = T.r(multiUploadStatus[data.anidex.status])
|
||||
}
|
||||
if (data.anidex.status != 1) count--
|
||||
if (data.ttosho.status > 0) {
|
||||
document.getElementById("ttoshoMess").value = data.ttosho.message
|
||||
document.querySelector("tr.upload-status.nyaasi .statusicon").className = "statusicon icon-"+multiUploadStatus[data.ttosho.status]+" "+multiUploadStatus[data.ttosho.status]
|
||||
document.querySelector("tr.upload-status.nyaasi .upload-progress").className = "upload-progress "+multiUploadStatus[data.ttosho.status]
|
||||
document.querySelector("tr.upload-status.nyaasi .upload-progress").textContent = T.r(multiUploadStatus[data.anidex.status])
|
||||
}
|
||||
if (data.anidex.status != 1) count--
|
||||
if (count > 0) setTimeout(checkStatus, 5000)
|
||||
})
|
||||
}
|
||||
checkStatus()
|
||||
</script>
|
||||
{{end}}
|
||||
|
|
|
@ -9,6 +9,7 @@ import (
|
|||
"path"
|
||||
"testing"
|
||||
|
||||
"github.com/NyaaPantsu/nyaa/utils/upload"
|
||||
"github.com/NyaaPantsu/nyaa/utils/validator/announcement"
|
||||
|
||||
"strings"
|
||||
|
@ -208,6 +209,10 @@ func walkDirTest(dir string, t *testing.T) {
|
|||
variables.Set("Form", models.Tags{*fakeTag, *fakeTag, *fakeTag})
|
||||
return variables
|
||||
},
|
||||
"upload_multiple.jet.html": func(variables jet.VarMap) jet.VarMap {
|
||||
variables.Set("UploadMultiple", upload.MultipleForm{})
|
||||
return variables
|
||||
},
|
||||
}
|
||||
|
||||
fmt.Printf("\nTesting Folder: %s\n", dir)
|
||||
|
|
|
@ -77,6 +77,16 @@
|
|||
* + read_notifications_cleared
|
||||
* + notifications_read
|
||||
* + mark_notifications_as_read
|
||||
## 2017/10/30
|
||||
* + status
|
||||
* + upload_status
|
||||
* + error
|
||||
* + pending
|
||||
* + finished
|
||||
* + upload_to
|
||||
* + upload_to_other
|
||||
* + choose_category
|
||||
* + choose_language
|
||||
## 2017/10/31
|
||||
* + followers
|
||||
* + comment_markdown_notice
|
||||
|
|
|
@ -2267,6 +2267,42 @@
|
|||
"id": "status",
|
||||
"translation": "Status"
|
||||
},
|
||||
{
|
||||
"id": "upload_status",
|
||||
"translation": "Upload status"
|
||||
},
|
||||
{
|
||||
"id": "pending",
|
||||
"translation": "Pending"
|
||||
},
|
||||
{
|
||||
"id": "error",
|
||||
"translation": "Error"
|
||||
},
|
||||
{
|
||||
"id": "finished",
|
||||
"translation": "Finished"
|
||||
},
|
||||
{
|
||||
"id": "upload_to",
|
||||
"translation": "Upload to"
|
||||
},
|
||||
{
|
||||
"id": "upload_to_other",
|
||||
"translation": "Upload to other websites"
|
||||
},
|
||||
{
|
||||
"id": "upload_to_other",
|
||||
"translation": "Upload to other websites"
|
||||
},
|
||||
{
|
||||
"id":"choose_category",
|
||||
"translation": "Choose a category (required)"
|
||||
},
|
||||
{
|
||||
"id":"choose_language",
|
||||
"translation":"Choose a language (required)"
|
||||
},
|
||||
{
|
||||
"id": "event",
|
||||
"translation": "Event"
|
||||
|
|
Fichier diff supprimé car celui-ci est trop grand
Voir la Diff
|
@ -0,0 +1,95 @@
|
|||
package upload
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/NyaaPantsu/nyaa/config"
|
||||
"github.com/NyaaPantsu/nyaa/models"
|
||||
)
|
||||
|
||||
// Convert automatically our sukebei cats to platform specified Hentai cats
|
||||
var sukebeiCategories = []map[string]string{
|
||||
ttosho: {
|
||||
"1_1": "12",
|
||||
"1_2": "12",
|
||||
"1_3": "14",
|
||||
"1_4": "13",
|
||||
"1_5": "4",
|
||||
"2_1": "4",
|
||||
"2_2": "15",
|
||||
},
|
||||
nyaasi: {
|
||||
"1_1": "1_1",
|
||||
"1_2": "1_2",
|
||||
"1_3": "1_3",
|
||||
"1_4": "1_4",
|
||||
"1_5": "1_5",
|
||||
"2_1": "2_1",
|
||||
"2_2": "2_2",
|
||||
},
|
||||
}
|
||||
|
||||
var normalCategories = []map[string]string{
|
||||
ttosho: {
|
||||
"3_12": "1",
|
||||
"3_5": "1",
|
||||
"3_13": "10",
|
||||
"3_6": "7",
|
||||
"2_3": "2",
|
||||
"2_4": "2",
|
||||
"4_7": "3",
|
||||
"4_8": "7",
|
||||
"4_14": "10",
|
||||
"5_9": "8",
|
||||
"5_10": "8",
|
||||
"5_18": "10",
|
||||
"5_11": "7",
|
||||
"6_15": "5",
|
||||
"6_16": "5",
|
||||
"1_1": "5",
|
||||
"1_2": "5",
|
||||
},
|
||||
nyaasi: {
|
||||
"3_12": "1_1",
|
||||
"3_5": "1_2",
|
||||
"3_13": "1_3",
|
||||
"3_6": "1_4",
|
||||
"2_3": "2_1",
|
||||
"2_4": "2_2",
|
||||
"4_7": "3_1",
|
||||
"4_8": "3_4",
|
||||
"4_14": "3_3",
|
||||
"5_9": "4_1",
|
||||
"5_10": "4_2",
|
||||
"5_18": "4_3",
|
||||
"5_11": "4_4",
|
||||
"6_15": "5_1",
|
||||
"6_16": "5_2",
|
||||
"1_1": "6_1",
|
||||
"1_2": "6_2",
|
||||
},
|
||||
}
|
||||
|
||||
// Category returns the category converted from nyaa one to tosho one
|
||||
func Category(platform int, t *models.Torrent) string {
|
||||
cat := fmt.Sprintf("%d_%d", t.Category, t.SubCategory)
|
||||
// if we are in sukebei, there are some categories
|
||||
if config.IsSukebei() {
|
||||
// check that platform exist in our map for sukebei categories
|
||||
if platform < len(sukebeiCategories) {
|
||||
// return the remaped category if it exists
|
||||
if val, ok := sukebeiCategories[platform][cat]; ok {
|
||||
return val
|
||||
}
|
||||
}
|
||||
}
|
||||
// check that platform exist in our map
|
||||
if platform >= len(normalCategories) {
|
||||
return ""
|
||||
}
|
||||
// return the remaped category if it exists
|
||||
if val, ok := normalCategories[platform][cat]; ok {
|
||||
return val
|
||||
}
|
||||
return ""
|
||||
}
|
|
@ -0,0 +1,38 @@
|
|||
package upload
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/NyaaPantsu/nyaa/config"
|
||||
"github.com/NyaaPantsu/nyaa/models"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestCategory(t *testing.T) {
|
||||
assert := assert.New(t)
|
||||
dummyTorrent := &models.Torrent{Category: 1, SubCategory: 1}
|
||||
tests := []struct {
|
||||
torrent *models.Torrent
|
||||
platform int
|
||||
sukebei bool
|
||||
expected string
|
||||
}{
|
||||
{dummyTorrent, anidex, false, ""},
|
||||
{dummyTorrent, ttosho, false, "5"},
|
||||
{dummyTorrent, ttosho, true, "12"},
|
||||
{dummyTorrent, nyaasi, false, "6_1"},
|
||||
{dummyTorrent, nyaasi, true, "1_1"},
|
||||
{dummyTorrent, 20, true, ""},
|
||||
{&models.Torrent{Category: 33, SubCategory: 33}, nyaasi, true, ""},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
if test.sukebei {
|
||||
// workaround to make the function believe we are in sukebei
|
||||
config.Get().Models.TorrentsTableName = "sukebei_torrents"
|
||||
} else {
|
||||
config.Get().Models.TorrentsTableName = "torrents"
|
||||
}
|
||||
assert.Equal(test.expected, Category(test.platform, test.torrent))
|
||||
}
|
||||
}
|
|
@ -0,0 +1,6 @@
|
|||
package upload
|
||||
|
||||
import "errors"
|
||||
|
||||
var errPending = errors.New("Magnet being generated already")
|
||||
var errConfig = errors.New("Magnet Empty or FileStorage not configured")
|
|
@ -1,12 +1,15 @@
|
|||
package upload
|
||||
|
||||
import (
|
||||
"strconv"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/NyaaPantsu/nyaa/config"
|
||||
"github.com/NyaaPantsu/nyaa/models"
|
||||
"github.com/NyaaPantsu/nyaa/utils/format"
|
||||
"github.com/NyaaPantsu/nyaa/utils/log"
|
||||
"github.com/anacrolix/dht"
|
||||
"github.com/anacrolix/torrent"
|
||||
|
@ -41,12 +44,12 @@ func GenerateTorrent(magnet string) error {
|
|||
}
|
||||
}
|
||||
if magnet == "" || len(config.Get().Torrents.FileStorage) == 0 {
|
||||
return errors.New("Magnet Empty or FileStorage not configured")
|
||||
return errConfig
|
||||
}
|
||||
if len(queue) > 0 {
|
||||
for _, m := range queue {
|
||||
if m == magnet {
|
||||
return errors.New("Magnet being generated already")
|
||||
return errPending
|
||||
}
|
||||
}
|
||||
}
|
||||
|
@ -57,28 +60,63 @@ func GenerateTorrent(magnet string) error {
|
|||
log.Errorf("error adding magnet to client: %s", err)
|
||||
return err
|
||||
}
|
||||
go func() {
|
||||
<-t.GotInfo()
|
||||
mi := t.Metainfo()
|
||||
t.Drop()
|
||||
file := fmt.Sprintf("%s%c%s.torrent", config.Get().Torrents.FileStorage, os.PathSeparator, t.InfoHash().String())
|
||||
f, err := os.Create(file)
|
||||
if err != nil {
|
||||
log.Errorf("error creating torrent metainfo file: %s", err)
|
||||
return
|
||||
<-t.GotInfo()
|
||||
mi := t.Metainfo()
|
||||
t.Drop()
|
||||
file := fmt.Sprintf("%s%c%s.torrent", config.Get().Torrents.FileStorage, os.PathSeparator, t.InfoHash().String())
|
||||
f, err := os.Create(file)
|
||||
if err != nil {
|
||||
log.Errorf("error creating torrent metainfo file: %s", err)
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
defaultTracker := config.Get().Torrents.Trackers.GetDefault()
|
||||
if defaultTracker != "" {
|
||||
mi.Announce = defaultTracker
|
||||
}
|
||||
err = bencode.NewEncoder(f).Encode(mi)
|
||||
if err != nil {
|
||||
log.Errorf("error writing torrent metainfo file: %s", err)
|
||||
return err
|
||||
}
|
||||
for k, m := range queue {
|
||||
if m == magnet {
|
||||
queue = append(queue[:k], queue[k+1:]...)
|
||||
}
|
||||
defer f.Close()
|
||||
err = bencode.NewEncoder(f).Encode(mi)
|
||||
if err != nil {
|
||||
log.Errorf("error writing torrent metainfo file: %s", err)
|
||||
return
|
||||
}
|
||||
for k, m := range queue {
|
||||
if m == magnet {
|
||||
queue = append(queue[:k], queue[k+1:]...)
|
||||
}
|
||||
}
|
||||
log.Infof("New torrent file generated in: %s", file)
|
||||
}()
|
||||
}
|
||||
log.Infof("New torrent file generated in: %s", file)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// GotFile will check if a torrent file exists and if not, try to generate it
|
||||
func GotFile(torrent *models.Torrent) error {
|
||||
var trackers []string
|
||||
if torrent.Trackers == "" {
|
||||
trackers = config.Get().Torrents.Trackers.Default
|
||||
} else {
|
||||
trackers = torrent.GetTrackersArray()
|
||||
}
|
||||
// We generate a new magnet link with all trackers (ours + ones from uploader)
|
||||
magnet := format.InfoHashToMagnet(strings.TrimSpace(torrent.Hash), torrent.Name, trackers...)
|
||||
//Check if file exists and open
|
||||
_, err := os.Open(torrent.GetPath())
|
||||
if err != nil {
|
||||
err := GenerateTorrent(magnet)
|
||||
if err != errPending && err != nil {
|
||||
return err
|
||||
}
|
||||
if err == errPending {
|
||||
for {
|
||||
//Check again if file exists and open
|
||||
_, err := os.Open(torrent.GetPath())
|
||||
if err == nil {
|
||||
break
|
||||
}
|
||||
// We check every 10 seconds
|
||||
time.Sleep(10000)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
|
|
@ -0,0 +1,367 @@
|
|||
package upload
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/ioutil"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/NyaaPantsu/nyaa/config"
|
||||
"github.com/NyaaPantsu/nyaa/models"
|
||||
"github.com/NyaaPantsu/nyaa/utils/cache"
|
||||
"github.com/NyaaPantsu/nyaa/utils/log"
|
||||
)
|
||||
|
||||
const (
|
||||
anidex = iota
|
||||
nyaasi
|
||||
ttosho
|
||||
)
|
||||
|
||||
const (
|
||||
pendingState = iota + 1
|
||||
errorState
|
||||
doneState
|
||||
)
|
||||
|
||||
// Each service gives a status and a message when uploading to them
|
||||
type service struct {
|
||||
Status int `json:"status"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
// MultipleForm is a struct used to follow the status of the uploads
|
||||
type MultipleForm struct {
|
||||
PantsuID uint `json:"id"`
|
||||
Anidex service `json:"anidex"`
|
||||
Nyaasi service `json:"nyaasi"`
|
||||
TTosho service `json:"ttosho"`
|
||||
}
|
||||
|
||||
// ToAnidex : function to upload a torrent to anidex
|
||||
// TODO: subCat and lang should be taken from torrent model and not asked to be typed again
|
||||
// so making the conversion here would be better
|
||||
func ToAnidex(torrent *models.Torrent, apiKey string, subCat string, lang string) {
|
||||
uploadMultiple := MultipleForm{PantsuID: torrent.ID, Anidex: service{Status: pendingState}}
|
||||
uploadMultiple.save(anidex)
|
||||
log.Info("Create anidex instance")
|
||||
|
||||
// If the torrent is posted as anonymous or apikey is not set, we set it with default value
|
||||
if apiKey == "" || (torrent.Hidden && apiKey != "") {
|
||||
apiKey = config.Get().Upload.DefaultAnidexToken
|
||||
}
|
||||
|
||||
if apiKey == "" { // You need to check that apikey is not empty even after config. Since it is left empty in config by default and is required
|
||||
log.Errorf("ApiKey is empty, we can't upload to anidex for torrent %d", torrent.ID)
|
||||
uploadMultiple.updateAndSave(anidex, errorState, "No ApiKey providen (required)")
|
||||
return
|
||||
}
|
||||
extraParams := map[string]string{
|
||||
//Required
|
||||
"api_key": apiKey,
|
||||
"subcat_id": subCat,
|
||||
"group_id": "0",
|
||||
"lang_id": lang,
|
||||
|
||||
//Optional
|
||||
"description": torrent.Description,
|
||||
"torrent_name": torrent.Name,
|
||||
}
|
||||
if config.IsSukebei() {
|
||||
extraParams["hentai"] = "1"
|
||||
}
|
||||
if torrent.IsRemake() {
|
||||
extraParams["reencode"] = "1"
|
||||
}
|
||||
if torrent.IsAnon() {
|
||||
extraParams["private"] = "1"
|
||||
}
|
||||
request, err := newfileUploadRequest("https://anidex.info/api/", extraParams, "file", torrent.GetPath())
|
||||
if err != nil {
|
||||
log.CheckError(err)
|
||||
return
|
||||
}
|
||||
client := &http.Client{}
|
||||
rsp, err := client.Do(request)
|
||||
if err != nil {
|
||||
log.CheckError(err)
|
||||
return
|
||||
}
|
||||
log.Info("Launch anidex http request")
|
||||
|
||||
if err != nil {
|
||||
uploadMultiple.updateAndSave(anidex, errorState, "Error during the HTTP POST request")
|
||||
log.CheckErrorWithMessage(err, "Error in request: %s")
|
||||
return
|
||||
}
|
||||
defer rsp.Body.Close()
|
||||
bodyByte, err := ioutil.ReadAll(rsp.Body)
|
||||
if err != nil {
|
||||
uploadMultiple.updateAndSave(anidex, errorState, "Unknown error")
|
||||
log.CheckErrorWithMessage(err, "Error in parsing request: %s")
|
||||
return
|
||||
}
|
||||
if uploadMultiple.Anidex.Status == pendingState {
|
||||
uploadMultiple.Anidex.Message = string(bodyByte)
|
||||
if strings.Contains(uploadMultiple.Anidex.Message, "http://") {
|
||||
uploadMultiple.Anidex.Status = doneState
|
||||
} else {
|
||||
uploadMultiple.Anidex.Status = errorState
|
||||
}
|
||||
uploadMultiple.save(anidex)
|
||||
log.Info("Anidex request done")
|
||||
fmt.Println(uploadMultiple)
|
||||
}
|
||||
}
|
||||
|
||||
// ToNyaasi : function to upload a torrent to anidex
|
||||
func ToNyaasi(username string, password string, torrent *models.Torrent) {
|
||||
uploadMultiple := MultipleForm{PantsuID: torrent.ID, Nyaasi: service{Status: pendingState}}
|
||||
uploadMultiple.Nyaasi.Message = "Sorry u are not allowed"
|
||||
uploadMultiple.save(nyaasi)
|
||||
log.Info("Create NyaaSi instance")
|
||||
|
||||
// If the torrent is posted as anonymous or apikey is not set, we set it with default value
|
||||
if username == "" || (torrent.Hidden && username != "") {
|
||||
username = config.Get().Upload.DefaultNyaasiUsername
|
||||
password = config.Get().Upload.DefaultNyaasiPassword
|
||||
}
|
||||
|
||||
if username == "" || password == "" { // You need to check that username AND password are not empty even after config. Since they are left empty in config by default and are required
|
||||
log.Errorf("Username or Password is empty, we can't upload to Nyaa.si for torrent %d", torrent.ID)
|
||||
uploadMultiple.updateAndSave(nyaasi, errorState, "No valid account providen (required)")
|
||||
return
|
||||
}
|
||||
|
||||
params := map[string]interface{}{
|
||||
"name": torrent.Name,
|
||||
"category": Category(nyaasi, torrent),
|
||||
"information": "",
|
||||
"description": torrent.Description,
|
||||
"anonymous": torrent.IsAnon(),
|
||||
"hidden": false,
|
||||
"remake": torrent.IsRemake(),
|
||||
"trusted": torrent.IsTrusted(),
|
||||
}
|
||||
torrentData, _ := json.Marshal(params)
|
||||
extraParams := map[string]string{
|
||||
"torrent_data": string(torrentData),
|
||||
}
|
||||
|
||||
request, err := newfileUploadRequest("https://nyaa.si/api/upload", extraParams, "torrent", torrent.GetPath())
|
||||
if err != nil {
|
||||
log.CheckError(err)
|
||||
return
|
||||
}
|
||||
request.SetBasicAuth(username, password)
|
||||
client := &http.Client{}
|
||||
rsp, err := client.Do(request)
|
||||
if err != nil {
|
||||
log.CheckError(err)
|
||||
return
|
||||
}
|
||||
log.Info("Launch Nyaa.Si http request")
|
||||
|
||||
if err != nil {
|
||||
uploadMultiple.updateAndSave(nyaasi, errorState, "Error during the HTTP POST request")
|
||||
log.CheckErrorWithMessage(err, "Error in request: %s")
|
||||
return
|
||||
}
|
||||
defer rsp.Body.Close()
|
||||
bodyByte, err := ioutil.ReadAll(rsp.Body)
|
||||
if err != nil {
|
||||
uploadMultiple.updateAndSave(nyaasi, errorState, "Unknown error")
|
||||
log.CheckErrorWithMessage(err, "Error in parsing request: %s")
|
||||
return
|
||||
}
|
||||
if uploadMultiple.Nyaasi.Status == pendingState {
|
||||
var data map[string]interface{}
|
||||
if err = json.Unmarshal(bodyByte, &data); err != nil {
|
||||
log.CheckErrorWithMessage(err, "Cannot unmarshal json Response after upload request to Nyaa.Si")
|
||||
uploadMultiple.Nyaasi.Status = errorState
|
||||
uploadMultiple.Nyaasi.Message = err.Error()
|
||||
}
|
||||
if _, ok := data["errors"]; ok {
|
||||
uploadMultiple.Nyaasi.Status = errorState
|
||||
uploadMultiple.Nyaasi.Message = string(bodyByte)
|
||||
} else {
|
||||
uploadMultiple.Nyaasi.Status = doneState
|
||||
uploadMultiple.Nyaasi.Message = fmt.Sprintf("%s", data["url"].(string))
|
||||
}
|
||||
uploadMultiple.save(nyaasi)
|
||||
log.Info("Nyaa.Si request done")
|
||||
fmt.Println(uploadMultiple)
|
||||
}
|
||||
}
|
||||
|
||||
// ToTTosho : function to upload a torrent to TokyoTosho
|
||||
func ToTTosho(apiKey string, torrent *models.Torrent) {
|
||||
uploadMultiple := MultipleForm{PantsuID: torrent.ID, TTosho: service{Status: pendingState}}
|
||||
uploadMultiple.save(ttosho)
|
||||
log.Info("Create TokyoTosho instance")
|
||||
|
||||
// If the torrent is posted as anonymous or apikey is not set, we set it with default value
|
||||
if apiKey == "" || (torrent.Hidden && apiKey != "") {
|
||||
apiKey = config.Get().Upload.DefaultTokyoTToken
|
||||
}
|
||||
|
||||
if apiKey == "" { // You need to check that apikey is not empty even after config. Since it is left empty in config by default and is required
|
||||
log.Errorf("ApiKey is empty, we can't upload to TokyoTosho for torrent %d", torrent.ID)
|
||||
uploadMultiple.updateAndSave(ttosho, errorState, "No ApiKey providen (required)")
|
||||
return
|
||||
}
|
||||
|
||||
extraParams := map[string]string{
|
||||
//Required
|
||||
"apikey": apiKey,
|
||||
"url": torrent.Download(),
|
||||
"type": Category(ttosho, torrent),
|
||||
"send": "true",
|
||||
|
||||
//Optional
|
||||
"website": torrent.WebsiteLink,
|
||||
"comment": torrent.Description,
|
||||
}
|
||||
request, err := newUploadRequest("https://www.tokyotosho.info/new.php", extraParams)
|
||||
if err != nil {
|
||||
log.CheckError(err)
|
||||
return
|
||||
}
|
||||
client := &http.Client{}
|
||||
rsp, err := client.Do(request)
|
||||
if err != nil {
|
||||
log.CheckError(err)
|
||||
return
|
||||
}
|
||||
log.Info("Launch TokyoTosho http request")
|
||||
|
||||
if err != nil {
|
||||
uploadMultiple.updateAndSave(ttosho, errorState, "Error during the HTTP POST request")
|
||||
log.CheckErrorWithMessage(err, "Error in request: %s")
|
||||
return
|
||||
}
|
||||
defer rsp.Body.Close()
|
||||
bodyByte, err := ioutil.ReadAll(rsp.Body)
|
||||
if err != nil {
|
||||
uploadMultiple.updateAndSave(ttosho, errorState, "Unknown error")
|
||||
log.CheckErrorWithMessage(err, "Error in parsing request: %s")
|
||||
return
|
||||
}
|
||||
if uploadMultiple.TTosho.Status == pendingState {
|
||||
if strings.Contains(string(bodyByte), "OK,") {
|
||||
uploadMultiple.TTosho.Status = doneState
|
||||
idnumber := strings.Split(string(bodyByte), ",")
|
||||
uploadMultiple.TTosho.Message = fmt.Sprintf("Upload done! https://www.tokyotosho.info/details.php?id=%s", idnumber[1])
|
||||
} else {
|
||||
uploadMultiple.TTosho.Status = errorState
|
||||
uploadMultiple.TTosho.Message = string(bodyByte)
|
||||
}
|
||||
uploadMultiple.save(ttosho)
|
||||
log.Info("TokyoTosho request done")
|
||||
fmt.Println(uploadMultiple)
|
||||
}
|
||||
}
|
||||
|
||||
// Saves the multipleform in each go routines and share the state of each upload for 5 minutes
|
||||
// After timeout, the multipleform is flushed from memory
|
||||
func (u *MultipleForm) save(which int) {
|
||||
// We check if there is already a variable shared, if there is we update only the status/message of one service
|
||||
if found, ok := cache.C.Get(fmt.Sprintf("tstatus_%d", u.PantsuID)); ok {
|
||||
uploadStatus := found.(MultipleForm)
|
||||
switch which {
|
||||
case anidex:
|
||||
uploadStatus.Anidex = u.Anidex
|
||||
break
|
||||
case nyaasi:
|
||||
uploadStatus.Nyaasi = u.Nyaasi
|
||||
break
|
||||
case ttosho:
|
||||
uploadStatus.TTosho = u.TTosho
|
||||
break
|
||||
}
|
||||
u = &uploadStatus
|
||||
}
|
||||
// And then we save the variable in cache
|
||||
cache.C.Set(fmt.Sprintf("tstatus_%d", u.PantsuID), *u, 5*time.Minute)
|
||||
}
|
||||
|
||||
// shortcut function to update and save a service
|
||||
func (u *MultipleForm) updateAndSave(which int, code int, message string) {
|
||||
switch which {
|
||||
case anidex:
|
||||
u.Anidex.update(code, message)
|
||||
break
|
||||
case nyaasi:
|
||||
u.Nyaasi.update(code, message)
|
||||
break
|
||||
case ttosho:
|
||||
u.TTosho.update(code, message)
|
||||
break
|
||||
}
|
||||
u.save(which)
|
||||
}
|
||||
|
||||
// shortcut function to update both code and message of a service
|
||||
func (s *service) update(code int, message string) {
|
||||
s.Status = code
|
||||
s.Message = message
|
||||
}
|
||||
|
||||
// Creates a new file upload http request with optional extra params
|
||||
func newfileUploadRequest(uri string, params map[string]string, paramName, path string) (*http.Request, error) {
|
||||
file, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer file.Close()
|
||||
fi, err := file.Stat()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
body := new(bytes.Buffer)
|
||||
writer := multipart.NewWriter(body)
|
||||
part, err := writer.CreateFormFile(paramName, fi.Name())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err = io.Copy(part, file); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for key, val := range params {
|
||||
_ = writer.WriteField(key, val)
|
||||
}
|
||||
err = writer.Close()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
request, err := http.NewRequest("POST", uri, body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
request.Header.Add("Content-Type", writer.FormDataContentType())
|
||||
return request, nil
|
||||
}
|
||||
|
||||
// Creates a new upload http request with optional extra params
|
||||
func newUploadRequest(uri string, params map[string]string) (*http.Request, error) {
|
||||
var form url.Values
|
||||
for key, val := range params {
|
||||
form[key] = append(form[key], val)
|
||||
}
|
||||
|
||||
body := bytes.NewBufferString(form.Encode())
|
||||
request, err := http.NewRequest("POST", uri, body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return request, nil
|
||||
}
|
|
@ -1,11 +1,6 @@
|
|||
package upload
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"io/ioutil"
|
||||
"mime/multipart"
|
||||
"os"
|
||||
"reflect"
|
||||
"strings"
|
||||
|
||||
|
@ -123,7 +118,7 @@ func ExtractInfo(c *gin.Context, r *torrentValidator.TorrentRequest) error {
|
|||
return err
|
||||
}
|
||||
|
||||
tfile, err := r.ValidateMultipartUpload(c, uploadFormTorrent)
|
||||
err = r.ValidateMultipartUpload(c, uploadFormTorrent)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
@ -133,16 +128,8 @@ func ExtractInfo(c *gin.Context, r *torrentValidator.TorrentRequest) error {
|
|||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// after data has been checked & extracted, write it to disk
|
||||
if len(config.Get().Torrents.FileStorage) > 0 && r.Filesize > 0 {
|
||||
err := writeTorrentToDisk(tfile, r.Infohash+".torrent", &r.Filepath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
r.Filepath = ""
|
||||
}
|
||||
// We are not saving the file here because we need to add our own tracker list to the torrent file, therefore, we generate the torrent file at upload time through GenerateTorrent()
|
||||
// when it is magnet
|
||||
|
||||
return nil
|
||||
}
|
||||
|
@ -213,19 +200,6 @@ func UpdateUnscopeTorrent(r *torrentValidator.UpdateRequest, t *models.Torrent,
|
|||
return t
|
||||
}
|
||||
|
||||
func writeTorrentToDisk(file multipart.File, name string, fullpath *string) error {
|
||||
_, seekErr := file.Seek(0, io.SeekStart)
|
||||
if seekErr != nil {
|
||||
return seekErr
|
||||
}
|
||||
b, err := ioutil.ReadAll(file)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*fullpath = fmt.Sprintf("%s%c%s", config.Get().Torrents.FileStorage, os.PathSeparator, name)
|
||||
return ioutil.WriteFile(*fullpath, b, 0644)
|
||||
}
|
||||
|
||||
// NewTorrentRequest : creates a new torrent request struc with some default value
|
||||
func NewTorrentRequest(params ...string) *torrentValidator.TorrentRequest {
|
||||
torrentRequest := &torrentValidator.TorrentRequest{}
|
||||
|
|
|
@ -15,3 +15,4 @@ var errTorrentHashInvalid = errors.New("torrent_hash_invalid")
|
|||
var errTorrentTagsInvalid = errors.New("torrent_tags_invalid")
|
||||
var errMissingFieldConfig = errors.New("Torrent Configuration in config.yml is invalid, missing a 'field' attribute in tags types")
|
||||
var errWrongFieldConfig = errors.New("Torrent Configuration in config.yml is invalid, wrong 'field' attribute in tags types")
|
||||
var errInvalidTorrentFile = errors.New("torrent_file_invalid")
|
||||
|
|
|
@ -4,9 +4,8 @@ import (
|
|||
"encoding/base32"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/url"
|
||||
"os"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"strconv"
|
||||
|
@ -15,13 +14,11 @@ import (
|
|||
"github.com/NyaaPantsu/nyaa/config"
|
||||
"github.com/NyaaPantsu/nyaa/utils/categories"
|
||||
"github.com/NyaaPantsu/nyaa/utils/cookies"
|
||||
"github.com/NyaaPantsu/nyaa/utils/format"
|
||||
msg "github.com/NyaaPantsu/nyaa/utils/messages"
|
||||
"github.com/NyaaPantsu/nyaa/utils/metainfo"
|
||||
"github.com/NyaaPantsu/nyaa/utils/torrentLanguages"
|
||||
"github.com/NyaaPantsu/nyaa/utils/validator/tag"
|
||||
"github.com/anacrolix/torrent/metainfo"
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/zeebo/bencode"
|
||||
)
|
||||
|
||||
// ValidateName is a function validating the torrent name
|
||||
|
@ -191,80 +188,90 @@ func (r *TorrentRequest) ExtractLanguage() error {
|
|||
}
|
||||
|
||||
// ValidateMultipartUpload : Check if multipart upload is valid
|
||||
func (r *TorrentRequest) ValidateMultipartUpload(c *gin.Context, uploadFormTorrent string) (multipart.File, error) {
|
||||
func (r *TorrentRequest) ValidateMultipartUpload(c *gin.Context, uploadFormTorrent string) error {
|
||||
// first: parse torrent file (if any) to fill missing information
|
||||
tfile, _, err := c.Request.FormFile(uploadFormTorrent)
|
||||
if err == nil {
|
||||
var torrent metainfo.TorrentFile
|
||||
|
||||
// decode torrent
|
||||
_, seekErr := tfile.Seek(0, io.SeekStart)
|
||||
if seekErr != nil {
|
||||
return tfile, seekErr
|
||||
}
|
||||
err = bencode.NewDecoder(tfile).Decode(&torrent)
|
||||
torrent, err := metainfo.Load(tfile)
|
||||
if err != nil {
|
||||
return tfile, metainfo.ErrInvalidTorrentFile
|
||||
return err
|
||||
}
|
||||
|
||||
torrentInfos, err := torrent.UnmarshalInfo()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// check a few things
|
||||
if torrent.IsPrivate() {
|
||||
return tfile, errTorrentPrivate
|
||||
if torrentInfos.Private != nil && *torrentInfos.Private {
|
||||
return errTorrentPrivate
|
||||
}
|
||||
trackers := torrent.GetAllAnnounceURLS()
|
||||
r.Trackers = CheckTrackers(trackers)
|
||||
// We check that the trackers are valid,
|
||||
// we filter the dead ones
|
||||
// and we add the needed ones from our config in the request and the torrent file
|
||||
r.Trackers = CheckTrackers(torrent)
|
||||
if len(r.Trackers) == 0 {
|
||||
return tfile, errTorrentNoTrackers
|
||||
return errTorrentNoTrackers
|
||||
}
|
||||
|
||||
// Name
|
||||
// For the name, if none is provided in request, we set the name from the torrent file
|
||||
if len(r.Name) == 0 {
|
||||
r.Name = torrent.TorrentName()
|
||||
r.Name = torrentInfos.Name
|
||||
}
|
||||
|
||||
// Magnet link: if a file is provided it should be empty
|
||||
if len(r.Magnet) != 0 {
|
||||
return tfile, errTorrentAndMagnet
|
||||
return errTorrentAndMagnet
|
||||
}
|
||||
|
||||
_, seekErr = tfile.Seek(0, io.SeekStart)
|
||||
if seekErr != nil {
|
||||
return tfile, seekErr
|
||||
}
|
||||
infohash, err := metainfo.DecodeInfohash(tfile)
|
||||
if err != nil {
|
||||
return tfile, metainfo.ErrInvalidTorrentFile
|
||||
}
|
||||
r.Infohash = infohash
|
||||
r.Magnet = format.InfoHashToMagnet(infohash, r.Name, trackers...)
|
||||
// We are using here default functions from anacrolix
|
||||
infohash := torrent.HashInfoBytes()
|
||||
|
||||
if infohash.String() == "" {
|
||||
return errInvalidTorrentFile
|
||||
}
|
||||
r.Infohash = infohash.String()
|
||||
r.Magnet = torrent.Magnet(r.Name, infohash).String()
|
||||
// extract filesize
|
||||
r.Filesize = int64(torrent.TotalSize())
|
||||
r.Filesize = torrentInfos.TotalLength()
|
||||
|
||||
// extract filelist
|
||||
fileInfos := torrent.Info.GetFiles()
|
||||
fileInfos := torrentInfos.Files
|
||||
for _, fileInfo := range fileInfos {
|
||||
r.FileList = append(r.FileList, uploadedFile{
|
||||
Path: fileInfo.Path,
|
||||
Filesize: int64(fileInfo.Length),
|
||||
Filesize: fileInfo.Length,
|
||||
})
|
||||
}
|
||||
// Since we can change with anacrolix the trackers of a torrent,
|
||||
// We will use this to generate directly the torrent file here
|
||||
file := fmt.Sprintf("%s%c%s.torrent", config.Get().Torrents.FileStorage, os.PathSeparator, infohash.String())
|
||||
f, err := os.Create(file)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
err = torrent.Write(f)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
} else {
|
||||
err = r.ValidateMagnet()
|
||||
if err != nil {
|
||||
return tfile, err
|
||||
return err
|
||||
}
|
||||
err = r.ValidateHash()
|
||||
if err != nil {
|
||||
return tfile, err
|
||||
return err
|
||||
}
|
||||
// TODO: Get Trackers from magnet URL
|
||||
r.Filesize = 0
|
||||
r.Filepath = ""
|
||||
|
||||
return tfile, nil
|
||||
return nil
|
||||
}
|
||||
return tfile, err
|
||||
return err
|
||||
}
|
||||
|
||||
// ExtractInfo : Function to assign values from request to ReassignForm
|
||||
|
|
|
@ -3,40 +3,61 @@ package torrentValidator
|
|||
import (
|
||||
"net/url"
|
||||
"strings"
|
||||
|
||||
"github.com/NyaaPantsu/nyaa/config"
|
||||
"github.com/anacrolix/torrent/metainfo"
|
||||
)
|
||||
|
||||
// CheckTrackers : Check if there is good trackers in torrent
|
||||
func CheckTrackers(trackers []string) []string {
|
||||
func CheckTrackers(t *metainfo.MetaInfo) []string {
|
||||
// TODO: move to runtime configuration
|
||||
var deadTrackers = []string{ // substring matches!
|
||||
"://open.nyaatorrents.info:6544",
|
||||
"://tracker.openbittorrent.com:80",
|
||||
"://tracker.publicbt.com:80",
|
||||
"://stats.anisource.net:2710",
|
||||
"://exodus.desync.com",
|
||||
"://open.demonii.com:1337",
|
||||
"://tracker.istole.it:80",
|
||||
"://tracker.ccc.de:80",
|
||||
"://bt2.careland.com.cn:6969",
|
||||
"://announce.torrentsmd.com:8080",
|
||||
"://open.demonii.com:1337",
|
||||
"://tracker.btcake.com",
|
||||
"://tracker.prq.to",
|
||||
"://bt.rghost.net"}
|
||||
var deadTrackers = config.Get().Torrents.Trackers.DeadTrackers
|
||||
|
||||
var trackerRet []string
|
||||
for _, t := range trackers {
|
||||
urlTracker, err := url.Parse(t)
|
||||
if err == nil {
|
||||
good := true
|
||||
for _, check := range deadTrackers {
|
||||
if strings.Contains(t, check) {
|
||||
good = false
|
||||
break // No need to continue the for loop
|
||||
trackerRet := []string{}
|
||||
tempList := metainfo.AnnounceList{}
|
||||
for _, group := range t.AnnounceList {
|
||||
var trackers []string
|
||||
for _, tracker := range group {
|
||||
urlTracker, err := url.ParseRequestURI(tracker)
|
||||
if err == nil {
|
||||
good := true
|
||||
for _, check := range deadTrackers {
|
||||
// the tracker is part of the deadtracker list
|
||||
// we don't keep it
|
||||
if strings.Contains(tracker, check) {
|
||||
good = false
|
||||
break // No need to continue the for loop
|
||||
}
|
||||
}
|
||||
if good {
|
||||
// We only keep the good trackers
|
||||
trackers = append(trackers, urlTracker.String())
|
||||
}
|
||||
}
|
||||
if good {
|
||||
trackerRet = append(trackerRet, urlTracker.String())
|
||||
}
|
||||
if len(trackers) > 0 {
|
||||
tempList = append(tempList, trackers)
|
||||
trackerRet = append(trackerRet, trackers...)
|
||||
}
|
||||
}
|
||||
t.AnnounceList = tempList
|
||||
defaultTracker := config.Get().Torrents.Trackers.GetDefault()
|
||||
if defaultTracker != "" {
|
||||
t.Announce = defaultTracker
|
||||
}
|
||||
|
||||
for _, key := range config.Get().Torrents.Trackers.NeededTrackers {
|
||||
inside := false
|
||||
if key < len(config.Get().Torrents.Trackers.Default) {
|
||||
tracker := config.Get().Torrents.Trackers.Default[key]
|
||||
for _, tr := range trackerRet {
|
||||
if tr == tracker {
|
||||
inside = true
|
||||
}
|
||||
}
|
||||
if !inside {
|
||||
trackerRet = append(trackerRet, tracker)
|
||||
t.AnnounceList = append(t.AnnounceList, []string{tracker})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
@ -0,0 +1,46 @@
|
|||
package torrentValidator
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/anacrolix/torrent/metainfo"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestCheckTrackers(t *testing.T) {
|
||||
assert := assert.New(t)
|
||||
tests := []struct {
|
||||
Info metainfo.MetaInfo
|
||||
Expected []string
|
||||
ExpectedList metainfo.AnnounceList
|
||||
}{
|
||||
{
|
||||
metainfo.MetaInfo{
|
||||
AnnounceList: [][]string{{"udp://nn.fof:4545/ano", "lol"}, {"://nyaa.tr/ann"}, {".co", "http://mont.co:444/ann"}},
|
||||
},
|
||||
[]string{"udp://nn.fof:4545/ano", "http://mont.co:444/ann", "udp://tracker.uw0.xyz:6969/announce", "http://anidex.moe:6969/announce"},
|
||||
[][]string{{"udp://nn.fof:4545/ano"}, {"http://mont.co:444/ann"}, {"udp://tracker.uw0.xyz:6969/announce"}, {"http://anidex.moe:6969/announce"}},
|
||||
},
|
||||
{
|
||||
metainfo.MetaInfo{
|
||||
AnnounceList: [][]string{{"http://open.nyaatorrents.info:6544", "http://mont.co:444/ann"}}, // dead tracker
|
||||
},
|
||||
[]string{"http://mont.co:444/ann", "udp://tracker.uw0.xyz:6969/announce", "http://anidex.moe:6969/announce"},
|
||||
[][]string{{"http://mont.co:444/ann"}, {"udp://tracker.uw0.xyz:6969/announce"}, {"http://anidex.moe:6969/announce"}},
|
||||
},
|
||||
{
|
||||
metainfo.MetaInfo{
|
||||
AnnounceList: [][]string{{"http://open.nyaatorrents.info:6544", "http://mont.co:444/ann", "http://mont.co:4434/ann"}, {"http://mont.co:444/anno"}}, // dead tracker
|
||||
},
|
||||
[]string{"http://mont.co:444/ann", "http://mont.co:4434/ann", "http://mont.co:444/anno", "udp://tracker.uw0.xyz:6969/announce", "http://anidex.moe:6969/announce"},
|
||||
[][]string{{"http://mont.co:444/ann", "http://mont.co:4434/ann"}, {"http://mont.co:444/anno"}, {"udp://tracker.uw0.xyz:6969/announce"}, {"http://anidex.moe:6969/announce"}},
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
trackers := CheckTrackers(&test.Info)
|
||||
assert.Equal(test.Expected, trackers)
|
||||
assert.Equal(test.ExpectedList, test.Info.AnnounceList)
|
||||
}
|
||||
|
||||
}
|
Référencer dans un nouveau ticket