Albirew/nyaa-pantsu
Albirew
/
nyaa-pantsu
Archivé
1
0
Bifurcation 0
Ce dépôt a été archivé le 2022-05-07. Vous pouvez voir ses fichiers ou le cloner, mais pas ouvrir de ticket ou de demandes d'ajout, ni soumettre de changements.
nyaa-pantsu/utils/upload/generate.go

123 lignes
2.8 KiB
Go
Brut Vue normale Historique

Torrent Generation on not found error (#1600) * [WIP] Torrent Generation on not found error As asked in #1517, it allows on-the-fly torrent generation. Since it uses magnet links, it needs some time to connect to peers. So it can't be instant generation, we need the user to wait and try after a minute at least. * Replace Fatal by simple error * attempt at fixing travis * del * Add Anacrolyx dependency * Add back difflib * Remove .torrent suffix in the url example * Add some explanations when file missing page shown * Ignore downloads directory * Either use cache (third-party site) or own download directory * Wrong import * If there is an error then it means we aren't generating a torrent file May it be "torrent not found" or "We do not store torrent files" which are the two only existing errors for this page * hash is never empty * TorrentLink may be empty at times So we add a /download/:hash link if it is * Update README.md * Made a mistake here, need to check if false * Update en-us.all.json * Update CHANGELOG.md * Torrent file generation can be triggered by click on button if JS enabled * Update download.go * Update download.go * Use c.JSON instead of text/template * Return to default behavior if we don't generate the file * Don't do the query if returned to default behavior * Add "Could not generate torrent file" error * Fix JS condition & lower delay until button updates * Start download automatically once torrent file is generated * Fix torrentFileExists() constantly returning false if external torrent download URL * torrent-view-data is two tables instead of one This allows the removal of useless things without any problem (e.g Website link), but also a better responsibe design since the previous one separated stats after a certain res looking very wonky * CSS changes to go along * Remove useless <b></b> * Update main.css * In torrentFileExists, check if filestorage path exists instead of looking at the domain in torrent link When checking if the file is stored on another server i used to simply check if the domain name was inside the torrent link, but we can straight up check for filestorage length * Fix JS of on-demand stat fetching * ScrapeAge variable accessible through view.jet.html Contains last scraped time in hours, is at -1 is torrent has never been scraped Stats will get updated if it's either at -1 or above 1460 (2 months old) * Refresh stats if older than two months OR unknown and older than 24h Show last scraped date even if stats are unknown * Add StatsObsolete variable to torrent Indicating if: - They can be shown - They need to be updated * Update scraped data even if Unknown, prevent users from trying to fetch stats every seconds * Torrent file stored locally by default * no need to do all of that if no filestorage * fix filestorage path * Fix torrent download button stuck on "Generating torrent file" at rare times * fix some css rules that didn't work on IE * Fix panic error Seems like this error is a known bug from anacrolyx torrent https://github.com/anacrolix/torrent/issues/83 To prevent it, I'm creating a single client and modifying the socket.go to make it not raise a panic but a simple error log.
2017-10-21 09:40:43 +02:00
package upload
import (
"fmt"
"os"
Upload to nyaa.si, anidex & TokyoTosho [done] (#1633) * Update upload.go * Update helpers.go * Update template_test.go * Update upload_multiple.jet.html * change variable names * wrong copypaste * change variable name * ditto * Fix travis * Update main.css * Update upload.go * Update upload.jet.html * Update main.css * Update upload.go * More compact form * CSS changes to go along * Update main.css * Update upload.jet.html * Slightly lower bottom margin * Update upload_multiple.jet.html * tomorrow color adjustements * small css adjustements * Update upload.go * Update default_config.yml * Update default_config.yml * Update structs.go * Update user.go * Update structs.go * Update upload.go * Update default_config.yml * Update upload_multiple.jet.html * Update upload.go * Update upload.jet.html * Update template_test.go * this one is optional for anidex * Update default_config.yml * Possible improvement As mentionned in my comment, it seems that having a checkbox already checked and disabled for logged in user + having the input text for apikey is a bit tedious. Moreover, asking to check and fill the input text to upload in anidex when you are not logged is doing things twice. If you don't want to upload to anidex, don't fill the input. So this commit push a new behaviour: * If you are logged in and you have filled your anidex/nyaasi api key : the form should only show the checkbox asking if you want to upload to nyaasi. * If you're not logged in or you don't have filled your api key in your profile settings: you only show the input text for the api key and not the checkbox. If someone wants to upload to nyaasi/anidex, he will just need to fill the input. * This adds back the support of anonymous upload when logged in or not. This works simply by checking if the user wants to upload as anon (when he checks the checkbox "upload as anonymous") or by checking the emptyness of apikey. + reverts the condition statements since it was needed to have apikey empty * Forgot to save this file * This commit adds the go routines for each upload service (anidex, nyaasi, tosho). New controller and url to check the upload status (/upload/status/:id). If you add ?json at the end, it outputs a json format of the multi upload status. To prevent memory overload, we only keep in memory the multiupload status for 5 minutes. Moved multipleform struct from templates to upload utils. * fix form display on classic theme * This commit adds: * Javascript refreshing of the upload status * Localization of the pages * Fixed some bugs This works, only need to get torrent file now. * Added a new function to get the Path to the torrent file. * Added a function to check that a torrent file exists and if not generate it on multiupload * Enabled file upload through http post request to anidex * Fixed bugs * Modified the behaviour to generate torrent preivously. It is a synchrone function now. So we need to make it asynchronous in download.go Now torrents file are always generated at upload time in a background processus with the correct trackers (needed ones + the ones provided by the user). I made the anidex tracker as a needed one to allow multiupload even if the user hasn't included in his torrent file/magnet url. * Moving deadtrackers list to default config Support torrent file creation when upload without scraping * Fix possible bug where no one is seeding a torrent yet (first time upload) Add primary tracker in torrent file * Fixing issue with boolean pointer for private torrent * fixing index out of range for announcelist filtering Fixing anidex tracker not found * Fixing lang id error * Anidex upload works * Fix for multiupload with magnet * This adds TokyoTosho support. Since TokyoTosho API only support upload by URL (and not torrent file upload). We need to make the download url available without ddos protection. Also TokyoTosho doesn't need a category select. Since it does have limited option, we can automatically convert our categories to the tokyotosho one. * Removing the folder tosho and put categories.go directly in upload package. Added category conversion for nyaasi too. Added tests on category conversion. * This should add nyaasi support. Is not tested though. * Updated defaut config with new nyaa.si behaviour? * Forgot to commit this file
2018-01-03 09:21:07 +01:00
"strconv"
"strings"
"time"
Torrent Generation on not found error (#1600) * [WIP] Torrent Generation on not found error As asked in #1517, it allows on-the-fly torrent generation. Since it uses magnet links, it needs some time to connect to peers. So it can't be instant generation, we need the user to wait and try after a minute at least. * Replace Fatal by simple error * attempt at fixing travis * del * Add Anacrolyx dependency * Add back difflib * Remove .torrent suffix in the url example * Add some explanations when file missing page shown * Ignore downloads directory * Either use cache (third-party site) or own download directory * Wrong import * If there is an error then it means we aren't generating a torrent file May it be "torrent not found" or "We do not store torrent files" which are the two only existing errors for this page * hash is never empty * TorrentLink may be empty at times So we add a /download/:hash link if it is * Update README.md * Made a mistake here, need to check if false * Update en-us.all.json * Update CHANGELOG.md * Torrent file generation can be triggered by click on button if JS enabled * Update download.go * Update download.go * Use c.JSON instead of text/template * Return to default behavior if we don't generate the file * Don't do the query if returned to default behavior * Add "Could not generate torrent file" error * Fix JS condition & lower delay until button updates * Start download automatically once torrent file is generated * Fix torrentFileExists() constantly returning false if external torrent download URL * torrent-view-data is two tables instead of one This allows the removal of useless things without any problem (e.g Website link), but also a better responsibe design since the previous one separated stats after a certain res looking very wonky * CSS changes to go along * Remove useless <b></b> * Update main.css * In torrentFileExists, check if filestorage path exists instead of looking at the domain in torrent link When checking if the file is stored on another server i used to simply check if the domain name was inside the torrent link, but we can straight up check for filestorage length * Fix JS of on-demand stat fetching * ScrapeAge variable accessible through view.jet.html Contains last scraped time in hours, is at -1 is torrent has never been scraped Stats will get updated if it's either at -1 or above 1460 (2 months old) * Refresh stats if older than two months OR unknown and older than 24h Show last scraped date even if stats are unknown * Add StatsObsolete variable to torrent Indicating if: - They can be shown - They need to be updated * Update scraped data even if Unknown, prevent users from trying to fetch stats every seconds * Torrent file stored locally by default * no need to do all of that if no filestorage * fix filestorage path * Fix torrent download button stuck on "Generating torrent file" at rare times * fix some css rules that didn't work on IE * Fix panic error Seems like this error is a known bug from anacrolyx torrent https://github.com/anacrolix/torrent/issues/83 To prevent it, I'm creating a single client and modifying the socket.go to make it not raise a panic but a simple error log.
2017-10-21 09:40:43 +02:00
"github.com/NyaaPantsu/nyaa/config"
Upload to nyaa.si, anidex & TokyoTosho [done] (#1633) * Update upload.go * Update helpers.go * Update template_test.go * Update upload_multiple.jet.html * change variable names * wrong copypaste * change variable name * ditto * Fix travis * Update main.css * Update upload.go * Update upload.jet.html * Update main.css * Update upload.go * More compact form * CSS changes to go along * Update main.css * Update upload.jet.html * Slightly lower bottom margin * Update upload_multiple.jet.html * tomorrow color adjustements * small css adjustements * Update upload.go * Update default_config.yml * Update default_config.yml * Update structs.go * Update user.go * Update structs.go * Update upload.go * Update default_config.yml * Update upload_multiple.jet.html * Update upload.go * Update upload.jet.html * Update template_test.go * this one is optional for anidex * Update default_config.yml * Possible improvement As mentionned in my comment, it seems that having a checkbox already checked and disabled for logged in user + having the input text for apikey is a bit tedious. Moreover, asking to check and fill the input text to upload in anidex when you are not logged is doing things twice. If you don't want to upload to anidex, don't fill the input. So this commit push a new behaviour: * If you are logged in and you have filled your anidex/nyaasi api key : the form should only show the checkbox asking if you want to upload to nyaasi. * If you're not logged in or you don't have filled your api key in your profile settings: you only show the input text for the api key and not the checkbox. If someone wants to upload to nyaasi/anidex, he will just need to fill the input. * This adds back the support of anonymous upload when logged in or not. This works simply by checking if the user wants to upload as anon (when he checks the checkbox "upload as anonymous") or by checking the emptyness of apikey. + reverts the condition statements since it was needed to have apikey empty * Forgot to save this file * This commit adds the go routines for each upload service (anidex, nyaasi, tosho). New controller and url to check the upload status (/upload/status/:id). If you add ?json at the end, it outputs a json format of the multi upload status. To prevent memory overload, we only keep in memory the multiupload status for 5 minutes. Moved multipleform struct from templates to upload utils. * fix form display on classic theme * This commit adds: * Javascript refreshing of the upload status * Localization of the pages * Fixed some bugs This works, only need to get torrent file now. * Added a new function to get the Path to the torrent file. * Added a function to check that a torrent file exists and if not generate it on multiupload * Enabled file upload through http post request to anidex * Fixed bugs * Modified the behaviour to generate torrent preivously. It is a synchrone function now. So we need to make it asynchronous in download.go Now torrents file are always generated at upload time in a background processus with the correct trackers (needed ones + the ones provided by the user). I made the anidex tracker as a needed one to allow multiupload even if the user hasn't included in his torrent file/magnet url. * Moving deadtrackers list to default config Support torrent file creation when upload without scraping * Fix possible bug where no one is seeding a torrent yet (first time upload) Add primary tracker in torrent file * Fixing issue with boolean pointer for private torrent * fixing index out of range for announcelist filtering Fixing anidex tracker not found * Fixing lang id error * Anidex upload works * Fix for multiupload with magnet * This adds TokyoTosho support. Since TokyoTosho API only support upload by URL (and not torrent file upload). We need to make the download url available without ddos protection. Also TokyoTosho doesn't need a category select. Since it does have limited option, we can automatically convert our categories to the tokyotosho one. * Removing the folder tosho and put categories.go directly in upload package. Added category conversion for nyaasi too. Added tests on category conversion. * This should add nyaasi support. Is not tested though. * Updated defaut config with new nyaa.si behaviour? * Forgot to commit this file
2018-01-03 09:21:07 +01:00
"github.com/NyaaPantsu/nyaa/models"
"github.com/NyaaPantsu/nyaa/utils/format"
Torrent Generation on not found error (#1600) * [WIP] Torrent Generation on not found error As asked in #1517, it allows on-the-fly torrent generation. Since it uses magnet links, it needs some time to connect to peers. So it can't be instant generation, we need the user to wait and try after a minute at least. * Replace Fatal by simple error * attempt at fixing travis * del * Add Anacrolyx dependency * Add back difflib * Remove .torrent suffix in the url example * Add some explanations when file missing page shown * Ignore downloads directory * Either use cache (third-party site) or own download directory * Wrong import * If there is an error then it means we aren't generating a torrent file May it be "torrent not found" or "We do not store torrent files" which are the two only existing errors for this page * hash is never empty * TorrentLink may be empty at times So we add a /download/:hash link if it is * Update README.md * Made a mistake here, need to check if false * Update en-us.all.json * Update CHANGELOG.md * Torrent file generation can be triggered by click on button if JS enabled * Update download.go * Update download.go * Use c.JSON instead of text/template * Return to default behavior if we don't generate the file * Don't do the query if returned to default behavior * Add "Could not generate torrent file" error * Fix JS condition & lower delay until button updates * Start download automatically once torrent file is generated * Fix torrentFileExists() constantly returning false if external torrent download URL * torrent-view-data is two tables instead of one This allows the removal of useless things without any problem (e.g Website link), but also a better responsibe design since the previous one separated stats after a certain res looking very wonky * CSS changes to go along * Remove useless <b></b> * Update main.css * In torrentFileExists, check if filestorage path exists instead of looking at the domain in torrent link When checking if the file is stored on another server i used to simply check if the domain name was inside the torrent link, but we can straight up check for filestorage length * Fix JS of on-demand stat fetching * ScrapeAge variable accessible through view.jet.html Contains last scraped time in hours, is at -1 is torrent has never been scraped Stats will get updated if it's either at -1 or above 1460 (2 months old) * Refresh stats if older than two months OR unknown and older than 24h Show last scraped date even if stats are unknown * Add StatsObsolete variable to torrent Indicating if: - They can be shown - They need to be updated * Update scraped data even if Unknown, prevent users from trying to fetch stats every seconds * Torrent file stored locally by default * no need to do all of that if no filestorage * fix filestorage path * Fix torrent download button stuck on "Generating torrent file" at rare times * fix some css rules that didn't work on IE * Fix panic error Seems like this error is a known bug from anacrolyx torrent https://github.com/anacrolix/torrent/issues/83 To prevent it, I'm creating a single client and modifying the socket.go to make it not raise a panic but a simple error log.
2017-10-21 09:40:43 +02:00
"github.com/NyaaPantsu/nyaa/utils/log"
"github.com/anacrolix/dht"
Torrent Generation on not found error (#1600) * [WIP] Torrent Generation on not found error As asked in #1517, it allows on-the-fly torrent generation. Since it uses magnet links, it needs some time to connect to peers. So it can't be instant generation, we need the user to wait and try after a minute at least. * Replace Fatal by simple error * attempt at fixing travis * del * Add Anacrolyx dependency * Add back difflib * Remove .torrent suffix in the url example * Add some explanations when file missing page shown * Ignore downloads directory * Either use cache (third-party site) or own download directory * Wrong import * If there is an error then it means we aren't generating a torrent file May it be "torrent not found" or "We do not store torrent files" which are the two only existing errors for this page * hash is never empty * TorrentLink may be empty at times So we add a /download/:hash link if it is * Update README.md * Made a mistake here, need to check if false * Update en-us.all.json * Update CHANGELOG.md * Torrent file generation can be triggered by click on button if JS enabled * Update download.go * Update download.go * Use c.JSON instead of text/template * Return to default behavior if we don't generate the file * Don't do the query if returned to default behavior * Add "Could not generate torrent file" error * Fix JS condition & lower delay until button updates * Start download automatically once torrent file is generated * Fix torrentFileExists() constantly returning false if external torrent download URL * torrent-view-data is two tables instead of one This allows the removal of useless things without any problem (e.g Website link), but also a better responsibe design since the previous one separated stats after a certain res looking very wonky * CSS changes to go along * Remove useless <b></b> * Update main.css * In torrentFileExists, check if filestorage path exists instead of looking at the domain in torrent link When checking if the file is stored on another server i used to simply check if the domain name was inside the torrent link, but we can straight up check for filestorage length * Fix JS of on-demand stat fetching * ScrapeAge variable accessible through view.jet.html Contains last scraped time in hours, is at -1 is torrent has never been scraped Stats will get updated if it's either at -1 or above 1460 (2 months old) * Refresh stats if older than two months OR unknown and older than 24h Show last scraped date even if stats are unknown * Add StatsObsolete variable to torrent Indicating if: - They can be shown - They need to be updated * Update scraped data even if Unknown, prevent users from trying to fetch stats every seconds * Torrent file stored locally by default * no need to do all of that if no filestorage * fix filestorage path * Fix torrent download button stuck on "Generating torrent file" at rare times * fix some css rules that didn't work on IE * Fix panic error Seems like this error is a known bug from anacrolyx torrent https://github.com/anacrolix/torrent/issues/83 To prevent it, I'm creating a single client and modifying the socket.go to make it not raise a panic but a simple error log.
2017-10-21 09:40:43 +02:00
"github.com/anacrolix/torrent"
"github.com/anacrolix/torrent/bencode"
)
var queue []string
var client *torrent.Client
func initClient() error {
clientConfig := torrent.Config{
DHTConfig: dht.ServerConfig{
StartingNodes: dht.GlobalBootstrapAddrs,
},
ListenAddr: ":" + strconv.Itoa(config.Get().Torrents.GenerationClientPort),
}
cl, err := torrent.NewClient(&clientConfig)
Torrent Generation on not found error (#1600) * [WIP] Torrent Generation on not found error As asked in #1517, it allows on-the-fly torrent generation. Since it uses magnet links, it needs some time to connect to peers. So it can't be instant generation, we need the user to wait and try after a minute at least. * Replace Fatal by simple error * attempt at fixing travis * del * Add Anacrolyx dependency * Add back difflib * Remove .torrent suffix in the url example * Add some explanations when file missing page shown * Ignore downloads directory * Either use cache (third-party site) or own download directory * Wrong import * If there is an error then it means we aren't generating a torrent file May it be "torrent not found" or "We do not store torrent files" which are the two only existing errors for this page * hash is never empty * TorrentLink may be empty at times So we add a /download/:hash link if it is * Update README.md * Made a mistake here, need to check if false * Update en-us.all.json * Update CHANGELOG.md * Torrent file generation can be triggered by click on button if JS enabled * Update download.go * Update download.go * Use c.JSON instead of text/template * Return to default behavior if we don't generate the file * Don't do the query if returned to default behavior * Add "Could not generate torrent file" error * Fix JS condition & lower delay until button updates * Start download automatically once torrent file is generated * Fix torrentFileExists() constantly returning false if external torrent download URL * torrent-view-data is two tables instead of one This allows the removal of useless things without any problem (e.g Website link), but also a better responsibe design since the previous one separated stats after a certain res looking very wonky * CSS changes to go along * Remove useless <b></b> * Update main.css * In torrentFileExists, check if filestorage path exists instead of looking at the domain in torrent link When checking if the file is stored on another server i used to simply check if the domain name was inside the torrent link, but we can straight up check for filestorage length * Fix JS of on-demand stat fetching * ScrapeAge variable accessible through view.jet.html Contains last scraped time in hours, is at -1 is torrent has never been scraped Stats will get updated if it's either at -1 or above 1460 (2 months old) * Refresh stats if older than two months OR unknown and older than 24h Show last scraped date even if stats are unknown * Add StatsObsolete variable to torrent Indicating if: - They can be shown - They need to be updated * Update scraped data even if Unknown, prevent users from trying to fetch stats every seconds * Torrent file stored locally by default * no need to do all of that if no filestorage * fix filestorage path * Fix torrent download button stuck on "Generating torrent file" at rare times * fix some css rules that didn't work on IE * Fix panic error Seems like this error is a known bug from anacrolyx torrent https://github.com/anacrolix/torrent/issues/83 To prevent it, I'm creating a single client and modifying the socket.go to make it not raise a panic but a simple error log.
2017-10-21 09:40:43 +02:00
if err != nil {
log.Errorf("error creating client: %s", err)
return err
}
client = cl
return nil
}
// GenerateTorrent generates a torrent file in the specified directory in config.yml from a magnet URI
func GenerateTorrent(magnet string) error {
if client == nil {
err := initClient()
if err != nil {
return err
}
}
if magnet == "" || len(config.Get().Torrents.FileStorage) == 0 {
Upload to nyaa.si, anidex & TokyoTosho [done] (#1633) * Update upload.go * Update helpers.go * Update template_test.go * Update upload_multiple.jet.html * change variable names * wrong copypaste * change variable name * ditto * Fix travis * Update main.css * Update upload.go * Update upload.jet.html * Update main.css * Update upload.go * More compact form * CSS changes to go along * Update main.css * Update upload.jet.html * Slightly lower bottom margin * Update upload_multiple.jet.html * tomorrow color adjustements * small css adjustements * Update upload.go * Update default_config.yml * Update default_config.yml * Update structs.go * Update user.go * Update structs.go * Update upload.go * Update default_config.yml * Update upload_multiple.jet.html * Update upload.go * Update upload.jet.html * Update template_test.go * this one is optional for anidex * Update default_config.yml * Possible improvement As mentionned in my comment, it seems that having a checkbox already checked and disabled for logged in user + having the input text for apikey is a bit tedious. Moreover, asking to check and fill the input text to upload in anidex when you are not logged is doing things twice. If you don't want to upload to anidex, don't fill the input. So this commit push a new behaviour: * If you are logged in and you have filled your anidex/nyaasi api key : the form should only show the checkbox asking if you want to upload to nyaasi. * If you're not logged in or you don't have filled your api key in your profile settings: you only show the input text for the api key and not the checkbox. If someone wants to upload to nyaasi/anidex, he will just need to fill the input. * This adds back the support of anonymous upload when logged in or not. This works simply by checking if the user wants to upload as anon (when he checks the checkbox "upload as anonymous") or by checking the emptyness of apikey. + reverts the condition statements since it was needed to have apikey empty * Forgot to save this file * This commit adds the go routines for each upload service (anidex, nyaasi, tosho). New controller and url to check the upload status (/upload/status/:id). If you add ?json at the end, it outputs a json format of the multi upload status. To prevent memory overload, we only keep in memory the multiupload status for 5 minutes. Moved multipleform struct from templates to upload utils. * fix form display on classic theme * This commit adds: * Javascript refreshing of the upload status * Localization of the pages * Fixed some bugs This works, only need to get torrent file now. * Added a new function to get the Path to the torrent file. * Added a function to check that a torrent file exists and if not generate it on multiupload * Enabled file upload through http post request to anidex * Fixed bugs * Modified the behaviour to generate torrent preivously. It is a synchrone function now. So we need to make it asynchronous in download.go Now torrents file are always generated at upload time in a background processus with the correct trackers (needed ones + the ones provided by the user). I made the anidex tracker as a needed one to allow multiupload even if the user hasn't included in his torrent file/magnet url. * Moving deadtrackers list to default config Support torrent file creation when upload without scraping * Fix possible bug where no one is seeding a torrent yet (first time upload) Add primary tracker in torrent file * Fixing issue with boolean pointer for private torrent * fixing index out of range for announcelist filtering Fixing anidex tracker not found * Fixing lang id error * Anidex upload works * Fix for multiupload with magnet * This adds TokyoTosho support. Since TokyoTosho API only support upload by URL (and not torrent file upload). We need to make the download url available without ddos protection. Also TokyoTosho doesn't need a category select. Since it does have limited option, we can automatically convert our categories to the tokyotosho one. * Removing the folder tosho and put categories.go directly in upload package. Added category conversion for nyaasi too. Added tests on category conversion. * This should add nyaasi support. Is not tested though. * Updated defaut config with new nyaa.si behaviour? * Forgot to commit this file
2018-01-03 09:21:07 +01:00
return errConfig
Torrent Generation on not found error (#1600) * [WIP] Torrent Generation on not found error As asked in #1517, it allows on-the-fly torrent generation. Since it uses magnet links, it needs some time to connect to peers. So it can't be instant generation, we need the user to wait and try after a minute at least. * Replace Fatal by simple error * attempt at fixing travis * del * Add Anacrolyx dependency * Add back difflib * Remove .torrent suffix in the url example * Add some explanations when file missing page shown * Ignore downloads directory * Either use cache (third-party site) or own download directory * Wrong import * If there is an error then it means we aren't generating a torrent file May it be "torrent not found" or "We do not store torrent files" which are the two only existing errors for this page * hash is never empty * TorrentLink may be empty at times So we add a /download/:hash link if it is * Update README.md * Made a mistake here, need to check if false * Update en-us.all.json * Update CHANGELOG.md * Torrent file generation can be triggered by click on button if JS enabled * Update download.go * Update download.go * Use c.JSON instead of text/template * Return to default behavior if we don't generate the file * Don't do the query if returned to default behavior * Add "Could not generate torrent file" error * Fix JS condition & lower delay until button updates * Start download automatically once torrent file is generated * Fix torrentFileExists() constantly returning false if external torrent download URL * torrent-view-data is two tables instead of one This allows the removal of useless things without any problem (e.g Website link), but also a better responsibe design since the previous one separated stats after a certain res looking very wonky * CSS changes to go along * Remove useless <b></b> * Update main.css * In torrentFileExists, check if filestorage path exists instead of looking at the domain in torrent link When checking if the file is stored on another server i used to simply check if the domain name was inside the torrent link, but we can straight up check for filestorage length * Fix JS of on-demand stat fetching * ScrapeAge variable accessible through view.jet.html Contains last scraped time in hours, is at -1 is torrent has never been scraped Stats will get updated if it's either at -1 or above 1460 (2 months old) * Refresh stats if older than two months OR unknown and older than 24h Show last scraped date even if stats are unknown * Add StatsObsolete variable to torrent Indicating if: - They can be shown - They need to be updated * Update scraped data even if Unknown, prevent users from trying to fetch stats every seconds * Torrent file stored locally by default * no need to do all of that if no filestorage * fix filestorage path * Fix torrent download button stuck on "Generating torrent file" at rare times * fix some css rules that didn't work on IE * Fix panic error Seems like this error is a known bug from anacrolyx torrent https://github.com/anacrolix/torrent/issues/83 To prevent it, I'm creating a single client and modifying the socket.go to make it not raise a panic but a simple error log.
2017-10-21 09:40:43 +02:00
}
if len(queue) > 0 {
for _, m := range queue {
if m == magnet {
Upload to nyaa.si, anidex & TokyoTosho [done] (#1633) * Update upload.go * Update helpers.go * Update template_test.go * Update upload_multiple.jet.html * change variable names * wrong copypaste * change variable name * ditto * Fix travis * Update main.css * Update upload.go * Update upload.jet.html * Update main.css * Update upload.go * More compact form * CSS changes to go along * Update main.css * Update upload.jet.html * Slightly lower bottom margin * Update upload_multiple.jet.html * tomorrow color adjustements * small css adjustements * Update upload.go * Update default_config.yml * Update default_config.yml * Update structs.go * Update user.go * Update structs.go * Update upload.go * Update default_config.yml * Update upload_multiple.jet.html * Update upload.go * Update upload.jet.html * Update template_test.go * this one is optional for anidex * Update default_config.yml * Possible improvement As mentionned in my comment, it seems that having a checkbox already checked and disabled for logged in user + having the input text for apikey is a bit tedious. Moreover, asking to check and fill the input text to upload in anidex when you are not logged is doing things twice. If you don't want to upload to anidex, don't fill the input. So this commit push a new behaviour: * If you are logged in and you have filled your anidex/nyaasi api key : the form should only show the checkbox asking if you want to upload to nyaasi. * If you're not logged in or you don't have filled your api key in your profile settings: you only show the input text for the api key and not the checkbox. If someone wants to upload to nyaasi/anidex, he will just need to fill the input. * This adds back the support of anonymous upload when logged in or not. This works simply by checking if the user wants to upload as anon (when he checks the checkbox "upload as anonymous") or by checking the emptyness of apikey. + reverts the condition statements since it was needed to have apikey empty * Forgot to save this file * This commit adds the go routines for each upload service (anidex, nyaasi, tosho). New controller and url to check the upload status (/upload/status/:id). If you add ?json at the end, it outputs a json format of the multi upload status. To prevent memory overload, we only keep in memory the multiupload status for 5 minutes. Moved multipleform struct from templates to upload utils. * fix form display on classic theme * This commit adds: * Javascript refreshing of the upload status * Localization of the pages * Fixed some bugs This works, only need to get torrent file now. * Added a new function to get the Path to the torrent file. * Added a function to check that a torrent file exists and if not generate it on multiupload * Enabled file upload through http post request to anidex * Fixed bugs * Modified the behaviour to generate torrent preivously. It is a synchrone function now. So we need to make it asynchronous in download.go Now torrents file are always generated at upload time in a background processus with the correct trackers (needed ones + the ones provided by the user). I made the anidex tracker as a needed one to allow multiupload even if the user hasn't included in his torrent file/magnet url. * Moving deadtrackers list to default config Support torrent file creation when upload without scraping * Fix possible bug where no one is seeding a torrent yet (first time upload) Add primary tracker in torrent file * Fixing issue with boolean pointer for private torrent * fixing index out of range for announcelist filtering Fixing anidex tracker not found * Fixing lang id error * Anidex upload works * Fix for multiupload with magnet * This adds TokyoTosho support. Since TokyoTosho API only support upload by URL (and not torrent file upload). We need to make the download url available without ddos protection. Also TokyoTosho doesn't need a category select. Since it does have limited option, we can automatically convert our categories to the tokyotosho one. * Removing the folder tosho and put categories.go directly in upload package. Added category conversion for nyaasi too. Added tests on category conversion. * This should add nyaasi support. Is not tested though. * Updated defaut config with new nyaa.si behaviour? * Forgot to commit this file
2018-01-03 09:21:07 +01:00
return errPending
Torrent Generation on not found error (#1600) * [WIP] Torrent Generation on not found error As asked in #1517, it allows on-the-fly torrent generation. Since it uses magnet links, it needs some time to connect to peers. So it can't be instant generation, we need the user to wait and try after a minute at least. * Replace Fatal by simple error * attempt at fixing travis * del * Add Anacrolyx dependency * Add back difflib * Remove .torrent suffix in the url example * Add some explanations when file missing page shown * Ignore downloads directory * Either use cache (third-party site) or own download directory * Wrong import * If there is an error then it means we aren't generating a torrent file May it be "torrent not found" or "We do not store torrent files" which are the two only existing errors for this page * hash is never empty * TorrentLink may be empty at times So we add a /download/:hash link if it is * Update README.md * Made a mistake here, need to check if false * Update en-us.all.json * Update CHANGELOG.md * Torrent file generation can be triggered by click on button if JS enabled * Update download.go * Update download.go * Use c.JSON instead of text/template * Return to default behavior if we don't generate the file * Don't do the query if returned to default behavior * Add "Could not generate torrent file" error * Fix JS condition & lower delay until button updates * Start download automatically once torrent file is generated * Fix torrentFileExists() constantly returning false if external torrent download URL * torrent-view-data is two tables instead of one This allows the removal of useless things without any problem (e.g Website link), but also a better responsibe design since the previous one separated stats after a certain res looking very wonky * CSS changes to go along * Remove useless <b></b> * Update main.css * In torrentFileExists, check if filestorage path exists instead of looking at the domain in torrent link When checking if the file is stored on another server i used to simply check if the domain name was inside the torrent link, but we can straight up check for filestorage length * Fix JS of on-demand stat fetching * ScrapeAge variable accessible through view.jet.html Contains last scraped time in hours, is at -1 is torrent has never been scraped Stats will get updated if it's either at -1 or above 1460 (2 months old) * Refresh stats if older than two months OR unknown and older than 24h Show last scraped date even if stats are unknown * Add StatsObsolete variable to torrent Indicating if: - They can be shown - They need to be updated * Update scraped data even if Unknown, prevent users from trying to fetch stats every seconds * Torrent file stored locally by default * no need to do all of that if no filestorage * fix filestorage path * Fix torrent download button stuck on "Generating torrent file" at rare times * fix some css rules that didn't work on IE * Fix panic error Seems like this error is a known bug from anacrolyx torrent https://github.com/anacrolix/torrent/issues/83 To prevent it, I'm creating a single client and modifying the socket.go to make it not raise a panic but a simple error log.
2017-10-21 09:40:43 +02:00
}
}
}
queue = append(queue, magnet)
t, err := client.AddMagnet(magnet)
if err != nil {
log.Errorf("error adding magnet to client: %s", err)
return err
}
Upload to nyaa.si, anidex & TokyoTosho [done] (#1633) * Update upload.go * Update helpers.go * Update template_test.go * Update upload_multiple.jet.html * change variable names * wrong copypaste * change variable name * ditto * Fix travis * Update main.css * Update upload.go * Update upload.jet.html * Update main.css * Update upload.go * More compact form * CSS changes to go along * Update main.css * Update upload.jet.html * Slightly lower bottom margin * Update upload_multiple.jet.html * tomorrow color adjustements * small css adjustements * Update upload.go * Update default_config.yml * Update default_config.yml * Update structs.go * Update user.go * Update structs.go * Update upload.go * Update default_config.yml * Update upload_multiple.jet.html * Update upload.go * Update upload.jet.html * Update template_test.go * this one is optional for anidex * Update default_config.yml * Possible improvement As mentionned in my comment, it seems that having a checkbox already checked and disabled for logged in user + having the input text for apikey is a bit tedious. Moreover, asking to check and fill the input text to upload in anidex when you are not logged is doing things twice. If you don't want to upload to anidex, don't fill the input. So this commit push a new behaviour: * If you are logged in and you have filled your anidex/nyaasi api key : the form should only show the checkbox asking if you want to upload to nyaasi. * If you're not logged in or you don't have filled your api key in your profile settings: you only show the input text for the api key and not the checkbox. If someone wants to upload to nyaasi/anidex, he will just need to fill the input. * This adds back the support of anonymous upload when logged in or not. This works simply by checking if the user wants to upload as anon (when he checks the checkbox "upload as anonymous") or by checking the emptyness of apikey. + reverts the condition statements since it was needed to have apikey empty * Forgot to save this file * This commit adds the go routines for each upload service (anidex, nyaasi, tosho). New controller and url to check the upload status (/upload/status/:id). If you add ?json at the end, it outputs a json format of the multi upload status. To prevent memory overload, we only keep in memory the multiupload status for 5 minutes. Moved multipleform struct from templates to upload utils. * fix form display on classic theme * This commit adds: * Javascript refreshing of the upload status * Localization of the pages * Fixed some bugs This works, only need to get torrent file now. * Added a new function to get the Path to the torrent file. * Added a function to check that a torrent file exists and if not generate it on multiupload * Enabled file upload through http post request to anidex * Fixed bugs * Modified the behaviour to generate torrent preivously. It is a synchrone function now. So we need to make it asynchronous in download.go Now torrents file are always generated at upload time in a background processus with the correct trackers (needed ones + the ones provided by the user). I made the anidex tracker as a needed one to allow multiupload even if the user hasn't included in his torrent file/magnet url. * Moving deadtrackers list to default config Support torrent file creation when upload without scraping * Fix possible bug where no one is seeding a torrent yet (first time upload) Add primary tracker in torrent file * Fixing issue with boolean pointer for private torrent * fixing index out of range for announcelist filtering Fixing anidex tracker not found * Fixing lang id error * Anidex upload works * Fix for multiupload with magnet * This adds TokyoTosho support. Since TokyoTosho API only support upload by URL (and not torrent file upload). We need to make the download url available without ddos protection. Also TokyoTosho doesn't need a category select. Since it does have limited option, we can automatically convert our categories to the tokyotosho one. * Removing the folder tosho and put categories.go directly in upload package. Added category conversion for nyaasi too. Added tests on category conversion. * This should add nyaasi support. Is not tested though. * Updated defaut config with new nyaa.si behaviour? * Forgot to commit this file
2018-01-03 09:21:07 +01:00
<-t.GotInfo()
mi := t.Metainfo()
t.Drop()
file := fmt.Sprintf("%s%c%s.torrent", config.Get().Torrents.FileStorage, os.PathSeparator, t.InfoHash().String())
f, err := os.Create(file)
if err != nil {
log.Errorf("error creating torrent metainfo file: %s", err)
return err
}
defer f.Close()
defaultTracker := config.Get().Torrents.Trackers.GetDefault()
if defaultTracker != "" {
mi.Announce = defaultTracker
}
err = bencode.NewEncoder(f).Encode(mi)
if err != nil {
log.Errorf("error writing torrent metainfo file: %s", err)
return err
}
for k, m := range queue {
if m == magnet {
queue = append(queue[:k], queue[k+1:]...)
Torrent Generation on not found error (#1600) * [WIP] Torrent Generation on not found error As asked in #1517, it allows on-the-fly torrent generation. Since it uses magnet links, it needs some time to connect to peers. So it can't be instant generation, we need the user to wait and try after a minute at least. * Replace Fatal by simple error * attempt at fixing travis * del * Add Anacrolyx dependency * Add back difflib * Remove .torrent suffix in the url example * Add some explanations when file missing page shown * Ignore downloads directory * Either use cache (third-party site) or own download directory * Wrong import * If there is an error then it means we aren't generating a torrent file May it be "torrent not found" or "We do not store torrent files" which are the two only existing errors for this page * hash is never empty * TorrentLink may be empty at times So we add a /download/:hash link if it is * Update README.md * Made a mistake here, need to check if false * Update en-us.all.json * Update CHANGELOG.md * Torrent file generation can be triggered by click on button if JS enabled * Update download.go * Update download.go * Use c.JSON instead of text/template * Return to default behavior if we don't generate the file * Don't do the query if returned to default behavior * Add "Could not generate torrent file" error * Fix JS condition & lower delay until button updates * Start download automatically once torrent file is generated * Fix torrentFileExists() constantly returning false if external torrent download URL * torrent-view-data is two tables instead of one This allows the removal of useless things without any problem (e.g Website link), but also a better responsibe design since the previous one separated stats after a certain res looking very wonky * CSS changes to go along * Remove useless <b></b> * Update main.css * In torrentFileExists, check if filestorage path exists instead of looking at the domain in torrent link When checking if the file is stored on another server i used to simply check if the domain name was inside the torrent link, but we can straight up check for filestorage length * Fix JS of on-demand stat fetching * ScrapeAge variable accessible through view.jet.html Contains last scraped time in hours, is at -1 is torrent has never been scraped Stats will get updated if it's either at -1 or above 1460 (2 months old) * Refresh stats if older than two months OR unknown and older than 24h Show last scraped date even if stats are unknown * Add StatsObsolete variable to torrent Indicating if: - They can be shown - They need to be updated * Update scraped data even if Unknown, prevent users from trying to fetch stats every seconds * Torrent file stored locally by default * no need to do all of that if no filestorage * fix filestorage path * Fix torrent download button stuck on "Generating torrent file" at rare times * fix some css rules that didn't work on IE * Fix panic error Seems like this error is a known bug from anacrolyx torrent https://github.com/anacrolix/torrent/issues/83 To prevent it, I'm creating a single client and modifying the socket.go to make it not raise a panic but a simple error log.
2017-10-21 09:40:43 +02:00
}
Upload to nyaa.si, anidex & TokyoTosho [done] (#1633) * Update upload.go * Update helpers.go * Update template_test.go * Update upload_multiple.jet.html * change variable names * wrong copypaste * change variable name * ditto * Fix travis * Update main.css * Update upload.go * Update upload.jet.html * Update main.css * Update upload.go * More compact form * CSS changes to go along * Update main.css * Update upload.jet.html * Slightly lower bottom margin * Update upload_multiple.jet.html * tomorrow color adjustements * small css adjustements * Update upload.go * Update default_config.yml * Update default_config.yml * Update structs.go * Update user.go * Update structs.go * Update upload.go * Update default_config.yml * Update upload_multiple.jet.html * Update upload.go * Update upload.jet.html * Update template_test.go * this one is optional for anidex * Update default_config.yml * Possible improvement As mentionned in my comment, it seems that having a checkbox already checked and disabled for logged in user + having the input text for apikey is a bit tedious. Moreover, asking to check and fill the input text to upload in anidex when you are not logged is doing things twice. If you don't want to upload to anidex, don't fill the input. So this commit push a new behaviour: * If you are logged in and you have filled your anidex/nyaasi api key : the form should only show the checkbox asking if you want to upload to nyaasi. * If you're not logged in or you don't have filled your api key in your profile settings: you only show the input text for the api key and not the checkbox. If someone wants to upload to nyaasi/anidex, he will just need to fill the input. * This adds back the support of anonymous upload when logged in or not. This works simply by checking if the user wants to upload as anon (when he checks the checkbox "upload as anonymous") or by checking the emptyness of apikey. + reverts the condition statements since it was needed to have apikey empty * Forgot to save this file * This commit adds the go routines for each upload service (anidex, nyaasi, tosho). New controller and url to check the upload status (/upload/status/:id). If you add ?json at the end, it outputs a json format of the multi upload status. To prevent memory overload, we only keep in memory the multiupload status for 5 minutes. Moved multipleform struct from templates to upload utils. * fix form display on classic theme * This commit adds: * Javascript refreshing of the upload status * Localization of the pages * Fixed some bugs This works, only need to get torrent file now. * Added a new function to get the Path to the torrent file. * Added a function to check that a torrent file exists and if not generate it on multiupload * Enabled file upload through http post request to anidex * Fixed bugs * Modified the behaviour to generate torrent preivously. It is a synchrone function now. So we need to make it asynchronous in download.go Now torrents file are always generated at upload time in a background processus with the correct trackers (needed ones + the ones provided by the user). I made the anidex tracker as a needed one to allow multiupload even if the user hasn't included in his torrent file/magnet url. * Moving deadtrackers list to default config Support torrent file creation when upload without scraping * Fix possible bug where no one is seeding a torrent yet (first time upload) Add primary tracker in torrent file * Fixing issue with boolean pointer for private torrent * fixing index out of range for announcelist filtering Fixing anidex tracker not found * Fixing lang id error * Anidex upload works * Fix for multiupload with magnet * This adds TokyoTosho support. Since TokyoTosho API only support upload by URL (and not torrent file upload). We need to make the download url available without ddos protection. Also TokyoTosho doesn't need a category select. Since it does have limited option, we can automatically convert our categories to the tokyotosho one. * Removing the folder tosho and put categories.go directly in upload package. Added category conversion for nyaasi too. Added tests on category conversion. * This should add nyaasi support. Is not tested though. * Updated defaut config with new nyaa.si behaviour? * Forgot to commit this file
2018-01-03 09:21:07 +01:00
}
log.Infof("New torrent file generated in: %s", file)
return nil
}
// GotFile will check if a torrent file exists and if not, try to generate it
func GotFile(torrent *models.Torrent) error {
var trackers []string
if torrent.Trackers == "" {
trackers = config.Get().Torrents.Trackers.Default
} else {
trackers = torrent.GetTrackersArray()
}
// We generate a new magnet link with all trackers (ours + ones from uploader)
magnet := format.InfoHashToMagnet(strings.TrimSpace(torrent.Hash), torrent.Name, trackers...)
//Check if file exists and open
_, err := os.Open(torrent.GetPath())
if err != nil {
err := GenerateTorrent(magnet)
if err != errPending && err != nil {
return err
Torrent Generation on not found error (#1600) * [WIP] Torrent Generation on not found error As asked in #1517, it allows on-the-fly torrent generation. Since it uses magnet links, it needs some time to connect to peers. So it can't be instant generation, we need the user to wait and try after a minute at least. * Replace Fatal by simple error * attempt at fixing travis * del * Add Anacrolyx dependency * Add back difflib * Remove .torrent suffix in the url example * Add some explanations when file missing page shown * Ignore downloads directory * Either use cache (third-party site) or own download directory * Wrong import * If there is an error then it means we aren't generating a torrent file May it be "torrent not found" or "We do not store torrent files" which are the two only existing errors for this page * hash is never empty * TorrentLink may be empty at times So we add a /download/:hash link if it is * Update README.md * Made a mistake here, need to check if false * Update en-us.all.json * Update CHANGELOG.md * Torrent file generation can be triggered by click on button if JS enabled * Update download.go * Update download.go * Use c.JSON instead of text/template * Return to default behavior if we don't generate the file * Don't do the query if returned to default behavior * Add "Could not generate torrent file" error * Fix JS condition & lower delay until button updates * Start download automatically once torrent file is generated * Fix torrentFileExists() constantly returning false if external torrent download URL * torrent-view-data is two tables instead of one This allows the removal of useless things without any problem (e.g Website link), but also a better responsibe design since the previous one separated stats after a certain res looking very wonky * CSS changes to go along * Remove useless <b></b> * Update main.css * In torrentFileExists, check if filestorage path exists instead of looking at the domain in torrent link When checking if the file is stored on another server i used to simply check if the domain name was inside the torrent link, but we can straight up check for filestorage length * Fix JS of on-demand stat fetching * ScrapeAge variable accessible through view.jet.html Contains last scraped time in hours, is at -1 is torrent has never been scraped Stats will get updated if it's either at -1 or above 1460 (2 months old) * Refresh stats if older than two months OR unknown and older than 24h Show last scraped date even if stats are unknown * Add StatsObsolete variable to torrent Indicating if: - They can be shown - They need to be updated * Update scraped data even if Unknown, prevent users from trying to fetch stats every seconds * Torrent file stored locally by default * no need to do all of that if no filestorage * fix filestorage path * Fix torrent download button stuck on "Generating torrent file" at rare times * fix some css rules that didn't work on IE * Fix panic error Seems like this error is a known bug from anacrolyx torrent https://github.com/anacrolix/torrent/issues/83 To prevent it, I'm creating a single client and modifying the socket.go to make it not raise a panic but a simple error log.
2017-10-21 09:40:43 +02:00
}
Upload to nyaa.si, anidex & TokyoTosho [done] (#1633) * Update upload.go * Update helpers.go * Update template_test.go * Update upload_multiple.jet.html * change variable names * wrong copypaste * change variable name * ditto * Fix travis * Update main.css * Update upload.go * Update upload.jet.html * Update main.css * Update upload.go * More compact form * CSS changes to go along * Update main.css * Update upload.jet.html * Slightly lower bottom margin * Update upload_multiple.jet.html * tomorrow color adjustements * small css adjustements * Update upload.go * Update default_config.yml * Update default_config.yml * Update structs.go * Update user.go * Update structs.go * Update upload.go * Update default_config.yml * Update upload_multiple.jet.html * Update upload.go * Update upload.jet.html * Update template_test.go * this one is optional for anidex * Update default_config.yml * Possible improvement As mentionned in my comment, it seems that having a checkbox already checked and disabled for logged in user + having the input text for apikey is a bit tedious. Moreover, asking to check and fill the input text to upload in anidex when you are not logged is doing things twice. If you don't want to upload to anidex, don't fill the input. So this commit push a new behaviour: * If you are logged in and you have filled your anidex/nyaasi api key : the form should only show the checkbox asking if you want to upload to nyaasi. * If you're not logged in or you don't have filled your api key in your profile settings: you only show the input text for the api key and not the checkbox. If someone wants to upload to nyaasi/anidex, he will just need to fill the input. * This adds back the support of anonymous upload when logged in or not. This works simply by checking if the user wants to upload as anon (when he checks the checkbox "upload as anonymous") or by checking the emptyness of apikey. + reverts the condition statements since it was needed to have apikey empty * Forgot to save this file * This commit adds the go routines for each upload service (anidex, nyaasi, tosho). New controller and url to check the upload status (/upload/status/:id). If you add ?json at the end, it outputs a json format of the multi upload status. To prevent memory overload, we only keep in memory the multiupload status for 5 minutes. Moved multipleform struct from templates to upload utils. * fix form display on classic theme * This commit adds: * Javascript refreshing of the upload status * Localization of the pages * Fixed some bugs This works, only need to get torrent file now. * Added a new function to get the Path to the torrent file. * Added a function to check that a torrent file exists and if not generate it on multiupload * Enabled file upload through http post request to anidex * Fixed bugs * Modified the behaviour to generate torrent preivously. It is a synchrone function now. So we need to make it asynchronous in download.go Now torrents file are always generated at upload time in a background processus with the correct trackers (needed ones + the ones provided by the user). I made the anidex tracker as a needed one to allow multiupload even if the user hasn't included in his torrent file/magnet url. * Moving deadtrackers list to default config Support torrent file creation when upload without scraping * Fix possible bug where no one is seeding a torrent yet (first time upload) Add primary tracker in torrent file * Fixing issue with boolean pointer for private torrent * fixing index out of range for announcelist filtering Fixing anidex tracker not found * Fixing lang id error * Anidex upload works * Fix for multiupload with magnet * This adds TokyoTosho support. Since TokyoTosho API only support upload by URL (and not torrent file upload). We need to make the download url available without ddos protection. Also TokyoTosho doesn't need a category select. Since it does have limited option, we can automatically convert our categories to the tokyotosho one. * Removing the folder tosho and put categories.go directly in upload package. Added category conversion for nyaasi too. Added tests on category conversion. * This should add nyaasi support. Is not tested though. * Updated defaut config with new nyaa.si behaviour? * Forgot to commit this file
2018-01-03 09:21:07 +01:00
if err == errPending {
for {
//Check again if file exists and open
_, err := os.Open(torrent.GetPath())
if err == nil {
break
}
// We check every 10 seconds
time.Sleep(10000)
Torrent Generation on not found error (#1600) * [WIP] Torrent Generation on not found error As asked in #1517, it allows on-the-fly torrent generation. Since it uses magnet links, it needs some time to connect to peers. So it can't be instant generation, we need the user to wait and try after a minute at least. * Replace Fatal by simple error * attempt at fixing travis * del * Add Anacrolyx dependency * Add back difflib * Remove .torrent suffix in the url example * Add some explanations when file missing page shown * Ignore downloads directory * Either use cache (third-party site) or own download directory * Wrong import * If there is an error then it means we aren't generating a torrent file May it be "torrent not found" or "We do not store torrent files" which are the two only existing errors for this page * hash is never empty * TorrentLink may be empty at times So we add a /download/:hash link if it is * Update README.md * Made a mistake here, need to check if false * Update en-us.all.json * Update CHANGELOG.md * Torrent file generation can be triggered by click on button if JS enabled * Update download.go * Update download.go * Use c.JSON instead of text/template * Return to default behavior if we don't generate the file * Don't do the query if returned to default behavior * Add "Could not generate torrent file" error * Fix JS condition & lower delay until button updates * Start download automatically once torrent file is generated * Fix torrentFileExists() constantly returning false if external torrent download URL * torrent-view-data is two tables instead of one This allows the removal of useless things without any problem (e.g Website link), but also a better responsibe design since the previous one separated stats after a certain res looking very wonky * CSS changes to go along * Remove useless <b></b> * Update main.css * In torrentFileExists, check if filestorage path exists instead of looking at the domain in torrent link When checking if the file is stored on another server i used to simply check if the domain name was inside the torrent link, but we can straight up check for filestorage length * Fix JS of on-demand stat fetching * ScrapeAge variable accessible through view.jet.html Contains last scraped time in hours, is at -1 is torrent has never been scraped Stats will get updated if it's either at -1 or above 1460 (2 months old) * Refresh stats if older than two months OR unknown and older than 24h Show last scraped date even if stats are unknown * Add StatsObsolete variable to torrent Indicating if: - They can be shown - They need to be updated * Update scraped data even if Unknown, prevent users from trying to fetch stats every seconds * Torrent file stored locally by default * no need to do all of that if no filestorage * fix filestorage path * Fix torrent download button stuck on "Generating torrent file" at rare times * fix some css rules that didn't work on IE * Fix panic error Seems like this error is a known bug from anacrolyx torrent https://github.com/anacrolix/torrent/issues/83 To prevent it, I'm creating a single client and modifying the socket.go to make it not raise a panic but a simple error log.
2017-10-21 09:40:43 +02:00
}
}
Upload to nyaa.si, anidex & TokyoTosho [done] (#1633) * Update upload.go * Update helpers.go * Update template_test.go * Update upload_multiple.jet.html * change variable names * wrong copypaste * change variable name * ditto * Fix travis * Update main.css * Update upload.go * Update upload.jet.html * Update main.css * Update upload.go * More compact form * CSS changes to go along * Update main.css * Update upload.jet.html * Slightly lower bottom margin * Update upload_multiple.jet.html * tomorrow color adjustements * small css adjustements * Update upload.go * Update default_config.yml * Update default_config.yml * Update structs.go * Update user.go * Update structs.go * Update upload.go * Update default_config.yml * Update upload_multiple.jet.html * Update upload.go * Update upload.jet.html * Update template_test.go * this one is optional for anidex * Update default_config.yml * Possible improvement As mentionned in my comment, it seems that having a checkbox already checked and disabled for logged in user + having the input text for apikey is a bit tedious. Moreover, asking to check and fill the input text to upload in anidex when you are not logged is doing things twice. If you don't want to upload to anidex, don't fill the input. So this commit push a new behaviour: * If you are logged in and you have filled your anidex/nyaasi api key : the form should only show the checkbox asking if you want to upload to nyaasi. * If you're not logged in or you don't have filled your api key in your profile settings: you only show the input text for the api key and not the checkbox. If someone wants to upload to nyaasi/anidex, he will just need to fill the input. * This adds back the support of anonymous upload when logged in or not. This works simply by checking if the user wants to upload as anon (when he checks the checkbox "upload as anonymous") or by checking the emptyness of apikey. + reverts the condition statements since it was needed to have apikey empty * Forgot to save this file * This commit adds the go routines for each upload service (anidex, nyaasi, tosho). New controller and url to check the upload status (/upload/status/:id). If you add ?json at the end, it outputs a json format of the multi upload status. To prevent memory overload, we only keep in memory the multiupload status for 5 minutes. Moved multipleform struct from templates to upload utils. * fix form display on classic theme * This commit adds: * Javascript refreshing of the upload status * Localization of the pages * Fixed some bugs This works, only need to get torrent file now. * Added a new function to get the Path to the torrent file. * Added a function to check that a torrent file exists and if not generate it on multiupload * Enabled file upload through http post request to anidex * Fixed bugs * Modified the behaviour to generate torrent preivously. It is a synchrone function now. So we need to make it asynchronous in download.go Now torrents file are always generated at upload time in a background processus with the correct trackers (needed ones + the ones provided by the user). I made the anidex tracker as a needed one to allow multiupload even if the user hasn't included in his torrent file/magnet url. * Moving deadtrackers list to default config Support torrent file creation when upload without scraping * Fix possible bug where no one is seeding a torrent yet (first time upload) Add primary tracker in torrent file * Fixing issue with boolean pointer for private torrent * fixing index out of range for announcelist filtering Fixing anidex tracker not found * Fixing lang id error * Anidex upload works * Fix for multiupload with magnet * This adds TokyoTosho support. Since TokyoTosho API only support upload by URL (and not torrent file upload). We need to make the download url available without ddos protection. Also TokyoTosho doesn't need a category select. Since it does have limited option, we can automatically convert our categories to the tokyotosho one. * Removing the folder tosho and put categories.go directly in upload package. Added category conversion for nyaasi too. Added tests on category conversion. * This should add nyaasi support. Is not tested though. * Updated defaut config with new nyaa.si behaviour? * Forgot to commit this file
2018-01-03 09:21:07 +01:00
}
Torrent Generation on not found error (#1600) * [WIP] Torrent Generation on not found error As asked in #1517, it allows on-the-fly torrent generation. Since it uses magnet links, it needs some time to connect to peers. So it can't be instant generation, we need the user to wait and try after a minute at least. * Replace Fatal by simple error * attempt at fixing travis * del * Add Anacrolyx dependency * Add back difflib * Remove .torrent suffix in the url example * Add some explanations when file missing page shown * Ignore downloads directory * Either use cache (third-party site) or own download directory * Wrong import * If there is an error then it means we aren't generating a torrent file May it be "torrent not found" or "We do not store torrent files" which are the two only existing errors for this page * hash is never empty * TorrentLink may be empty at times So we add a /download/:hash link if it is * Update README.md * Made a mistake here, need to check if false * Update en-us.all.json * Update CHANGELOG.md * Torrent file generation can be triggered by click on button if JS enabled * Update download.go * Update download.go * Use c.JSON instead of text/template * Return to default behavior if we don't generate the file * Don't do the query if returned to default behavior * Add "Could not generate torrent file" error * Fix JS condition & lower delay until button updates * Start download automatically once torrent file is generated * Fix torrentFileExists() constantly returning false if external torrent download URL * torrent-view-data is two tables instead of one This allows the removal of useless things without any problem (e.g Website link), but also a better responsibe design since the previous one separated stats after a certain res looking very wonky * CSS changes to go along * Remove useless <b></b> * Update main.css * In torrentFileExists, check if filestorage path exists instead of looking at the domain in torrent link When checking if the file is stored on another server i used to simply check if the domain name was inside the torrent link, but we can straight up check for filestorage length * Fix JS of on-demand stat fetching * ScrapeAge variable accessible through view.jet.html Contains last scraped time in hours, is at -1 is torrent has never been scraped Stats will get updated if it's either at -1 or above 1460 (2 months old) * Refresh stats if older than two months OR unknown and older than 24h Show last scraped date even if stats are unknown * Add StatsObsolete variable to torrent Indicating if: - They can be shown - They need to be updated * Update scraped data even if Unknown, prevent users from trying to fetch stats every seconds * Torrent file stored locally by default * no need to do all of that if no filestorage * fix filestorage path * Fix torrent download button stuck on "Generating torrent file" at rare times * fix some css rules that didn't work on IE * Fix panic error Seems like this error is a known bug from anacrolyx torrent https://github.com/anacrolix/torrent/issues/83 To prevent it, I'm creating a single client and modifying the socket.go to make it not raise a panic but a simple error log.
2017-10-21 09:40:43 +02:00
return nil
}