- completato il supporto al futuro Kodi 19\n- ridisegnato infoplus\n- fix vari ed eventuali\n
This commit is contained in:
marco
2020-09-29 21:08:25 +02:00
parent d153ac5918
commit 8a8d1e4f5e
195 changed files with 20697 additions and 23038 deletions
-46
View File
@@ -1,46 +0,0 @@
# This is a basic workflow to help you get started with Actions
name: Test Suite
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
workflow_dispatch:
schedule:
- cron: '30 17 * * *'
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
tests:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:
python-version: 2.7
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install sakee
pip install html-testRunner
pip install parameterized
- name: Run tests
run: |
export PYTHONPATH=$GITHUB_WORKSPACE
export KODI_INTERACTIVE=0
export KODI_HOME=$GITHUB_WORKSPACE/tests/home
python tests/test_generic.py
- uses: actions/upload-artifact@v2
with:
name: report
path: reports/report.html
-2
View File
@@ -8,8 +8,6 @@ KoD, come Alfa, è sotto licenza GPL v3, pertanto siete liberi di utilizzare par
- Il tuo addon deve essere rilasciando secondo la stessa licenza, ovvero essere open source (il fatto che lo zip sia visibile da chiunque non ha importanza, è necessario avere un repository git come questo)
- Aggiungere i crediti a tutto ciò che copiate/modificate, ad esempio aggiungendo un commento nel file in questione o, meglio, facendo un cherry-pick (in modo da preservarnee lo storico)
Nota: KoD attualmente funziona con Kodi fino alla versione 18 (Python 2).
### Come contribuire o fare segnalazioni?
Ti piace il progetto e vuoi dare una mano? Leggi [qui](https://github.com/kodiondemand/addon/blob/master/CONTRIBUTING.md)
+7 -9
View File
@@ -1,4 +1,4 @@
<addon id="plugin.video.kod" name="Kodi on Demand" version="1.3.1" provider-name="KoD Team">
<addon id="plugin.video.kod" name="Kodi on Demand" version="1.4" provider-name="KoD Team">
<requires>
<!-- <import addon="script.module.libtorrent" optional="true"/> -->
<import addon="metadata.themoviedb.org"/>
@@ -10,7 +10,7 @@
</extension>
<extension point="kodi.context.item">
<menu id="kodi.core.main">
<item library="updatetvshow.py">
<item library="platformcode/updatetvshow.py">
<label>70269</label>
<visible>String.IsEqual(ListItem.dbtype,tvshow)</visible>
</item>
@@ -20,17 +20,15 @@
<summary lang="en">Kodi on Demand is a Kodi add-on to search and watch contents on the web.</summary>
<summary lang="it">Kodi on Demand è un addon di Kodi per cercare e guardare contenuti sul web.</summary>
<assets>
<icon>logo.png</icon>
<fanart>fanart.jpg</fanart>
<icon>resources/media/logo.png</icon>
<fanart>resources/media/fanart.jpg</fanart>
<screenshot>resources/media/themes/ss/1.png</screenshot>
<screenshot>resources/media/themes/ss/2.png</screenshot>
<screenshot>resources/media/themes/ss/3.png</screenshot>
</assets>
<news>- aggiunti nuovi canali: film4k, animealtadefinizione, streamingcommunity, animeuniverse , guardaserieICU
- HDmario ora supporta l'utilizzo di account
- Miglioramenti sezione news, è ora possibile raggruppare per canale o per contenuto, e settare l'ordinamento
- risolto il fastidioso problema per cui poteva capitare che la ricerca ripartisse dopo un refresh di kodi (tipicamente quando l'aggiornamento della videoteca finiva)
- alcuni fix ai canali</news>
<news>- completato il supporto al futuro Kodi 19
- ridisegnato infoplus
- fix vari ed eventuali</news>
<description lang="it">Naviga velocemente sul web e guarda i contenuti presenti</description>
<disclaimer>[COLOR red]The owners and submitters to this addon do not host or distribute any of the content displayed by these addons nor do they have any affiliation with the content providers.[/COLOR]
[COLOR yellow]Kodi © is a registered trademark of the XBMC Foundation. We are not connected to or in any other way affiliated with Kodi, Team Kodi, or the XBMC Foundation. Furthermore, any software, addons, or products offered by us will receive no support in official Kodi channels, including the Kodi forums and various social networks.[/COLOR]</disclaimer>
+18 -19
View File
@@ -1,49 +1,48 @@
{
"altadefinizione01": "https://www.altadefinizione01.photo",
"altadefinizione01_link": "https://altadefinizione01.wine",
"altadefinizioneclick": "https://altadefinizione.group",
"altadefinizione01": "https://www.altadefinizione01.green",
"altadefinizione01_link": "https://altadefinizione01.energy",
"altadefinizioneclick": "https://altadefinizione.vote",
"animealtadefinizione": "https://www.animealtadefinizione.it",
"animeforce": "https://ww1.animeforce.org",
"animeleggendari": "https://animeora.com",
"animesaturn": "https://www.animesaturn.com",
"animestream": "https://www.animeworld.it",
"animesaturn": "https://www.animesaturn.it",
"animestream": "https://www.animeworld.tv",
"animesubita": "http://www.animesubita.org",
"animetubeita": "http://www.animetubeita.com",
"animeunity": "https://www.animeunity.it",
"animeuniverse" : "https://www.animeuniverse.it/",
"animeuniverse": "https://www.animeuniverse.it/",
"animeworld": "https://www.animeworld.tv",
"casacinema": "https://www.casacinema.page",
"cb01anime": "https://www.cineblog01.red",
"cinemalibero": "https://cinemalibero.plus",
"cinemalibero": "https://www.cinemalibero.run",
"cinetecadibologna": "http://cinestore.cinetecadibologna.it",
"dreamsub": "https://dreamsub.stream",
"dsda": "https://www.dsda.press",
"fastsubita": "https://fastsubita.online",
"filmgratis": "https://www.filmaltadefinizione.tv",
"filmigratis": "https://filmigratis.org",
"filmpertutti": "https://www.filmpertutti.site",
"filmpertutti": "https://www.filmpertutti.icu",
"filmsenzalimiticc": "https://www.filmsenzalimiti01.casa",
"filmstreaming01": "https://filmstreaming01.com",
"guardaseriecam": "https://guardaserie.cam",
"guardaserieicu": "https://guardaserie.icu",
"guardaserie_stream": "https://guardaserie.store",
"guardaserieclick": "https://www.guardaserie.fit",
"guardaseriecam": "https://guardaserie.cam",
"guardaserieclick": "https://www.guardaserie.guru",
"guardaserieicu": "https://guardaserie.icu",
"hd4me": "https://hd4me.net",
"ilgeniodellostreaming": "https://ilgeniodellostreaming.gy",
"ilgeniodellostreaming": "https://ilgeniodellostreaming.lc",
"ilgeniodellostreaming_cam": "https://ilgeniodellostreaming.cam",
"italiaserie": "https://italiaserie.org",
"mondoserietv": "https://mondoserietv.com",
"netfreex": "https://www.netfreex.top",
"piratestreaming": "https://www.piratestreaming.movie",
"piratestreaming": "https://www.piratestreaming.cloud",
"polpotv": "https://polpotv.life",
"raiplay": "https://www.raiplay.it",
"seriehd": "https://seriehd.productions",
"serietvonline": "https://serietvonline.press",
"seriehd": "https://seriehd.group",
"serietvonline": "https://serietvonline.pw",
"serietvsubita": "http://serietvsubita.xyz",
"serietvu": "https://www.serietvu.link",
"streamingcommunity":"https://streamingcommunity.to",
"streamingcommunity": "https://streamingcommunity.to",
"streamtime": "https://t.me/s/StreamTime",
"tantifilm": "https://www.tantifilm.rest",
"tantifilm": "https://www.tantifilm.fit",
"toonitalia": "https://toonitalia.org",
"vedohd": "https://vedohd.uno",
"vvvvid": "https://www.vvvvid.it"
}
+11 -11
View File
@@ -76,7 +76,7 @@ headers = [['Referer', host]]
@support.menu
def mainlist(item):
support.log(item)
support.info(item)
# Ordine delle voci
# Voce FILM, puoi solo impostare l'url
@@ -147,7 +147,7 @@ def mainlist(item):
# AVVERTENZE: Se il titolo è trovato nella ricerca TMDB/TVDB/Altro allora le locandine e altre info non saranno quelle recuperate nel sito.!!!!
@support.scrape
def peliculas(item):
support.log(item)
support.info(item)
#support.dbg() # decommentare per attivare web_pdb
action = ''
@@ -162,7 +162,7 @@ def peliculas(item):
@support.scrape
def episodios(item):
support.log(item)
support.info(item)
#support.dbg()
action = ''
@@ -179,7 +179,7 @@ def episodios(item):
# per genere, per anno, per lettera, per qualità ecc ecc
@support.scrape
def genres(item):
support.log(item)
support.info(item)
#support.dbg()
action = ''
@@ -199,7 +199,7 @@ def genres(item):
# e la ricerca porta i titoli mischiati senza poterli distinguere tra loro
# andranno modificate anche le def peliculas e episodios ove occorre
def select(item):
support.log('select --->', item)
support.info('select --->', item)
#support.dbg()
data = httptools.downloadpage(item.url, headers=headers).data
# pulizia di data, in caso commentare le prossime 2 righe
@@ -207,7 +207,7 @@ def select(item):
data = re.sub(r'>\s+<', '> <', data)
block = scrapertools.find_single_match(data, r'')
if re.findall('', data, re.IGNORECASE):
support.log('select = ### è una serie ###')
support.info('select = ### è una serie ###')
return episodios(Item(channel=item.channel,
title=item.title,
fulltitle=item.fulltitle,
@@ -220,7 +220,7 @@ def select(item):
############## Fondo Pagina
# da adattare al canale
def search(item, text):
support.log('search', item)
support.info('search', item)
itemlist = []
text = text.replace(' ', '+')
item.url = host + '/index.php?do=search&story=%s&subaction=search' % (text)
@@ -233,7 +233,7 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
log('search log:', line)
info('search log:', line)
return []
@@ -241,7 +241,7 @@ def search(item, text):
# inserire newest solo se il sito ha la pagina con le ultime novità/aggiunte
# altrimenti NON inserirlo
def newest(categoria):
support.log('newest ->', categoria)
support.info('newest ->', categoria)
itemlist = []
item = Item()
try:
@@ -256,7 +256,7 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
support.log('newest log: ', {0}.format(line))
support.info('newest log: ', {0}.format(line))
return []
return itemlist
@@ -266,5 +266,5 @@ def newest(categoria):
# sia per i siti con hdpass
#support.server(item, data='', itemlist=[], headers='', AutoPlay=True, CheckLinks=True)
def findvideos(item):
support.log('findvideos ->', item)
support.info('findvideos ->', item)
return support.server(item, headers=headers)
+6 -6
View File
@@ -47,7 +47,7 @@ def mainlist(item):
@support.scrape
def peliculas(item):
support.log('peliculas',item)
support.info('peliculas', item)
## deflang = 'ITA'
action="findvideos"
@@ -69,7 +69,7 @@ def peliculas(item):
@support.scrape
def genres(item):
support.log('genres',item)
support.info('genres',item)
if item.args != 'orderalf': action = "peliculas"
else: action = 'orderalf'
@@ -90,7 +90,7 @@ def genres(item):
@support.scrape
def orderalf(item):
support.log('orderalf',item)
support.info('orderalf',item)
action= 'findvideos'
patron = r'<td class="mlnh-thumb"><a href="(?P<url>[^"]+)".*?src="(?P<thumb>[^"]+)"'\
@@ -102,7 +102,7 @@ def orderalf(item):
def search(item, text):
support.log(item, text)
support.info(item, text)
itemlist = []
@@ -119,7 +119,7 @@ def search(item, text):
return []
def newest(categoria):
support.log(categoria)
support.info(categoria)
itemlist = []
item = Item()
@@ -141,5 +141,5 @@ def newest(categoria):
return itemlist
def findvideos(item):
support.log('findvideos', item)
support.info('findvideos', item)
return support.server(item, headers=headers)
+6 -6
View File
@@ -15,7 +15,7 @@ headers = [['Referer', host]]
# =========== home menu ===================
@support.menu
def mainlist(item):
support.log('mainlist',item)
support.info('mainlist',item)
film = [
('Al Cinema', ['/film-del-cinema', 'peliculas', '']),
@@ -33,7 +33,7 @@ def mainlist(item):
@support.scrape
def peliculas(item):
# debug = True
support.log('peliculas',item)
support.info('peliculas',item)
patron = r'<a href="(?P<url>[^"]+)">(?P<title>[^<]+)[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>\s*<div class="[^"]+" style="background-image:url\((?P<thumb>[^\)]+)[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>\s*(?P<year>\d{4})[^>]+>[^>]+>(?:\s*(?P<duration>\d+)[^>]+>[^>]+>)?\s*(?P<quality>[a-zA-Z]+) [^>]+>[^>]+> (?P<lang>[^>]+) [^>]+>'
patronNext = r'<span>\d</span> <a href="([^"]+)">'
@@ -43,7 +43,7 @@ def peliculas(item):
# =========== def pagina categorie ======================================
@support.scrape
def genres(item):
support.log('genres',item)
support.info('genres',item)
action = 'peliculas'
if item.args == 'genres':
@@ -63,7 +63,7 @@ def genres(item):
# =========== def per cercare film/serietv =============
#host+/index.php?do=search&story=avatar&subaction=search
def search(item, text):
support.log('search', item)
support.info('search', item)
itemlist = []
text = text.replace(" ", "+")
item.url = host+"/index.php?do=search&story=%s&subaction=search" % (text)
@@ -79,7 +79,7 @@ def search(item, text):
# =========== def per le novità nel menu principale =============
def newest(categoria):
support.log('newest', categoria)
support.info('newest', categoria)
itemlist = []
item = Item()
try:
@@ -101,5 +101,5 @@ def newest(categoria):
return itemlist
def findvideos(item):
support.log('findvideos', item)
support.info('findvideos', item)
return support.server(item, support.match(item, patron='<ul class="playernav">.*?</ul>', headers=headers).match)
+3 -3
View File
@@ -96,7 +96,7 @@ def genres(item):
return locals()
def search(item, texto):
support.log("search ", texto)
support.info("search ", texto)
item.args = 'search'
item.url = host + "/?s=" + texto
@@ -110,7 +110,7 @@ def search(item, texto):
return []
def newest(categoria):
support.log(categoria)
support.info(categoria)
itemlist = []
item = Item()
try:
@@ -135,7 +135,7 @@ def newest(categoria):
return itemlist
def findvideos(item):
support.log('findvideos', item)
support.info('findvideos', item)
return support.hdpass_get_servers(item)
def play(item):
+5 -6
View File
@@ -32,7 +32,7 @@ def menu(item):
def search(item, texto):
support.log(texto)
support.info(texto)
item.search = texto
try:
return peliculas(item)
@@ -45,7 +45,7 @@ def search(item, texto):
def newest(categoria):
support.log(categoria)
support.info(categoria)
item = support.Item()
try:
if categoria == "anime":
@@ -78,11 +78,10 @@ def peliculas(item):
query='category_name'
searchtext = item.url.split('/')[-2]
if not item.pag: item.pag = 1
anime=True
# debug = True
anime = True
data = support.match(host + '/wp-admin/admin-ajax.php', post='action=itajax-sort&loop=main+loop&location=&thumbnail=1&rating=1sorter=recent&columns=4&numarticles='+perpage+'&paginated='+str(item.pag)+'&currentquery%5B'+query+'%5D='+searchtext).data.replace('\\','')
patron=r'<a href="(?P<url>[^"]+)"><img width="[^"]+" height="[^"]+" src="(?P<thumb>[^"]+)" class="[^"]+" alt="" title="(?P<title>.*?)\s+(?P<type>Movie)?\s*(?P<lang>Sub Ita|Ita)'
patron = '<a href="(?P<url>[^"]+)"><img width="[^"]+" height="[^"]+" src="(?P<thumb>[^"]+)" class="[^"]+" alt="" title="(?P<title>.*?)\s+(?P<type>Movie)?\s*(?P<lang>Sub Ita|Ita)?\s*[sS]treaming'
typeContentDict = {'movie':['movie']}
typeActionDict = {'findvideos':['movie']}
+4 -4
View File
@@ -36,7 +36,7 @@ def submenu(item):
def newest(categoria):
support.log(categoria)
support.info(categoria)
itemlist = []
item = support.Item()
try:
@@ -55,7 +55,7 @@ def newest(categoria):
return itemlist
def search(item, texto):
support.log(texto)
support.info(texto)
item.args = 'noorder'
item.url = host + '/?s=' + texto + '&cat=6010'
item.contentType = 'tvshow'
@@ -114,7 +114,7 @@ def episodios(item):
else:
patron = r'<a\s*href="(?P<url>[^"]+)"\s*title="(?P<title>[^"]+)"\s*class="btn btn-dark mb-1">'
def itemHook(item):
support.log(item)
support.info(item)
if item.url.startswith('//'): item.url= 'https:' + item.url
elif item.url.startswith('/'): item.url= 'https:/' + item.url
return item
@@ -123,7 +123,7 @@ def episodios(item):
def findvideos(item):
support.log(item)
support.info(item)
itemlist = []
if 'adf.ly' in item.url:
+3 -3
View File
@@ -31,7 +31,7 @@ def mainlist(item):
def search(item, texto):
support.log(texto)
support.info(texto)
item.url = host + "/?s=" + texto
try:
@@ -83,7 +83,7 @@ def peliculas(item):
def episodios(item):
data = support.match(item, headers=headers, patronBlock=r'entry-content clearfix">(.*?)class="mh-widget mh-posts-2 widget_text').block
if not 'pagination clearfix' in data:
support.log('NOT IN DATA')
support.info('NOT IN DATA')
patron = r'<iframe.*?src="(?P<url>[^"]+)"'
title = item.title
def fullItemlistHook(itemlist):
@@ -124,7 +124,7 @@ def check(item):
return data
def findvideos(item):
support.log()
support.info()
if item.data:
data = item.data
else:
+39 -50
View File
@@ -10,50 +10,12 @@ from platformcode import config
host = support.config.get_channel_url()
headers={'X-Requested-With': 'XMLHttpRequest'}
def get_data(item, head=[]):
global headers
jstr = ''
for h in head:
headers[h[0]] = h[1]
if not item.count: item.count = 0
if not config.get_setting('key', item.channel):
matches = support.match(item, patron=r'<script>(.*?location.href=".*?(http[^"]+)";)</').match
if matches:
jstr, location = matches
item.url=support.re.sub(r':\d+', '', location).replace('http://','https://')
if jstr:
jshe = 'var document = {}, location = {}'
aesjs = str(support.match(host + '/aes.min.js').data)
js_fix = 'window.toHex = window.toHex || function(){for(var d=[],d=1==arguments.length&&arguments[0].constructor==Array?arguments[0]:arguments,e="",f=0;f<d.length;f++)e+=(16>d[f]?"0":"")+d[f].toString(16);return e.toLowerCase()}'
jsret = 'return document.cookie'
key_data = js2py.eval_js( 'function (){ ' + jshe + '\n' + aesjs + '\n' + js_fix + '\n' + jstr + '\n' + jsret + '}' )()
key = key_data.split(';')[0]
# save Key in settings
config.set_setting('key', key, item.channel)
# set cookie
headers['cookie'] = config.get_setting('key', item.channel)
res = support.match(item, headers=headers, patron=r';\s*location.href=".*?(http[^"]+)"')
if res.match:
item.url= res.match.replace('http://','https://')
data = support.match(item, headers=headers).data
else:
data = res.data
#check that the key is still valid
if 'document.cookie=' in data and item.count < 3:
item.count += 1
config.set_setting('key', '', item.channel)
return get_data(item)
return data
@support.menu
def mainlist(item):
anime = ['/animelist?load_all=1&d=1',
('ITA',['', 'submenu', '/filter?language%5B0%5D=1']),
('SUB-ITA',['', 'submenu', '/filter?language%5B0%5D=0']),
('Più Votati',['/toplist','menu', 'top']),
('In Corso',['/animeincorso','peliculas','incorso']),
('Ultimi Episodi',['/fetch_pages.php?request=episodes&d=1','peliculas','updated'])]
@@ -62,7 +24,7 @@ def mainlist(item):
def search(item, texto):
support.log(texto)
support.info(texto)
item.url = host + '/animelist?search=' + texto
item.contentType = 'tvshow'
try:
@@ -76,7 +38,7 @@ def search(item, texto):
def newest(categoria):
support.log()
support.info()
itemlist = []
item = support.Item()
try:
@@ -94,9 +56,28 @@ def newest(categoria):
return itemlist
@support.scrape
def submenu(item):
data = support.match(item.url + item.args).data
action = 'filter'
patronMenu = r'<h5 class="[^"]+">(?P<title>[^<]+)[^>]+>[^>]+><select id="(?P<parameter>[^"]+)"[^>]+>(?P<url>.*?)</select>'
def itemlistHook(itemlist):
itemlist.insert(0, item.clone(title=support.typo('Tutti','bold'), url=item.url + item.args, action='peliculas'))
return itemlist[:-1]
return locals()
def filter(item):
itemlist = []
matches = support.match(item.url, patron=r'<option value="(?P<value>[^"]+)"[^>]*>(?P<title>[^<]+)').matches
for value, title in matches:
itemlist.append(item.clone(title= support.typo(title,'bold'), url='{}{}&{}%5B0%5D={}'.format(host, item.args, item.parameter, value), action='peliculas', args='filter'))
support.thumb(itemlist, genre=True)
return itemlist
@support.scrape
def menu(item):
data=get_data(item)
patronMenu = r'<div class="col-md-13 bg-dark-as-box-shadow p-2 text-white text-center">(?P<title>[^"<]+)<(?P<other>.*?)(?:"lista-top"|"clearfix")'
action = 'peliculas'
item.args = 'top'
@@ -109,12 +90,10 @@ def menu(item):
@support.scrape
def peliculas(item):
data = get_data(item)
anime = True
deflang= 'Sub-ITA'
action = 'check'
page = None
post = "page=" + str(item.page if item.page else 1) if item.page > 1 else None
@@ -124,7 +103,7 @@ def peliculas(item):
else:
data = support.match(item, post=post, headers=headers).data
if item.args == 'updated':
page= support.match(data, patron=r'data-page="(\d+)" title="Next">').match
page = support.match(data, patron=r'data-page="(\d+)" title="Next">').match
patron = r'<a href="(?P<url>[^"]+)" title="(?P<title>[^"(]+)(?:\s*\((?P<year>\d+)\))?(?:\s*\((?P<lang>[A-Za-z-]+)\))?"><img src="(?P<thumb>[^"]+)"[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>\s\s*(?P<type>[^\s]+)\s*(?P<episode>\d+)'
typeContentDict = {'Movie':'movie', 'Episodio':'episode'} #item.contentType='episode'
action = 'findvideos'
@@ -132,18 +111,29 @@ def peliculas(item):
if page:
itemlist.append(item.clone(title=support.typo(support.config.get_localized_string(30992), 'color kod bold'),page= page, thumbnail=support.thumb()))
return itemlist
elif 'filter' in item.args:
page = support.match(data, patron=r'totalPages:\s*(\d+)').match
patron = r'<a href="(?P<url>[^"]+)" title="(?P<title>[^"(]+)(?:\s*\((?P<year>\d+)\))?(?:\s*\((?P<lang>[A-Za-z-]+)\))?"><img src="(?P<thumb>[^"]+)"'
def itemlistHook(itemlist):
if item.nextpage: item.nextpage += 1
else: item.nextpage = 2
if page and item.nextpage < int(page):
itemlist.append(item.clone(title=support.typo(support.config.get_localized_string(30992), 'color kod bold'), url= '{}&page={}'.format(item.url, item.nextpage), infoLabels={}, thumbnail=support.thumb()))
return itemlist
else:
pagination = ''
if item.args == 'incorso':
patron = r'<a href="(?P<url>[^"]+)"[^>]+>(?P<title>[^<(]+)(?:\s*\((?P<year>\d+)\))?(?:\s*\((?P<lang>[A-za-z-]+)\))?</a>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>\s*<img width="[^"]+" height="[^"]+" src="(?P<thumb>[^"]+)"[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>(?P<plot>[^<]+)<'
else:
# debug=True
patron = r'<img src="(?P<thumb>[^"]+)" alt="(?P<title>[^"\(]+)(?:\((?P<lang>[Ii][Tt][Aa])\))?(?:\s*\((?P<year>\d+)\))?[^"]*"[^>]+>[^>]+>[^>]+>[^>]+>[^>]+><a class="[^"]+" href="(?P<url>[^"]+)">[^>]+>[^>]+>[^>]+><p[^>]+>(?:(?P<plot>[^<]+))?<'
return locals()
def check(item):
movie = support.match(get_data(item), patron=r'Episodi:</b> (\d*) Movie')
movie = support.match(item, patron=r'Episodi:</b> (\d*) Movie')
if movie.match:
episodes = episodios(item)
if len(episodes) > 0:
@@ -156,18 +146,17 @@ def check(item):
@support.scrape
def episodios(item):
data = get_data(item)
if item.contentType != 'movie': anime = True
patron = r'episodi-link-button">\s*<a href="(?P<url>[^"]+)"[^>]+>\s*(?P<title>[^<]+)</a>'
return locals()
def findvideos(item):
support.log()
support.info()
itemlist = []
page_data = ''
titles =['Primario', 'Secondario', 'Alternativo Primario', 'Alternativo Secondario']
url = support.match(get_data(item), patron=r'<a href="([^"]+)">[^>]+>[^>]+>G', headers=headers).match
url = support.match(item, patron=r'<a href="([^"]+)">[^>]+>[^>]+>G', headers=headers).match
urls = [url, url+'&extra=1', url+'&s=alt', url+'&s=alt&extra=1']
links = []
for i, url in enumerate(urls):
+13 -13
View File
@@ -5,7 +5,7 @@
import requests, json, copy
from core import support
from specials import autorenumber
from platformcode import autorenumber
try: from lib import cloudscraper
except: from lib import cloudscraper
@@ -46,12 +46,12 @@ def menu(item):
if item.contentType == 'tvshow':
itemlist += [item.clone(title=support.typo('In Corso','bold'), args=InCorso),
item.clone(title=support.typo('Terminato','bold'), args=Terminato)]
itemlist +=[item.clone(title=support.typo('Cerca...','bold'), action='search', thumbnail=support.thumb(thumb='search.png'))]
itemlist +=[item.clone(title=support.typo('Cerca...','bold'), action='search', thumbnail=support.thumb('search'))]
return itemlist
def genres(item):
support.log()
support.info()
# support.dbg()
itemlist = []
@@ -63,7 +63,7 @@ def genres(item):
return support.thumb(itemlist)
def years(item):
support.log()
support.info()
itemlist = []
from datetime import datetime
@@ -77,7 +77,7 @@ def years(item):
def search(item, text):
support.log('search', item)
support.info('search', item)
if not item.args:
item.args = {'title':text}
else:
@@ -90,12 +90,12 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
support.log('search log:', line)
support.info('search log:', line)
return []
def newest(categoria):
support.log(categoria)
support.info(categoria)
itemlist = []
item = support.Item()
item.url = host
@@ -109,13 +109,13 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
support.log(line)
support.info(line)
return []
return itemlist
def news(item):
support.log()
support.info()
item.contentType = 'tvshow'
itemlist = []
@@ -140,7 +140,7 @@ def news(item):
def peliculas(item):
support.log()
support.info()
itemlist = []
page = item.page if item.page else 0
@@ -154,7 +154,7 @@ def peliculas(item):
payload = json.dumps(item.args)
records = requests.post(host + '/archivio/get-animes', headers=headers, data=payload).json()['records']
# js = []
# support.log(records)
# support.info(records)
# for record in records:
# js += record
for it in records:
@@ -194,7 +194,7 @@ def peliculas(item):
return itemlist
def episodios(item):
support.log()
support.info()
itemlist = []
title = 'Parte ' if item.type.lower() == 'movie' else 'Episodio '
for it in item.episodes:
@@ -219,5 +219,5 @@ def episodios(item):
def findvideos(item):
support.log()
support.info()
return support.server(item,itemlist=[item.clone(title=support.config.get_localized_string(30137), server='directo', action='play')])
+2 -2
View File
@@ -33,7 +33,7 @@ def menu(item):
def search(item, texto):
support.log(texto)
support.info(texto)
item.search = texto
try:
return peliculas(item)
@@ -46,7 +46,7 @@ def search(item, texto):
def newest(categoria):
support.log(categoria)
support.info(categoria)
item = support.Item()
try:
if categoria == "anime":
+48 -46
View File
@@ -18,44 +18,44 @@ def order():
# Seleziona l'ordinamento dei risultati
return str(support.config.get_setting("order", __channel__))
def get_data(item, head=[]):
global headers
jstr = ''
for h in head:
headers[h[0]] = h[1]
if not item.count: item.count = 0
if not config.get_setting('key', item.channel):
matches = support.match(item, patron=r'<script>(.*?location.href=".*?(http[^"]+)";)</').match
if matches:
jstr, location = matches
item.url=support.re.sub(r':\d+', '', location).replace('http://','https://')
if jstr:
jshe = 'var document = {}, location = {}'
aesjs = str(support.match(host + '/aes.min.js').data)
js_fix = 'window.toHex = window.toHex || function(){for(var d=[],d=1==arguments.length&&arguments[0].constructor==Array?arguments[0]:arguments,e="",f=0;f<d.length;f++)e+=(16>d[f]?"0":"")+d[f].toString(16);return e.toLowerCase()}'
jsret = 'return document.cookie'
key_data = js2py.eval_js( 'function (){ ' + jshe + '\n' + aesjs + '\n' + js_fix + '\n' + jstr + '\n' + jsret + '}' )()
key = key_data.split(';')[0]
# def get_data(item, head=[]):
# global headers
# jstr = ''
# for h in head:
# headers[h[0]] = h[1]
# if not item.count: item.count = 0
# if not config.get_setting('key', item.channel):
# matches = support.match(item, patron=r'<script>(.*?location.href=".*?(http[^"]+)";)</').match
# if matches:
# jstr, location = matches
# item.url=support.re.sub(r':\d+', '', location).replace('http://','https://')
# if jstr:
# jshe = 'var document = {}, location = {}'
# aesjs = str(support.match(host + '/aes.min.js').data)
# js_fix = 'window.toHex = window.toHex || function(){for(var d=[],d=1==arguments.length&&arguments[0].constructor==Array?arguments[0]:arguments,e="",f=0;f<d.length;f++)e+=(16>d[f]?"0":"")+d[f].toString(16);return e.toLowerCase()}'
# jsret = 'return document.cookie'
# key_data = js2py.eval_js( 'function (){ ' + jshe + '\n' + aesjs + '\n' + js_fix + '\n' + jstr + '\n' + jsret + '}' )()
# key = key_data.split(';')[0]
# save Key in settings
config.set_setting('key', key, item.channel)
# # save Key in settings
# config.set_setting('key', key, item.channel)
# set cookie
headers['cookie'] = config.get_setting('key', item.channel)
res = support.match(item, headers=headers, patron=r';\s*location.href=".*?(http[^"]+)"')
if res.match:
item.url= res.match.replace('http://','https://')
data = support.match(item, headers=headers).data
else:
data = res.data
# # set cookie
# headers['cookie'] = config.get_setting('key', item.channel)
# res = support.match(item, headers=headers, patron=r';\s*location.href=".*?(http[^"]+)"')
# if res.match:
# item.url= res.match.replace('http://','https://')
# data = support.match(item, headers=headers).data
# else:
# data = res.data
#check that the key is still valid
if 'document.cookie=' in data and item.count < 3:
item.count += 1
config.set_setting('key', '', item.channel)
return get_data(item)
return data
# #check that the key is still valid
# if 'document.cookie=' in data and item.count < 3:
# item.count += 1
# config.set_setting('key', '', item.channel)
# return get_data(item)
# return data
@support.menu
@@ -73,7 +73,7 @@ def mainlist(item):
@support.scrape
def genres(item):
action = 'peliculas'
data = get_data(item)
# data = get_data(item)
patronBlock = r'dropdown[^>]*>\s*Generi\s*<span.[^>]+>(?P<block>.*?)</ul>'
patronMenu = r'<input.*?name="(?P<name>[^"]+)" value="(?P<value>[^"]+)"\s*>[^>]+>(?P<title>[^<]+)</label>'
@@ -86,11 +86,11 @@ def genres(item):
@support.scrape
def menu(item):
action = 'submenu'
data = get_data(item)
# data = get_data(item)
patronMenu=r'<button[^>]+>\s*(?P<title>[A-Za-z0-9]+)\s*<span.[^>]+>(?P<other>.*?)</ul>'
def ItemItemlistHook(item, itemlist):
itemlist.insert(0, item.clone(title=support.typo('Tutti','bold'), action='peliculas'))
itemlist.append(item.clone(title=support.typo('Cerca...','bold'), action='search', search=True, thumbnail=support.thumb(thumb='search.png')))
itemlist.append(item.clone(title=support.typo('Cerca...','bold'), action='search', search=True, thumbnail=support.thumb('search.png')))
return itemlist
return locals()
@@ -107,7 +107,7 @@ def submenu(item):
def newest(categoria):
support.log(categoria)
support.info(categoria)
item = support.Item()
try:
if categoria == "anime":
@@ -123,7 +123,7 @@ def newest(categoria):
def search(item, texto):
support.log(texto)
support.info(texto)
if item.search:
item.url = host + '/filter?dub=' + item.args + '&keyword=' + texto + '&sort='
else:
@@ -143,19 +143,20 @@ def search(item, texto):
@support.scrape
def peliculas(item):
anime=True
# debug =True
if item.args not in ['noorder', 'updated'] and not item.url[-1].isdigit(): item.url += order() # usa l'ordinamento di configura canale
data = get_data(item)
# data = get_data(item)
if item.args == 'updated':
data = get_data(item)
# data = get_data(item)
item.contentType='episode'
patron=r'<div class="inner">\s*<a href="(?P<url>[^"]+)" class[^>]+>\s*<img.*?src="(?P<thumb>[^"]+)" alt?="(?P<title>[^\("]+)(?:\((?P<lang>[^\)]+)\))?"[^>]+>[^>]+>\s*(?:<div class="[^"]+">(?P<type>[^<]+)</div>)?[^>]+>[^>]+>\s*<div class="ep">[^\d]+(?P<episode>\d+)[^<]*</div>'
action='findvideos'
else:
patron= r'<div class="inner">\s*<a href="(?P<url>[^"]+)" class[^>]+>\s*<img.*?src="(?P<thumb>[^"]+)" alt?="(?P<title>[^\("]+)(?:\((?P<year>\d+)\) )?(?:\((?P<lang>[^\)]+)\))?"[^>]+>[^>]+>(?:\s*<div class="(?P<l>[^"]+)">[^>]+>)?\s*(?:<div class="[^"]+">(?P<type>[^<]+)</div>)?'
patron= r'<div class="inner">\s*<a href="(?P<url>[^"]+)" class[^>]+>\s*<img.*?src="(?P<thumb>[^"]+)" alt?="(?P<title>[^\("]+)(?:\((?P<year>\d+)\) )?(?:\((?P<lang>[^\)]+)\))?(?P<title2>[^"]+)?[^>]+>[^>]+>(?:\s*<div class="(?P<l>[^"]+)">[^>]+>)?\s*(?:<div class="[^"]+">(?P<type>[^<]+)</div>)?'
action='episodios'
# Controlla la lingua se assente
patronNext=r'</span></a><a href="([^"]+)"'
patronNext=r'<a href="([^"]+)" class="[^"]+" id="go-next'
typeContentDict={'movie':['movie', 'special']}
typeActionDict={'findvideos':['movie', 'special']}
def itemHook(item):
@@ -174,7 +175,7 @@ def peliculas(item):
def episodios(item):
anime=True
pagination = 50
data = get_data(item)
# data = get_data(item)
patronBlock= r'<div class="server\s*active\s*"(?P<block>.*?)(?:<div class="server|<link)'
patron = r'<li[^>]*>\s*<a.*?href="(?P<url>[^"]+)"[^>]*>(?P<episode>[^<]+)<'
def itemHook(item):
@@ -187,10 +188,11 @@ def episodios(item):
def findvideos(item):
import time
support.log(item)
support.info(item)
itemlist = []
urls = []
resp = support.match(get_data(item), headers=headers, patron=r'data-name="(\d+)">([^<]+)<')
# resp = support.match(get_data(item), headers=headers, patron=r'data-name="(\d+)">([^<]+)<')
resp = support.match(item, headers=headers, patron=r'data-name="(\d+)">([^<]+)<')
data = resp.data
for ID, name in resp.matches:
if not item.number: item.number = support.match(item.title, patron=r'(\d+) -').match
+7 -7
View File
@@ -39,17 +39,17 @@ def genres(item):
def select(item):
item.data = support.match(item).data
if 'continua con il video' in item.data.lower():
support.log('select = ### è un film ###')
support.info('select = ### è un film ###')
item.contentType = 'movie'
return findvideos(item)
else:
support.log('select = ### è una serie ###')
support.info('select = ### è una serie ###')
item.contentType = 'tvshow'
return episodios(item)
def search(item, text):
support.log(text)
support.info(text)
text = text.replace(' ', '+')
item.url = host + '/?s=' + text
item.args = 'search'
@@ -59,7 +59,7 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
support.log('search log:', line)
support.info('search log:', line)
return []
@@ -87,7 +87,7 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
support.log("%s" % line)
support.info("%s" % line)
return []
return itemlist
@@ -107,7 +107,7 @@ def peliculas(item):
if item.args == 'newest':
patron = r'<li><a href="(?P<url>[^"]+)"[^=]+="(?P<thumb>[^"]+)"><div> <div[^>]+>(?P<title>[^\(\[<]+)(?:\[(?P<quality1>HD)\])?[ ]?(?:\(|\[)?(?P<lang>Sub-ITA)?(?:\)|\])?[ ]?(?:\[(?P<quality>.+?)\])?[ ]?(?:\((?P<year>\d+)\))?<(?:[^>]+>.+?(?:title="Nuovi episodi">(?P<episode>\d+x\d+)[ ]?(?P<lang2>Sub-Ita)?|title="IMDb">(?P<rating>[^<]+)))?'
else:
patron = r'<li><a href="(?P<url>[^"]+)"[^=]+="(?P<thumb>[^"]+)"><div> <div[^>]+>(?P<title>[^\(\[<]+)(?:\[(?P<quality1>HD)\])?[ ]?(?:\(|\[)?(?P<lang>Sub-ITA)?(?:\)|\])?[ ]?(?:\[(?P<quality>.+?)\])?[ ]?(?:\((?P<year>\d+)\))?<'
patron = r'<li><a href="(?P<url>[^"]+)"[^=]+="(?P<thumb>[^"]+)"><div> <div[^>]+>(?P<title>[^\(\[<]+)(?:\[(?P<quality1>HD)\])?[ ]?(?:\(|\[)?(?P<lang>Sub-ITA)?(?:\)|\])?[ ]?(?:\[(?P<quality>.+?)\])?[ ]?(?:\((?P<year>\d+)\))?'
patronNext = r'<a href="([^"]+)" >Pagina'
# debug = True
@@ -148,7 +148,7 @@ def findvideos(item):
data = ''
from lib.unshortenit import unshorten_only
for link in links:
support.log('URL=',link)
support.info('URL=',link)
url, c = unshorten_only(link.replace('#', 'speedvideo.net'))
data += url + '\n'
return support.server(item, data)
+3 -3
View File
@@ -35,12 +35,12 @@ def menu(item):
def search(item, texto):
support.log(texto)
support.info(texto)
item.url = host + "/?s=" + texto
return peliculas(item)
def newest(categoria):
support.log(categoria)
support.info(categoria)
itemlist = []
item = support.Item()
try:
@@ -80,7 +80,7 @@ def check(item):
@support.scrape
def episodios(item):
support.log('EPISODIOS ', item.data)
support.info('EPISODIOS ', item.data)
data = ''
matches = item.data
season = 1
+14 -12
View File
@@ -55,7 +55,7 @@ def menu(item):
def newest(categoria):
support.log(categoria)
support.info(categoria)
item = support.Item()
try:
@@ -71,15 +71,16 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
support.logger.error("{0}".format(line))
logger.error("{0}".format(line))
return []
def search(item, text):
support.log(item.url, "search", text)
logger.info(item, "search", text)
if item.contentType == 'tvshow': item.url = host + '/serietv/'
else: item.url = host
try:
item.url = item.url + "/?s=" + text.replace(' ', '+')
item.url = item.url + "?s=" + text.replace(' ', '+')
return peliculas(item)
# Continua la ricerca in caso di errore
@@ -100,6 +101,7 @@ def peliculas(item):
# debug= True
if 'newest' in item.args:
if '/serietv/' not in item.url:
# debug = True
pagination = ''
patronBlock = r'Ultimi 100 film [^:]+:(?P<block>.*?)<\/td>'
patron = r'<a href="?(?P<url>[^">]+)"?>(?P<title>[^<([]+)(?:\[(?P<lang>Sub-ITA|B/N|SUB-ITA)\])?\s*(?:\[(?P<quality>HD|SD|HD/3D)\])?\s*\((?P<year>[0-9]{4})\)<\/a>'
@@ -127,7 +129,7 @@ def peliculas(item):
def episodios(item):
# support.dbg()
data = support.match(item.url, headers=headers).data
support.log(data)
support.info(data)
if 'TUTTA LA ' in data:
folderUrl = scrapertools.find_single_match(data, r'TUTTA LA \w+\s+(?:&#8211;|-)\s+<a href="?([^" ]+)')
data = httptools.downloadpage(folderUrl).data
@@ -170,14 +172,14 @@ def findvideos(item):
def load_links(itemlist, re_txt, desc_txt, quality=""):
streaming = scrapertools.find_single_match(data, re_txt).replace('"', '')
support.log('STREAMING', streaming)
support.log('STREAMING=', streaming)
support.info('STREAMING', streaming)
support.info('STREAMING=', streaming)
matches = support.match(streaming, patron = r'<td><a.*?href=([^ ]+) [^>]+>([^<]+)<').matches
for scrapedurl, scrapedtitle in matches:
logger.debug("##### findvideos %s ## %s ## %s ##" % (desc_txt, scrapedurl, scrapedtitle))
itemlist.append(item.clone(action="play", title=scrapedtitle, url=scrapedurl, server=scrapedtitle, quality=quality))
support.log()
support.info()
itemlist = []
@@ -210,7 +212,7 @@ def findvideos(item):
def findvid_serie(item):
def load_vid_series(html, item, itemlist, blktxt):
support.log('HTML',html)
support.info('HTML',html)
# Estrae i contenuti
matches = support.match(html, patron=r'<a href=(?:")?([^ "]+)[^>]+>(?!<!--)(.*?)(?:</a>|<img)').matches
for url, server in matches:
@@ -218,7 +220,7 @@ def findvid_serie(item):
if 'swzz' in item.url: item.url = support.swzz_get_url(item)
itemlist.append(item)
support.log()
support.info()
itemlist = []
@@ -238,5 +240,5 @@ def findvid_serie(item):
def play(item):
support.log()
support.info()
return servertools.find_video_items(item, data=item.url)
+8 -8
View File
@@ -89,13 +89,13 @@ def episodios(item):
data=item.data
# debugBlock=True
if item.args == 'anime':
support.log("Anime :", item)
support.info("Anime :", item)
# blacklist = ['Clipwatching', 'Verystream', 'Easybytez', 'Flix555', 'Cloudvideo']
patron = r'<a target=(?P<url>[^>]+>(?P<title>Episodio\s(?P<episode>\d+))(?::)?(?:(?P<title2>[^<]+))?.*?(?:<br|</p))'
patronBlock = r'(?:Stagione (?P<season>\d+))?(?:</span><br />|</span></p>|strong></p>)(?P<block>.*?)(?:<div style="margin-left|<span class="txt_dow">)'
item.contentType = 'tvshow'
else:# item.extra == 'serie':
support.log("Serie :", item)
support.info("Serie :", item)
patron = r'(?:>| )(?P<episode>\d+(?:x|×|&#215;)\d+)[;]?[ ]?(?:(?P<title>[^<-]+)(?P<url>.*?)|(\2[ ])(?:<(\3.*?)))(?:</a><br /|</a></p|$)'
patronBlock = r'>(?:[^<]+[Ss]tagione\s|[Ss]tagione [Uu]nica)(?:(?P<lang>iTA|ITA|Sub-ITA|Sub-iTA))?.*?</strong>(?P<block>.+?)(?:<strong|<div class="at-below)'
item.contentType = 'tvshow'
@@ -118,7 +118,7 @@ def genres(item):
def search(item, texto):
support.log(item.url,texto)
support.info(item.url,texto)
texto = texto.replace(' ', '+')
item.url = host + "/?s=" + texto
# item.contentType = 'tv'
@@ -129,11 +129,11 @@ def search(item, texto):
except:
import sys
for line in sys.exc_info():
support.log("%s" % line)
support.info("%s" % line)
return []
def newest(categoria):
support.log('newest ->', categoria)
support.info('newest ->', categoria)
itemlist = []
item = Item()
item.args = 'newest'
@@ -147,13 +147,13 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
support.log('newest log: ', (line))
support.info('newest log: ', (line))
return []
return itemlist
def check(item):
support.log()
support.info()
data = support.match(item.url, headers=headers).data
if data:
blockAnime = support.match(data, patron=r'<div id="container" class="container">(.+?<div style="margin-left)').match
@@ -175,6 +175,6 @@ def check(item):
return findvideos(item)
def findvideos(item):
support.log()
support.info()
item.url = item.url.replace('http://rapidcrypt.net/verys/', '').replace('http://rapidcrypt.net/open/', '') #blocca la ricerca
return support.server(item, data= item.url)
+2 -2
View File
@@ -34,7 +34,7 @@ def menu(item):
def search(item, text):
support.log(text)
support.info(text)
item.args = 'noorder'
item.url = host + '/ricerca/type_ALL/ricerca_' + text
item.contentType = 'movie'
@@ -62,7 +62,7 @@ def peliculas(item):
def findvideos(item):
support.log()
support.info()
itemlist = []
matches = support.match(item, patron=r'filename: "(.*?)"').matches
+7 -7
View File
@@ -10,7 +10,7 @@ headers = [['Referer', host]]
@support.menu
def mainlist(item):
support.log(item)
support.info(item)
anime = ['/search?typeY=tv',
('Movie', ['/search?typeY=movie', 'peliculas', '', 'movie']),
@@ -34,7 +34,7 @@ def menu(item):
patronMenu = r'<a class="[^"]+" data-state="[^"]+" (?P<other>[^>]+)>[^>]+></i>[^>]+></i>[^>]+></i>(?P<title>[^>]+)</a>'
def itemHook(item):
support.log(item.type)
support.info(item.type)
for Type, ID in support.match(item.other, patron=r'data-type="([^"]+)" data-id="([^"]+)"').matches:
item.url = host + '/search?' + Type + 'Y=' + ID
return item
@@ -42,7 +42,7 @@ def menu(item):
def search(item, text):
support.log(text)
support.info(text)
text = text.replace(' ', '+')
item.url = host + '/search/' + text
@@ -53,12 +53,12 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
support.log('search log:', line)
support.info('search log:', line)
return []
def newest(categoria):
support.log(categoria)
support.info(categoria)
item = support.Item()
try:
if categoria == "anime":
@@ -110,7 +110,7 @@ def episodios(item):
def findvideos(item):
itemlist = []
support.log()
support.info()
# support.dbg()
matches = support.match(item, patron=r'href="([^"]+)"', patronBlock=r'<div style="white-space: (.*?)<div id="main-content"')
@@ -124,7 +124,7 @@ def findvideos(item):
itemlist.append(item.clone(action="play", title='VVVVID', url=support.match(matches.data, patron=r'(http://www.vvvvid[^"]+)').match, server='vvvvid'))
else:
# matches.matches.sort()
support.log('VIDEO')
support.info('VIDEO')
for url in matches.matches:
lang = url.split('/')[-2]
if 'ita' in lang.lower():
+3 -3
View File
@@ -33,7 +33,7 @@ def menu(item):
return locals()
def newest(categoria):
support.log()
support.info()
item = Item()
try:
if categoria == "documentales":
@@ -50,7 +50,7 @@ def newest(categoria):
def search(item, texto):
support.log(texto)
support.info(texto)
item.url = host + "/?s=" + texto
try:
return peliculas(item)
@@ -133,7 +133,7 @@ def episodios(item):
def findvideos(item):
support.log()
support.info()
if item.args == 'raccolta' or item.contentType == 'episode':
return support.server(item, item.url)
else:
+6 -6
View File
@@ -17,7 +17,7 @@ headers = [['Referer', host]]
@support.menu
def mainlist(item):
support.log()
support.info()
tvshow = []
anime = ['/category/anime-cartoni-animati/']
mix = [('Aggiornamenti {bullet bold} {TV}', ['/aggiornamento-episodi/', 'peliculas', 'newest']),
@@ -57,7 +57,7 @@ def episodios(item):
def search(item, texto):
support.log()
support.info()
item.url = "%s/?s=%s" % (host, texto)
item.contentType = 'tvshow'
@@ -69,12 +69,12 @@ def search(item, texto):
except:
import sys
for line in sys.exc_info():
support.log(line)
support.info(line)
return []
def newest(categoria):
support.log()
support.info()
itemlist = []
item = Item()
@@ -88,12 +88,12 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
support.log("{0}".format(line))
support.info("{0}".format(line))
return []
return itemlist
def findvideos(item):
support.log()
support.info()
return support.server(item, item.url)
+9 -9
View File
@@ -18,7 +18,7 @@
"""
from core import support, httptools, scrapertools
from core.item import Item
from core.support import log
from core.support import info
from platformcode import config
host = config.get_channel_url()
@@ -42,7 +42,7 @@ def mainlist(item):
@support.scrape
def peliculas(item):
support.log(item)
support.info(item)
# support.dbg()
deflang = 'Sub-ITA'
@@ -67,7 +67,7 @@ def peliculas(item):
@support.scrape
def episodios(item):
support.log(item)
support.info(item)
#support.dbg()
deflang = 'Sub-ITA'
@@ -82,7 +82,7 @@ def episodios(item):
@support.scrape
def genres(item):
support.log()
support.info()
#support.dbg()
action = 'peliculas'
@@ -99,7 +99,7 @@ def genres(item):
def search(item, text):
support.log('search', item)
support.info('search', item)
text = text.replace(' ', '+')
item.url = host + '?s=' + text
try:
@@ -110,12 +110,12 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
log('search log:', line)
info('search log:', line)
return []
def newest(categoria):
support.log('newest ->', categoria)
support.info('newest ->', categoria)
itemlist = []
item = Item()
if categoria == 'series':
@@ -132,14 +132,14 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
support.log('newest log: ', line)
support.info('newest log: ', line)
return []
return itemlist
def findvideos(item):
support.log('findvideos ->', item)
support.info('findvideos ->', item)
itemlist = []
patronBlock = '<div class="entry-content">(?P<block>.*)<footer class="entry-footer">'
patron = r'<a href="([^"]+)">'
+8 -8
View File
@@ -41,7 +41,7 @@ def mainlist(item):
@support.scrape
def peliculas(item):
support.log()
support.info()
if item.args == 'search':
action = ''
@@ -88,7 +88,7 @@ def peliculas(item):
@support.scrape
def episodios(item):
support.log()
support.info()
action = 'findvideos'
patronBlock = r'<div class="row">(?P<block>.*?)<section class="main-content">'
@@ -98,7 +98,7 @@ def episodios(item):
@support.scrape
def genres(item):
support.log()
support.info()
if item.contentType == 'movie':
action = 'peliculas'
@@ -115,7 +115,7 @@ def genres(item):
def search(item, text):
support.log('search', item)
support.info('search', item)
text = text.replace(' ', '+')
item.url = host + '/search/?s=' + text
@@ -126,11 +126,11 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
support.log('search log:', line)
support.info('search log:', line)
return []
def newest(categoria):
support.log('newest ->', categoria)
support.info('newest ->', categoria)
itemlist = []
item = Item()
try:
@@ -146,11 +146,11 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
support.log({0}.format(line))
support.info({0}.format(line))
return []
return itemlist
def findvideos(item):
support.log()
support.info()
return support.server(item)
+8 -8
View File
@@ -33,7 +33,7 @@ def mainlist(item):
@support.scrape
def peliculas(item):
support.log()
support.info()
if item.args != 'newest':
patronBlock = r'<ul class="posts">(?P<block>.*)<\/ul>'
@@ -94,21 +94,21 @@ def genres(item):
def select(item):
support.log()
support.info()
patron=r'class="taxonomy category" ><span property="name">([^>]+)</span></a><meta property="position" content="2">'
block = support.match(item.url, patron=patron,headers=headers).match
if block.lower() != 'film':
support.log('select = ### è una serie ###')
support.info('select = ### è una serie ###')
item.contentType='tvshow'
return episodios(item)
else:
support.log('select = ### è un movie ###')
support.info('select = ### è un movie ###')
item.contentType='movie'
return findvideos(item)
def search(item, texto):
support.log()
support.info()
item.url = host + "/?s=" + texto
item.contentType = 'episode'
item.args = 'search'
@@ -118,12 +118,12 @@ def search(item, texto):
except:
import sys
for line in sys.exc_info():
support.log("%s" % line)
support.info("%s" % line)
return []
def newest(categoria):
support.log()
support.info()
itemlist = []
item = Item()
try:
@@ -144,7 +144,7 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
support.log("{0}".format(line))
support.info("{0}".format(line))
return []
return itemlist
+3 -3
View File
@@ -11,7 +11,7 @@
# possibilità di miglioramento: inserire menu per genere - lista serie tv e gestire le novità
from core import support
from core.support import log
from core.support import info
from platformcode import logger, config
host = config.get_channel_url()
@@ -51,7 +51,7 @@ def episodios(item):
def search(item, text):
support.log('search', item)
support.info('search', item)
item.contentType = 'tvshow'
itemlist = []
text = text.replace(' ', '+')
@@ -61,7 +61,7 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
log('search log:', line)
info('search log:', line)
return []
+10 -10
View File
@@ -17,7 +17,7 @@
from core import support
from core.item import Item
from platformcode import config
from core.support import log
from core.support import info
host = config.get_channel_url()
headers = [['Referer', host]]
@@ -40,7 +40,7 @@ def mainlist(item):
##@support.scrape
##def peliculas(item):
#### import web_pdb; web_pdb.set_trace()
## log('peliculas ->\n', item)
## info('peliculas ->\n', item)
##
## action = 'episodios'
## block = r'(?P<block>.*?)<div\s+class="btn btn-lg btn-default btn-load-other-series">'
@@ -77,7 +77,7 @@ def mainlist(item):
@support.scrape
def peliculas(item):
## import web_pdb; web_pdb.set_trace()
log('peliculas ->\n', item)
info('peliculas ->\n', item)
action = 'episodios'
blacklist = ['DMCA']
@@ -122,7 +122,7 @@ def peliculas(item):
@support.scrape
def episodios(item):
log()
info()
action = 'findvideos'
patron = r'<div class="number-episodes-on-img">\s?\d+.\d+\s?(?:\((?P<lang>[a-zA-Z\-]+)\))?</div>.+?(?:<span class="pull-left bottom-year">(?P<title2>[^<]+)<[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>(?P<plot>[^<]+)<[^>]+>[^>]+>[^>]+>\s?)?<span(?: meta-nextep="[^"]+")? class="[^"]+" meta-serie="(?P<title>[^"]+)" meta-stag="(?P<season>\d+)" meta-ep="(?P<episode>\d+)" meta-embed="(?P<url>[^>]+)">'
@@ -137,7 +137,7 @@ def episodios(item):
@support.scrape
def genres(item):
log()
info()
action = 'peliculas'
patron = r'<li>\s<a\shref="(?P<url>[^"]+)"[^>]+>(?P<title>[^<]+)</a></li>'
@@ -148,7 +148,7 @@ def genres(item):
def search(item, text):
log(text)
info(text)
item.url = host + "/?s=" + text
item.contentType = 'tvshow'
item.args = 'search'
@@ -158,11 +158,11 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
log("%s" % line)
info("%s" % line)
return []
def newest(categoria):
log()
info()
itemlist = []
item = Item()
item.contentType= 'tvshow'
@@ -177,12 +177,12 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
log("{0}".format(line))
info("{0}".format(line))
return []
return itemlist
def findvideos(item):
log('--->', item)
info('--->', item)
return support.server(item, item.url)
+4 -4
View File
@@ -10,7 +10,7 @@
# possibilità di miglioramento: gestire le novità (sezione Ultimi episodi sul sito)
from core.support import log
from core.support import info
from core import support
from platformcode import config
@@ -40,7 +40,7 @@ def episodios(item):
return locals()
def search(item, text):
log(text)
info(text)
itemlist = []
text = text.replace(' ', '+')
item.url = host + "/?s=" + text
@@ -50,10 +50,10 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
log("%s" % line)
info("%s" % line)
return []
def findvideos(item):
support.log('findvideos', item)
support.info('findvideos', item)
return support.server(item, headers=headers)
+1 -1
View File
@@ -42,7 +42,7 @@ def genre(item):
def search(item, text):
support.log(text)
support.info(text)
item.url = host + '/?s=' + text
try:
return peliculas(item)
+2 -2
View File
@@ -11,7 +11,7 @@ def findhost():
return url[:-1] if url.endswith('/') else url
host = support.config.get_channel_url(findhost)
support.log('HOST',host)
support.info('HOST',host)
# host = 'https://ilcorsaronero.xyz'
headers = [['Referer', host]]
@@ -64,7 +64,7 @@ def peliculas(item):
def search(item, text):
support.log(item, text)
support.info(item, text)
if 'all' in item.args:
item.url += text
else:
+10 -10
View File
@@ -6,7 +6,7 @@
from core import support
from core.support import log
from core.support import info
from core.item import Item
from platformcode import config
@@ -15,7 +15,7 @@ headers = [['Referer', host]]
@support.menu
def mainlist(item):
support.log(item)
support.info(item)
film = ['/film/',
('Generi',['', 'genres', 'genres']),
@@ -46,7 +46,7 @@ def mainlist(item):
@support.scrape
def peliculas(item):
log()
info()
# debug = True
if item.args == 'search':
@@ -96,7 +96,7 @@ def peliculas(item):
@support.scrape
def episodios(item):
log()
info()
patronBlock = r'<h1>.*?[ ]?(?:\[(?P<lang>.+?\]))?</h1>.+?<div class="se-a" '\
'style="display:block"><ul class="episodios">(?P<block>.*?)</ul>'\
@@ -109,7 +109,7 @@ def episodios(item):
@support.scrape
def genres(item):
log(item)
info(item)
action='peliculas'
if item.args == 'genres':
@@ -125,7 +125,7 @@ def genres(item):
return locals()
def search(item, text):
log(text)
info(text)
itemlist = []
text = text.replace(' ', '+')
item.url = host + "/search/" + text
@@ -135,12 +135,12 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
log("%s" % line)
info("%s" % line)
return []
def newest(categoria):
log(categoria)
info(categoria)
itemlist = []
item = Item()
@@ -161,14 +161,14 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
log("{0}".format(line))
info("{0}".format(line))
return []
return itemlist
def findvideos(item):
log()
info()
matches = support.match(item, patron=[r'class="metaframe rptss" src="([^"]+)"',r' href="#option-\d">([^\s]+)\s*([^\s]+)']).matches
itemlist = []
list_url = []
+10
View File
@@ -0,0 +1,10 @@
{
"id": "ilgeniodellostreaming_cam",
"name": "IlGenioDelloStreaming CAM",
"active": true,
"language": ["ita"],
"thumbnail": "ilgeniodellostreaming.png",
"banner": "ilgeniodellostreaming.png",
"categories": ["movie"],
"settings": []
}
+92
View File
@@ -0,0 +1,92 @@
# -*- coding: utf-8 -*-
# ------------------------------------------------------------
# Canale per ilgeniodellostreaming_cam
# ------------------------------------------------------------
from core import support
from core.support import info
from core.item import Item
from platformcode import config
host = config.get_channel_url()
headers = [['Referer', host]]
@support.menu
def mainlist(item):
film = ['/film/',
('In Sala', ['', 'peliculas', 'sala']),
('Generi',['', 'genres', 'genres']),
('Per Lettera',['/catalog/all', 'genres', 'az']),
('Anni',['', 'genres', 'year'])]
return locals()
@support.scrape
def peliculas(item):
if item.args == 'sala':
patronBlock = r'insala(?P<block>.*?)<header>'
patron = r'<img src="(?P<thumb>[^"]+)[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>\s*(?P<rating>[^<]+)[^>]+>[^>]+>(?P<quality>[^<]+)[^>]+>[^>]+>[^>]+>[^>]+><a href="(?P<url>[^"]+)">(?P<title>[^<]+)[^>]+>[^>]+>[^>]+>(?P<year>\d{4})'
elif item.args == 'az':
patron = r'<img src="(?P<thumb>[^"]+)[^>]+>[^>]+>[^>]+>[^>]+><a href="(?P<url>[^"]+)[^>]+>(?P<title>[^<]+)<[^>]+>[^>]+>[^>]+>.*?<span class="labelimdb">(?P<rating>[^>]+)<'
else:
patron = r'<img src="(?P<thumb>[^"]+)[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>\s*(?P<rating>[^<]+)[^>]+>[^>]+>(?P<quality>[^<]+)[^>]+>[^>]+>[^>]+>[^>]+><a href="(?P<url>[^"]+)">(?P<title>[^<]+)[^>]+>[^>]+>[^>]+>(?P<year>\d{4})[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>\s*(?P<plot>[^<]+)<[^>]+>'
patronNext = 'href="([^>]+)"'
return locals()
@support.scrape
def genres(item):
action='peliculas'
if item.args == 'genres':
patronBlock = r'<div class="sidemenu">\s*<h2>Genere</h2>(?P<block>.*?)</ul'
elif item.args == 'year':
item.args = 'genres'
patronBlock = r'<div class="sidemenu">\s*<h2>Anno di uscita</h2>(?P<block>.*?)</ul'
elif item.args == 'az':
patronBlock = r'<div class="movies-letter">(?P<block>.*?)<div class="clearfix">'
patronMenu = r'<a(?:.+?)?href="(?P<url>.*?)"[ ]?>(?P<title>.*?)<\/a>'
return locals()
def search(item, text):
info(text)
text = text.replace(' ', '+')
item.url = host + "/search/" + text
try:
return peliculas(item)
except:
import sys
for line in sys.exc_info():
info("%s" % line)
return []
def newest(categoria):
info(categoria)
itemlist = []
item = Item()
if categoria == 'peliculas':
item.contentType = 'movie'
item.url = host + '/film/'
try:
item.action = 'peliculas'
itemlist = peliculas(item)
except:
import sys
for line in sys.exc_info():
info("{0}".format(line))
return []
return itemlist
def findvideos(item):
info()
return support.server(item)
+9 -9
View File
@@ -25,7 +25,7 @@ headers = [['Referer', host]]
@support.menu
def mainlist(item):
support.log()
support.info()
tvshow = ['/category/serie-tv/',
('Aggiornamenti', ['/ultimi-episodi/', 'peliculas', 'update']),
@@ -37,7 +37,7 @@ def mainlist(item):
@support.scrape
def peliculas(item):
support.log()
support.info()
action = 'episodios'
patron = r'<div class="post-thumb">\s*<a href="(?P<url>[^"]+)" '\
@@ -54,7 +54,7 @@ def peliculas(item):
@support.scrape
def episodios(item):
support.log()
support.info()
patronBlock = r'</i> Stagione (?P<block>(?P<season>\d+)</div> '\
'<div class="su-spoiler-content".*?)<div class="clearfix">'
@@ -70,7 +70,7 @@ def episodios(item):
@support.scrape
def category(item):
support.log()
support.info()
action = 'peliculas'
patron = r'<li class="cat-item.*?href="(?P<url>[^"]+)".*?>(?P<title>.*?)</a>'
@@ -79,7 +79,7 @@ def category(item):
def search(item, texto):
support.log("s=", texto)
support.info("s=", texto)
item.url = host + "/?s=" + texto
item.contentType = 'tvshow'
try:
@@ -88,12 +88,12 @@ def search(item, texto):
except:
import sys
for line in sys.exc_info():
support.log("%s" % line)
support.info("%s" % line)
return []
def newest(categoria):
support.log("newest", categoria)
support.info("newest", categoria)
itemlist = []
item = Item()
try:
@@ -111,14 +111,14 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
support.log("{0}".format(line))
support.info("{0}".format(line))
return []
return itemlist
def findvideos(item):
support.log()
support.info()
if item.args == 'update':
itemlist = []
+1 -1
View File
@@ -117,7 +117,7 @@ def episodios(item):
def play(item):
support.log()
support.info()
data = support.match(item).data
match = support.match(data, patron='/content/entry/data/(.*?).mp4').match
if match:
+11 -11
View File
@@ -49,7 +49,7 @@ def mainlist(item):
def search(item, text):
support.log(text)
support.info(text)
item.search = text
if not item.args:
item.contentType = 'undefined'
@@ -64,7 +64,7 @@ def search(item, text):
return itemlist
def menu(item):
support.log()
support.info()
itemlist = [item.clone(title=support.typo(item.args[0],'bullet bold'), url='', action='peliculas')]
if item.url:
json = get_from_id(item)
@@ -74,7 +74,7 @@ def menu(item):
return itemlist
def live(item):
support.log()
support.info()
itemlist = []
json = current_session.get(item.url).json()['entries']
for it in json:
@@ -99,7 +99,7 @@ def live(item):
return support.thumb(itemlist, live=True)
def peliculas(item):
support.log()
support.info()
itemlist = []
contentType = ''
json = get_programs(item)
@@ -144,7 +144,7 @@ def peliculas(item):
return itemlist
def epmenu(item):
support.log()
support.info()
itemlist = []
entries = current_session.get('https://feed.entertainment.tv.theplatform.eu/f/PR1GhC/mediaset-prod-all-brands?byCustomValue={brandId}{' + item.urls + '}').json()['entries']
for entry in entries:
@@ -159,7 +159,7 @@ def epmenu(item):
def episodios(item):
support.log()
support.info()
itemlist = []
episode = ''
@@ -188,12 +188,12 @@ def episodios(item):
return sorted(itemlist, key=lambda it: it.title)
def findvideos(item):
support.log()
support.info()
itemlist = [support.Item(server = 'directo', title = 'Direct', url = item.urls, action = 'play')]
return support.server(item, itemlist=itemlist, Download=False)
def play(item):
support.log()
support.info()
if not item.urls: urls = item.url
else: urls = item.urls
for url in urls:
@@ -209,13 +209,13 @@ def play(item):
return support.servertools.find_video_items(item, data=data)
def subBrand(json):
support.log()
support.info()
subBrandId = current_session.get('https://feed.entertainment.tv.theplatform.eu/f/PR1GhC/mediaset-prod-all-brands?byCustomValue={brandId}{' + json + '}').json()['entries'][-1]['mediasetprogram$subBrandId']
json = current_session.get('https://feed.entertainment.tv.theplatform.eu/f/PR1GhC/mediaset-prod-all-programs?byCustomValue={subBrandId}{' + subBrandId + '}').json()['entries']
return json
def get_from_id(item):
support.log()
support.info()
json = current_session.get(entry.format(id=item.url)).json()
if 'components' in json:
id = quote(",".join(json["components"]))
@@ -225,7 +225,7 @@ def get_from_id(item):
return {}
def get_programs(item, ret=[], args={}):
support.log()
support.info()
hasMore = False
if not args:
if item.url:
+3 -3
View File
@@ -24,7 +24,7 @@ def mainlist(item):
patronBlock = r'<ul class="dropdown-menu(?P<block>.*?)</ul> </div'
patron = r'<a href="(?P<url>[^"]+)"(?: class="")?>(?P<title>[^<]+)<'
def itemHook(item):
item.thumbnail = support.thumb(thumb='music.png')
item.thumbnail = support.thumb('music')
item.contentType = 'music'
return item
def itemlistHook(itemlist):
@@ -36,7 +36,7 @@ def mainlist(item):
contentType='music',
url=item.url,
action='search',
thumbnail=support.thumb(thumb='search.png')))
thumbnail=support.thumb('search')))
support.channel_config(item, itemlist)
return itemlist
@@ -56,7 +56,7 @@ def findvideos(item):
def search(item, text):
support.log(text)
support.info(text)
item.url = host + '/search.php?keywords=' + text + '&video-id='
try:
return peliculas(item)
+2 -2
View File
@@ -35,7 +35,7 @@ def mainlist(item):
def search(item, text):
support.log(text)
support.info(text)
if item.contentType == 'movie' or item.extra == 'movie':
action = 'findvideos'
else:
@@ -53,7 +53,7 @@ def search(item, text):
def newest(categoria):
support.log(categoria)
support.info(categoria)
item = support.Item()
try:
if categoria == "series":
+22
View File
@@ -0,0 +1,22 @@
{
"id": "paramount",
"name": "Paramount Network",
"active": false,
"language": ["ita"],
"thumbnail": "paramount.png",
"banner": "paramount.png",
"categories": ["movie", "tvshow", "documentary", "live"],
"not_active": ["include_in_newest"],
"default_off": ["include_in_global_search"],
"settings": [
{
"id": "pagination",
"type": "list",
"label": "Elementi per pagina",
"default": 1,
"enabled": true,
"visible": true,
"lvalues": ["20", "40", "60", "80", "100"]
}
]
}
+176
View File
@@ -0,0 +1,176 @@
# -*- coding: utf-8 -*-
# ------------------------------------------------------------
# Canale per Paramount Network
# ------------------------------------------------------------
from core import support, jsontools
from platformcode import autorenumber
# host = support.config.get_channel_url()
host = 'https://www.paramountnetwork.it'
headers = [['Referer', host]]
@support.menu
def mainlist(item):
top = [('Dirette {bold}', ['/dl/RaiPlay/2016/PublishingBlock-9a2ff311-fcf0-4539-8f8f-c4fee2a71d58.html?json', 'live'])]
film = []
tvshow = []
return locals()
@support.scrape
def menu(item):
action='peliculas'
blacklist=['Tutti']
patronMenu = r'<a data-display-name="Link" href="(?P<url>[^"]+)" class="[^"]+">(?P<title>[^<]+)'
return locals()
def search(item, text):
support.info(text)
item.search = text.replace(' ','+')
try:
return peliculas(item)
# Continua la ricerca in caso di errore .
except:
import sys
for line in sys.exc_info():
support.logger.error("%s" % line)
return []
def live(item):
itemlist=[]
urls=[]
matches = support.match(host, patron=r'(/diretta-tv/[^"]+)"[^>]+>([^ ]+)').matches
from datetime import date
today = date.today()
channels = jsontools.load(support.match(host + '/api/more/tvschedule/' + str(today.year) + str(today.month) + str(today.day)).data)['channels']
ch_dict = {}
for channel in channels:
ch_dict[channel['label']] = channel['channelId']
for url, title in matches:
if url not in urls:
urls.append(url)
info = jsontools.load(support.match(host +'/api/on-air?channelId=' + ch_dict[title]).data)
support.info(info)
plot= '[B]' + info['seriesTitle'] +'[/B]\n' + info['description'] if 'seriesTitle' in info else ''
itemlist.append(item.clone(title=support.typo(title,'bold'), contentTitle=title, url=host+url, plot=plot, action='findvideos'))
return itemlist
def peliculas(item):
itemlist = []
if item.contentType == 'movie':
Type = 'Movie'
action = 'findvideos'
else:
Type = 'Series'
action = 'episodios'
if not item.page: item.page = 1
pagination_values = [20, 40, 60, 80, 100]
pagination = pagination_values[support.config.get_setting('pagination','paramount')]
item.url = host + '/api/search?activeTab=' + Type + '&searchFilter=site&pageNumber=0&rowsPerPage=10000'
data = jsontools.load(support.match(item).data)['response']['items']
for it in data:
title = it['meta']['header']['title']
support.info(title, it)
d = it['meta']['date'].split('/') if it['meta']['date'] else ['0000','00','00']
date = int(d[2] + d[1] + d[0])
if item.search.lower() in title.lower() \
and 'stagione' not in it['url'] \
and 'season' not in it['url'] \
and title not in ['Serie TV']:
itemlist.append(
item.clone(title=support.typo(title,'bold'),
action=action,
fulltitle=title,
show=title,
contentTitle=title if it['type'] == 'movie' else '',
contentSerieName=title if it['type'] != 'movie' else '',
plot= it['meta']['description'] if 'description' in it['meta'] else '',
url=host + it['url'],
date=date,
thumbnail='https:' + it['media']['image']['url'] if 'url' in it['media']['image'] else item.thumbnail))
itemlist.sort(key=lambda item: item.fulltitle)
if not item.search:
itlist = []
for i, it in enumerate(itemlist):
if pagination and (item.page - 1) * pagination > i and not item.search: continue # pagination
if pagination and i >= item.page * pagination and not item.search: break # pagination
itlist.append(it)
if pagination and len(itemlist) >= item.page * pagination and not item.search:
itlist.append(item.clone(channel=item.channel, action = 'peliculas', title=support.typo(support.config.get_localized_string(30992), 'color kod bold'), page=item.page + 1, thumbnail=support.thumb()))
itemlist = itlist
autorenumber.renumber(itemlist, item, 'bold')
return itemlist
def episodios(item):
def load_more(url):
second_url = host if url.startswith('/') else '' + url.replace('\u002F','/').replace('%5C','/')
new_data = support.match(host + second_url).data
match = support.scrapertools.decodeHtmlentities(support.match(new_data, headers=headers, patron=r'"items":([^\]]+])').match.replace('\x01','l').replace('\x02','a'))
return jsontools.load(match)
itemlist = []
data = []
page_data = support.match(item.url).data
seasons = support.match(page_data, patron=r'href="([^"]+)"[^>]+>Stagione\s*\d+').matches
more = support.match(page_data, patron=r'loadingTitle":[^,]+,"url":"([^"]+)"').match
data = jsontools.load(support.scrapertools.decodeHtmlentities(support.match(page_data, patron=r'"isEpisodes":[^,]+,"items":(.*?),"as"').match))
if data:
if more:
data += load_more(more)
if seasons:
for url in seasons:
new_data = support.match(host + url).data
data += jsontools.load(support.scrapertools.decodeHtmlentities(support.match(new_data, patron=r'isEpisodes":[^,]+,"items":(.*?),"as"').match.replace('\x01','l').replace('\x02','a')))
match = support.match(new_data, patron=r'loadingTitle":[^,]+,"url":"([^"]+)"').match
if match and match != load_more:
data += load_more(match)
for it in data:
if 'text' in it['meta']['header']['title']:
se = it['meta']['header']['title']['text']
s = support.match(se, patron=r'S\s*(?P<season>\d+)').match
e = support.match(se, patron=r'E\s*(?P<episode>\d+)').match
if not e: e = support.match(it['meta']['subHeader'], patron=r'(\d+)').match
title = support.typo((s + 'x' if s else 'Episodio ') + e.zfill(2) + ' - ' + it['meta']['subHeader'],'bold')
else:
s = e = '0'
title = support.typo(it['meta']['header']['title'],'bold')
itemlist.append(
item.clone(title=title,
season=int(s) if s else '',
episode=int(e),
url=host + it['url'] if it['url'].startswith('/') else it['url'],
thumbnail=it['media']['image']['url'],
fanart=it['media']['image']['url'],
plot=it['meta']['description'],
contentType='episode',
action='findvideos'))
itemlist.sort(key=lambda item: (item.season, item.episode))
autorenumber.renumber(itemlist, item, 'bold')
return support.videolibrary(itemlist, item)
def findvideos(item):
itemlist = []
qualities = []
mgid = support.match(item, patron=r'uri":"([^"]+)"').match
url = 'https://media.mtvnservices.com/pmt/e1/access/index.html?uri=' + mgid + '&configtype=edge&ref=' + item.url
ID, rootUrl = support.match(url, patron=[r'"id":"([^"]+)",',r'brightcove_mediagenRootURL":"([^"]+)"']).matches
url = jsontools.load(support.match(rootUrl.replace('&device={device}','').format(uri = ID)).data)['package']['video']['item'][0]['rendition'][0]['src']
video_urls = support.match(url, patron=r'RESOLUTION=(\d+x\d+).*?(http[^ ]+)').matches
for quality, url in video_urls:
if quality not in qualities:
qualities.append(quality)
itemlist.append(item.clone(title=support.config.get_localized_string(30137), server='directo', action='play', url=url, quality=quality))
itemlist.sort(key=lambda item: item.quality)
return support.server(item, itemlist=itemlist, Download=False)
+3 -3
View File
@@ -5,7 +5,7 @@
from core import support
from core.support import config, log
from core.support import config, info
host = config.get_channel_url()
headers = [['Referer', host]]
@@ -23,7 +23,7 @@ def mainlist(item):
def search(item, texto):
log(texto)
info(texto)
item.url = host + "/?s=" + texto
try:
return peliculas(item)
@@ -36,7 +36,7 @@ def search(item, texto):
def newest(categoria):
support.log(categoria)
support.info(categoria)
itemlist = []
item = support.Item()
try:
+9 -9
View File
@@ -36,7 +36,7 @@ def mainlist(item):
return locals()
def newest(categoria):
support.log()
support.info()
item = Item()
if categoria == 'peliculas':
item.contentType = 'movie'
@@ -47,7 +47,7 @@ def newest(categoria):
return peliculas(item)
def peliculas(item):
support.log()
support.info()
itemlist = []
data = support.match(item.url, headers=headers).data
@@ -68,7 +68,7 @@ def peliculas(item):
return itemlist
def episodios(item):
support.log()
support.info()
itemlist = []
data = support.match(item.url, headers=headers).data
json_object = jsontools.load(data)
@@ -84,7 +84,7 @@ def episodios(item):
return itemlist
def get_season(item, seas_url, seasonNumber):
support.log()
support.info()
itemlist = []
data = support.match(seas_url, headers=headers).data
json_object = jsontools.load(data)
@@ -98,7 +98,7 @@ def get_season(item, seas_url, seasonNumber):
return itemlist[::-1]
def search(item, texto):
support.log(item.url, "search", texto)
support.info(item.url, "search", texto)
itemlist=[]
try:
item.url = host + "/api/movies?originalTitle="+texto+"&translations.name=" +texto
@@ -122,7 +122,7 @@ def search(item, texto):
return []
def search_movie_by_genre(item):
support.log()
support.info()
itemlist = []
data = support.match(item.url, headers=headers).data
json_object = jsontools.load(data)
@@ -135,7 +135,7 @@ def search_movie_by_genre(item):
return support.thumb(itemlist, True)
def search_movie_by_year(item):
support.log()
support.info()
now = datetime.datetime.now()
year = int(now.year)
itemlist = []
@@ -150,7 +150,7 @@ def search_movie_by_year(item):
return itemlist
def findvideos(item):
support.log()
support.info()
itemlist = []
try:
data = support.match(item.url, headers=headers).data
@@ -171,7 +171,7 @@ def findvideos(item):
return support.server(item, itemlist=itemlist)
def get_itemlist_element(element,item):
support.log()
support.info()
itemlist=[]
contentSerieName = ''
contentTitle =''
+4 -4
View File
@@ -38,7 +38,7 @@ def menu(item):
def search(item, text):
support.log('search', item)
support.info('search', item)
itemlist = []
text = text.replace(' ', '+')
item.url = host + '/search/keyword/' + text
@@ -52,12 +52,12 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
support.log('search log:', line)
support.info('search log:', line)
return []
def newest(categoria):
support.log(categoria)
support.info(categoria)
itemlist = []
item = support.Item()
item.url = host
@@ -109,6 +109,6 @@ def episodios(item):
def findvideos(item):
support.log()
support.info()
# match = support.match(item, patron='wstream', debug=True)
return support.server(item)
+18 -18
View File
@@ -38,7 +38,7 @@ def mainlist(item):
def menu(item):
support.log()
support.info()
itemlist = [item.clone(title = support.typo('Tutti','bullet bold'), action = 'peliculas'),
item.clone(title = support.typo('Generi','submenu'), args = 'genre', action = 'submenu'),
item.clone(title = support.typo('A-Z','submenu'), args = 'az', action = 'submenu'),
@@ -48,7 +48,7 @@ def menu(item):
def learning(item):
support.log()
support.info()
itemlist =[]
json = current_session.get(item.url).json()['contents']
for key in json:
@@ -58,7 +58,7 @@ def learning(item):
def submenu(item):
support.log()
support.info()
itemlist = []
json = current_session.get(item.url).json()['contents'][-1]['contents']
if item.args == 'az':
@@ -76,7 +76,7 @@ def submenu(item):
def replay_menu(item):
support.log()
support.info()
import datetime, xbmc
# create day and month list
@@ -91,14 +91,14 @@ def replay_menu(item):
today = datetime.date.today()
for d in range(7):
day = today - datetime.timedelta(days=d)
support.log(day)
support.info(day)
itemlist.append(item.clone(action = 'replay_channels', date = day.strftime("%d-%m-%Y"),
title = support.typo(days[int(day.strftime("%w"))] + " " + day.strftime("%d") + " " + months[int(day.strftime("%m"))-1], 'bold')))
return itemlist
def replay_channels(item):
support.log()
support.info()
itemlist = []
json = current_session.get(item.url).json()['dirette']
for key in json:
@@ -108,18 +108,18 @@ def replay_channels(item):
def replay(item):
support.log()
support.info()
itemlist = []
json = current_session.get(item.url).json()[item.fulltitle][0]['palinsesto'][0]['programmi']
for key in json:
support.log('KEY=',key)
support.info('KEY=',key)
if key and key['pathID']: itemlist.append(item.clone(thumbnail = getUrl(key['images']['landscape']), fanart = getUrl(key['images']['landscape']), url = getUrl(key['pathID']), fulltitle = key['name'], show = key['name'],
title = support.typo(key['timePublished'], 'color kod bold') + support.typo(' | ' + key['name'], ' bold'), plot = key['testoBreve'], action = 'findvideos'))
return itemlist
def search(item, text):
# support.dbg()
support.log()
support.info()
itemlist =[]
try:
if item.url != host:
@@ -153,12 +153,12 @@ def Type(item):
def live(item):
support.log()
support.info()
itemlist =[]
info={}
json = current_session.get(item.url).json()['dirette']
onAir = current_session.get(onair).json()['on_air']
support.log(onAir)
support.info(onAir)
for key in onAir:
channel = key['channel']
info[channel] = {}
@@ -174,7 +174,7 @@ def live(item):
def peliculas(item):
support.log()
support.info()
itemlist = []
keys = []
key_list = []
@@ -222,7 +222,7 @@ def peliculas(item):
def select(item):
support.log()
support.info()
itemlist = []
json = current_session.get(item.url).json()['blocks']
for key in json:
@@ -234,7 +234,7 @@ def select(item):
def episodios(item):
support.log()
support.info()
itemlist = []
if type(item.url) in [list, dict] and len(item.url) > 1 and ('name' in item.url[0] and 'stagione' not in item.url[0]['name'].lower()):
for key in item.url:
@@ -276,7 +276,7 @@ def episodios(item):
def findvideos(item):
support.log()
support.info()
itemlist = []
if item.url.endswith('json'):
json = current_session.get(item.url).json()
@@ -293,7 +293,7 @@ def findvideos(item):
def getUrl(pathId):
support.log()
support.info()
url = pathId.replace(" ", "%20")
if url.startswith("/raiplay/"):
url = url.replace("/raiplay/",host +'/')
@@ -315,7 +315,7 @@ def getUrl(pathId):
def addinfo(key, item):
support.log()
support.info()
info = current_session.get(getUrl(key['info_url'])).json()
if not item.search or item.search.lower() in key['name'].lower():
it = item.clone(title = support.typo(key['name'],'bold'), fulltitle = key['name'], show = key['name'],
@@ -333,7 +333,7 @@ def addinfo(key, item):
def load_episodes(key, item):
support.log()
support.info()
itemlist = []
json = current_session.get(getUrl(key['path_id'])).json()['items']
order = 0
+6 -7
View File
@@ -5,14 +5,13 @@
from core import support
def findhost():
return support.match('https://nuovoindirizzo.info/seriehd/', patron=r'<h2[^>]+><a href="([^"]+)"').match
host = support.config.get_channel_url()
host = support.config.get_channel_url(findhost)
headers = [['Referer', host]]
@support.menu
def mainlist(item):
@@ -28,7 +27,7 @@ def mainlist(item):
def search(item, texto):
support.log(texto)
support.info(texto)
item.contentType = 'tvshow'
@@ -44,7 +43,7 @@ def search(item, texto):
def newest(categoria):
support.log(categoria)
support.info(categoria)
itemlist = []
item = support.Item()
@@ -137,7 +136,7 @@ def menu(item):
def findvideos(item):
item.url = item.url.replace('&amp;', '&')
support.log(item)
support.info(item)
if item.args == 'last':
url = support.match(item, patron = r'<iframe id="iframeVid" width="[^"]+" height="[^"]+" src="([^"]+)" allowfullscreen').match
matches = support.match(url,patron=r'<a href="([^"]+)">(\d+)<', patronBlock=r'<h3>EPISODIO</h3><ul>(.*?)</ul>').matches
+11 -11
View File
@@ -31,7 +31,7 @@ headers = [['Referer', host]]
@support.menu
def mainlist(item):
support.log()
support.info()
film = ['/ultimi-film-aggiunti/',
@@ -56,7 +56,7 @@ def mainlist(item):
@support.scrape
def peliculas(item):
support.log()
support.info()
blacklist = ['DMCA', 'Contatti', 'Attenzione NON FARTI OSCURARE', 'Lista Cartoni Animati e Anime']
patronBlock = r'<h1>.+?</h1>(?P<block>.*?)<div class="footer_c">'
@@ -76,13 +76,13 @@ def peliculas(item):
pagination = 35
if not item.args and 'anime' not in item.url:
patron = r'<div class="movie">[^>]+>.+?src="(?P<thumb>[^"]+)" alt="[^"]+".+?href="(?P<url>[^"]+)">[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[^>]+>[ ](?P<rating>\d+.\d+|\d+)<[^>]+>[^>]+><h2>(?P<title>[^"]+)</h2>[ ]?(?:<span class="year">(?P<year>\d+|\-\d+))?<'
patron = r'<div class="movie">[^>]+>.+?src="(?P<thumb>[^"]+)" alt="[^"]+".+? href="(?P<url>[^"]+)">.*?<h2>(?P<title>[^"]+)</h2>\s?(?:<span class="year">(?P<year>\d+|\-\d+))?<'
else:
anime = True
patron = r'(?:<td>)?<a href="(?P<url>[^"]+)"(?:[^>]+)?>\s?(?P<title>[^<]+)(?P<episode>[\d\-x]+)?(?P<title2>[^<]+)?<'
else:
# SEZIONE FILM
# pagination = 25
pagination = 25
if item.args == 'lista':
patron = r'href="(?P<url>[^"]+)"[^>]+>(?P<title>.*?)(?P<year>\d{4})?<'
@@ -105,7 +105,7 @@ def peliculas(item):
@support.scrape
def episodios(item):
support.log()
support.info()
action = 'findvideos'
patronBlock = r'<table>(?P<block>.*?)<\/table>'
patron = r'<tr><td>(?P<title>.*?)?[ ](?:Parte)?(?P<episode>\d+x\d+|\d+)(?:|[ ]?(?P<title2>.+?)?(?:avi)?)<(?P<url>.*?)</td><tr>'
@@ -119,7 +119,7 @@ def episodios(item):
def search(item, text):
support.log("CERCA :" ,text, item)
support.info("CERCA :" ,text, item)
item.url = "%s/?s=%s" % (host, text)
@@ -130,11 +130,11 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
support.log("%s" % line)
support.info("%s" % line)
return []
def newest(categoria):
support.log(categoria)
support.info(categoria)
itemlist = []
item = Item()
@@ -153,13 +153,13 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
support.log("{0}".format(line))
support.info("{0}".format(line))
return []
return itemlist
def findvideos(item):
support.log()
support.info()
if item.contentType == 'movie':
return support.server(item, headers=headers)
else:
@@ -173,7 +173,7 @@ def findvideos(item):
data = httptools.downloadpage(item.url, headers=headers).data
data = re.sub('\n|\t', ' ', data)
data = re.sub(r'>\s+<', '> <', data)
#support.log("DATA - HTML:\n", data)
#support.info("DATA - HTML:\n", data)
url_video = scrapertools.find_single_match(data, r'<tr><td>(.+?)</td><tr>', -1)
url_serie = scrapertools.find_single_match(data, r'<link rel="canonical" href="([^"]+)"\s?/>')
goseries = support.typo("Vai alla Serie:", ' bold')
+14 -16
View File
@@ -9,7 +9,7 @@ import time
from core import httptools, tmdb, scrapertools, support
from core.item import Item
from core.support import log
from core.support import info
from platformcode import logger, config
host = config.get_channel_url()
@@ -21,7 +21,7 @@ list_language = IDIOMAS.values()
@support.menu
def mainlist(item):
log()
info()
itemlist = []
tvshowSub = [
('Novità {bold}',[ '', 'peliculas_tv', '', 'tvshow']),
@@ -52,7 +52,7 @@ def cleantitle(scrapedtitle):
# ----------------------------------------------------------------------------------------------------------------
def findvideos(item):
log()
info()
data = httptools.downloadpage(item.url, headers=headers, ignore_response_code=True).data
data = re.sub(r'\n|\t|\s+', ' ', data)
# recupero il blocco contenente i link
@@ -66,8 +66,8 @@ def findvideos(item):
episodio = item.infoLabels['episode']
patron = r'\.\.:: Episodio %s([\s\S]*?)(<div class="post|..:: Episodio)' % episodio
log(patron)
log(blocco)
info(patron)
info(blocco)
matches = scrapertools.find_multiple_matches(blocco, patron)
if len(matches):
@@ -89,15 +89,12 @@ def findvideos(item):
# ----------------------------------------------------------------------------------------------------------------
def lista_serie(item):
log()
info()
itemlist = []
PERPAGE = 15
p = 1
if '{}' in item.url:
item.url, p = item.url.split('{}')
p = int(p)
p = 1 if not item.args else int(item.args)
if '||' in item.data:
series = item.data.split('\n\n')
@@ -129,7 +126,8 @@ def lista_serie(item):
# Paginazione
if len(matches) >= p * PERPAGE:
support.nextPage(itemlist, item, next_page=(item.url + '{}' + str(p + 1)))
item.args = p + 1
support.nextPage(itemlist, item, next_page=item.url)
return itemlist
@@ -139,7 +137,7 @@ def lista_serie(item):
# ----------------------------------------------------------------------------------------------------------------
def episodios(item, itemlist=[]):
log()
info()
patron = r'<div class="post-meta">\s*<a href="([^"]+)"\s*title="([^"]+)"\s*class=".*?"></a>.*?'
patron += r'<p><a href="([^"]+)">'
@@ -214,7 +212,7 @@ def episodios(item, itemlist=[]):
# ----------------------------------------------------------------------------------------------------------------
def peliculas_tv(item):
log()
info()
itemlist = []
patron = r'<div class="post-meta">\s*<a href="([^"]+)"\s*title="([^"]+)"\s*class=".*?"></a>'
@@ -267,7 +265,7 @@ def peliculas_tv(item):
# ----------------------------------------------------------------------------------------------------------------
def newest(categoria):
log(categoria)
info(categoria)
itemlist = []
item = Item()
item.url = host
@@ -291,7 +289,7 @@ def newest(categoria):
# ----------------------------------------------------------------------------------------------------------------
def search(item, texto):
log(texto)
info(texto)
itemlist = []
patron = r'<li class="cat-item cat-item-\d+"><a href="([^"]+)"\s?>([^<]+)</a>'
@@ -322,7 +320,7 @@ def search(item, texto):
def list_az(item):
log()
info()
itemlist = []
alphabet = dict()
+9 -9
View File
@@ -10,7 +10,7 @@ import re
from core import support, httptools, scrapertools
from core.item import Item
from core.support import log
from core.support import info
from platformcode import config
host = config.get_channel_url()
@@ -54,8 +54,8 @@ def episodios(item):
patron = r'(?:<div class="list (?:active)?")?\s*<a data-id="\d+(?:[ ](?P<lang>[SuUbBiItTaA\-]+))?"(?P<other>[^>]+)>.*?Episodio [0-9]+\s?(?:<br>(?P<title>[^<]+))?.*?Stagione (?P<season>[0-9]+) , Episodio - (?P<episode>[0-9]+).*?<(?P<url>.*?<iframe)'
def itemHook(item):
for value, season in seasons:
log(value)
log(season)
info(value)
info(season)
item.title = item.title.replace(value+'x',season+'x')
item.url += '\n' + item.other
return item
@@ -72,7 +72,7 @@ def genres(item):
def search(item, text):
log(text)
info(text)
item.url = host + "/?s=" + text
try:
item.contentType = 'tvshow'
@@ -81,12 +81,12 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
log("%s" % line)
info("%s" % line)
return []
def newest(categoria):
log(categoria)
info(categoria)
itemlist = []
item = Item()
try:
@@ -101,14 +101,14 @@ def newest(categoria):
except:
import sys
for line in sys.exc_info():
log("{0}".format(line))
info("{0}".format(line))
return []
return itemlist
def findvideos(item):
log(item)
info(item)
if item.args != 'update':
return support.server(item, data=item.url)
else:
@@ -126,6 +126,6 @@ def findvideos(item):
contentType='tvshow',
url=url_serie,
action='episodios',
thumbnail = support.thumb(thumb='tvshow.png')))
thumbnail = support.thumb('tvshow')))
return itemlist
+1 -1
View File
@@ -32,7 +32,7 @@ def mainlist(item):
def search(item, text):
support.log("[streamingaltadefinizione.py] " + item.url + " search " + text)
support.info("[streamingaltadefinizione.py] " + item.url + " search " + text)
item.url = item.url + "/?s=" + text
try:
return support.dooplay_search(item)
+12 -13
View File
@@ -3,9 +3,8 @@
# Canale per AnimeUnity
# ------------------------------------------------------------
import requests, json, copy
from core import support, jsontools
from specials import autorenumber
import requests, json
from core import support
try: from lib import cloudscraper
except: from lib import cloudscraper
@@ -37,7 +36,7 @@ def mainlist(item):
def genres(item):
support.log()
support.info()
itemlist = []
data = support.scrapertools.decodeHtmlentities(support.match(item).data)
args = support.match(data, patronBlock=r'genre-options-json="([^\]]+)\]', patron=r'name"\s*:\s*"([^"]+)').matches
@@ -48,7 +47,7 @@ def genres(item):
def search(item, text):
support.log('search', item)
support.info('search', item)
item.search = text
try:
@@ -57,12 +56,12 @@ def search(item, text):
except:
import sys
for line in sys.exc_info():
support.log('search log:', line)
support.info('search log:', line)
return []
def newest(category):
support.log(category)
support.info(category)
itemlist = []
item = support.Item()
item.args = 1
@@ -80,7 +79,7 @@ def newest(category):
except:
import sys
for line in sys.exc_info():
support.log(line)
support.info(line)
return []
return itemlist
@@ -88,7 +87,7 @@ def newest(category):
def peliculas(item):
support.log()
support.info()
itemlist = []
videoType = 'movie' if item.contentType == 'movie' else 'tv'
@@ -147,15 +146,15 @@ def peliculas(item):
return itemlist
def episodios(item):
support.log()
support.info()
itemlist = []
js = json.loads(support.match(item.url, patron=r'seasons="([^"]+)').match.replace('&quot;','"'))
support.log(js)
support.info(js)
for episodes in js:
for it in episodes['episodes']:
support.log(it)
support.info(it)
itemlist.append(
support.Item(channel=item.channel,
title=support.typo(str(episodes['number']) + 'x' + str(it['number']).zfill(2) + ' - ' + it['name'], 'bold'),
@@ -174,7 +173,7 @@ def episodios(item):
def findvideos(item):
support.log()
support.info()
itemlist=[]
url = support.match(support.match(item).data.replace('&quot;','"').replace('\\',''), patron=r'video_url"\s*:\s*"([^"]+)"').match
playlist = support.match(url.replace('https','http'), patron=r'\./([^.]+)').matches
+1 -1
View File
@@ -2,7 +2,7 @@
"id": "tantifilm",
"name": "Tantifilm",
"language": ["ita"],
"active": true,
"active": false,
"thumbnail": "tantifilm.png",
"banner": "tantifilm.png",
"categories": ["tvshow", "movie", "anime"],
+11 -11
View File
@@ -7,7 +7,7 @@ import re
from core import scrapertools, httptools, support
from core.item import Item
from core.support import log
from core.support import info
from platformcode import logger
from platformcode import config, unify
@@ -24,7 +24,7 @@ player_iframe = r'<iframe src="([^"]+)"[^>]+></iframe>\s?<div class="player'
@support.menu
def mainlist(item):
log()
info()
top = [('Generi', ['', 'category'])]
film = ['/film',
@@ -65,7 +65,7 @@ def peliculas(item):
@support.scrape
def episodios(item):
log()
info()
if not item.data:
data_check = httptools.downloadpage(item.url, headers=headers).data
data_check = re.sub('\n|\t', ' ', data_check)
@@ -125,7 +125,7 @@ def category(item):
def search(item, texto):
log(texto)
info(texto)
item.url = host + "/?s=" + texto
@@ -166,17 +166,17 @@ def hdpass(item):
def findvideos(item):
log()
support.log("ITEMLIST: ", item)
info()
support.info("ITEMLIST: ", item)
data = support.match(item.url, headers=headers).data
check = support.match(data, patron=r'<div class="category-film">(.*?)</div>').match
if 'sub' in check.lower():
item.contentLanguage = 'Sub-ITA'
support.log("CHECK : ", check)
support.info("CHECK : ", check)
if 'anime' in check.lower():
item.contentType = 'tvshow'
item.data = data
support.log('select = ### è una anime ###')
support.info('select = ### è una anime ###')
try:
return episodios(item)
except:
@@ -188,7 +188,7 @@ def findvideos(item):
# if 'protectlink' in data:
# urls = scrapertools.find_multiple_matches(data, r'<iframe src="[^=]+=(.*?)"')
# support.log("SONO QUI: ", urls)
# support.info("SONO QUI: ", urls)
# for url in urls:
# url = url.decode('base64')
# # tiro via l'ultimo carattere perchè non c'entra
@@ -199,7 +199,7 @@ def findvideos(item):
# if url:
# listurl.add(url)
# data += '\n'.join(listurl)
log(data)
info(data)
itemlist = []
# support.dbg()
@@ -211,7 +211,7 @@ def findvideos(item):
if item.otherLinks:
urls += support.match(item.otherLinks, patron=r'href="([^"]+)').matches
log('URLS', urls)
info('URLS', urls)
for u in urls:
if 'hdplayer.casa/series/' in u:
urls.remove(u)
+3 -3
View File
@@ -27,7 +27,7 @@ def mainlist(item):
def search(item, texto):
support.log(texto)
support.info(texto)
item.args='search'
item.contentType='tvshow'
item.url = host + '/?s=' + texto
@@ -42,7 +42,7 @@ def search(item, texto):
def newest(categoria):
support.log(categoria)
support.info(categoria)
item = support.Item()
try:
item.contentType = 'tvshow'
@@ -86,7 +86,7 @@ def peliculas(item):
patron = r'<li ><a href="(?P<url>[^"]+)" title="[^>]+">(?P<title>[^<|\(]+)?(?:\([^\d]*(?P<year>\d+)\))?[^<]*</a>'
def itemHook(item):
support.log(item.title)
support.info(item.title)
item.title = support.re.sub(' (?:- )?[Ss]erie [Tt][Vv]', '', item.title)
if item.args == 'sub':
#corregge l'esatta lang per quelle pagine in cui c'è
+4 -4
View File
@@ -22,19 +22,19 @@ def mainlist(item):
action = 'radio'
patron = r'text="(?P<title>[^"]+)" URL="(?P<url>[^"]+)"'
def itemHook(item):
item.thumbnail = support.thumb(thumb='music.png')
item.thumbnail = support.thumb('music')
item.contentType = 'music'
return item
def itemlistHook(itemlist):
itemlist.append(
item.clone(title=support.typo('Cerca...', 'bold color kod'), action='search', thumbnail=support.thumb(thumb='search.png')))
item.clone(title=support.typo('Cerca...', 'bold color kod'), action='search', thumbnail=support.thumb('search')))
support.channel_config(item, itemlist)
return itemlist
return locals()
def radio(item):
support.log()
support.info()
itemlist = []
data = support.match(item, patron= r'text="(?P<title>[^\("]+)(?:\((?P<location>[^\)]+)\))?" URL="(?P<url>[^"]+)" bitrate="(?P<quality>[^"]+)" reliability="[^"]+" guide_id="[^"]+" subtext="(?P<song>[^"]+)" genre_id="[^"]+" formats="(?P<type>[^"]+)" (?:playing="[^"]+" )?(?:playing_image="[^"]+" )?(?:show_id="[^"]+" )?(?:item="[^"]+" )?image="(?P<thumb>[^"]+)"')
if data.matches:
@@ -85,7 +85,7 @@ def findvideos(item):
def search(item, text):
support.log(text)
support.info(text)
item.url = host + '/Search.ashx?query=' +text
try:
return radio(item)
+1 -3
View File
@@ -3,10 +3,8 @@
# Canale per vedohd
# ------------------------------------------------------------
from core import scrapertools, httptools, support
from core.item import Item
from core import scrapertools, support, autoplay
from platformcode import logger, config
from specials import autoplay
host = config.get_channel_url()
headers = ""
+5 -10
View File
@@ -4,12 +4,7 @@
# ----------------------------------------------------------
import requests, sys
from core import support, tmdb
from specials import autorenumber
if sys.version_info[0] >= 3:
from concurrent import futures
else:
from concurrent_py2 import futures
from platformcode import autorenumber
host = support.config.get_channel_url()
@@ -71,7 +66,7 @@ def mainlist(item):
def search(item, text):
support.log(text)
support.info(text)
itemlist = []
if conn_id:
if 'film' in item.url: item.contentType = 'movie'
@@ -106,7 +101,7 @@ def peliculas(item):
itemlist = []
if not item.args:
json_file =loadjs(item.url + 'channel/10005/last/')
support.log(json_file)
support.logger.debug(json_file)
make_itemlist(itemlist, item, json_file)
elif ('=' not in item.args) and ('=' not in item.url):
@@ -142,7 +137,7 @@ def episodios(item):
show_id = str(json_file['data'][0]['show_id'])
season_id = str(json_file['data'][0]['season_id'])
episodes = []
support.log('SEASON ID= ',season_id)
support.info('SEASON ID= ',season_id)
for episode in json_file['data']:
episodes.append(episode['episodes'])
for episode in episodes:
@@ -233,7 +228,7 @@ def make_itemlist(itemlist, item, data):
def loadjs(url):
if '?category' not in url:
url += '?full=true'
support.log('Json URL;',url)
support.info('Json URL;',url)
json = current_session.get(url, headers=headers, params=payload).json()
return json
+16 -133
View File
@@ -2,37 +2,32 @@
import glob, os
from core import channeltools
from core.item import Item
from platformcode.unify import thumb_dict
from platformcode import config, logger, unify
from platformcode import config, logger
addon = config.__settings__
downloadenabled = addon.getSetting('downloadenabled')
def getmainlist(view="thumb_"):
logger.info()
itemlist = list()
if config.dev_mode():
itemlist.append(Item(title="Redirect", channel="checkhost", action="check_channels",
thumbnail='',
itemlist.append(Item(title="Redirect", action="check_channels", thumbnail='',
category=config.get_localized_string(30119), viewmode="thumbnails"))
# Main Menu Channels
if addon.getSetting('enable_news_menu') == "true":
itemlist.append(Item(title=config.get_localized_string(30130), channel="news", action="mainlist",
thumbnail=get_thumb("news.png", view),
category=config.get_localized_string(30119), viewmode="thumbnails",
thumbnail=get_thumb("news.png", view), category=config.get_localized_string(30119), viewmode="thumbnails",
context=[{"title": config.get_localized_string(70285), "channel": "shortcuts", "action": "SettingOnPosition", "category":7, "setting":1}]))
if addon.getSetting('enable_channels_menu') == "true":
itemlist.append(Item(title=config.get_localized_string(30118), channel="channelselector", action="getchanneltypes",
thumbnail=get_thumb("channels.png", view), view=view,
category=config.get_localized_string(30119), viewmode="thumbnails"))
thumbnail=get_thumb("channels.png", view), view=view, category=config.get_localized_string(30119), viewmode="thumbnails"))
if addon.getSetting('enable_search_menu') == "true":
itemlist.append(Item(title=config.get_localized_string(30103), channel="search", path='special', action="mainlist",
thumbnail=get_thumb("search.png", view),
category=config.get_localized_string(30119), viewmode="list",
thumbnail=get_thumb("search.png", view), category=config.get_localized_string(30119), viewmode="list",
context = [{"title": config.get_localized_string(60412), "action": "setting_channel_new", "channel": "search"},
{"title": config.get_localized_string(70286), "channel": "shortcuts", "action": "SettingOnPosition", "category":5 , "setting":1}]))
@@ -41,34 +36,28 @@ def getmainlist(view="thumb_"):
thumbnail=get_thumb("on_the_air.png"), viewmode="thumbnails"))
if addon.getSetting('enable_link_menu') == "true":
itemlist.append(Item(title=config.get_localized_string(70527), channel="kodfavorites", action="mainlist",
thumbnail=get_thumb("mylink.png", view), view=view,
category=config.get_localized_string(70527), viewmode="thumbnails"))
itemlist.append(Item(title=config.get_localized_string(70527), channel="kodfavorites", action="mainlist", thumbnail=get_thumb("mylink.png", view),
view=view, category=config.get_localized_string(70527), viewmode="thumbnails"))
if addon.getSetting('enable_fav_menu') == "true":
itemlist.append(Item(title=config.get_localized_string(30102), channel="favorites", action="mainlist",
thumbnail=get_thumb("favorites.png", view),
category=config.get_localized_string(30102), viewmode="thumbnails"))
thumbnail=get_thumb("favorites.png", view), category=config.get_localized_string(30102), viewmode="thumbnails"))
if config.get_videolibrary_support() and addon.getSetting('enable_library_menu') == "true":
itemlist.append(Item(title=config.get_localized_string(30131), channel="videolibrary", action="mainlist",
thumbnail=get_thumb("videolibrary.png", view),
category=config.get_localized_string(30119), viewmode="thumbnails",
thumbnail=get_thumb("videolibrary.png", view), category=config.get_localized_string(30119), viewmode="thumbnails",
context=[{"title": config.get_localized_string(70287), "channel": "shortcuts", "action": "SettingOnPosition", "category":2, "setting":1},
{"title": config.get_localized_string(60568), "channel": "videolibrary", "action": "update_videolibrary"}]))
if downloadenabled != "false":
itemlist.append(Item(title=config.get_localized_string(30101), channel="downloads", action="mainlist",
thumbnail=get_thumb("downloads.png", view), viewmode="list",
itemlist.append(Item(title=config.get_localized_string(30101), channel="downloads", action="mainlist", thumbnail=get_thumb("downloads.png", view), viewmode="list",
context=[{"title": config.get_localized_string(70288), "channel": "shortcuts", "action": "SettingOnPosition", "category":6}]))
thumb_setting = "setting_%s.png" % 0 # config.get_setting("plugin_updates_available")
itemlist.append(Item(title=config.get_localized_string(30100), channel="setting", action="settings",
thumbnail=get_thumb(thumb_setting, view),
category=config.get_localized_string(30100), viewmode="list"))
thumbnail=get_thumb(thumb_setting, view), category=config.get_localized_string(30100), viewmode="list"))
itemlist.append(Item(title=config.get_localized_string(30104) + " (v" + config.get_addon_version(with_fix=True) + ")", channel="help", action="mainlist",
thumbnail=get_thumb("help.png", view),
category=config.get_localized_string(30104), viewmode="list"))
thumbnail=get_thumb("help.png", view), category=config.get_localized_string(30104), viewmode="list"))
return itemlist
@@ -102,6 +91,7 @@ def getchanneltypes(view="thumb_"):
def filterchannels(category, view="thumb_"):
from core import channeltools
logger.info('Filter Channels ' + category)
channelslist = []
@@ -123,7 +113,7 @@ def filterchannels(category, view="thumb_"):
logger.info("channel_language=%s" % channel_language)
for channel_path in channel_files:
logger.info("channel in for = %s" % channel_path)
logger.debug("channel in for = %s" % channel_path)
channel = os.path.basename(channel_path).replace(".json", "")
@@ -136,7 +126,7 @@ def filterchannels(category, view="thumb_"):
# If it's not a channel we skip it
if not channel_parameters["channel"]:
continue
logger.info("channel_parameters=%s" % repr(channel_parameters))
logger.debug("channel_parameters=%s" % repr(channel_parameters))
# If you prefer the banner and the channel has it, now change your mind
if view == "banner_" and "banner" in channel_parameters:
@@ -275,110 +265,3 @@ def auto_filter(auto_lang=False):
lang = 'all'
return lang
def thumb(item_or_itemlist=None, genre=False, live=False, thumb=''):
if live:
if type(item_or_itemlist) == list:
for item in item_or_itemlist:
item.thumbnail = "https://raw.githubusercontent.com/kodiondemand/media/master/live/" + item.fulltitle.lower().replace(' ','_') + '.png'
else:
item_or_itemlist.thumbnail = "https://raw.githubusercontent.com/kodiondemand/media/master/live/" + item.fulltitle.lower().replace(' ','_') + '.png'
return item_or_itemlist
import re
icon_dict = {'movie':['film', 'movie'],
'tvshow':['serie','tv','episodi','episodio','fiction', 'show'],
'documentary':['documentari','documentario', 'documentary', 'documentaristico'],
'teenager':['ragazzi','teenager', 'teen'],
'learning':['learning'],
'all':['tutti', 'all'],
'news':['novità', "novita'", 'aggiornamenti', 'nuovi', 'nuove', 'new', 'newest', 'news', 'ultimi'],
'now_playing':['cinema', 'in sala'],
'anime':['anime'],
'genres':['genere', 'generi', 'categorie', 'categoria', 'category'],
'animation': ['animazione', 'cartoni', 'cartoon', 'animation'],
'action':['azione', 'arti marziali', 'action'],
'adventure': ['avventura', 'adventure'],
'biographical':['biografico', 'biographical'],
'comedy':['comico', 'commedia', 'demenziale', 'comedy', 'brillante'],
'adult':['erotico', 'hentai', 'harem', 'ecchi', 'adult'],
'drama':['drammatico', 'drama', 'dramma'],
'syfy':['fantascienza', 'science fiction', 'syfy', 'sci'],
'fantasy':['fantasy', 'magia', 'magic', 'fantastico'],
'crime':['gangster','poliziesco', 'crime', 'crimine'],
'grotesque':['grottesco', 'grotesque'],
'war':['guerra', 'war'],
'children':['bambini', 'kids'],
'horror':['horror'],
'music':['musical', 'musica', 'music', 'musicale'],
'mistery':['mistero', 'giallo', 'mystery'],
'noir':['noir'],
'popular' : ['popolari','popolare', 'più visti'],
'thriller':['thriller'],
'top_rated' : ['fortunato', 'votati', 'lucky', 'top'],
'on_the_air' : ['corso', 'onda', 'diretta', 'dirette'],
'western':['western'],
'vos':['sub','sub-ita'],
'romance':['romantico','sentimentale', 'romance', 'soap'],
'family':['famiglia','famiglie', 'family', 'historical'],
'historical':['storico', 'history', 'storia'],
'az':['lettera','lista','alfabetico','a-z', 'alphabetical'],
'year':['anno', 'anni', 'year'],
'update':['replay', 'update'],
'videolibrary':['teche'],
'autoplay':[config.get_localized_string(60071)]
}
suffix_dict = {'_hd':['hd','altadefinizione','alta definizione'],
'_4k':['4K'],
'_az':['lettera','lista','alfabetico','a-z', 'alphabetical'],
'_year':['anno', 'anni', 'year'],
'_genre':['genere', 'generi', 'categorie', 'categoria']}
search = ['cerca', 'search']
search_suffix ={'_movie':['film', 'movie'],
'_tvshow':['serie','tv', 'fiction']}
def autoselect_thumb(item, genre):
if genre == False:
for thumb, titles in icon_dict.items():
if any( word in re.split(r'\.|\{|\}|\[|\]|\(|\)| ',item.title.lower()) for word in search):
thumb = 'search'
for suffix, titles in search_suffix.items():
if any( word in re.split(r'\.|\{|\}|\[|\]|\(|\)| ',item.title.lower()) for word in titles ):
thumb = thumb + suffix
item.thumbnail = get_thumb(thumb + '.png')
elif any( word in re.split(r'\.|\{|\}|\[|\]|\(|\)| ',item.title.lower()) for word in titles ):
if thumb == 'movie' or thumb == 'tvshow':
for suffix, titles in suffix_dict.items():
if any( word in re.split(r'\.|\{|\}|\[|\]|\(|\)| ',item.title.lower()) for word in titles ):
thumb = thumb + suffix
item.thumbnail = get_thumb(thumb + '.png')
else: item.thumbnail = get_thumb(thumb + '.png')
else:
thumb = item.thumbnail
else:
for thumb, titles in icon_dict.items():
if any(word in re.split(r'\.|\{|\}|\[|\]|\(|\)| ',item.title.lower()) for word in titles ):
item.thumbnail = get_thumb(thumb + '.png')
else:
thumb = item.thumbnail
item.title = re.sub(r'\s*\{[^\}]+\}','',item.title)
return item
if item_or_itemlist:
if type(item_or_itemlist) == list:
for item in item_or_itemlist:
autoselect_thumb(item, genre)
return item_or_itemlist
else:
return autoselect_thumb(item_or_itemlist, genre)
elif thumb:
return get_thumb(thumb)
else:
return get_thumb('next.png')
+5 -7
View File
@@ -44,20 +44,16 @@ def start(itemlist, item):
autoplay_list = []
autoplay_b = []
favorite_quality = []
blacklisted_servers = []
favorite_servers = []
blacklisted_servers = config.get_setting("black_list", server='servers')
if not blacklisted_servers: blacklisted_servers = []
from core import servertools
servers_list = list(servertools.get_servers_list().items())
for server, server_parameters in servers_list:
if config.get_setting('black_list', server=server):
blacklisted_servers.append(server.lower())
if config.get_setting('favorites_servers_list', server=server):
favorite_servers.append(server.lower())
if not blacklisted_servers:
config.set_setting('black_list', [], server='servers')
blacklisted_servers = []
if not favorite_servers:
config.set_setting('favorites_servers_list', [], server='servers')
favorite_servers = []
@@ -283,5 +279,7 @@ def play_multi_channel(item, itemlist):
def servername(server):
from core.servertools import translate_server_name
path = filetools.join(config.get_runtime_path(), 'servers', server.lower() + '.json')
return jsontools.load(open(path, "r").read())['name'].upper()
name = jsontools.load(open(path, "r").read())['name'].upper()
return translate_server_name(name)
+1 -1
View File
@@ -113,7 +113,7 @@ class Downloader(object):
line2 = config.get_localized_string(59983) % ( self.downloaded[1], self.downloaded[2], self.size[1], self.size[2], self.speed[1], self.speed[2], self.connections[0], self.connections[1])
line3 = config.get_localized_string(60202) % (self.remaining_time)
progreso.update(int(self.progress), line1, line2 + " " + line3)
progreso.update(int(self.progress), line1 + '\n' + line2 + " " + line3)
self.__update_json()
finally:
progreso.close()
+4 -4
View File
@@ -271,7 +271,7 @@ def downloadfile(url, nombrefichero, headers=None, silent=False, continuar=False
# Create the progress dialog
if not silent:
progreso = platformtools.dialog_progress(header, "Downloading...", url, nombrefichero)
progreso = platformtools.dialog_progress(header, "Downloading..." + '\n' + url + '\n' + nombrefichero)
# If the platform does not return a valid dialog box, it assumes silent mode
if progreso is None:
@@ -408,7 +408,7 @@ def downloadfile(url, nombrefichero, headers=None, silent=False, continuar=False
error = downloadfileRTMP(url, nombrefichero, silent)
if error and not silent:
from platformcode import platformtools
platformtools.dialog_ok("You cannot download that video "," RTMP downloads not yet "," are supported")
platformtools.dialog_ok("You cannot download that video "," RTMP downloads not yet supported")
else:
import traceback
from pprint import pprint
@@ -480,7 +480,7 @@ def downloadfileRTMP(url, nombrefichero, silent):
rtmpdump_exit = spawnv(P_NOWAIT, rtmpdump_cmd, rtmpdump_args)
if not silent:
from platformcode import platformtools
advertencia = platformtools.dialog_ok("RTMP download option is experimental", "and the video will download in the background.", "No progress bar will be displayed.")
advertencia = platformtools.dialog_ok("RTMP download option is experimental", "and the video will download in the background. \n No progress bar will be displayed.")
except:
return True
@@ -520,7 +520,7 @@ def downloadfileGzipped(url, pathfichero):
# Create the progress dialog
from platformcode import platformtools
progreso = platformtools.dialog_progress("addon", config.get_localized_string(60200), url.split("|")[0], nombrefichero)
progreso = platformtools.dialog_progress("addon", config.get_localized_string(60200) + '\n' + url.split("|")[0] + '\n' + nombrefichero)
# Socket timeout at 60 seconds
socket.setdefaulttimeout(10)
+19 -16
View File
@@ -137,7 +137,7 @@ load_cookies()
def save_cookies(alfa_s=False):
cookies_lock.acquire()
if not alfa_s: logger.info("Saving cookies...")
if not alfa_s: logger.debug("Saving cookies...")
cj.save(cookies_file, ignore_discard=True)
cookies_lock.release()
@@ -161,7 +161,7 @@ def random_useragent():
def show_infobox(info_dict):
logger.info()
logger.debug()
from textwrap import wrap
box_items_kodi = {'r_up_corner': u'\u250c',
@@ -186,16 +186,16 @@ def show_infobox(info_dict):
width = 60
width = 100
version = '%s: %s' % (config.get_localized_string(20000), __version)
if config.is_xbmc():
box = box_items_kodi
else:
box = box_items
logger.info('%s%s%s' % (box['r_up_corner'], box['fill'] * width, box['l_up_corner']))
logger.info('%s%s%s' % (box['center'], version.center(width), box['center']))
logger.info('%s%s%s' % (box['r_center'], box['fill'] * width, box['l_center']))
logger.debug('%s%s%s' % (box['r_up_corner'], box['fill'] * width, box['l_up_corner']))
logger.debug('%s%s%s' % (box['center'], version.center(width), box['center']))
logger.debug('%s%s%s' % (box['r_center'], box['fill'] * width, box['l_center']))
count = 0
for key, value in info_dict:
@@ -210,13 +210,13 @@ def show_infobox(info_dict):
for line in text:
if len(line) < width:
line = line.ljust(width, ' ')
logger.info('%s%s%s' % (box['center'], line, box['center']))
logger.debug('%s%s%s' % (box['center'], line, box['center']))
else:
logger.info('%s%s%s' % (box['center'], text, box['center']))
logger.debug('%s%s%s' % (box['center'], text, box['center']))
if count < len(info_dict):
logger.info('%s%s%s' % (box['r_center'], box['fill'] * width, box['l_center']))
logger.debug('%s%s%s' % (box['r_center'], box['fill'] * width, box['l_center']))
else:
logger.info('%s%s%s' % (box['r_dn_corner'], box['fill'] * width, box['l_dn_corner']))
logger.debug('%s%s%s' % (box['r_dn_corner'], box['fill'] * width, box['l_dn_corner']))
return
@@ -284,7 +284,7 @@ def downloadpage(url, **opt):
CF = True
if config.get_setting('resolver_dns') and not opt.get('use_requests', False):
from specials import resolverdns
from core import resolverdns
session.mount('https://', resolverdns.CipherSuiteAdapter(domain, CF))
req_headers = default_headers.copy()
@@ -402,6 +402,13 @@ def downloadpage(url, **opt):
response['data'] = req.content if req.content else ''
response['url'] = req.url
if type(response['data']) != str:
try: response['data'] = response['data'].decode('utf-8')
except: response['data'] = response['data'].decode('ISO-8859-1')
if not response['data']:
response['data'] = ''
if req.headers.get('Server', '').startswith('cloudflare') and response_code in [429, 503, 403]\
and not opt.get('CF', False) and 'Please turn JavaScript on and reload the page' in response['data']:
# if domain not in CF_LIST:
@@ -416,15 +423,11 @@ def downloadpage(url, **opt):
response['data'] = re.sub('["|\']/save/[^"]*(https?://[^"]+)', '"\\1', response['data'])
response['url'] = response['url'].replace('https://web.archive.org/save/', '')
if type(response['data']) != str:
response['data'] = response['data'].decode('UTF-8')
if not response['data']:
response['data'] = ''
try:
response['json'] = to_utf8(req.json())
except:
response['json'] = dict()
response['code'] = response_code
response['headers'] = req.headers
response['cookies'] = req.cookies
+2 -2
View File
@@ -90,7 +90,7 @@ def get_node_from_file(name_file, node, path=None):
@return: dict with the node to return
@rtype: dict
"""
logger.info()
logger.debug()
from platformcode import config
from core import filetools
@@ -129,7 +129,7 @@ def check_to_backup(data, fname, dict_data):
@param dict_data: dictionary name
@type dict_data: dict
"""
logger.info()
logger.debug()
if not dict_data:
logger.error("Error loading json from file %s" % fname)
+18 -18
View File
@@ -1,10 +1,14 @@
# -*- coding: utf-8 -*-
import os
import ssl
try:
import urlparse
except:
import os, sys, ssl
PY3 = False
if sys.version_info[0] >= 3: PY3 = True; unicode = str; unichr = chr; long = int
if PY3:
import urllib.parse as urlparse
import _ssl
DEFAULT_CIPHERS = _ssl._DEFAULT_CIPHERS
else:
import urlparse
DEFAULT_CIPHERS = ssl._DEFAULT_CIPHERS
from lib.requests_toolbelt.adapters import host_header_ssl
from lib import doh
@@ -25,24 +29,20 @@ elif 'PROTOCOL_SSLv23' in ssl.__dict__:
else:
protocol = ssl.PROTOCOL_SSLv3
class CustomSocket(ssl.SSLSocket):
def __init__(self, *args, **kwargs):
super(CustomSocket, self).__init__(*args, **kwargs)
class CustomContext(ssl.SSLContext):
def __init__(self, protocol, hostname, *args, **kwargs):
self.hostname = hostname
if PY3:
super(CustomContext, self).__init__()
else:
super(CustomContext, self).__init__(protocol)
self.verify_mode = ssl.CERT_NONE
def wrap_socket(self, sock, server_side=False,
do_handshake_on_connect=True,
suppress_ragged_eofs=True,
server_hostname=None):
return CustomSocket(sock=sock, server_side=server_side,
do_handshake_on_connect=do_handshake_on_connect,
suppress_ragged_eofs=suppress_ragged_eofs,
server_hostname=self.hostname,
_context=self)
def wrap_socket(self, *args, **kwargs):
kwargs['server_hostname'] = self.hostname
self.verify_mode = ssl.CERT_NONE
return super(CustomContext, self).wrap_socket(*args, **kwargs)
class CipherSuiteAdapter(host_header_ssl.HostHeaderSSLAdapter):
@@ -52,7 +52,7 @@ class CipherSuiteAdapter(host_header_ssl.HostHeaderSSLAdapter):
self.cur = self.conn.cursor()
self.ssl_context = CustomContext(protocol, domain)
self.CF = CF # if cloudscrape is in action
self.cipherSuite = kwargs.pop('cipherSuite', ssl._DEFAULT_CIPHERS)
self.cipherSuite = kwargs.pop('cipherSuite', DEFAULT_CIPHERS)
super(CipherSuiteAdapter, self).__init__(**kwargs)
+7 -4
View File
@@ -85,11 +85,12 @@ def decodeHtmlentities(data):
if match.group(1) == "#" and ent.replace(";", "").isdigit():
ent = unichr(int(ent.replace(";", "")))
return ent.encode('utf-8')
return ent if PY3 else ent.encode('utf-8')
else:
cp = html5.get(ent)
if cp:
return cp.decode("unicode-escape").encode('utf-8') + res
if PY3: return cp + res
else: return cp.decode("unicode-escape").encode('utf-8') + res
else:
return match.group()
@@ -112,9 +113,11 @@ def unescape(text):
# character reference
try:
if text[:3] == "&#x":
return unichr(int(text[3:-1], 16)).encode("utf-8")
ret = unichr(int(text[3:-1], 16))
return ret if PY3 else ret.encode("utf-8")
else:
return unichr(int(text[2:-1])).encode("utf-8")
ret = unichr(int(text[2:-1]))
return ret if PY3 else ret.encode("utf-8")
except ValueError:
logger.error("error de valor")
+3 -16
View File
@@ -148,26 +148,13 @@ def findvideos(data, skip=False):
devuelve = []
skip = int(skip)
servers_list = list(get_servers_list().keys())
# is_filter_servers = False
# Run findvideos on each active server
for serverid in servers_list:
'''if not is_server_enabled(serverid):
continue'''
if config.get_setting('servers_blacklist') and serverid not in config.get_setting("black_list", server='servers'):
# if config.get_setting("filter_servers") == True and config.get_setting("black_list", server=serverid):
# is_filter_servers = True
continue
if is_server_enabled(serverid) :
devuelve.extend(findvideosbyserver(data, serverid))
if skip and len(devuelve) >= skip:
devuelve = devuelve[:skip]
break
# if config.get_setting("filter_servers") == False: is_filter_servers = False
# logger.info('DEVUELVE: ' + str(devuelve))
# if not devuelve and is_filter_servers:
# platformtools.dialog_ok(config.get_localized_string(60000), config.get_localized_string(60001))
return devuelve
@@ -710,9 +697,9 @@ def sort_servers(servers_list):
"""
if servers_list and config.get_setting('favorites_servers'):
if isinstance(servers_list[0], Item):
servers_list = sorted(servers_list, key=lambda x: config.get_setting("favorites_servers_list", server=x.server) or 100)
servers_list = sorted(servers_list, key=lambda x: config.get_setting("favorites_servers_list", server=x.server))
else:
servers_list = sorted(servers_list, key=lambda x: config.get_setting("favorites_servers_list", server=x) or 100)
servers_list = sorted(servers_list, key=lambda x: config.get_setting("favorites_servers_list", server=x))
return servers_list
+188 -111
View File
@@ -1,34 +1,29 @@
# -*- coding: utf-8 -*-
# -----------------------------------------------------------
# support functions that are needed by many channels, to no repeat the same code
import base64
import inspect
import os
import re
import sys
from time import time
import base64, inspect, os, re, sys
PY3 = False
if sys.version_info[0] >= 3: PY3 = True; unicode = str; unichr = chr; long = int
if PY3:
from concurrent import futures
else:
from concurrent_py2 import futures
try:
import urllib.request as urllib
from urllib.request import Request, urlopen
import urllib.parse as urlparse
from urllib.parse import urlencode
except ImportError:
import urllib, urlparse
else:
from concurrent_py2 import futures
import urlparse
from urllib2 import Request, urlopen
from urllib import urlencode
from channelselector import thumb
from core import httptools, scrapertools, servertools, tmdb, channeltools
from time import time
from core import httptools, scrapertools, servertools, tmdb, channeltools, autoplay
from core.item import Item
from lib import unshortenit
from platformcode import logger, config
from specials import autoplay
from platformcode import config
from platformcode.logger import info
from platformcode import logger
def hdpass_get_servers(item):
def get_hosts(url, quality):
@@ -38,29 +33,21 @@ def hdpass_get_servers(item):
for mir_url, srv in scrapertools.find_multiple_matches(mir, patron_option):
mir_url = scrapertools.decodeHtmlentities(mir_url)
log(mir_url)
it = item.clone(action="play",
quality=quality,
title=srv,
server=srv,
url= mir_url)
if not servertools.get_server_parameters(srv.lower()): # do not exists or it's empty
it = hdpass_get_url(it)[0]
info(mir_url)
it = item.clone(action="play", quality=quality, title=srv, server=srv, url= mir_url)
if not servertools.get_server_parameters(srv.lower()): it = hdpass_get_url(it)[0] # do not exists or it's empty
ret.append(it)
return ret
# Carica la pagina
itemlist = []
if 'hdpass' in item.url or 'hdplayer' in item.url:
url = item.url
if 'hdpass' in item.url or 'hdplayer' in item.url: url = item.url
else:
data = httptools.downloadpage(item.url, CF=False).data.replace('\n', '')
patron = r'<iframe(?: id="[^"]+")? width="[^"]+" height="[^"]+" src="([^"]+)"[^>]+><\/iframe>'
url = scrapertools.find_single_match(data, patron)
url = url.replace("&download=1", "")
if 'hdpass' not in url and 'hdplayer' not in url:
return itemlist
if not url.startswith('http'):
url = 'https:' + url
if 'hdpass' not in url and 'hdplayer' not in url: return itemlist
if not url.startswith('http'): url = 'https:' + url
data = httptools.downloadpage(url, CF=False).data
patron_res = '<div class="buttons-bar resolutions-bar">(.*?)<div class="buttons-bar'
@@ -68,7 +55,6 @@ def hdpass_get_servers(item):
patron_option = r'<a href="([^"]+?)"[^>]+>([^<]+?)</a'
res = scrapertools.find_single_match(data, patron_res)
# dbg()
with futures.ThreadPoolExecutor() as executor:
thL = []
@@ -78,15 +64,14 @@ def hdpass_get_servers(item):
for res in futures.as_completed(thL):
if res.result():
itemlist.extend(res.result())
return server(item, itemlist=itemlist)
def hdpass_get_url(item):
data = httptools.downloadpage(item.url, CF=False).data
src = scrapertools.find_single_match(data, r'<iframe allowfullscreen custom-src="([^"]+)')
if src:
item.url = base64.b64decode(src)
else:
item.url = scrapertools.find_single_match(data, r'<iframe allowfullscreen src="([^"]+)')
if src: item.url = base64.b64decode(src)
else: item.url = scrapertools.find_single_match(data, r'<iframe allowfullscreen src="([^"]+)')
item.url, c = unshortenit.unshorten_only(item.url)
return [item]
@@ -96,9 +81,8 @@ def color(text, color):
def search(channel, item, texto):
log(item.url + " search " + texto)
if 'findhost' in dir(channel):
channel.findhost()
info(item.url + " search " + texto)
if 'findhost' in dir(channel): channel.findhost()
item.url = channel.host + "/?s=" + texto
try:
return channel.peliculas(item)
@@ -121,7 +105,7 @@ def dbg():
def regexDbg(item, patron, headers, data=''):
if config.dev_mode():
import json, urllib2, webbrowser
import json, webbrowser
url = 'https://regex101.com'
if not data:
@@ -132,14 +116,15 @@ def regexDbg(item, patron, headers, data=''):
html = data
headers = {'content-type': 'application/json'}
data = {
'regex': patron.decode('utf-8'),
'regex': patron if PY3 else patron.decode('utf-8'),
'flags': 'gm',
'testString': html.decode('utf-8'),
'testString': html if PY3 else html.decode('utf-8'),
'delimiter': '"""',
'flavor': 'python'
}
r = urllib2.Request(url + '/api/regex', json.dumps(data, encoding='latin1'), headers=headers)
r = urllib2.urlopen(r).read()
data = json.dumps(data).encode() if PY3 else json.dumps(data, encoding='latin1')
r = Request(url + '/api/regex', data, headers=headers)
r = urlopen(r).read()
permaLink = json.loads(r)['permalinkFragment']
webbrowser.open(url + "/r/" + permaLink)
@@ -152,10 +137,8 @@ def scrapeLang(scraped, lang, longtitle):
language = ''
if scraped['lang']:
if 'ita' in scraped['lang'].lower():
language = 'ITA'
if 'sub' in scraped['lang'].lower():
language = 'Sub-' + language
if 'ita' in scraped['lang'].lower(): language = 'ITA'
if 'sub' in scraped['lang'].lower(): language = 'Sub-' + language
if not language: language = lang
if language: longtitle += typo(language, '_ [] color kod')
@@ -177,11 +160,10 @@ def unifyEp(ep):
def scrapeBlock(item, args, block, patron, headers, action, pagination, debug, typeContentDict, typeActionDict, blacklist, search, pag, function, lang, sceneTitle):
itemlist = []
log("scrapeBlock qui")
if debug:
regexDbg(item, patron, headers, block)
matches = scrapertools.find_multiple_matches_groups(block, patron)
log('MATCHES =', matches)
logger.debug('MATCHES =', matches)
known_keys = ['url', 'title', 'title2', 'season', 'episode', 'thumb', 'quality', 'year', 'plot', 'duration', 'genere', 'rating', 'type', 'lang', 'other', 'size', 'seed']
# Legenda known_keys per i groups nei patron
@@ -298,7 +280,7 @@ def scrapeBlock(item, args, block, patron, headers, action, pagination, debug, t
try:
parsedTitle = guessit(title)
title = longtitle = parsedTitle.get('title', '')
log('TITOLO',title)
logger.debug('TITOLO',title)
if parsedTitle.get('source'):
quality = str(parsedTitle.get('source'))
if parsedTitle.get('screen_size'):
@@ -332,7 +314,7 @@ def scrapeBlock(item, args, block, patron, headers, action, pagination, debug, t
longtitle += s + parsedTitle.get('episode_title')
item.contentEpisodeTitle = parsedTitle.get('episode_title')
except:
log('Error')
logger.debug('Error')
longtitle = typo(longtitle, 'bold')
lang1, longtitle = scrapeLang(scraped, lang, longtitle)
@@ -419,7 +401,7 @@ def scrape(func):
args = func(*args)
function = func.__name__ if not 'actLike' in args else args['actLike']
# log('STACK= ',inspect.stack()[1][3])
# info('STACK= ',inspect.stack()[1][3])
item = args['item']
@@ -451,12 +433,13 @@ def scrape(func):
matches = []
for n in range(2):
log('PATRON= ', patron)
logger.debug('PATRON= ', patron)
if not data:
page = httptools.downloadpage(item.url, headers=headers, ignore_response_code=True)
data = re.sub("='([^']+)'", '="\\1"', page.data)
data = data.replace('\n', ' ')
data = data.replace('\t', ' ')
data = data.replace('&nbsp;', ' ')
data = re.sub(r'>\s+<', '> <', data)
# replace all ' with " and eliminate newline, so we don't need to worry about
scrapingTime = time()
@@ -466,7 +449,7 @@ def scrape(func):
blocks = scrapertools.find_multiple_matches_groups(data, patronBlock)
block = ""
for bl in blocks:
# log(len(blocks),bl)
# info(len(blocks),bl)
if 'season' in bl and bl['season']:
item.season = bl['season']
blockItemlist, blockMatches = scrapeBlock(item, args, bl['block'], patron, headers, action, pagination, debug,
@@ -491,7 +474,7 @@ def scrape(func):
# if url may be changed and channel has findhost to update
if 'findhost' in func.__globals__ and not itemlist:
logger.info('running findhost ' + func.__module__)
info('running findhost ' + func.__module__)
host = func.__globals__['findhost']()
parse = list(urlparse.urlparse(item.url))
from core import jsontools
@@ -505,7 +488,7 @@ def scrape(func):
break
if (pagination and len(matches) <= pag * pagination) or not pagination: # next page with pagination
if patronNext and inspect.stack()[1][3] not in ['newest', 'search']:
if patronNext and inspect.stack()[1][3] not in ['newest']:
nextPage(itemlist, item, data, patronNext, function)
# next page for pagination
@@ -526,7 +509,7 @@ def scrape(func):
tmdb.set_infoLabels_itemlist(itemlist, seekTmdb=True)
if anime:
from specials import autorenumber
from platformcode import autorenumber
if function == 'episodios' or item.action == 'episodios': autorenumber.renumber(itemlist, item, 'bold')
else: autorenumber.renumber(itemlist)
# if anime and autorenumber.check(item) == False and len(itemlist)>0 and not scrapertools.find_single_match(itemlist[0].title, r'(\d+.\d+)'):
@@ -548,7 +531,7 @@ def scrape(func):
if config.get_setting('trakt_sync'):
from core import trakt_tools
trakt_tools.trakt_check(itemlist)
log('scraping time: ', time()-scrapingTime)
logger.debug('scraping time: ', time()-scrapingTime)
return itemlist
return wrapper
@@ -726,15 +709,11 @@ def menuItem(itemlist, filename, title='', action='', url='', contentType='undef
def menu(func):
def wrapper(*args):
log()
args = func(*args)
item = args['item']
log(item.channel + ' start')
logger.debug(item.channel + ' menu start')
host = func.__globals__['host']
list_servers = func.__globals__['list_servers'] if 'list_servers' in func.__globals__ else ['directo']
list_quality = func.__globals__['list_quality'] if 'list_quality' in func.__globals__ else ['default']
log('LIST QUALITY', list_quality)
filename = func.__module__.split('.')[1]
single_search = False
# listUrls = ['film', 'filmSub', 'tvshow', 'tvshowSub', 'anime', 'animeSub', 'search', 'top', 'topSub']
@@ -749,7 +728,7 @@ def menu(func):
for name in listUrls:
dictUrl[name] = args[name] if name in args else None
log(dictUrl[name])
logger.debug(dictUrl[name])
if name == 'film': title = 'Film'
if name == 'tvshow': title = 'Serie TV'
if name == 'anime': title = 'Anime'
@@ -816,9 +795,8 @@ def menu(func):
channel_config(item, itemlist)
# Apply auto Thumbnails at the menus
from channelselector import thumb
thumb(itemlist)
log(item.channel + ' end')
logger.debug(item.channel + ' menu end')
return itemlist
return wrapper
@@ -944,6 +922,7 @@ def match(item_url_string, **args):
data = re.sub("='([^']+)'", '="\\1"', data)
data = data.replace('\n', ' ')
data = data.replace('\t', ' ')
data = data.replace('&nbsp;', ' ')
data = re.sub(r'>\s+<', '><', data)
data = re.sub(r'([a-zA-Z])"([a-zA-Z])', "\1'\2", data)
@@ -984,7 +963,7 @@ def match(item_url_string, **args):
def match_dbg(data, patron):
import json, urllib2, webbrowser
import json, webbrowser
url = 'https://regex101.com'
headers = {'content-type': 'application/json'}
data = {
@@ -994,8 +973,9 @@ def match_dbg(data, patron):
'delimiter': '"""',
'flavor': 'python'
}
r = urllib2.Request(url + '/api/regex', json.dumps(data, encoding='latin1'), headers=headers)
r = urllib2.urlopen(r).read()
js = json.dumps(data).encode() if PY3 else json.dumps(data, encoding='latin1')
r = Request(url + '/api/regex', js, headers=headers)
r = urlopen(r).read()
permaLink = json.loads(r)['permalinkFragment']
webbrowser.open(url + "/r/" + permaLink)
@@ -1051,7 +1031,7 @@ def download(itemlist, item, typography='', function_level=1, function=''):
from_action=from_action,
contentTitle=contentTitle,
path=item.path,
thumbnail=thumb(thumb='downloads.png'),
thumbnail=thumb('downloads'),
downloadItemlist=downloadItemlist
))
if from_action == 'episodios':
@@ -1068,7 +1048,7 @@ def download(itemlist, item, typography='', function_level=1, function=''):
from_action=from_action,
contentTitle=contentTitle,
download='season',
thumbnail=thumb(thumb='downloads.png'),
thumbnail=thumb('downloads'),
downloadItemlist=downloadItemlist
))
@@ -1079,7 +1059,7 @@ def videolibrary(itemlist, item, typography='', function_level=1, function=''):
# Simply add this function to add video library support
# Function_level is useful if the function is called by another function.
# If the call is direct, leave it blank
log()
info()
if item.contentType == 'movie':
action = 'add_pelicula_to_library'
@@ -1108,7 +1088,6 @@ def videolibrary(itemlist, item, typography='', function_level=1, function=''):
if (function == 'findvideos' and contentType == 'movie') \
or (function == 'episodios' and contentType != 'movie'):
if config.get_videolibrary_support() and len(itemlist) > 0:
from channelselector import get_thumb
itemlist.append(
Item(channel=item.channel,
title=title,
@@ -1122,7 +1101,7 @@ def videolibrary(itemlist, item, typography='', function_level=1, function=''):
from_action=item.action,
extra=extra,
path=item.path,
thumbnail=get_thumb('add_to_videolibrary.png')
thumbnail=thumb('add_to_videolibrary')
))
return itemlist
@@ -1130,7 +1109,7 @@ def videolibrary(itemlist, item, typography='', function_level=1, function=''):
def nextPage(itemlist, item, data='', patron='', function_or_level=1, next_page='', resub=[]):
# Function_level is useful if the function is called by another function.
# If the call is direct, leave it blank
log()
info()
action = inspect.stack()[function_or_level][3] if type(function_or_level) == int else function_or_level
if next_page == '':
next_page = scrapertools.find_single_match(data, patron)
@@ -1140,7 +1119,7 @@ def nextPage(itemlist, item, data='', patron='', function_or_level=1, next_page=
if 'http' not in next_page:
next_page = scrapertools.find_single_match(item.url, 'https?://[a-z0-9.-]+') + (next_page if next_page.startswith('/') else '/' + next_page)
next_page = next_page.replace('&amp;', '&')
log('NEXT= ', next_page)
info('NEXT= ', next_page)
itemlist.append(
Item(channel=item.channel,
action = action,
@@ -1168,7 +1147,9 @@ def pagination(itemlist, item, page, perpage, function_level=1):
def server(item, data='', itemlist=[], headers='', AutoPlay=True, CheckLinks=True, Download=True, patronTag=None, Videolibrary=True):
log()
info()
blacklisted_servers = config.get_setting("black_list", server='servers')
if not blacklisted_servers: blacklisted_servers = []
if not data and not itemlist:
data = httptools.downloadpage(item.url, headers=headers, ignore_response_code=True).data
if data:
@@ -1179,7 +1160,7 @@ def server(item, data='', itemlist=[], headers='', AutoPlay=True, CheckLinks=Tru
def getItem(videoitem):
if not servertools.get_server_parameters(videoitem.server.lower()): # do not exists or it's empty
findS = servertools.get_server_from_url(videoitem.url)
log(findS)
info(findS)
if not findS:
if item.channel == 'community':
findS= (config.get_localized_string(30137), videoitem.url, 'directo')
@@ -1187,7 +1168,7 @@ def server(item, data='', itemlist=[], headers='', AutoPlay=True, CheckLinks=Tru
videoitem.url = unshortenit.unshorten_only(videoitem.url)[0]
findS = servertools.get_server_from_url(videoitem.url)
if not findS:
log(videoitem, 'Non supportato')
info(videoitem, 'Non supportato')
return
videoitem.server = findS[2]
videoitem.title = findS[0]
@@ -1213,7 +1194,7 @@ def server(item, data='', itemlist=[], headers='', AutoPlay=True, CheckLinks=Tru
with futures.ThreadPoolExecutor() as executor:
thL = [executor.submit(getItem, videoitem) for videoitem in itemlist if videoitem.url]
for it in futures.as_completed(thL):
if it.result():
if it.result() and it.result().server.lower() not in blacklisted_servers:
verifiedItemlist.append(it.result())
try:
verifiedItemlist.sort(key=lambda it: int(re.sub(r'\D','',it.quality)))
@@ -1247,39 +1228,19 @@ def filterLang(item, itemlist):
# import channeltools
list_language = channeltools.get_lang(item.channel)
if len(list_language) > 1:
from specials import filtertools
from core import filtertools
itemlist = filtertools.get_links(itemlist, item, list_language)
return itemlist
# def aplay(item, itemlist, list_servers='', list_quality=''):
# if inspect.stack()[1][3] == 'mainlist':
# autoplay.init(item.channel, list_servers, list_quality)
# autoplay.show_option(item.channel, itemlist)
# else:
# autoplay.start(itemlist, item)
def log(*args):
# Function to simplify the log
# Automatically returns File Name and Function Name
string = ''
for arg in args:
string += ' '+str(arg)
frame = inspect.stack()[1]
filename = frame[0].f_code.co_filename
filename = os.path.basename(filename)
logger.info("[" + filename + "] - [" + inspect.stack()[1][3] + "] " + string)
def channel_config(item, itemlist):
from channelselector import get_thumb
itemlist.append(
Item(channel='setting',
action="channel_config",
title=typo(config.get_localized_string(60587), 'color kod bold'),
config=item.channel,
folder=False,
thumbnail=get_thumb('setting_0.png'))
thumbnail=thumb('setting_0'))
)
@@ -1288,6 +1249,7 @@ def extract_wrapped(decorated):
closure = (c.cell_contents for c in decorated.__closure__)
return next((c for c in closure if isinstance(c, FunctionType)), None)
def addQualityTag(item, itemlist, data, patron):
if itemlist:
defQualVideo = {
@@ -1332,10 +1294,8 @@ def addQualityTag(item, itemlist, data, patron):
"RESYNC": "il film è stato lavorato e re sincronizzato con una traccia audio. A volte potresti riscontrare una mancata sincronizzazione tra audio e video.",
}
qualityStr = scrapertools.find_single_match(data, patron).strip().upper()
if PY3:
qualityStr = qualityStr.encode('ascii', 'ignore')
else:
qualityStr = qualityStr.decode('unicode_escape').encode('ascii', 'ignore')
# if PY3: qualityStr = qualityStr.encode('ascii', 'ignore')
if not PY3: qualityStr = qualityStr.decode('unicode_escape').encode('ascii', 'ignore')
if qualityStr:
try:
@@ -1354,14 +1314,14 @@ def addQualityTag(item, itemlist, data, patron):
descr += typo(audio + ': ', 'color kod') + defQualAudio.get(audio, '') + '\n'
except:
descr = ''
itemlist.insert(0,
Item(channel=item.channel,
itemlist.insert(0,Item(channel=item.channel,
action="",
title=typo(qualityStr, '[] color kod bold'),
plot=descr,
folder=False))
folder=False,
thumbnail=thumb('info')))
else:
log('nessun tag qualità trovato')
info('nessun tag qualità trovato')
def get_jwplayer_mediaurl(data, srvName, onlyHttp=False):
video_urls = []
@@ -1379,3 +1339,120 @@ def get_jwplayer_mediaurl(data, srvName, onlyHttp=False):
video_urls.sort(key=lambda x: x[0].split()[1])
return video_urls
def thumb(item_itemlist_string=None, genre=False, live=False):
from channelselector import get_thumb
if live:
if type(item_itemlist_string) == list:
for item in item_itemlist_string:
item.thumbnail = "https://raw.githubusercontent.com/kodiondemand/media/master/live/" + item.fulltitle.lower().replace(' ','_') + '.png'
else:
item_itemlist_string.thumbnail = "https://raw.githubusercontent.com/kodiondemand/media/master/live/" + item.fulltitle.lower().replace(' ','_') + '.png'
return item_itemlist_string
icon_dict = {'movie':['film', 'movie'],
'tvshow':['serie','tv','episodi','episodio','fiction', 'show'],
'documentary':['documentari','documentario', 'documentary', 'documentaristico'],
'teenager':['ragazzi','teenager', 'teen'],
'learning':['learning', 'school', 'scuola'],
'all':['tutti', 'all'],
'news':['novità', "novita'", 'aggiornamenti', 'nuovi', 'nuove', 'new', 'newest', 'news', 'ultimi', 'notizie'],
'now_playing':['cinema', 'in sala'],
'anime':['anime'],
'genres':['genere', 'generi', 'categorie', 'categoria', 'category'],
'animation': ['animazione', 'cartoni', 'cartoon', 'animation'],
'action':['azione', 'marziali', 'action', 'martial'],
'adventure': ['avventura', 'adventure'],
'biographical':['biografico', 'biographical', 'biografia'],
'comedy':['comico', 'commedia', 'demenziale', 'comedy', 'brillante', 'demential', 'parody'],
'adult':['erotico', 'hentai', 'harem', 'ecchi', 'adult'],
'drama':['drammatico', 'drama', 'dramma'],
'syfy':['fantascienza', 'science fiction', 'syfy', 'sci-fi'],
'fantasy':['fantasy', 'magia', 'magic', 'fantastico'],
'crime':['gangster','poliziesco', 'crime', 'crimine', 'police'],
'grotesque':['grottesco', 'grotesque'],
'war':['guerra', 'war', 'military'],
'children':['bambini', 'kids'],
'horror':['horror', 'orrore'],
'music':['musical', 'musica', 'music', 'musicale'],
'mistery':['mistero', 'giallo', 'mystery'],
'noir':['noir'],
'popular':['popolari','popolare', 'più visti', 'raccomandati', 'raccomandazioni' 'recommendations'],
'thriller':['thriller'],
'top_rated' : ['fortunato', 'votati', 'lucky', 'top'],
'on_the_air' : ['corso', 'onda', 'diretta', 'dirette'],
'western':['western'],
'vos':['sub','sub-ita'],
'romance':['romantico','sentimentale', 'romance', 'soap'],
'family':['famiglia','famiglie', 'family'],
'historical':['storico', 'history', 'storia', 'historical'],
'az':['lettera','lista','alfabetico','a-z', 'alphabetical'],
'year':['anno', 'anni', 'year'],
'update':['replay', 'update'],
'videolibrary':['teche'],
'info':['info','information','informazioni'],
'star':['star', 'personaggi', 'interpreti', 'stars', 'characters', 'performers', 'staff', 'actors', 'attori'],
'winter':['inverno', 'winter'],
'spring':['primavera', 'spring'],
'summer':['estate', 'summer'],
'autumn':['autunno', 'autumn'],
'autoplay':[config.get_localized_string(60071)]
}
suffix_dict = {'_hd':['hd','altadefinizione','alta definizione'],
'_4k':['4K'],
'_az':['lettera','lista','alfabetico','a-z', 'alphabetical'],
'_year':['anno', 'anni', 'year'],
'_genre':['genere', 'generi', 'categorie', 'categoria']}
search = ['cerca', 'search']
search_suffix ={'_movie':['film', 'movie'],
'_tvshow':['serie','tv', 'fiction']}
def autoselect_thumb(item, genre):
info('SPLIT',re.split(r'\.|\{|\}|\[|\]|\(|\)|/| ',item.title.lower()))
if genre == False:
for thumb, titles in icon_dict.items():
if any(word in re.split(r'\.|\{|\}|\[|\]|\(|\)|/| ',item.title.lower()) for word in search):
thumb = 'search'
for suffix, titles in search_suffix.items():
if any(word in re.split(r'\.|\{|\}|\[|\]|\(|\)|/| ',item.title.lower()) for word in titles ):
thumb = thumb + suffix
item.thumbnail = get_thumb(thumb + '.png')
elif any(word in re.split(r'\.|\{|\}|\[|\]|\(|\)| ',item.title.lower()) for word in titles ):
if thumb == 'movie' or thumb == 'tvshow':
for suffix, titles in suffix_dict.items():
if any(word in re.split(r'\.|\{|\}|\[|\]|\(|\)|/| ',item.title.lower()) for word in titles ):
thumb = thumb + suffix
item.thumbnail = get_thumb(thumb + '.png')
else: item.thumbnail = get_thumb(thumb + '.png')
else:
thumb = item.thumbnail
else:
for thumb, titles in icon_dict.items():
if any(word in re.split(r'\.|\{|\}|\[|\]|\(|\)|/| ',item.title.lower()) for word in titles ):
item.thumbnail = get_thumb(thumb + '.png')
else:
thumb = item.thumbnail
item.title = re.sub(r'\s*\{[^\}]+\}','',item.title)
return item
if item_itemlist_string:
if type(item_itemlist_string) == list:
for item in item_itemlist_string:
autoselect_thumb(item, genre)
return item_itemlist_string
elif type(item_itemlist_string) == str:
filename, file_extension = os.path.splitext(item_itemlist_string)
if not file_extension: item_itemlist_string += '.png'
return get_thumb(item_itemlist_string)
else:
return autoselect_thumb(item_itemlist_string, genre)
else:
return get_thumb('next.png')
+3
View File
@@ -950,7 +950,10 @@ class Tmdb(object):
if self.busqueda_tipo == "movie":
resultado = resultado["movie_results"][0]
else:
if resultado["tv_results"]:
resultado = resultado["tv_results"][0]
else:
resultado = resultado['tv_episode_results'][0]
self.results = [resultado]
self.total_results = 1
+4 -3
View File
@@ -71,9 +71,8 @@ def token_trakt(item):
else:
import time
dialog_auth = platformtools.dialog_progress(config.get_localized_string(60251),
config.get_localized_string(60252) % item.verify_url,
config.get_localized_string(60253)
% item.user_code,
config.get_localized_string(60252) % item.verify_url + '\n' +
config.get_localized_string(60253) % item.user_code + '\n' +
config.get_localized_string(60254))
# Generalmente cada 5 segundos se intenta comprobar si el usuario ha introducido el código
@@ -199,6 +198,8 @@ def get_trakt_watched(id_type, mediatype, update=False):
def trakt_check(itemlist):
if type(itemlist) != list:
return
def sync(item, id_result):
info = item.infoLabels
try:
+50 -99
View File
@@ -6,13 +6,17 @@
# Used to obtain series data for the video library
# ------------------------------------------------------------
import sys
if sys.version_info[0] >= 3: PY3 = True
else: PY3 = False
from future import standard_library
standard_library.install_aliases()
from future.builtins import object
import urllib.request, urllib.error, urllib.parse
import re
import re, requests
from core import jsontools
from core import scrapertools
@@ -100,12 +104,12 @@ def find_and_set_infoLabels(item):
otvdb_global = Tvdb(tvdb_id=item.infoLabels['tvdb_id'])
if not item.contentSeason:
p_dialog.update(50, config.get_localized_string(60296), config.get_localized_string(60295))
p_dialog.update(50, config.get_localized_string(60296) + '\n' + config.get_localized_string(60295))
results, info_load = otvdb_global.get_list_results()
logger.debug("results: %s" % results)
if not item.contentSeason:
p_dialog.update(100, config.get_localized_string(60296), config.get_localized_string(60297) % len(results))
p_dialog.update(100, config.get_localized_string(60296) + '\n' + config.get_localized_string(60297) % len(results))
p_dialog.close()
if len(results) > 1:
@@ -335,7 +339,7 @@ class Tvdb(object):
self.list_results = []
self.lang = ""
self.search_name = kwargs['search'] = \
re.sub('\[\\\?(B|I|COLOR)\s?[^\]]*\]', '', kwargs.get('search', ''))
re.sub(r'\[\\\?(B|I|COLOR)\s?[^\]]*\]', '', kwargs.get('search', ''))
self.list_episodes = {}
self.episodes = {}
@@ -390,9 +394,11 @@ class Tvdb(object):
url = HOST + "/login"
params = {"apikey": apikey}
if PY3: params = jsontools.dump(params).encode()
else: params = jsontools.dump(params)
try:
req = urllib.request.Request(url, data=jsontools.dump(params), headers=DEFAULT_HEADERS)
req = urllib.request.Request(url, data=params, headers=DEFAULT_HEADERS)
response = urllib.request.urlopen(req)
html = response.read()
response.close()
@@ -743,6 +749,7 @@ class Tvdb(object):
req = urllib.request.Request(url, headers=DEFAULT_HEADERS)
response = urllib.request.urlopen(req)
html = response.read()
logger.info(html)
response.close()
except Exception as ex:
@@ -836,7 +843,7 @@ class Tvdb(object):
except Exception as ex:
# if isinstance(ex, urllib).HTTPError:
logger.debug("code %s " % ex.code)
logger.debug("code %s " % ex)
message = "An exception of type %s occured. Arguments:\n%s" % (type(ex).__name__, repr(ex.args))
logger.error("error: %s" % message)
@@ -1021,102 +1028,46 @@ class Tvdb(object):
if not origen:
origen = self.result
# todo revisar
# if 'credits' in origen.keys():
# dic_origen_credits = origen['credits']
# origen['credits_cast'] = dic_origen_credits.get('cast', [])
# origen['credits_crew'] = dic_origen_credits.get('crew', [])
# del origen['credits']
items = list(origen.items())
for k, v in items:
if not v:
continue
if k == 'overview':
ret_infoLabels['plot'] = v
elif k == 'runtime':
ret_infoLabels['duration'] = int(v) * 60
elif k == 'firstAired':
ret_infoLabels['year'] = int(v[:4])
ret_infoLabels['premiered'] = v.split("-")[2] + "/" + v.split("-")[1] + "/" + v.split("-")[0]
# todo revisar
# elif k == 'original_title' or k == 'original_name':
# ret_infoLabels['originaltitle'] = v
elif k == 'siteRating':
ret_infoLabels['rating'] = float(v)
elif k == 'siteRatingCount':
ret_infoLabels['votes'] = v
elif k == 'status':
# se traduce los estados de una serie
ret_infoLabels['status'] = v
# I am not in favor of putting the chain as a studio but it is how the scraper does it in a generic way
elif k == 'network':
ret_infoLabels['studio'] = v
elif k == 'image_poster':
# obtenemos la primera imagen de la lista
ret_infoLabels['thumbnail'] = HOST_IMAGE + v[0]['fileName']
elif k == 'image_fanart':
# obtenemos la primera imagen de la lista
ret_infoLabels['fanart'] = HOST_IMAGE + v[0]['fileName']
# # no disponemos de la imagen de fondo
# elif k == 'banner':
# ret_infoLabels['fanart'] = HOST_IMAGE + v
elif k == 'id':
ret_infoLabels['tvdb_id'] = v
elif k == 'imdbId':
ret_infoLabels['imdb_id'] = v
# no se muestra
# ret_infoLabels['code'] = v
elif k in "rating":
# we translate the age rating (content rating system)
ret_infoLabels['mpaa'] = v
elif k in "genre":
ret_infoLabels['title'] = origen['seriesName']
ret_infoLabels['tvdb_id'] = origen['id']
thumbs = requests.get(HOST + '/series/' + str(origen['id']) + '/images/query?keyType=poster').json()
if 'data' in thumbs:
ret_infoLabels['thumbnail'] = HOST_IMAGE + thumbs['data'][0]['fileName']
elif 'poster' in origen and origen['poster']:
ret_infoLabels['thumbnail'] = origen['poster']
fanarts = requests.get(HOST + '/series/' + str(origen['id']) + '/images/query?keyType=fanart').json()
if 'data' in fanarts:
ret_infoLabels['fanart'] = HOST_IMAGE + fanarts['data'][0]['fileName']
elif 'fanart' in origen and origen['fanart']:
ret_infoLabels['thumbnail'] = origen['fanart']
if 'overview' in origen and origen['overview']:
ret_infoLabels['plot'] = origen['overview']
if 'duration' in origen and origen['duration']:
ret_infoLabels['duration'] = int(origen['duration']) * 60
if 'firstAired' in origen and origen['firstAired']:
ret_infoLabels['year'] = int(origen['firstAired'][:4])
ret_infoLabels['premiered'] = origen['firstAired'].split("-")[2] + "/" + origen['firstAired'].split("-")[1] + "/" + origen['firstAired'].split("-")[0]
if 'siteRating' in origen and origen['siteRating']:
ret_infoLabels['rating'] = float(origen['siteRating'])
if 'siteRatingCount' in origen and origen['siteRatingCount']:
ret_infoLabels['votes'] = origen['siteRatingCount']
if 'status' in origen and origen['status']:
ret_infoLabels['status'] = origen['status']
if 'network' in origen and origen['network']:
ret_infoLabels['studio'] = origen['network']
if 'imdbId' in origen and origen['rating']:
ret_infoLabels['imdb_id'] = origen['imdbId']
if 'rating' in origen and origen['rating']:
ret_infoLabels['mpaa'] = origen['rating']
if 'genre' in origen and origen['genre']:
for genre in origen['genre']:
genre_list = ""
for index, i in enumerate(v):
if index > 0:
genre_list += ", "
# traducimos los generos
genre_list += i
ret_infoLabels['genre'] = genre_list
elif k == 'seriesName': # or k == 'name' or k == 'title':
# if len(origen.get('aliases', [])) > 0:
# ret_infoLabels['title'] = v + " " + origen.get('aliases', [''])[0]
# else:
# ret_infoLabels['title'] = v
# logger.info("el titulo es %s " % ret_infoLabels['title'])
ret_infoLabels['title'] = v
elif k == 'cast':
genre_list += genre + ', '
ret_infoLabels['genre'] = genre_list.rstrip(', ')
if 'cast' in origen and origen['cast']:
dic_aux = dict((name, character) for (name, character) in l_castandrole)
l_castandrole.extend([(p['name'], p['role']) for p in v if p['name'] not in list(dic_aux.keys())])
else:
logger.debug("Attributes not added: %s=%s" % (k, v))
pass
# Sort the lists and convert them to str if necessary
if l_castandrole:
l_castandrole.extend([(p['name'], p['role']) for p in origen['cast'] if p['name'] not in list(dic_aux.keys())])
ret_infoLabels['castandrole'] = l_castandrole
logger.debug("ret_infoLabels %s" % ret_infoLabels)
return ret_infoLabels
+10 -10
View File
@@ -205,7 +205,7 @@ def save_movie(item, silent=False):
logger.error(traceback.format_exc())
if filetools.write(json_path, item.tojson()):
if not silent: p_dialog.update(100, config.get_localized_string(60062), item.contentTitle)
if not silent: p_dialog.update(100, item.contentTitle)
item_nfo.library_urls[item.channel] = item.url
if filetools.write(nfo_path, head_nfo + item_nfo.tojson()):
@@ -221,7 +221,7 @@ def save_movie(item, silent=False):
# If we get to this point it is because something has gone wrong
logger.error("Could not save %s in the video library" % item.contentTitle)
if not silent:
p_dialog.update(100, config.get_localized_string(60063), item.contentTitle)
p_dialog.update(100, item.contentTitle)
p_dialog.close()
return 0, 0, -1, path
@@ -254,7 +254,7 @@ def add_renumber_options(item, head_nfo, path):
return ret
def check_renumber_options(item):
from specials.autorenumber import load, write
from platformcode.autorenumber import load, write
for key in item.channel_prefs:
if 'TVSHOW_AUTORENUMBER' in item.channel_prefs[key]:
item.channel = key
@@ -645,10 +645,10 @@ def save_episodes(path, episodelist, serie, silent=False, overwrite=True):
# Silent is to show no progress (for service)
if not silent:
# progress dialog
p_dialog = platformtools.dialog_progress(config.get_localized_string(20000), config.get_localized_string(60064))
p_dialog.update(0, config.get_localized_string(60065))
p_dialog = platformtools.dialog_progress(config.get_localized_string(60064) ,'')
# p_dialog.update(0, config.get_localized_string(60065))
channel_alt = serie.channels # We prepare to add the emergency urls
channel_alt = serie.channel # We prepare to add the emergency urls
emergency_urls_stat = config.get_setting("emergency_urls", channel_alt) # Does the channel want emergency urls?
emergency_urls_succ = False
try: channel = __import__('specials.%s' % channel_alt, fromlist=["specials.%s" % channel_alt])
@@ -673,7 +673,7 @@ def save_episodes(path, episodelist, serie, silent=False, overwrite=True):
json_path = filetools.join(path, ("%s [%s].json" % (season_episode, e.channel)).lower()) # Path of the episode .json
if emergency_urls_stat == 1 and not e.emergency_urls and e.contentType == 'episode': # Do we keep emergency urls?
if not silent:
p_dialog.update(0, 'Caching links and .torren filest...', e.title) # progress dialog
p_dialog.update(0, 'Caching links and .torren filest...\n' + e.title) # progress dialog
if json_path in ficheros: # If there is the .json we get the urls from there
if overwrite: # but only if .json are overwritten
json_epi = Item().fromjson(filetools.read(json_path)) #We read the .json
@@ -690,7 +690,7 @@ def save_episodes(path, episodelist, serie, silent=False, overwrite=True):
emergency_urls_succ = True # ... is a success and we are going to mark the .nfo
elif emergency_urls_stat == 3 and e.contentType == 'episode': # Do we update emergency urls?
if not silent:
p_dialog.update(0, 'Caching links and .torrent files...', e.title) # progress dialog
p_dialog.update(0, 'Caching links and .torrent files...\n' + e.title) # progress dialog
e = emergency_urls(e, channel, json_path, headers=headers) # we generate the urls
if e.emergency_urls: # If we already have urls...
emergency_urls_succ = True # ... is a success and we are going to mark the .nfo
@@ -722,7 +722,7 @@ def save_episodes(path, episodelist, serie, silent=False, overwrite=True):
t = 0
for i, e in enumerate(scraper.sort_episode_list(new_episodelist)):
if not silent:
p_dialog.update(int(math.ceil((i + 1) * t)), config.get_localized_string(60064), e.title)
p_dialog.update(int(math.ceil((i + 1) * t)), e.title)
high_sea = e.contentSeason
high_epi = e.contentEpisodeNumber
@@ -1073,7 +1073,7 @@ def add_tvshow(item, channel=None):
# Get the episode list
itemlist = getattr(channel, item.action)(item)
if itemlist and not scrapertools.find_single_match(itemlist[0].title, r'(\d+x\d+)'):
from specials.autorenumber import select_type, renumber, check
from platformcode.autorenumber import select_type, renumber, check
if not check(item):
action = item.action
select_type(item)
-1
View File
@@ -245,7 +245,6 @@ class ConverterManager(object):
def parse(str):
import re
match = re.match('(?P<name>\w+) = (?P<module>[a-z0-9.]+):(?P<class>\w+)', str)
print match.groupdict()
return match.groupdict()
for ep in (parse(c) for c in self.registered_converters + self.internal_converters):
if ep.get('name') == name:
+3 -3
View File
@@ -8,7 +8,7 @@ from __future__ import unicode_literals
from collections import namedtuple
from functools import partial
# from pkg_resources import resource_stream # @UnresolvedImport
import os
import os, io
from .converters import ConverterManager
from . import basestr
@@ -19,10 +19,10 @@ COUNTRY_MATRIX = []
#: The namedtuple used in the :data:`COUNTRY_MATRIX`
IsoCountry = namedtuple('IsoCountry', ['name', 'alpha2'])
f = open(os.path.join(os.path.dirname(__file__), 'data/iso-3166-1.txt'))
f = io.open(os.path.join(os.path.dirname(__file__), 'data/iso-3166-1.txt'), encoding='utf-8')
f.readline()
for l in f:
iso_country = IsoCountry(*l.decode('utf-8').strip().split(';'))
iso_country = IsoCountry(*l.strip().split(';'))
COUNTRIES[iso_country.alpha2] = iso_country.name
COUNTRY_MATRIX.append(iso_country)
f.close()
+3 -3
View File
@@ -7,7 +7,7 @@
from __future__ import unicode_literals
from collections import namedtuple
from functools import partial
import os
import os, io
# from pkg_resources import resource_stream # @UnresolvedImport
from .converters import ConverterManager
from .country import Country
@@ -22,10 +22,10 @@ LANGUAGE_MATRIX = []
#: The namedtuple used in the :data:`LANGUAGE_MATRIX`
IsoLanguage = namedtuple('IsoLanguage', ['alpha3', 'alpha3b', 'alpha3t', 'alpha2', 'scope', 'type', 'name', 'comment'])
f = open(os.path.join(os.path.dirname(__file__), 'data/iso-639-3.tab'))
f = io.open(os.path.join(os.path.dirname(__file__), 'data/iso-639-3.tab'), encoding='utf-8')
f.readline()
for l in f:
iso_language = IsoLanguage(*l.decode('utf-8').split('\t'))
iso_language = IsoLanguage(*l.split('\t'))
LANGUAGES.add(iso_language.alpha3)
LANGUAGE_MATRIX.append(iso_language)
f.close()
+3 -3
View File
@@ -6,7 +6,7 @@
#
from __future__ import unicode_literals
import os
import os, io
from collections import namedtuple
# from pkg_resources import resource_stream # @UnresolvedImport
from . import basestr
@@ -20,10 +20,10 @@ SCRIPT_MATRIX = []
#: The namedtuple used in the :data:`SCRIPT_MATRIX`
IsoScript = namedtuple('IsoScript', ['code', 'number', 'name', 'french_name', 'pva', 'date'])
f = open(os.path.join(os.path.dirname(__file__), 'data/iso15924-utf8-20131012.txt'))
f = io.open(os.path.join(os.path.dirname(__file__), 'data/iso15924-utf8-20131012.txt'), encoding='utf-8')
f.readline()
for l in f:
l = l.decode('utf-8').strip()
l = l.strip()
if not l or l.startswith('#'):
continue
script = IsoScript._make(l.split(';'))
+124 -38
View File
@@ -46,18 +46,19 @@ from .exceptions import (
CloudflareLoopProtection,
CloudflareCode1020,
CloudflareIUAMError,
CloudflareSolveError,
CloudflareChallengeError,
CloudflareReCaptchaError,
CloudflareReCaptchaProvider
CloudflareCaptchaError,
CloudflareCaptchaProvider
)
from .interpreters import JavaScriptInterpreter
from .reCaptcha import reCaptcha
from .captcha import Captcha
from .user_agent import User_Agent
# ------------------------------------------------------------------------------- #
__version__ = '1.2.40'
__version__ = '1.2.46'
# ------------------------------------------------------------------------------- #
@@ -70,12 +71,23 @@ class CipherSuiteAdapter(HTTPAdapter):
'config',
'_pool_connections',
'_pool_maxsize',
'_pool_block'
'_pool_block',
'source_address'
]
def __init__(self, *args, **kwargs):
self.ssl_context = kwargs.pop('ssl_context', None)
self.cipherSuite = kwargs.pop('cipherSuite', None)
self.source_address = kwargs.pop('source_address', None)
if self.source_address:
if isinstance(self.source_address, str):
self.source_address = (self.source_address, 0)
if not isinstance(self.source_address, tuple):
raise TypeError(
"source_address must be IP address string or (ip, port) tuple"
)
if not self.ssl_context:
self.ssl_context = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
@@ -89,12 +101,14 @@ class CipherSuiteAdapter(HTTPAdapter):
def init_poolmanager(self, *args, **kwargs):
kwargs['ssl_context'] = self.ssl_context
kwargs['source_address'] = self.source_address
return super(CipherSuiteAdapter, self).init_poolmanager(*args, **kwargs)
# ------------------------------------------------------------------------------- #
def proxy_manager_for(self, *args, **kwargs):
kwargs['ssl_context'] = self.ssl_context
kwargs['source_address'] = self.source_address
return super(CipherSuiteAdapter, self).proxy_manager_for(*args, **kwargs)
# ------------------------------------------------------------------------------- #
@@ -108,9 +122,11 @@ class CloudScraper(Session):
self.cipherSuite = kwargs.pop('cipherSuite', None)
self.ssl_context = kwargs.pop('ssl_context', None)
self.interpreter = kwargs.pop('interpreter', 'native')
self.recaptcha = kwargs.pop('recaptcha', {})
self.captcha = kwargs.pop('captcha', {})
self.requestPreHook = kwargs.pop('requestPreHook', None)
self.requestPostHook = kwargs.pop('requestPostHook', None)
self.source_address = kwargs.pop('source_address', None)
self.doubleDown = kwargs.pop('doubleDown', True)
self.allow_brotli = kwargs.pop(
'allow_brotli',
@@ -143,7 +159,8 @@ class CloudScraper(Session):
'https://',
CipherSuiteAdapter(
cipherSuite=self.cipherSuite,
ssl_context=self.ssl_context
ssl_context=self.ssl_context,
source_address=self.source_address
)
)
@@ -157,6 +174,13 @@ class CloudScraper(Session):
def __getstate__(self):
return self.__dict__
# ------------------------------------------------------------------------------- #
# Allow replacing actual web request call via subclassing
# ------------------------------------------------------------------------------- #
def perform_request(self, method, url, *args, **kwargs):
return super(CloudScraper, self).request(method, url, *args, **kwargs)
# ------------------------------------------------------------------------------- #
# Raise an Exception with no stacktrace and reset depth counter.
# ------------------------------------------------------------------------------- #
@@ -236,7 +260,7 @@ class CloudScraper(Session):
# ------------------------------------------------------------------------------- #
response = self.decodeBrotli(
super(CloudScraper, self).request(method, url, *args, **kwargs)
self.perform_request(method, url, *args, **kwargs)
)
# ------------------------------------------------------------------------------- #
@@ -314,6 +338,7 @@ class CloudScraper(Session):
resp.text,
re.M | re.S
)
and re.search(r'window._cf_chl_enter\(', resp.text, re.M | re.S)
)
except AttributeError:
pass
@@ -321,17 +346,38 @@ class CloudScraper(Session):
return False
# ------------------------------------------------------------------------------- #
# check if the response contains a valid Cloudflare reCaptcha challenge
# check if the response contains a v2 hCaptcha Cloudflare challenge
# ------------------------------------------------------------------------------- #
@staticmethod
def is_reCaptcha_Challenge(resp):
def is_New_Captcha_Challenge(resp):
try:
return (
CloudScraper.is_Captcha_Challenge(resp)
and re.search(
r'cpo.src\s*=\s*"/cdn-cgi/challenge-platform/orchestrate/captcha/v1"',
resp.text,
re.M | re.S
)
and re.search(r'window._cf_chl_enter\(', resp.text, re.M | re.S)
)
except AttributeError:
pass
return False
# ------------------------------------------------------------------------------- #
# check if the response contains a Cloudflare hCaptcha challenge
# ------------------------------------------------------------------------------- #
@staticmethod
def is_Captcha_Challenge(resp):
try:
return (
resp.headers.get('Server', '').startswith('cloudflare')
and resp.status_code == 403
and re.search(
r'action="/.*?__cf_chl_captcha_tk__=\S+".*?data\-sitekey=.*?',
r'action="/\S+__cf_chl_captcha_tk__=\S+',
resp.text,
re.M | re.DOTALL
)
@@ -363,7 +409,7 @@ class CloudScraper(Session):
return False
# ------------------------------------------------------------------------------- #
# Wrapper for is_reCaptcha_Challenge, is_IUAM_Challenge, is_Firewall_Blocked
# Wrapper for is_Captcha_Challenge, is_IUAM_Challenge, is_Firewall_Blocked
# ------------------------------------------------------------------------------- #
def is_Challenge_Request(self, resp):
@@ -373,15 +419,21 @@ class CloudScraper(Session):
'Cloudflare has blocked this request (Code 1020 Detected).'
)
if self.is_New_Captcha_Challenge(resp):
self.simpleException(
CloudflareChallengeError,
'Detected a Cloudflare version 2 challenge, This feature is not available in the opensource (free) version.'
)
if self.is_New_IUAM_Challenge(resp):
self.simpleException(
CloudflareChallengeError,
'Detected the new Cloudflare challenge.'
'Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version.'
)
if self.is_reCaptcha_Challenge(resp) or self.is_IUAM_Challenge(resp):
if self.is_Captcha_Challenge(resp) or self.is_IUAM_Challenge(resp):
if self.debug:
print('Detected Challenge.')
print('Detected a Cloudflare version 1 challenge.')
return True
return False
@@ -442,10 +494,10 @@ class CloudScraper(Session):
}
# ------------------------------------------------------------------------------- #
# Try to solve the reCaptcha challenge via 3rd party.
# Try to solve the Captcha challenge via 3rd party.
# ------------------------------------------------------------------------------- #
def reCaptcha_Challenge_Response(self, provider, provider_params, body, url):
def captcha_Challenge_Response(self, provider, provider_params, body, url):
try:
formPayload = re.search(
r'<form (?P<form>.*?="challenge-form" '
@@ -456,8 +508,8 @@ class CloudScraper(Session):
if not all(key in formPayload for key in ['form', 'challengeUUID']):
self.simpleException(
CloudflareReCaptchaError,
"Cloudflare reCaptcha detected, unfortunately we can't extract the parameters correctly."
CloudflareCaptchaError,
"Cloudflare Captcha detected, unfortunately we can't extract the parameters correctly."
)
payload = OrderedDict(
@@ -471,11 +523,28 @@ class CloudScraper(Session):
except (AttributeError, KeyError):
self.simpleException(
CloudflareReCaptchaError,
"Cloudflare reCaptcha detected, unfortunately we can't extract the parameters correctly."
CloudflareCaptchaError,
"Cloudflare Captcha detected, unfortunately we can't extract the parameters correctly."
)
captchaResponse = reCaptcha.dynamicImport(
# ------------------------------------------------------------------------------- #
# Pass proxy parameter to provider to solve captcha.
# ------------------------------------------------------------------------------- #
if self.proxies and self.proxies != self.captcha.get('proxy'):
self.captcha['proxy'] = self.proxies
# ------------------------------------------------------------------------------- #
# Pass User-Agent if provider supports it to solve captcha.
# ------------------------------------------------------------------------------- #
self.captcha['User-Agent'] = self.headers['User-Agent']
# ------------------------------------------------------------------------------- #
# Submit job to provider to request captcha solve.
# ------------------------------------------------------------------------------- #
captchaResponse = Captcha.dynamicImport(
provider.lower()
).solveCaptcha(
captchaType,
@@ -484,6 +553,10 @@ class CloudScraper(Session):
provider_params
)
# ------------------------------------------------------------------------------- #
# Parse and handle the response of solved captcha.
# ------------------------------------------------------------------------------- #
dataPayload = OrderedDict([
('r', payload.get('name="r" value', '')),
('cf_captcha_kind', payload['name="cf_captcha_kind" value']),
@@ -510,41 +583,45 @@ class CloudScraper(Session):
# ------------------------------------------------------------------------------- #
def Challenge_Response(self, resp, **kwargs):
if self.is_reCaptcha_Challenge(resp):
if self.is_Captcha_Challenge(resp):
# ------------------------------------------------------------------------------- #
# double down on the request as some websites are only checking
# if cfuid is populated before issuing reCaptcha.
# if cfuid is populated before issuing Captcha.
# ------------------------------------------------------------------------------- #
if self.doubleDown:
resp = self.decodeBrotli(
super(CloudScraper, self).request(resp.request.method, resp.url, **kwargs)
self.perform_request(resp.request.method, resp.url, **kwargs)
)
if not self.is_reCaptcha_Challenge(resp):
if not self.is_Captcha_Challenge(resp):
return resp
# ------------------------------------------------------------------------------- #
# if no reCaptcha provider raise a runtime error.
# if no captcha provider raise a runtime error.
# ------------------------------------------------------------------------------- #
if not self.recaptcha or not isinstance(self.recaptcha, dict) or not self.recaptcha.get('provider'):
if not self.captcha or not isinstance(self.captcha, dict) or not self.captcha.get('provider'):
self.simpleException(
CloudflareReCaptchaProvider,
"Cloudflare reCaptcha detected, unfortunately you haven't loaded an anti reCaptcha provider "
"correctly via the 'recaptcha' parameter."
CloudflareCaptchaProvider,
"Cloudflare Captcha detected, unfortunately you haven't loaded an anti Captcha provider "
"correctly via the 'captcha' parameter."
)
# ------------------------------------------------------------------------------- #
# if provider is return_response, return the response without doing anything.
# ------------------------------------------------------------------------------- #
if self.recaptcha.get('provider') == 'return_response':
if self.captcha.get('provider') == 'return_response':
return resp
self.recaptcha['proxies'] = self.proxies
submit_url = self.reCaptcha_Challenge_Response(
self.recaptcha.get('provider'),
self.recaptcha,
# ------------------------------------------------------------------------------- #
# Submit request to parser wrapper to solve captcha
# ------------------------------------------------------------------------------- #
submit_url = self.captcha_Challenge_Response(
self.captcha.get('provider'),
self.captcha,
resp.text,
resp.url
)
@@ -618,6 +695,12 @@ class CloudScraper(Session):
**cloudflare_kwargs
)
if challengeSubmitResponse.status_code == 400:
self.simpleException(
CloudflareSolveError,
'Invalid challenge answer detected, Cloudflare broken?'
)
# ------------------------------------------------------------------------------- #
# Return response if Cloudflare is doing content pass through instead of 3xx
# else request with redirect URL also handle protocol scheme change http -> https
@@ -686,7 +769,10 @@ class CloudScraper(Session):
'debug',
'delay',
'interpreter',
'recaptcha'
'captcha',
'requestPreHook',
'requestPostHook',
'source_address'
] if field in kwargs
}
)
@@ -738,7 +824,7 @@ if ssl.OPENSSL_VERSION_INFO < (1, 1, 1):
print(
"DEPRECATION: The OpenSSL being used by this python install ({}) does not meet the minimum supported "
"version (>= OpenSSL 1.1.1) in order to support TLS 1.3 required by Cloudflare, "
"You may encounter an unexpected reCaptcha or cloudflare 1020 blocks.".format(
"You may encounter an unexpected Captcha or cloudflare 1020 blocks.".format(
ssl.OPENSSL_VERSION
)
)
@@ -1,14 +1,18 @@
from __future__ import absolute_import
import requests
try:
from urlparse import urlparse
except ImportError:
from urllib.parse import urlparse
from ..exceptions import (
reCaptchaServiceUnavailable,
reCaptchaAPIError,
reCaptchaTimeout,
reCaptchaParameter,
reCaptchaBadJobID,
reCaptchaReportError
CaptchaServiceUnavailable,
CaptchaAPIError,
CaptchaTimeout,
CaptchaParameter,
CaptchaBadJobID,
CaptchaReportError
)
try:
@@ -19,10 +23,10 @@ except ImportError:
"https://github.com/justiniso/polling/"
)
from . import reCaptcha
from . import Captcha
class captchaSolver(reCaptcha):
class captchaSolver(Captcha):
def __init__(self):
super(captchaSolver, self).__init__('2captcha')
@@ -34,7 +38,7 @@ class captchaSolver(reCaptcha):
@staticmethod
def checkErrorStatus(response, request_type):
if response.status_code in [500, 502]:
raise reCaptchaServiceUnavailable('2Captcha: Server Side Error {}'.format(response.status_code))
raise CaptchaServiceUnavailable('2Captcha: Server Side Error {}'.format(response.status_code))
errors = {
'in.php': {
@@ -81,7 +85,7 @@ class captchaSolver(reCaptcha):
}
if response.json().get('status') == 0 and response.json().get('request') in errors.get(request_type):
raise reCaptchaAPIError(
raise CaptchaAPIError(
'{} {}'.format(
response.json().get('request'),
errors.get(request_type).get(response.json().get('request'))
@@ -92,8 +96,8 @@ class captchaSolver(reCaptcha):
def reportJob(self, jobID):
if not jobID:
raise reCaptchaBadJobID(
"2Captcha: Error bad job id to request reCaptcha."
raise CaptchaBadJobID(
"2Captcha: Error bad job id to request Captcha."
)
def _checkRequest(response):
@@ -123,15 +127,15 @@ class captchaSolver(reCaptcha):
if response:
return True
else:
raise reCaptchaReportError(
"2Captcha: Error - Failed to report bad reCaptcha solve."
raise CaptchaReportError(
"2Captcha: Error - Failed to report bad Captcha solve."
)
# ------------------------------------------------------------------------------- #
def requestJob(self, jobID):
if not jobID:
raise reCaptchaBadJobID("2Captcha: Error bad job id to request reCaptcha.")
raise CaptchaBadJobID("2Captcha: Error bad job id to request Captcha.")
def _checkRequest(response):
if response.ok and response.json().get('status') == 1:
@@ -160,8 +164,8 @@ class captchaSolver(reCaptcha):
if response:
return response.json().get('request')
else:
raise reCaptchaTimeout(
"2Captcha: Error failed to solve reCaptcha."
raise CaptchaTimeout(
"2Captcha: Error failed to solve Captcha."
)
# ------------------------------------------------------------------------------- #
@@ -192,6 +196,14 @@ class captchaSolver(reCaptcha):
}
)
if self.proxy:
data.update(
{
'proxy': self.proxy,
'proxytype': self.proxyType
}
)
response = polling.poll(
lambda: self.session.post(
'{}/in.php'.format(self.host),
@@ -207,24 +219,35 @@ class captchaSolver(reCaptcha):
if response:
return response.json().get('request')
else:
raise reCaptchaBadJobID(
raise CaptchaBadJobID(
'2Captcha: Error no job id was returned.'
)
# ------------------------------------------------------------------------------- #
def getCaptchaAnswer(self, captchaType, url, siteKey, reCaptchaParams):
def getCaptchaAnswer(self, captchaType, url, siteKey, captchaParams):
jobID = None
if not reCaptchaParams.get('api_key'):
raise reCaptchaParameter(
if not captchaParams.get('api_key'):
raise CaptchaParameter(
"2Captcha: Missing api_key parameter."
)
self.api_key = reCaptchaParams.get('api_key')
self.api_key = captchaParams.get('api_key')
if reCaptchaParams.get('proxy'):
self.session.proxies = reCaptchaParams.get('proxies')
if captchaParams.get('proxy') and not captchaParams.get('no_proxy'):
hostParsed = urlparse(captchaParams.get('proxy', {}).get('https'))
if not hostParsed.scheme:
raise CaptchaParameter('Cannot parse proxy correctly, bad scheme')
if not hostParsed.netloc:
raise CaptchaParameter('Cannot parse proxy correctly, bad netloc')
self.proxyType = hostParsed.scheme
self.proxy = hostParsed.netloc
else:
self.proxy = None
try:
jobID = self.requestSolve(captchaType, url, siteKey)
@@ -234,12 +257,12 @@ class captchaSolver(reCaptcha):
if jobID:
self.reportJob(jobID)
except polling.TimeoutException:
raise reCaptchaTimeout(
"2Captcha: reCaptcha solve took to long and also failed reporting the job the job id {}.".format(jobID)
raise CaptchaTimeout(
"2Captcha: Captcha solve took to long and also failed reporting the job the job id {}.".format(jobID)
)
raise reCaptchaTimeout(
"2Captcha: reCaptcha solve took to long to execute job id {}, aborting.".format(jobID)
raise CaptchaTimeout(
"2Captcha: Captcha solve took to long to execute job id {}, aborting.".format(jobID)
)
@@ -14,7 +14,7 @@ captchaSolvers = {}
# ------------------------------------------------------------------------------- #
class reCaptcha(ABC):
class Captcha(ABC):
@abc.abstractmethod
def __init__(self, name):
captchaSolvers[name] = self
@@ -26,10 +26,10 @@ class reCaptcha(ABC):
if name not in captchaSolvers:
try:
__import__('{}.{}'.format(cls.__module__, name))
if not isinstance(captchaSolvers.get(name), reCaptcha):
raise ImportError('The anti reCaptcha provider was not initialized.')
if not isinstance(captchaSolvers.get(name), Captcha):
raise ImportError('The anti captcha provider was not initialized.')
except ImportError:
logging.error("Unable to load {} anti reCaptcha provider".format(name))
logging.error("Unable to load {} anti captcha provider".format(name))
raise
return captchaSolvers[name]
@@ -37,10 +37,10 @@ class reCaptcha(ABC):
# ------------------------------------------------------------------------------- #
@abc.abstractmethod
def getCaptchaAnswer(self, captchaType, url, siteKey, reCaptchaParams):
def getCaptchaAnswer(self, captchaType, url, siteKey, captchaParams):
pass
# ------------------------------------------------------------------------------- #
def solveCaptcha(self, captchaType, url, siteKey, reCaptchaParams):
return self.getCaptchaAnswer(captchaType, url, siteKey, reCaptchaParams)
def solveCaptcha(self, captchaType, url, siteKey, captchaParams):
return self.getCaptchaAnswer(captchaType, url, siteKey, captchaParams)
+109
View File
@@ -0,0 +1,109 @@
from __future__ import absolute_import
from ..exceptions import (
CaptchaParameter,
CaptchaTimeout,
CaptchaAPIError
)
try:
from urlparse import urlparse
except ImportError:
from urllib.parse import urlparse
try:
from python_anticaptcha import (
AnticaptchaClient,
NoCaptchaTaskProxylessTask,
HCaptchaTaskProxyless,
NoCaptchaTask,
HCaptchaTask,
AnticaptchaException
)
except ImportError:
raise ImportError(
"Please install/upgrade the python module 'python_anticaptcha' via "
"pip install python-anticaptcha or https://github.com/ad-m/python-anticaptcha/"
)
import sys
from . import Captcha
class captchaSolver(Captcha):
def __init__(self):
if sys.modules['python_anticaptcha'].__version__ < '0.6':
raise ImportError(
"Please upgrade the python module 'python_anticaptcha' via "
"pip install -U python-anticaptcha or https://github.com/ad-m/python-anticaptcha/"
)
super(captchaSolver, self).__init__('anticaptcha')
# ------------------------------------------------------------------------------- #
def parseProxy(self, url, user_agent):
parsed = urlparse(url)
return dict(
proxy_type=parsed.scheme,
proxy_address=parsed.hostname,
proxy_port=parsed.port,
proxy_login=parsed.username,
proxy_password=parsed.password,
user_agent=user_agent
)
# ------------------------------------------------------------------------------- #
def getCaptchaAnswer(self, captchaType, url, siteKey, captchaParams):
if not captchaParams.get('api_key'):
raise CaptchaParameter("anticaptcha: Missing api_key parameter.")
client = AnticaptchaClient(captchaParams.get('api_key'))
if captchaParams.get('proxy') and not captchaParams.get('no_proxy'):
captchaMap = {
'reCaptcha': NoCaptchaTask,
'hCaptcha': HCaptchaTask
}
proxy = self.parseProxy(
captchaParams.get('proxy', {}).get('https'),
captchaParams.get('User-Agent', '')
)
task = captchaMap[captchaType](
url,
siteKey,
**proxy
)
else:
captchaMap = {
'reCaptcha': NoCaptchaTaskProxylessTask,
'hCaptcha': HCaptchaTaskProxyless
}
task = captchaMap[captchaType](url, siteKey)
if not hasattr(client, 'createTaskSmee'):
raise NotImplementedError(
"Please upgrade 'python_anticaptcha' via pip or download it from "
"https://github.com/ad-m/python-anticaptcha/tree/hcaptcha"
)
job = client.createTaskSmee(task, timeout=180)
try:
job.join(maximum_time=180)
except (AnticaptchaException) as e:
raise CaptchaTimeout('{}'.format(getattr(e, 'message', e)))
if 'solution' in job._last_result:
return job.get_solution_response()
else:
raise CaptchaAPIError('Job did not return `solution` key in payload.')
# ------------------------------------------------------------------------------- #
captchaSolver()
+16 -16
View File
@@ -48,64 +48,64 @@ class CloudflareSolveError(CloudflareException):
"""
class CloudflareReCaptchaError(CloudflareException):
class CloudflareCaptchaError(CloudflareException):
"""
Raise an error for problem extracting reCaptcha paramters
Raise an error for problem extracting Captcha paramters
from Cloudflare payload
"""
class CloudflareReCaptchaProvider(CloudflareException):
class CloudflareCaptchaProvider(CloudflareException):
"""
Raise an exception for no reCaptcha provider loaded for Cloudflare.
Raise an exception for no Captcha provider loaded for Cloudflare.
"""
# ------------------------------------------------------------------------------- #
class reCaptchaException(Exception):
class CaptchaException(Exception):
"""
Base exception class for cloudscraper reCaptcha Providers
Base exception class for cloudscraper captcha Providers
"""
class reCaptchaServiceUnavailable(reCaptchaException):
class CaptchaServiceUnavailable(CaptchaException):
"""
Raise an exception for external services that cannot be reached
"""
class reCaptchaAPIError(reCaptchaException):
class CaptchaAPIError(CaptchaException):
"""
Raise an error for error from API response.
"""
class reCaptchaAccountError(reCaptchaException):
class CaptchaAccountError(CaptchaException):
"""
Raise an error for reCaptcha provider account problem.
Raise an error for captcha provider account problem.
"""
class reCaptchaTimeout(reCaptchaException):
class CaptchaTimeout(CaptchaException):
"""
Raise an exception for reCaptcha provider taking too long.
Raise an exception for captcha provider taking too long.
"""
class reCaptchaParameter(reCaptchaException):
class CaptchaParameter(CaptchaException):
"""
Raise an exception for bad or missing Parameter.
"""
class reCaptchaBadJobID(reCaptchaException):
class CaptchaBadJobID(CaptchaException):
"""
Raise an exception for invalid job id.
"""
class reCaptchaReportError(reCaptchaException):
class CaptchaReportError(CaptchaException):
"""
Raise an error for reCaptcha provider unable to report bad solve.
Raise an error for captcha provider unable to report bad solve.
"""
-75
View File
@@ -1,75 +0,0 @@
from __future__ import absolute_import
from ..exceptions import (
reCaptchaParameter,
reCaptchaTimeout,
reCaptchaAPIError
)
try:
from python_anticaptcha import (
AnticaptchaClient,
NoCaptchaTaskProxylessTask,
HCaptchaTaskProxyless,
AnticaptchaException
)
except ImportError:
raise ImportError(
"Please install/upgrade the python module 'python_anticaptcha' via "
"pip install python-anticaptcha or https://github.com/ad-m/python-anticaptcha/"
)
import sys
from . import reCaptcha
class captchaSolver(reCaptcha):
def __init__(self):
if sys.modules['python_anticaptcha'].__version__ < '0.6':
raise ImportError(
"Please upgrade the python module 'python_anticaptcha' via "
"pip install -U python-anticaptcha or https://github.com/ad-m/python-anticaptcha/"
)
super(captchaSolver, self).__init__('anticaptcha')
# ------------------------------------------------------------------------------- #
def getCaptchaAnswer(self, captchaType, url, siteKey, reCaptchaParams):
if not reCaptchaParams.get('api_key'):
raise reCaptchaParameter("anticaptcha: Missing api_key parameter.")
client = AnticaptchaClient(reCaptchaParams.get('api_key'))
if reCaptchaParams.get('proxy'):
client.session.proxies = reCaptchaParams.get('proxies')
captchaMap = {
'reCaptcha': NoCaptchaTaskProxylessTask,
'hCaptcha': HCaptchaTaskProxyless
}
task = captchaMap[captchaType](url, siteKey)
if not hasattr(client, 'createTaskSmee'):
raise NotImplementedError(
"Please upgrade 'python_anticaptcha' via pip or download it from "
"https://github.com/ad-m/python-anticaptcha/tree/hcaptcha"
)
job = client.createTaskSmee(task, timeout=180)
try:
job.join(maximum_time=180)
except (AnticaptchaException) as e:
raise reCaptchaTimeout('{}'.format(getattr(e, 'message', e)))
if 'solution' in job._last_result:
return job.get_solution_response()
else:
raise reCaptchaAPIError('Job did not return `solution` key in payload.')
# ------------------------------------------------------------------------------- #
captchaSolver()
+38 -31
View File
@@ -21,37 +21,29 @@ class User_Agent():
# ------------------------------------------------------------------------------- #
def loadHeaders(self, user_agents, user_agent_version):
if user_agents.get(self.browser).get('releases').get(user_agent_version).get('headers'):
self.headers = user_agents.get(self.browser).get('releases').get(user_agent_version).get('headers')
else:
self.headers = user_agents.get(self.browser).get('default_headers')
# ------------------------------------------------------------------------------- #
def filterAgents(self, releases):
def filterAgents(self, user_agents):
filtered = {}
for release in releases:
if self.mobile and releases[release]['User-Agent']['mobile']:
filtered[release] = filtered.get(release, []) + releases[release]['User-Agent']['mobile']
if self.mobile:
if self.platform in user_agents['mobile'] and user_agents['mobile'][self.platform]:
filtered.update(user_agents['mobile'][self.platform])
if self.desktop and releases[release]['User-Agent']['desktop']:
filtered[release] = filtered.get(release, []) + releases[release]['User-Agent']['desktop']
if self.desktop:
if self.platform in user_agents['desktop'] and user_agents['desktop'][self.platform]:
filtered.update(user_agents['desktop'][self.platform])
return filtered
# ------------------------------------------------------------------------------- #
def tryMatchCustom(self, user_agents):
for browser in user_agents:
for release in user_agents[browser]['releases']:
for platform in ['mobile', 'desktop']:
if re.search(re.escape(self.custom), ' '.join(user_agents[browser]['releases'][release]['User-Agent'][platform])):
self.browser = browser
self.loadHeaders(user_agents, release)
for device_type in user_agents['user_agents']:
for platform in user_agents['user_agents'][device_type]:
for browser in user_agents['user_agents'][device_type][platform]:
if re.search(re.escape(self.custom), ' '.join(user_agents['user_agents'][device_type][platform][browser])):
self.headers = user_agents['headers'][browser]
self.headers['User-Agent'] = self.custom
self.cipherSuite = user_agents[self.browser].get('cipherSuite', [])
self.cipherSuite = user_agents['cipherSuite'][browser]
return True
return False
@@ -60,13 +52,18 @@ class User_Agent():
def loadUserAgent(self, *args, **kwargs):
self.browser = kwargs.pop('browser', None)
self.platforms = ['linux', 'windows', 'darwin', 'android', 'ios']
self.browsers = ['chrome', 'firefox']
if isinstance(self.browser, dict):
self.custom = self.browser.get('custom', None)
self.platform = self.browser.get('platform', None)
self.desktop = self.browser.get('desktop', True)
self.mobile = self.browser.get('mobile', True)
self.browser = self.browser.get('browser', None)
else:
self.custom = kwargs.pop('custom', None)
self.platform = kwargs.pop('platform', None)
self.desktop = kwargs.pop('desktop', True)
self.mobile = kwargs.pop('mobile', True)
@@ -94,22 +91,32 @@ class User_Agent():
('Accept-Encoding', 'gzip, deflate, br')
])
else:
if self.browser and not user_agents.get(self.browser):
if self.browser and self.browser not in self.browsers:
sys.tracebacklimit = 0
raise RuntimeError('Sorry "{}" browser User-Agent was not found.'.format(self.browser))
raise RuntimeError('Sorry "{}" browser is not valid, valid browsers are [{}].'.format(self.browser, ', '.join(self.browsers)))
if not self.platform:
self.platform = random.SystemRandom().choice(self.platforms)
if self.platform not in self.platforms:
sys.tracebacklimit = 0
raise RuntimeError('Sorry the platform "{}" is not valid, valid platforms are [{}]'.format(self.platform, ', '.join(self.platforms)))
filteredAgents = self.filterAgents(user_agents['user_agents'])
if not self.browser:
self.browser = random.SystemRandom().choice(list(user_agents))
# has to be at least one in there...
while not filteredAgents.get(self.browser):
self.browser = random.SystemRandom().choice(list(filteredAgents.keys()))
self.cipherSuite = user_agents.get(self.browser).get('cipherSuite', [])
if not filteredAgents[self.browser]:
sys.tracebacklimit = 0
raise RuntimeError('Sorry "{}" browser was not found with a platform of "{}".'.format(self.browser, self.platform))
filteredAgents = self.filterAgents(user_agents.get(self.browser).get('releases'))
self.cipherSuite = user_agents['cipherSuite'][self.browser]
self.headers = user_agents['headers'][self.browser]
user_agent_version = random.SystemRandom().choice(list(filteredAgents))
self.loadHeaders(user_agents, user_agent_version)
self.headers['User-Agent'] = random.SystemRandom().choice(filteredAgents[user_agent_version])
self.headers['User-Agent'] = random.SystemRandom().choice(filteredAgents[self.browser])
if not kwargs.get('allow_brotli', False) and 'br' in self.headers['Accept-Encoding']:
self.headers['Accept-Encoding'] = ','.join([
File diff suppressed because it is too large Load Diff
+7 -8
View File
@@ -1,9 +1,8 @@
"""
Copyright (c) 2003-2010 Gustavo Niemeyer <gustavo@niemeyer.net>
# -*- coding: utf-8 -*-
try:
from ._version import version as __version__
except ImportError:
__version__ = 'unknown'
This module offers extensions to the standard python 2.3+
datetime module.
"""
__author__ = "Gustavo Niemeyer <gustavo@niemeyer.net>"
__license__ = "PSF License"
__version__ = "1.5.0.1"
__all__ = ['easter', 'parser', 'relativedelta', 'rrule', 'tz',
'utils', 'zoneinfo']
+43
View File
@@ -0,0 +1,43 @@
"""
Common code used in multiple modules.
"""
class weekday(object):
__slots__ = ["weekday", "n"]
def __init__(self, weekday, n=None):
self.weekday = weekday
self.n = n
def __call__(self, n):
if n == self.n:
return self
else:
return self.__class__(self.weekday, n)
def __eq__(self, other):
try:
if self.weekday != other.weekday or self.n != other.n:
return False
except AttributeError:
return False
return True
def __hash__(self):
return hash((
self.weekday,
self.n,
))
def __ne__(self, other):
return not (self == other)
def __repr__(self):
s = ("MO", "TU", "WE", "TH", "FR", "SA", "SU")[self.weekday]
if not self.n:
return s
else:
return "%s(%+d)" % (s, self.n)
# vim:ts=4:sw=4:et
+4
View File
@@ -0,0 +1,4 @@
# coding: utf-8
# file generated by setuptools_scm
# don't change, don't track in version control
version = '2.8.1'
+20 -23
View File
@@ -1,11 +1,8 @@
# -*- coding: utf-8 -*-
"""
Copyright (c) 2003-2007 Gustavo Niemeyer <gustavo@niemeyer.net>
This module offers extensions to the standard python 2.3+
datetime module.
This module offers a generic easter computing method for any given year, using
Western, Orthodox or Julian algorithms.
"""
__author__ = "Gustavo Niemeyer <gustavo@niemeyer.net>"
__license__ = "PSF License"
import datetime
@@ -15,6 +12,7 @@ EASTER_JULIAN = 1
EASTER_ORTHODOX = 2
EASTER_WESTERN = 3
def easter(year, method=EASTER_WESTERN):
"""
This method was ported from the work done by GM Arts,
@@ -35,24 +33,24 @@ def easter(year, method=EASTER_WESTERN):
These methods are represented by the constants:
EASTER_JULIAN = 1
EASTER_ORTHODOX = 2
EASTER_WESTERN = 3
* ``EASTER_JULIAN = 1``
* ``EASTER_ORTHODOX = 2``
* ``EASTER_WESTERN = 3``
The default method is method 3.
More about the algorithm may be found at:
http://users.chariot.net.au/~gmarts/eastalg.htm
`GM Arts: Easter Algorithms <http://www.gmarts.org/index.php?go=415>`_
and
http://www.tondering.dk/claus/calendar.html
`The Calendar FAQ: Easter <https://www.tondering.dk/claus/cal/easter.php>`_
"""
if not (1 <= method <= 3):
raise ValueError, "invalid method"
raise ValueError("invalid method")
# g - Golden year - 1
# c - Century
@@ -69,24 +67,23 @@ def easter(year, method=EASTER_WESTERN):
e = 0
if method < 3:
# Old method
i = (19*g+15)%30
j = (y+y//4+i)%7
i = (19*g + 15) % 30
j = (y + y//4 + i) % 7
if method == 2:
# Extra dates to convert Julian to Gregorian date
e = 10
if y > 1600:
e = e+y//100-16-(y//100-16)//4
e = e + y//100 - 16 - (y//100 - 16)//4
else:
# New method
c = y//100
h = (c-c//4-(8*c+13)//25+19*g+15)%30
i = h-(h//28)*(1-(h//28)*(29//(h+1))*((21-g)//11))
j = (y+y//4+i+2-c+c//4)%7
h = (c - c//4 - (8*c + 13)//25 + 19*g + 15) % 30
i = h - (h//28)*(1 - (h//28)*(29//(h + 1))*((21 - g)//11))
j = (y + y//4 + i + 2 - c + c//4) % 7
# p can be from -6 to 56 corresponding to dates 22 March to 23 May
# (later dates apply to method 2, although 23 May never actually occurs)
p = i-j+e
d = 1+(p+27+(p+6)//40)%31
m = 3+(p+26)//30
return datetime.date(int(y),int(m),int(d))
p = i - j + e
d = 1 + (p + 27 + (p + 6)//40) % 31
m = 3 + (p + 26)//30
return datetime.date(int(y), int(m), int(d))
-886
View File
@@ -1,886 +0,0 @@
# -*- coding:iso-8859-1 -*-
"""
Copyright (c) 2003-2007 Gustavo Niemeyer <gustavo@niemeyer.net>
This module offers extensions to the standard python 2.3+
datetime module.
"""
__author__ = "Gustavo Niemeyer <gustavo@niemeyer.net>"
__license__ = "PSF License"
import datetime
import string
import time
import sys
import os
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
import relativedelta
import tz
__all__ = ["parse", "parserinfo"]
# Some pointers:
#
# http://www.cl.cam.ac.uk/~mgk25/iso-time.html
# http://www.iso.ch/iso/en/prods-services/popstds/datesandtime.html
# http://www.w3.org/TR/NOTE-datetime
# http://ringmaster.arc.nasa.gov/tools/time_formats.html
# http://search.cpan.org/author/MUIR/Time-modules-2003.0211/lib/Time/ParseDate.pm
# http://stein.cshl.org/jade/distrib/docs/java.text.SimpleDateFormat.html
class _timelex(object):
def __init__(self, instream):
if isinstance(instream, basestring):
instream = StringIO(instream)
self.instream = instream
self.wordchars = ('abcdfeghijklmnopqrstuvwxyz'
'ABCDEFGHIJKLMNOPQRSTUVWXYZ_'
'ßàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿ'
'ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝÞ')
self.numchars = '0123456789'
self.whitespace = ' \t\r\n'
self.charstack = []
self.tokenstack = []
self.eof = False
def get_token(self):
if self.tokenstack:
return self.tokenstack.pop(0)
seenletters = False
token = None
state = None
wordchars = self.wordchars
numchars = self.numchars
whitespace = self.whitespace
while not self.eof:
if self.charstack:
nextchar = self.charstack.pop(0)
else:
nextchar = self.instream.read(1)
while nextchar == '\x00':
nextchar = self.instream.read(1)
if not nextchar:
self.eof = True
break
elif not state:
token = nextchar
if nextchar in wordchars:
state = 'a'
elif nextchar in numchars:
state = '0'
elif nextchar in whitespace:
token = ' '
break # emit token
else:
break # emit token
elif state == 'a':
seenletters = True
if nextchar in wordchars:
token += nextchar
elif nextchar == '.':
token += nextchar
state = 'a.'
else:
self.charstack.append(nextchar)
break # emit token
elif state == '0':
if nextchar in numchars:
token += nextchar
elif nextchar == '.':
token += nextchar
state = '0.'
else:
self.charstack.append(nextchar)
break # emit token
elif state == 'a.':
seenletters = True
if nextchar == '.' or nextchar in wordchars:
token += nextchar
elif nextchar in numchars and token[-1] == '.':
token += nextchar
state = '0.'
else:
self.charstack.append(nextchar)
break # emit token
elif state == '0.':
if nextchar == '.' or nextchar in numchars:
token += nextchar
elif nextchar in wordchars and token[-1] == '.':
token += nextchar
state = 'a.'
else:
self.charstack.append(nextchar)
break # emit token
if (state in ('a.', '0.') and
(seenletters or token.count('.') > 1 or token[-1] == '.')):
l = token.split('.')
token = l[0]
for tok in l[1:]:
self.tokenstack.append('.')
if tok:
self.tokenstack.append(tok)
return token
def __iter__(self):
return self
def next(self):
token = self.get_token()
if token is None:
raise StopIteration
return token
def split(cls, s):
return list(cls(s))
split = classmethod(split)
class _resultbase(object):
def __init__(self):
for attr in self.__slots__:
setattr(self, attr, None)
def _repr(self, classname):
l = []
for attr in self.__slots__:
value = getattr(self, attr)
if value is not None:
l.append("%s=%s" % (attr, `value`))
return "%s(%s)" % (classname, ", ".join(l))
def __repr__(self):
return self._repr(self.__class__.__name__)
class parserinfo(object):
# m from a.m/p.m, t from ISO T separator
JUMP = [" ", ".", ",", ";", "-", "/", "'",
"at", "on", "and", "ad", "m", "t", "of",
"st", "nd", "rd", "th"]
WEEKDAYS = [("Mon", "Monday"),
("Tue", "Tuesday"),
("Wed", "Wednesday"),
("Thu", "Thursday"),
("Fri", "Friday"),
("Sat", "Saturday"),
("Sun", "Sunday")]
MONTHS = [("Jan", "January"),
("Feb", "February"),
("Mar", "March"),
("Apr", "April"),
("May", "May"),
("Jun", "June"),
("Jul", "July"),
("Aug", "August"),
("Sep", "September"),
("Oct", "October"),
("Nov", "November"),
("Dec", "December")]
HMS = [("h", "hour", "hours"),
("m", "minute", "minutes"),
("s", "second", "seconds")]
AMPM = [("am", "a"),
("pm", "p")]
UTCZONE = ["UTC", "GMT", "Z"]
PERTAIN = ["of"]
TZOFFSET = {}
def __init__(self, dayfirst=False, yearfirst=False):
self._jump = self._convert(self.JUMP)
self._weekdays = self._convert(self.WEEKDAYS)
self._months = self._convert(self.MONTHS)
self._hms = self._convert(self.HMS)
self._ampm = self._convert(self.AMPM)
self._utczone = self._convert(self.UTCZONE)
self._pertain = self._convert(self.PERTAIN)
self.dayfirst = dayfirst
self.yearfirst = yearfirst
self._year = time.localtime().tm_year
self._century = self._year//100*100
def _convert(self, lst):
dct = {}
for i in range(len(lst)):
v = lst[i]
if isinstance(v, tuple):
for v in v:
dct[v.lower()] = i
else:
dct[v.lower()] = i
return dct
def jump(self, name):
return name.lower() in self._jump
def weekday(self, name):
if len(name) >= 3:
try:
return self._weekdays[name.lower()]
except KeyError:
pass
return None
def month(self, name):
if len(name) >= 3:
try:
return self._months[name.lower()]+1
except KeyError:
pass
return None
def hms(self, name):
try:
return self._hms[name.lower()]
except KeyError:
return None
def ampm(self, name):
try:
return self._ampm[name.lower()]
except KeyError:
return None
def pertain(self, name):
return name.lower() in self._pertain
def utczone(self, name):
return name.lower() in self._utczone
def tzoffset(self, name):
if name in self._utczone:
return 0
return self.TZOFFSET.get(name)
def convertyear(self, year):
if year < 100:
year += self._century
if abs(year-self._year) >= 50:
if year < self._year:
year += 100
else:
year -= 100
return year
def validate(self, res):
# move to info
if res.year is not None:
res.year = self.convertyear(res.year)
if res.tzoffset == 0 and not res.tzname or res.tzname == 'Z':
res.tzname = "UTC"
res.tzoffset = 0
elif res.tzoffset != 0 and res.tzname and self.utczone(res.tzname):
res.tzoffset = 0
return True
class parser(object):
def __init__(self, info=None):
self.info = info or parserinfo()
def parse(self, timestr, default=None,
ignoretz=False, tzinfos=None,
**kwargs):
if not default:
default = datetime.datetime.now().replace(hour=0, minute=0,
second=0, microsecond=0)
res = self._parse(timestr, **kwargs)
if res is None:
raise ValueError, "unknown string format"
repl = {}
for attr in ["year", "month", "day", "hour",
"minute", "second", "microsecond"]:
value = getattr(res, attr)
if value is not None:
repl[attr] = value
ret = default.replace(**repl)
if res.weekday is not None and not res.day:
ret = ret+relativedelta.relativedelta(weekday=res.weekday)
if not ignoretz:
if callable(tzinfos) or tzinfos and res.tzname in tzinfos:
if callable(tzinfos):
tzdata = tzinfos(res.tzname, res.tzoffset)
else:
tzdata = tzinfos.get(res.tzname)
if isinstance(tzdata, datetime.tzinfo):
tzinfo = tzdata
elif isinstance(tzdata, basestring):
tzinfo = tz.tzstr(tzdata)
elif isinstance(tzdata, int):
tzinfo = tz.tzoffset(res.tzname, tzdata)
else:
raise ValueError, "offset must be tzinfo subclass, " \
"tz string, or int offset"
ret = ret.replace(tzinfo=tzinfo)
elif res.tzname and res.tzname in time.tzname:
ret = ret.replace(tzinfo=tz.tzlocal())
elif res.tzoffset == 0:
ret = ret.replace(tzinfo=tz.tzutc())
elif res.tzoffset:
ret = ret.replace(tzinfo=tz.tzoffset(res.tzname, res.tzoffset))
return ret
class _result(_resultbase):
__slots__ = ["year", "month", "day", "weekday",
"hour", "minute", "second", "microsecond",
"tzname", "tzoffset"]
def _parse(self, timestr, dayfirst=None, yearfirst=None, fuzzy=False):
info = self.info
if dayfirst is None:
dayfirst = info.dayfirst
if yearfirst is None:
yearfirst = info.yearfirst
res = self._result()
l = _timelex.split(timestr)
try:
# year/month/day list
ymd = []
# Index of the month string in ymd
mstridx = -1
len_l = len(l)
i = 0
while i < len_l:
# Check if it's a number
try:
value_repr = l[i]
value = float(value_repr)
except ValueError:
value = None
if value is not None:
# Token is a number
len_li = len(l[i])
i += 1
if (len(ymd) == 3 and len_li in (2, 4)
and (i >= len_l or (l[i] != ':' and
info.hms(l[i]) is None))):
# 19990101T23[59]
s = l[i-1]
res.hour = int(s[:2])
if len_li == 4:
res.minute = int(s[2:])
elif len_li == 6 or (len_li > 6 and l[i-1].find('.') == 6):
# YYMMDD or HHMMSS[.ss]
s = l[i-1]
if not ymd and l[i-1].find('.') == -1:
ymd.append(info.convertyear(int(s[:2])))
ymd.append(int(s[2:4]))
ymd.append(int(s[4:]))
else:
# 19990101T235959[.59]
res.hour = int(s[:2])
res.minute = int(s[2:4])
res.second, res.microsecond = _parsems(s[4:])
elif len_li == 8:
# YYYYMMDD
s = l[i-1]
ymd.append(int(s[:4]))
ymd.append(int(s[4:6]))
ymd.append(int(s[6:]))
elif len_li in (12, 14):
# YYYYMMDDhhmm[ss]
s = l[i-1]
ymd.append(int(s[:4]))
ymd.append(int(s[4:6]))
ymd.append(int(s[6:8]))
res.hour = int(s[8:10])
res.minute = int(s[10:12])
if len_li == 14:
res.second = int(s[12:])
elif ((i < len_l and info.hms(l[i]) is not None) or
(i+1 < len_l and l[i] == ' ' and
info.hms(l[i+1]) is not None)):
# HH[ ]h or MM[ ]m or SS[.ss][ ]s
if l[i] == ' ':
i += 1
idx = info.hms(l[i])
while True:
if idx == 0:
res.hour = int(value)
if value%1:
res.minute = int(60*(value%1))
elif idx == 1:
res.minute = int(value)
if value%1:
res.second = int(60*(value%1))
elif idx == 2:
res.second, res.microsecond = \
_parsems(value_repr)
i += 1
if i >= len_l or idx == 2:
break
# 12h00
try:
value_repr = l[i]
value = float(value_repr)
except ValueError:
break
else:
i += 1
idx += 1
if i < len_l:
newidx = info.hms(l[i])
if newidx is not None:
idx = newidx
elif i+1 < len_l and l[i] == ':':
# HH:MM[:SS[.ss]]
res.hour = int(value)
i += 1
value = float(l[i])
res.minute = int(value)
if value%1:
res.second = int(60*(value%1))
i += 1
if i < len_l and l[i] == ':':
res.second, res.microsecond = _parsems(l[i+1])
i += 2
elif i < len_l and l[i] in ('-', '/', '.'):
sep = l[i]
ymd.append(int(value))
i += 1
if i < len_l and not info.jump(l[i]):
try:
# 01-01[-01]
ymd.append(int(l[i]))
except ValueError:
# 01-Jan[-01]
value = info.month(l[i])
if value is not None:
ymd.append(value)
assert mstridx == -1
mstridx = len(ymd)-1
else:
return None
i += 1
if i < len_l and l[i] == sep:
# We have three members
i += 1
value = info.month(l[i])
if value is not None:
ymd.append(value)
mstridx = len(ymd)-1
assert mstridx == -1
else:
ymd.append(int(l[i]))
i += 1
elif i >= len_l or info.jump(l[i]):
if i+1 < len_l and info.ampm(l[i+1]) is not None:
# 12 am
res.hour = int(value)
if res.hour < 12 and info.ampm(l[i+1]) == 1:
res.hour += 12
elif res.hour == 12 and info.ampm(l[i+1]) == 0:
res.hour = 0
i += 1
else:
# Year, month or day
ymd.append(int(value))
i += 1
elif info.ampm(l[i]) is not None:
# 12am
res.hour = int(value)
if res.hour < 12 and info.ampm(l[i]) == 1:
res.hour += 12
elif res.hour == 12 and info.ampm(l[i]) == 0:
res.hour = 0
i += 1
elif not fuzzy:
return None
else:
i += 1
continue
# Check weekday
value = info.weekday(l[i])
if value is not None:
res.weekday = value
i += 1
continue
# Check month name
value = info.month(l[i])
if value is not None:
ymd.append(value)
assert mstridx == -1
mstridx = len(ymd)-1
i += 1
if i < len_l:
if l[i] in ('-', '/'):
# Jan-01[-99]
sep = l[i]
i += 1
ymd.append(int(l[i]))
i += 1
if i < len_l and l[i] == sep:
# Jan-01-99
i += 1
ymd.append(int(l[i]))
i += 1
elif (i+3 < len_l and l[i] == l[i+2] == ' '
and info.pertain(l[i+1])):
# Jan of 01
# In this case, 01 is clearly year
try:
value = int(l[i+3])
except ValueError:
# Wrong guess
pass
else:
# Convert it here to become unambiguous
ymd.append(info.convertyear(value))
i += 4
continue
# Check am/pm
value = info.ampm(l[i])
if value is not None:
if value == 1 and res.hour < 12:
res.hour += 12
elif value == 0 and res.hour == 12:
res.hour = 0
i += 1
continue
# Check for a timezone name
if (res.hour is not None and len(l[i]) <= 5 and
res.tzname is None and res.tzoffset is None and
not [x for x in l[i] if x not in string.ascii_uppercase]):
res.tzname = l[i]
res.tzoffset = info.tzoffset(res.tzname)
i += 1
# Check for something like GMT+3, or BRST+3. Notice
# that it doesn't mean "I am 3 hours after GMT", but
# "my time +3 is GMT". If found, we reverse the
# logic so that timezone parsing code will get it
# right.
if i < len_l and l[i] in ('+', '-'):
l[i] = ('+', '-')[l[i] == '+']
res.tzoffset = None
if info.utczone(res.tzname):
# With something like GMT+3, the timezone
# is *not* GMT.
res.tzname = None
continue
# Check for a numbered timezone
if res.hour is not None and l[i] in ('+', '-'):
signal = (-1,1)[l[i] == '+']
i += 1
len_li = len(l[i])
if len_li == 4:
# -0300
res.tzoffset = int(l[i][:2])*3600+int(l[i][2:])*60
elif i+1 < len_l and l[i+1] == ':':
# -03:00
res.tzoffset = int(l[i])*3600+int(l[i+2])*60
i += 2
elif len_li <= 2:
# -[0]3
res.tzoffset = int(l[i][:2])*3600
else:
return None
i += 1
res.tzoffset *= signal
# Look for a timezone name between parenthesis
if (i+3 < len_l and
info.jump(l[i]) and l[i+1] == '(' and l[i+3] == ')' and
3 <= len(l[i+2]) <= 5 and
not [x for x in l[i+2]
if x not in string.ascii_uppercase]):
# -0300 (BRST)
res.tzname = l[i+2]
i += 4
continue
# Check jumps
if not (info.jump(l[i]) or fuzzy):
return None
i += 1
# Process year/month/day
len_ymd = len(ymd)
if len_ymd > 3:
# More than three members!?
return None
elif len_ymd == 1 or (mstridx != -1 and len_ymd == 2):
# One member, or two members with a month string
if mstridx != -1:
res.month = ymd[mstridx]
del ymd[mstridx]
if len_ymd > 1 or mstridx == -1:
if ymd[0] > 31:
res.year = ymd[0]
else:
res.day = ymd[0]
elif len_ymd == 2:
# Two members with numbers
if ymd[0] > 31:
# 99-01
res.year, res.month = ymd
elif ymd[1] > 31:
# 01-99
res.month, res.year = ymd
elif dayfirst and ymd[1] <= 12:
# 13-01
res.day, res.month = ymd
else:
# 01-13
res.month, res.day = ymd
if len_ymd == 3:
# Three members
if mstridx == 0:
res.month, res.day, res.year = ymd
elif mstridx == 1:
if ymd[0] > 31 or (yearfirst and ymd[2] <= 31):
# 99-Jan-01
res.year, res.month, res.day = ymd
else:
# 01-Jan-01
# Give precendence to day-first, since
# two-digit years is usually hand-written.
res.day, res.month, res.year = ymd
elif mstridx == 2:
# WTF!?
if ymd[1] > 31:
# 01-99-Jan
res.day, res.year, res.month = ymd
else:
# 99-01-Jan
res.year, res.day, res.month = ymd
else:
if ymd[0] > 31 or \
(yearfirst and ymd[1] <= 12 and ymd[2] <= 31):
# 99-01-01
res.year, res.month, res.day = ymd
elif ymd[0] > 12 or (dayfirst and ymd[1] <= 12):
# 13-01-01
res.day, res.month, res.year = ymd
else:
# 01-13-01
res.month, res.day, res.year = ymd
except (IndexError, ValueError, AssertionError):
return None
if not info.validate(res):
return None
return res
DEFAULTPARSER = parser()
def parse(timestr, parserinfo=None, **kwargs):
if parserinfo:
return parser(parserinfo).parse(timestr, **kwargs)
else:
return DEFAULTPARSER.parse(timestr, **kwargs)
class _tzparser(object):
class _result(_resultbase):
__slots__ = ["stdabbr", "stdoffset", "dstabbr", "dstoffset",
"start", "end"]
class _attr(_resultbase):
__slots__ = ["month", "week", "weekday",
"yday", "jyday", "day", "time"]
def __repr__(self):
return self._repr("")
def __init__(self):
_resultbase.__init__(self)
self.start = self._attr()
self.end = self._attr()
def parse(self, tzstr):
res = self._result()
l = _timelex.split(tzstr)
try:
len_l = len(l)
i = 0
while i < len_l:
# BRST+3[BRDT[+2]]
j = i
while j < len_l and not [x for x in l[j]
if x in "0123456789:,-+"]:
j += 1
if j != i:
if not res.stdabbr:
offattr = "stdoffset"
res.stdabbr = "".join(l[i:j])
else:
offattr = "dstoffset"
res.dstabbr = "".join(l[i:j])
i = j
if (i < len_l and
(l[i] in ('+', '-') or l[i][0] in "0123456789")):
if l[i] in ('+', '-'):
# Yes, that's right. See the TZ variable
# documentation.
signal = (1,-1)[l[i] == '+']
i += 1
else:
signal = -1
len_li = len(l[i])
if len_li == 4:
# -0300
setattr(res, offattr,
(int(l[i][:2])*3600+int(l[i][2:])*60)*signal)
elif i+1 < len_l and l[i+1] == ':':
# -03:00
setattr(res, offattr,
(int(l[i])*3600+int(l[i+2])*60)*signal)
i += 2
elif len_li <= 2:
# -[0]3
setattr(res, offattr,
int(l[i][:2])*3600*signal)
else:
return None
i += 1
if res.dstabbr:
break
else:
break
if i < len_l:
for j in range(i, len_l):
if l[j] == ';': l[j] = ','
assert l[i] == ','
i += 1
if i >= len_l:
pass
elif (8 <= l.count(',') <= 9 and
not [y for x in l[i:] if x != ','
for y in x if y not in "0123456789"]):
# GMT0BST,3,0,30,3600,10,0,26,7200[,3600]
for x in (res.start, res.end):
x.month = int(l[i])
i += 2
if l[i] == '-':
value = int(l[i+1])*-1
i += 1
else:
value = int(l[i])
i += 2
if value:
x.week = value
x.weekday = (int(l[i])-1)%7
else:
x.day = int(l[i])
i += 2
x.time = int(l[i])
i += 2
if i < len_l:
if l[i] in ('-','+'):
signal = (-1,1)[l[i] == "+"]
i += 1
else:
signal = 1
res.dstoffset = (res.stdoffset+int(l[i]))*signal
elif (l.count(',') == 2 and l[i:].count('/') <= 2 and
not [y for x in l[i:] if x not in (',','/','J','M',
'.','-',':')
for y in x if y not in "0123456789"]):
for x in (res.start, res.end):
if l[i] == 'J':
# non-leap year day (1 based)
i += 1
x.jyday = int(l[i])
elif l[i] == 'M':
# month[-.]week[-.]weekday
i += 1
x.month = int(l[i])
i += 1
assert l[i] in ('-', '.')
i += 1
x.week = int(l[i])
if x.week == 5:
x.week = -1
i += 1
assert l[i] in ('-', '.')
i += 1
x.weekday = (int(l[i])-1)%7
else:
# year day (zero based)
x.yday = int(l[i])+1
i += 1
if i < len_l and l[i] == '/':
i += 1
# start time
len_li = len(l[i])
if len_li == 4:
# -0300
x.time = (int(l[i][:2])*3600+int(l[i][2:])*60)
elif i+1 < len_l and l[i+1] == ':':
# -03:00
x.time = int(l[i])*3600+int(l[i+2])*60
i += 2
if i+1 < len_l and l[i+1] == ':':
i += 2
x.time += int(l[i])
elif len_li <= 2:
# -[0]3
x.time = (int(l[i][:2])*3600)
else:
return None
i += 1
assert i == len_l or l[i] == ','
i += 1
assert i >= len_l
except (IndexError, ValueError, AssertionError):
return None
return res
DEFAULTTZPARSER = _tzparser()
def _parsetz(tzstr):
return DEFAULTTZPARSER.parse(tzstr)
def _parsems(value):
"""Parse a I[.F] seconds value into (seconds, microseconds)."""
if "." not in value:
return int(value), 0
else:
i, f = value.split(".")
return int(i), int(f.ljust(6, "0")[:6])
# vim:ts=4:sw=4:et
+61
View File
@@ -0,0 +1,61 @@
# -*- coding: utf-8 -*-
from ._parser import parse, parser, parserinfo, ParserError
from ._parser import DEFAULTPARSER, DEFAULTTZPARSER
from ._parser import UnknownTimezoneWarning
from ._parser import __doc__
from .isoparser import isoparser, isoparse
__all__ = ['parse', 'parser', 'parserinfo',
'isoparse', 'isoparser',
'ParserError',
'UnknownTimezoneWarning']
###
# Deprecate portions of the private interface so that downstream code that
# is improperly relying on it is given *some* notice.
def __deprecated_private_func(f):
from functools import wraps
import warnings
msg = ('{name} is a private function and may break without warning, '
'it will be moved and or renamed in future versions.')
msg = msg.format(name=f.__name__)
@wraps(f)
def deprecated_func(*args, **kwargs):
warnings.warn(msg, DeprecationWarning)
return f(*args, **kwargs)
return deprecated_func
def __deprecate_private_class(c):
import warnings
msg = ('{name} is a private class and may break without warning, '
'it will be moved and or renamed in future versions.')
msg = msg.format(name=c.__name__)
class private_class(c):
__doc__ = c.__doc__
def __init__(self, *args, **kwargs):
warnings.warn(msg, DeprecationWarning)
super(private_class, self).__init__(*args, **kwargs)
private_class.__name__ = c.__name__
return private_class
from ._parser import _timelex, _resultbase
from ._parser import _tzparser, _parsetz
_timelex = __deprecate_private_class(_timelex)
_tzparser = __deprecate_private_class(_tzparser)
_resultbase = __deprecate_private_class(_resultbase)
_parsetz = __deprecated_private_func(_parsetz)
File diff suppressed because it is too large Load Diff
+411
View File
@@ -0,0 +1,411 @@
# -*- coding: utf-8 -*-
"""
This module offers a parser for ISO-8601 strings
It is intended to support all valid date, time and datetime formats per the
ISO-8601 specification.
..versionadded:: 2.7.0
"""
from datetime import datetime, timedelta, time, date
import calendar
from dateutil import tz
from functools import wraps
import re
import six
__all__ = ["isoparse", "isoparser"]
def _takes_ascii(f):
@wraps(f)
def func(self, str_in, *args, **kwargs):
# If it's a stream, read the whole thing
str_in = getattr(str_in, 'read', lambda: str_in)()
# If it's unicode, turn it into bytes, since ISO-8601 only covers ASCII
if isinstance(str_in, six.text_type):
# ASCII is the same in UTF-8
try:
str_in = str_in.encode('ascii')
except UnicodeEncodeError as e:
msg = 'ISO-8601 strings should contain only ASCII characters'
six.raise_from(ValueError(msg), e)
return f(self, str_in, *args, **kwargs)
return func
class isoparser(object):
def __init__(self, sep=None):
"""
:param sep:
A single character that separates date and time portions. If
``None``, the parser will accept any single character.
For strict ISO-8601 adherence, pass ``'T'``.
"""
if sep is not None:
if (len(sep) != 1 or ord(sep) >= 128 or sep in '0123456789'):
raise ValueError('Separator must be a single, non-numeric ' +
'ASCII character')
sep = sep.encode('ascii')
self._sep = sep
@_takes_ascii
def isoparse(self, dt_str):
"""
Parse an ISO-8601 datetime string into a :class:`datetime.datetime`.
An ISO-8601 datetime string consists of a date portion, followed
optionally by a time portion - the date and time portions are separated
by a single character separator, which is ``T`` in the official
standard. Incomplete date formats (such as ``YYYY-MM``) may *not* be
combined with a time portion.
Supported date formats are:
Common:
- ``YYYY``
- ``YYYY-MM`` or ``YYYYMM``
- ``YYYY-MM-DD`` or ``YYYYMMDD``
Uncommon:
- ``YYYY-Www`` or ``YYYYWww`` - ISO week (day defaults to 0)
- ``YYYY-Www-D`` or ``YYYYWwwD`` - ISO week and day
The ISO week and day numbering follows the same logic as
:func:`datetime.date.isocalendar`.
Supported time formats are:
- ``hh``
- ``hh:mm`` or ``hhmm``
- ``hh:mm:ss`` or ``hhmmss``
- ``hh:mm:ss.ssssss`` (Up to 6 sub-second digits)
Midnight is a special case for `hh`, as the standard supports both
00:00 and 24:00 as a representation. The decimal separator can be
either a dot or a comma.
.. caution::
Support for fractional components other than seconds is part of the
ISO-8601 standard, but is not currently implemented in this parser.
Supported time zone offset formats are:
- `Z` (UTC)
- `±HH:MM`
- `±HHMM`
- `±HH`
Offsets will be represented as :class:`dateutil.tz.tzoffset` objects,
with the exception of UTC, which will be represented as
:class:`dateutil.tz.tzutc`. Time zone offsets equivalent to UTC (such
as `+00:00`) will also be represented as :class:`dateutil.tz.tzutc`.
:param dt_str:
A string or stream containing only an ISO-8601 datetime string
:return:
Returns a :class:`datetime.datetime` representing the string.
Unspecified components default to their lowest value.
.. warning::
As of version 2.7.0, the strictness of the parser should not be
considered a stable part of the contract. Any valid ISO-8601 string
that parses correctly with the default settings will continue to
parse correctly in future versions, but invalid strings that
currently fail (e.g. ``2017-01-01T00:00+00:00:00``) are not
guaranteed to continue failing in future versions if they encode
a valid date.
.. versionadded:: 2.7.0
"""
components, pos = self._parse_isodate(dt_str)
if len(dt_str) > pos:
if self._sep is None or dt_str[pos:pos + 1] == self._sep:
components += self._parse_isotime(dt_str[pos + 1:])
else:
raise ValueError('String contains unknown ISO components')
if len(components) > 3 and components[3] == 24:
components[3] = 0
return datetime(*components) + timedelta(days=1)
return datetime(*components)
@_takes_ascii
def parse_isodate(self, datestr):
"""
Parse the date portion of an ISO string.
:param datestr:
The string portion of an ISO string, without a separator
:return:
Returns a :class:`datetime.date` object
"""
components, pos = self._parse_isodate(datestr)
if pos < len(datestr):
raise ValueError('String contains unknown ISO ' +
'components: {}'.format(datestr))
return date(*components)
@_takes_ascii
def parse_isotime(self, timestr):
"""
Parse the time portion of an ISO string.
:param timestr:
The time portion of an ISO string, without a separator
:return:
Returns a :class:`datetime.time` object
"""
components = self._parse_isotime(timestr)
if components[0] == 24:
components[0] = 0
return time(*components)
@_takes_ascii
def parse_tzstr(self, tzstr, zero_as_utc=True):
"""
Parse a valid ISO time zone string.
See :func:`isoparser.isoparse` for details on supported formats.
:param tzstr:
A string representing an ISO time zone offset
:param zero_as_utc:
Whether to return :class:`dateutil.tz.tzutc` for zero-offset zones
:return:
Returns :class:`dateutil.tz.tzoffset` for offsets and
:class:`dateutil.tz.tzutc` for ``Z`` and (if ``zero_as_utc`` is
specified) offsets equivalent to UTC.
"""
return self._parse_tzstr(tzstr, zero_as_utc=zero_as_utc)
# Constants
_DATE_SEP = b'-'
_TIME_SEP = b':'
_FRACTION_REGEX = re.compile(b'[\\.,]([0-9]+)')
def _parse_isodate(self, dt_str):
try:
return self._parse_isodate_common(dt_str)
except ValueError:
return self._parse_isodate_uncommon(dt_str)
def _parse_isodate_common(self, dt_str):
len_str = len(dt_str)
components = [1, 1, 1]
if len_str < 4:
raise ValueError('ISO string too short')
# Year
components[0] = int(dt_str[0:4])
pos = 4
if pos >= len_str:
return components, pos
has_sep = dt_str[pos:pos + 1] == self._DATE_SEP
if has_sep:
pos += 1
# Month
if len_str - pos < 2:
raise ValueError('Invalid common month')
components[1] = int(dt_str[pos:pos + 2])
pos += 2
if pos >= len_str:
if has_sep:
return components, pos
else:
raise ValueError('Invalid ISO format')
if has_sep:
if dt_str[pos:pos + 1] != self._DATE_SEP:
raise ValueError('Invalid separator in ISO string')
pos += 1
# Day
if len_str - pos < 2:
raise ValueError('Invalid common day')
components[2] = int(dt_str[pos:pos + 2])
return components, pos + 2
def _parse_isodate_uncommon(self, dt_str):
if len(dt_str) < 4:
raise ValueError('ISO string too short')
# All ISO formats start with the year
year = int(dt_str[0:4])
has_sep = dt_str[4:5] == self._DATE_SEP
pos = 4 + has_sep # Skip '-' if it's there
if dt_str[pos:pos + 1] == b'W':
# YYYY-?Www-?D?
pos += 1
weekno = int(dt_str[pos:pos + 2])
pos += 2
dayno = 1
if len(dt_str) > pos:
if (dt_str[pos:pos + 1] == self._DATE_SEP) != has_sep:
raise ValueError('Inconsistent use of dash separator')
pos += has_sep
dayno = int(dt_str[pos:pos + 1])
pos += 1
base_date = self._calculate_weekdate(year, weekno, dayno)
else:
# YYYYDDD or YYYY-DDD
if len(dt_str) - pos < 3:
raise ValueError('Invalid ordinal day')
ordinal_day = int(dt_str[pos:pos + 3])
pos += 3
if ordinal_day < 1 or ordinal_day > (365 + calendar.isleap(year)):
raise ValueError('Invalid ordinal day' +
' {} for year {}'.format(ordinal_day, year))
base_date = date(year, 1, 1) + timedelta(days=ordinal_day - 1)
components = [base_date.year, base_date.month, base_date.day]
return components, pos
def _calculate_weekdate(self, year, week, day):
"""
Calculate the day of corresponding to the ISO year-week-day calendar.
This function is effectively the inverse of
:func:`datetime.date.isocalendar`.
:param year:
The year in the ISO calendar
:param week:
The week in the ISO calendar - range is [1, 53]
:param day:
The day in the ISO calendar - range is [1 (MON), 7 (SUN)]
:return:
Returns a :class:`datetime.date`
"""
if not 0 < week < 54:
raise ValueError('Invalid week: {}'.format(week))
if not 0 < day < 8: # Range is 1-7
raise ValueError('Invalid weekday: {}'.format(day))
# Get week 1 for the specific year:
jan_4 = date(year, 1, 4) # Week 1 always has January 4th in it
week_1 = jan_4 - timedelta(days=jan_4.isocalendar()[2] - 1)
# Now add the specific number of weeks and days to get what we want
week_offset = (week - 1) * 7 + (day - 1)
return week_1 + timedelta(days=week_offset)
def _parse_isotime(self, timestr):
len_str = len(timestr)
components = [0, 0, 0, 0, None]
pos = 0
comp = -1
if len(timestr) < 2:
raise ValueError('ISO time too short')
has_sep = len_str >= 3 and timestr[2:3] == self._TIME_SEP
while pos < len_str and comp < 5:
comp += 1
if timestr[pos:pos + 1] in b'-+Zz':
# Detect time zone boundary
components[-1] = self._parse_tzstr(timestr[pos:])
pos = len_str
break
if comp < 3:
# Hour, minute, second
components[comp] = int(timestr[pos:pos + 2])
pos += 2
if (has_sep and pos < len_str and
timestr[pos:pos + 1] == self._TIME_SEP):
pos += 1
if comp == 3:
# Fraction of a second
frac = self._FRACTION_REGEX.match(timestr[pos:])
if not frac:
continue
us_str = frac.group(1)[:6] # Truncate to microseconds
components[comp] = int(us_str) * 10**(6 - len(us_str))
pos += len(frac.group())
if pos < len_str:
raise ValueError('Unused components in ISO string')
if components[0] == 24:
# Standard supports 00:00 and 24:00 as representations of midnight
if any(component != 0 for component in components[1:4]):
raise ValueError('Hour may only be 24 at 24:00:00.000')
return components
def _parse_tzstr(self, tzstr, zero_as_utc=True):
if tzstr == b'Z' or tzstr == b'z':
return tz.UTC
if len(tzstr) not in {3, 5, 6}:
raise ValueError('Time zone offset must be 1, 3, 5 or 6 characters')
if tzstr[0:1] == b'-':
mult = -1
elif tzstr[0:1] == b'+':
mult = 1
else:
raise ValueError('Time zone offset requires sign')
hours = int(tzstr[1:3])
if len(tzstr) == 3:
minutes = 0
else:
minutes = int(tzstr[(4 if tzstr[3:4] == self._TIME_SEP else 3):])
if zero_as_utc and hours == 0 and minutes == 0:
return tz.UTC
else:
if minutes > 59:
raise ValueError('Invalid minutes in time zone offset')
if hours > 23:
raise ValueError('Invalid hours in time zone offset')
return tz.tzoffset(None, mult * (hours * 60 + minutes) * 60)
DEFAULT_ISOPARSER = isoparser()
isoparse = DEFAULT_ISOPARSER.isoparse
+349 -182
View File
@@ -1,73 +1,63 @@
"""
Copyright (c) 2003-2010 Gustavo Niemeyer <gustavo@niemeyer.net>
This module offers extensions to the standard python 2.3+
datetime module.
"""
__author__ = "Gustavo Niemeyer <gustavo@niemeyer.net>"
__license__ = "PSF License"
# -*- coding: utf-8 -*-
import datetime
import calendar
import operator
from math import copysign
from six import integer_types
from warnings import warn
from ._common import weekday
MO, TU, WE, TH, FR, SA, SU = weekdays = tuple(weekday(x) for x in range(7))
__all__ = ["relativedelta", "MO", "TU", "WE", "TH", "FR", "SA", "SU"]
class weekday(object):
__slots__ = ["weekday", "n"]
def __init__(self, weekday, n=None):
self.weekday = weekday
self.n = n
def __call__(self, n):
if n == self.n:
return self
else:
return self.__class__(self.weekday, n)
def __eq__(self, other):
try:
if self.weekday != other.weekday or self.n != other.n:
return False
except AttributeError:
return False
return True
def __repr__(self):
s = ("MO", "TU", "WE", "TH", "FR", "SA", "SU")[self.weekday]
if not self.n:
return s
else:
return "%s(%+d)" % (s, self.n)
MO, TU, WE, TH, FR, SA, SU = weekdays = tuple([weekday(x) for x in range(7)])
class relativedelta:
class relativedelta(object):
"""
The relativedelta type is based on the specification of the excelent
work done by M.-A. Lemburg in his mx.DateTime extension. However,
notice that this type does *NOT* implement the same algorithm as
his work. Do *NOT* expect it to behave like mx.DateTime's counterpart.
The relativedelta type is designed to be applied to an existing datetime and
can replace specific components of that datetime, or represents an interval
of time.
There's two different ways to build a relativedelta instance. The
first one is passing it two date/datetime classes:
It is based on the specification of the excellent work done by M.-A. Lemburg
in his
`mx.DateTime <https://www.egenix.com/products/python/mxBase/mxDateTime/>`_ extension.
However, notice that this type does *NOT* implement the same algorithm as
his work. Do *NOT* expect it to behave like mx.DateTime's counterpart.
There are two different ways to build a relativedelta instance. The
first one is passing it two date/datetime classes::
relativedelta(datetime1, datetime2)
And the other way is to use the following keyword arguments:
The second one is passing it any number of the following keyword arguments::
relativedelta(arg1=x,arg2=y,arg3=z...)
year, month, day, hour, minute, second, microsecond:
Absolute information.
Absolute information (argument is singular); adding or subtracting a
relativedelta with absolute information does not perform an arithmetic
operation, but rather REPLACES the corresponding value in the
original datetime with the value(s) in relativedelta.
years, months, weeks, days, hours, minutes, seconds, microseconds:
Relative information, may be negative.
Relative information, may be negative (argument is plural); adding
or subtracting a relativedelta with relative information performs
the corresponding arithmetic operation on the original datetime value
with the information in the relativedelta.
weekday:
One of the weekday instances (MO, TU, etc). These instances may
receive a parameter N, specifying the Nth weekday, which could
be positive or negative (like MO(+1) or MO(-2). Not specifying
it is the same as specifying +1. You can also use an integer,
where 0=MO.
One of the weekday instances (MO, TU, etc) available in the
relativedelta module. These instances may receive a parameter N,
specifying the Nth weekday, which could be positive or negative
(like MO(+1) or MO(-2)). Not specifying it is the same as specifying
+1. You can also use an integer, where 0=MO. This argument is always
relative e.g. if the calculated date is already Monday, using MO(1)
or MO(-1) won't change the day. To effectively make it absolute, use
it in combination with the day argument (e.g. day=1, MO(1) for first
Monday of the month).
leapdays:
Will add given days to the date found, if year is a leap
@@ -77,33 +67,39 @@ And the other way is to use the following keyword arguments:
Set the yearday or the non-leap year day (jump leap days).
These are converted to day/month/leapdays information.
Here is the behavior of operations with relativedelta:
There are relative and absolute forms of the keyword
arguments. The plural is relative, and the singular is
absolute. For each argument in the order below, the absolute form
is applied first (by setting each attribute to that value) and
then the relative form (by adding the value to the attribute).
1) Calculate the absolute year, using the 'year' argument, or the
original datetime year, if the argument is not present.
The order of attributes considered when this relativedelta is
added to a datetime is:
2) Add the relative 'years' argument to the absolute year.
1. Year
2. Month
3. Day
4. Hours
5. Minutes
6. Seconds
7. Microseconds
3) Do steps 1 and 2 for month/months.
Finally, weekday is applied, using the rule described above.
4) Calculate the absolute day, using the 'day' argument, or the
original datetime day, if the argument is not present. Then,
subtract from the day until it fits in the year and month
found after their operations.
For example
5) Add the relative 'days' argument to the absolute day. Notice
that the 'weeks' argument is multiplied by 7 and added to
'days'.
>>> from datetime import datetime
>>> from dateutil.relativedelta import relativedelta, MO
>>> dt = datetime(2018, 4, 9, 13, 37, 0)
>>> delta = relativedelta(hours=25, day=1, weekday=MO(1))
>>> dt + delta
datetime.datetime(2018, 4, 2, 14, 37)
6) Do steps 1 and 2 for hour/hours, minute/minutes, second/seconds,
microsecond/microseconds.
First, the day is set to 1 (the first of the month), then 25 hours
are added, to get to the 2nd day and 14th hour, finally the
weekday is applied, but since the 2nd is already a Monday there is
no effect.
7) If the 'weekday' argument is present, calculate the weekday,
with the given (wday, nth) tuple. wday is the index of the
weekday (0-6, 0=Mon), and nth is the number of weeks to add
forward or backward, depending on its signal. Notice that if
the calculated date is already Monday, for example, using
(0, 1) or (0, -1) won't change the day.
"""
def __init__(self, dt1=None, dt2=None,
@@ -112,15 +108,22 @@ Here is the behavior of operations with relativedelta:
year=None, month=None, day=None, weekday=None,
yearday=None, nlyearday=None,
hour=None, minute=None, second=None, microsecond=None):
if dt1 and dt2:
if not isinstance(dt1, datetime.date) or \
not isinstance(dt2, datetime.date):
raise TypeError, "relativedelta only diffs datetime/date"
if type(dt1) is not type(dt2):
# datetime is a subclass of date. So both must be date
if not (isinstance(dt1, datetime.date) and
isinstance(dt2, datetime.date)):
raise TypeError("relativedelta only diffs datetime/date")
# We allow two dates, or two datetimes, so we coerce them to be
# of the same type
if (isinstance(dt1, datetime.datetime) !=
isinstance(dt2, datetime.datetime)):
if not isinstance(dt1, datetime.datetime):
dt1 = datetime.datetime.fromordinal(dt1.toordinal())
elif not isinstance(dt2, datetime.datetime):
dt2 = datetime.datetime.fromordinal(dt2.toordinal())
self.years = 0
self.months = 0
self.days = 0
@@ -139,31 +142,48 @@ Here is the behavior of operations with relativedelta:
self.microsecond = None
self._has_time = 0
months = (dt1.year*12+dt1.month)-(dt2.year*12+dt2.month)
# Get year / month delta between the two
months = (dt1.year - dt2.year) * 12 + (dt1.month - dt2.month)
self._set_months(months)
# Remove the year/month delta so the timedelta is just well-defined
# time units (seconds, days and microseconds)
dtm = self.__radd__(dt2)
# If we've overshot our target, make an adjustment
if dt1 < dt2:
while dt1 > dtm:
months += 1
self._set_months(months)
dtm = self.__radd__(dt2)
compare = operator.gt
increment = 1
else:
while dt1 < dtm:
months -= 1
compare = operator.lt
increment = -1
while compare(dt1, dtm):
months += increment
self._set_months(months)
dtm = self.__radd__(dt2)
# Get the timedelta between the "months-adjusted" date and dt1
delta = dt1 - dtm
self.seconds = delta.seconds+delta.days*86400
self.seconds = delta.seconds + delta.days * 86400
self.microseconds = delta.microseconds
else:
self.years = years
self.months = months
self.days = days+weeks*7
# Check for non-integer values in integer-only quantities
if any(x is not None and x != int(x) for x in (years, months)):
raise ValueError("Non-integer years and months are "
"ambiguous and not currently supported.")
# Relative information
self.years = int(years)
self.months = int(months)
self.days = days + weeks * 7
self.leapdays = leapdays
self.hours = hours
self.minutes = minutes
self.seconds = seconds
self.microseconds = microseconds
# Absolute information
self.year = year
self.month = month
self.day = day
@@ -172,7 +192,15 @@ Here is the behavior of operations with relativedelta:
self.second = second
self.microsecond = microsecond
if type(weekday) is int:
if any(x is not None and int(x) != x
for x in (year, month, day, hour,
minute, second, microsecond)):
# For now we'll deprecate floats - later it'll be an error.
warn("Non-integer value passed as absolute information. " +
"This is not a well-defined condition and will raise " +
"errors in future versions.", DeprecationWarning)
if isinstance(weekday, integer_types):
self.weekday = weekdays[weekday]
else:
self.weekday = weekday
@@ -185,7 +213,8 @@ Here is the behavior of operations with relativedelta:
if yearday > 59:
self.leapdays = -1
if yday:
ydayidx = [31,59,90,120,151,181,212,243,273,304,334,366]
ydayidx = [31, 59, 90, 120, 151, 181, 212,
243, 273, 304, 334, 366]
for idx, ydays in enumerate(ydayidx):
if yday <= ydays:
self.month = idx+1
@@ -195,56 +224,143 @@ Here is the behavior of operations with relativedelta:
self.day = yday-ydayidx[idx-1]
break
else:
raise ValueError, "invalid year day (%d)" % yday
raise ValueError("invalid year day (%d)" % yday)
self._fix()
def _fix(self):
if abs(self.microseconds) > 999999:
s = self.microseconds//abs(self.microseconds)
div, mod = divmod(self.microseconds*s, 1000000)
self.microseconds = mod*s
self.seconds += div*s
s = _sign(self.microseconds)
div, mod = divmod(self.microseconds * s, 1000000)
self.microseconds = mod * s
self.seconds += div * s
if abs(self.seconds) > 59:
s = self.seconds//abs(self.seconds)
div, mod = divmod(self.seconds*s, 60)
self.seconds = mod*s
self.minutes += div*s
s = _sign(self.seconds)
div, mod = divmod(self.seconds * s, 60)
self.seconds = mod * s
self.minutes += div * s
if abs(self.minutes) > 59:
s = self.minutes//abs(self.minutes)
div, mod = divmod(self.minutes*s, 60)
self.minutes = mod*s
self.hours += div*s
s = _sign(self.minutes)
div, mod = divmod(self.minutes * s, 60)
self.minutes = mod * s
self.hours += div * s
if abs(self.hours) > 23:
s = self.hours//abs(self.hours)
div, mod = divmod(self.hours*s, 24)
self.hours = mod*s
self.days += div*s
s = _sign(self.hours)
div, mod = divmod(self.hours * s, 24)
self.hours = mod * s
self.days += div * s
if abs(self.months) > 11:
s = self.months//abs(self.months)
div, mod = divmod(self.months*s, 12)
self.months = mod*s
self.years += div*s
if (self.hours or self.minutes or self.seconds or self.microseconds or
self.hour is not None or self.minute is not None or
s = _sign(self.months)
div, mod = divmod(self.months * s, 12)
self.months = mod * s
self.years += div * s
if (self.hours or self.minutes or self.seconds or self.microseconds
or self.hour is not None or self.minute is not None or
self.second is not None or self.microsecond is not None):
self._has_time = 1
else:
self._has_time = 0
@property
def weeks(self):
return int(self.days / 7.0)
@weeks.setter
def weeks(self, value):
self.days = self.days - (self.weeks * 7) + value * 7
def _set_months(self, months):
self.months = months
if abs(self.months) > 11:
s = self.months//abs(self.months)
div, mod = divmod(self.months*s, 12)
self.months = mod*s
self.years = div*s
s = _sign(self.months)
div, mod = divmod(self.months * s, 12)
self.months = mod * s
self.years = div * s
else:
self.years = 0
def __radd__(self, other):
def normalized(self):
"""
Return a version of this object represented entirely using integer
values for the relative attributes.
>>> relativedelta(days=1.5, hours=2).normalized()
relativedelta(days=+1, hours=+14)
:return:
Returns a :class:`dateutil.relativedelta.relativedelta` object.
"""
# Cascade remainders down (rounding each to roughly nearest microsecond)
days = int(self.days)
hours_f = round(self.hours + 24 * (self.days - days), 11)
hours = int(hours_f)
minutes_f = round(self.minutes + 60 * (hours_f - hours), 10)
minutes = int(minutes_f)
seconds_f = round(self.seconds + 60 * (minutes_f - minutes), 8)
seconds = int(seconds_f)
microseconds = round(self.microseconds + 1e6 * (seconds_f - seconds))
# Constructor carries overflow back up with call to _fix()
return self.__class__(years=self.years, months=self.months,
days=days, hours=hours, minutes=minutes,
seconds=seconds, microseconds=microseconds,
leapdays=self.leapdays, year=self.year,
month=self.month, day=self.day,
weekday=self.weekday, hour=self.hour,
minute=self.minute, second=self.second,
microsecond=self.microsecond)
def __add__(self, other):
if isinstance(other, relativedelta):
return self.__class__(years=other.years + self.years,
months=other.months + self.months,
days=other.days + self.days,
hours=other.hours + self.hours,
minutes=other.minutes + self.minutes,
seconds=other.seconds + self.seconds,
microseconds=(other.microseconds +
self.microseconds),
leapdays=other.leapdays or self.leapdays,
year=(other.year if other.year is not None
else self.year),
month=(other.month if other.month is not None
else self.month),
day=(other.day if other.day is not None
else self.day),
weekday=(other.weekday if other.weekday is not None
else self.weekday),
hour=(other.hour if other.hour is not None
else self.hour),
minute=(other.minute if other.minute is not None
else self.minute),
second=(other.second if other.second is not None
else self.second),
microsecond=(other.microsecond if other.microsecond
is not None else
self.microsecond))
if isinstance(other, datetime.timedelta):
return self.__class__(years=self.years,
months=self.months,
days=self.days + other.days,
hours=self.hours,
minutes=self.minutes,
seconds=self.seconds + other.seconds,
microseconds=self.microseconds + other.microseconds,
leapdays=self.leapdays,
year=self.year,
month=self.month,
day=self.day,
weekday=self.weekday,
hour=self.hour,
minute=self.minute,
second=self.second,
microsecond=self.microsecond)
if not isinstance(other, datetime.date):
raise TypeError, "unsupported type for add operation"
return NotImplemented
elif self._has_time and not isinstance(other, datetime.datetime):
other = datetime.datetime.fromordinal(other.toordinal())
year = (self.year or other.year)+self.years
@@ -276,60 +392,70 @@ Here is the behavior of operations with relativedelta:
microseconds=self.microseconds))
if self.weekday:
weekday, nth = self.weekday.weekday, self.weekday.n or 1
jumpdays = (abs(nth)-1)*7
jumpdays = (abs(nth) - 1) * 7
if nth > 0:
jumpdays += (7-ret.weekday()+weekday)%7
jumpdays += (7 - ret.weekday() + weekday) % 7
else:
jumpdays += (ret.weekday()-weekday)%7
jumpdays += (ret.weekday() - weekday) % 7
jumpdays *= -1
ret += datetime.timedelta(days=jumpdays)
return ret
def __radd__(self, other):
return self.__add__(other)
def __rsub__(self, other):
return self.__neg__().__radd__(other)
def __add__(self, other):
if not isinstance(other, relativedelta):
raise TypeError, "unsupported type for add operation"
return relativedelta(years=other.years+self.years,
months=other.months+self.months,
days=other.days+self.days,
hours=other.hours+self.hours,
minutes=other.minutes+self.minutes,
seconds=other.seconds+self.seconds,
microseconds=other.microseconds+self.microseconds,
leapdays=other.leapdays or self.leapdays,
year=other.year or self.year,
month=other.month or self.month,
day=other.day or self.day,
weekday=other.weekday or self.weekday,
hour=other.hour or self.hour,
minute=other.minute or self.minute,
second=other.second or self.second,
microsecond=other.second or self.microsecond)
def __sub__(self, other):
if not isinstance(other, relativedelta):
raise TypeError, "unsupported type for sub operation"
return relativedelta(years=other.years-self.years,
months=other.months-self.months,
days=other.days-self.days,
hours=other.hours-self.hours,
minutes=other.minutes-self.minutes,
seconds=other.seconds-self.seconds,
microseconds=other.microseconds-self.microseconds,
leapdays=other.leapdays or self.leapdays,
year=other.year or self.year,
month=other.month or self.month,
day=other.day or self.day,
weekday=other.weekday or self.weekday,
hour=other.hour or self.hour,
minute=other.minute or self.minute,
second=other.second or self.second,
microsecond=other.second or self.microsecond)
return NotImplemented # In case the other object defines __rsub__
return self.__class__(years=self.years - other.years,
months=self.months - other.months,
days=self.days - other.days,
hours=self.hours - other.hours,
minutes=self.minutes - other.minutes,
seconds=self.seconds - other.seconds,
microseconds=self.microseconds - other.microseconds,
leapdays=self.leapdays or other.leapdays,
year=(self.year if self.year is not None
else other.year),
month=(self.month if self.month is not None else
other.month),
day=(self.day if self.day is not None else
other.day),
weekday=(self.weekday if self.weekday is not None else
other.weekday),
hour=(self.hour if self.hour is not None else
other.hour),
minute=(self.minute if self.minute is not None else
other.minute),
second=(self.second if self.second is not None else
other.second),
microsecond=(self.microsecond if self.microsecond
is not None else
other.microsecond))
def __abs__(self):
return self.__class__(years=abs(self.years),
months=abs(self.months),
days=abs(self.days),
hours=abs(self.hours),
minutes=abs(self.minutes),
seconds=abs(self.seconds),
microseconds=abs(self.microseconds),
leapdays=self.leapdays,
year=self.year,
month=self.month,
day=self.day,
weekday=self.weekday,
hour=self.hour,
minute=self.minute,
second=self.second,
microsecond=self.microsecond)
def __neg__(self):
return relativedelta(years=-self.years,
return self.__class__(years=-self.years,
months=-self.months,
days=-self.days,
hours=-self.hours,
@@ -346,7 +472,7 @@ Here is the behavior of operations with relativedelta:
second=self.second,
microsecond=self.microsecond)
def __nonzero__(self):
def __bool__(self):
return not (not self.years and
not self.months and
not self.days and
@@ -363,29 +489,37 @@ Here is the behavior of operations with relativedelta:
self.minute is None and
self.second is None and
self.microsecond is None)
# Compatibility with Python 2.x
__nonzero__ = __bool__
def __mul__(self, other):
try:
f = float(other)
return relativedelta(years = int(round(self.years*f)),
months = int(round(self.months*f)),
days = int(round(self.days*f)),
hours = int(round(self.hours*f)),
minutes = int(round(self.minutes*f)),
seconds = int(round(self.seconds*f)),
microseconds = self.microseconds*f,
leapdays = self.leapdays,
year = self.year,
month = self.month,
day = self.day,
weekday = self.weekday,
hour = self.hour,
minute = self.minute,
second = self.second,
microsecond = self.microsecond)
except TypeError:
return NotImplemented
return self.__class__(years=int(self.years * f),
months=int(self.months * f),
days=int(self.days * f),
hours=int(self.hours * f),
minutes=int(self.minutes * f),
seconds=int(self.seconds * f),
microseconds=int(self.microseconds * f),
leapdays=self.leapdays,
year=self.year,
month=self.month,
day=self.day,
weekday=self.weekday,
hour=self.hour,
minute=self.minute,
second=self.second,
microsecond=self.microsecond)
__rmul__ = __mul__
def __eq__(self, other):
if not isinstance(other, relativedelta):
return False
return NotImplemented
if self.weekday or other.weekday:
if not self.weekday or not other.weekday:
return False
@@ -400,6 +534,7 @@ Here is the behavior of operations with relativedelta:
self.hours == other.hours and
self.minutes == other.minutes and
self.seconds == other.seconds and
self.microseconds == other.microseconds and
self.leapdays == other.leapdays and
self.year == other.year and
self.month == other.month and
@@ -409,11 +544,38 @@ Here is the behavior of operations with relativedelta:
self.second == other.second and
self.microsecond == other.microsecond)
def __hash__(self):
return hash((
self.weekday,
self.years,
self.months,
self.days,
self.hours,
self.minutes,
self.seconds,
self.microseconds,
self.leapdays,
self.year,
self.month,
self.day,
self.hour,
self.minute,
self.second,
self.microsecond,
))
def __ne__(self, other):
return not self.__eq__(other)
def __div__(self, other):
return self.__mul__(1/float(other))
try:
reciprocal = 1 / float(other)
except TypeError:
return NotImplemented
return self.__mul__(reciprocal)
__truediv__ = __div__
def __repr__(self):
l = []
@@ -421,12 +583,17 @@ Here is the behavior of operations with relativedelta:
"hours", "minutes", "seconds", "microseconds"]:
value = getattr(self, attr)
if value:
l.append("%s=%+d" % (attr, value))
l.append("{attr}={value:+g}".format(attr=attr, value=value))
for attr in ["year", "month", "day", "weekday",
"hour", "minute", "second", "microsecond"]:
value = getattr(self, attr)
if value is not None:
l.append("%s=%s" % (attr, `value`))
return "%s(%s)" % (self.__class__.__name__, ", ".join(l))
l.append("{attr}={value}".format(attr=attr, value=repr(value)))
return "{classname}({attrs})".format(classname=self.__class__.__name__,
attrs=", ".join(l))
def _sign(x):
return int(copysign(1, x))
# vim:ts=4:sw=4:et
+883 -256
View File
File diff suppressed because it is too large Load Diff
-958
View File
@@ -1,958 +0,0 @@
"""
Copyright (c) 2003-2007 Gustavo Niemeyer <gustavo@niemeyer.net>
This module offers extensions to the standard python 2.3+
datetime module.
"""
__author__ = "Gustavo Niemeyer <gustavo@niemeyer.net>"
__license__ = "PSF License"
import datetime
import struct
import time
import sys
import os
relativedelta = None
parser = None
rrule = None
__all__ = ["tzutc", "tzoffset", "tzlocal", "tzfile", "tzrange",
"tzstr", "tzical", "tzwin", "tzwinlocal", "gettz"]
try:
from dateutil.tzwin import tzwin, tzwinlocal
except (ImportError, OSError):
tzwin, tzwinlocal = None, None
ZERO = datetime.timedelta(0)
EPOCHORDINAL = datetime.datetime.utcfromtimestamp(0).toordinal()
class tzutc(datetime.tzinfo):
def utcoffset(self, dt):
return ZERO
def dst(self, dt):
return ZERO
def tzname(self, dt):
return "UTC"
def __eq__(self, other):
return (isinstance(other, tzutc) or
(isinstance(other, tzoffset) and other._offset == ZERO))
def __ne__(self, other):
return not self.__eq__(other)
def __repr__(self):
return "%s()" % self.__class__.__name__
__reduce__ = object.__reduce__
class tzoffset(datetime.tzinfo):
def __init__(self, name, offset):
self._name = name
self._offset = datetime.timedelta(seconds=offset)
def utcoffset(self, dt):
return self._offset
def dst(self, dt):
return ZERO
def tzname(self, dt):
return self._name
def __eq__(self, other):
return (isinstance(other, tzoffset) and
self._offset == other._offset)
def __ne__(self, other):
return not self.__eq__(other)
def __repr__(self):
return "%s(%s, %s)" % (self.__class__.__name__,
`self._name`,
self._offset.days*86400+self._offset.seconds)
__reduce__ = object.__reduce__
class tzlocal(datetime.tzinfo):
_std_offset = datetime.timedelta(seconds=-time.timezone)
if time.daylight:
_dst_offset = datetime.timedelta(seconds=-time.altzone)
else:
_dst_offset = _std_offset
def utcoffset(self, dt):
if self._isdst(dt):
return self._dst_offset
else:
return self._std_offset
def dst(self, dt):
if self._isdst(dt):
return self._dst_offset-self._std_offset
else:
return ZERO
def tzname(self, dt):
return time.tzname[self._isdst(dt)]
def _isdst(self, dt):
# We can't use mktime here. It is unstable when deciding if
# the hour near to a change is DST or not.
#
# timestamp = time.mktime((dt.year, dt.month, dt.day, dt.hour,
# dt.minute, dt.second, dt.weekday(), 0, -1))
# return time.localtime(timestamp).tm_isdst
#
# The code above yields the following result:
#
#>>> import tz, datetime
#>>> t = tz.tzlocal()
#>>> datetime.datetime(2003,2,15,23,tzinfo=t).tzname()
#'BRDT'
#>>> datetime.datetime(2003,2,16,0,tzinfo=t).tzname()
#'BRST'
#>>> datetime.datetime(2003,2,15,23,tzinfo=t).tzname()
#'BRST'
#>>> datetime.datetime(2003,2,15,22,tzinfo=t).tzname()
#'BRDT'
#>>> datetime.datetime(2003,2,15,23,tzinfo=t).tzname()
#'BRDT'
#
# Here is a more stable implementation:
#
timestamp = ((dt.toordinal() - EPOCHORDINAL) * 86400
+ dt.hour * 3600
+ dt.minute * 60
+ dt.second)
return time.localtime(timestamp+time.timezone).tm_isdst
def __eq__(self, other):
if not isinstance(other, tzlocal):
return False
return (self._std_offset == other._std_offset and
self._dst_offset == other._dst_offset)
return True
def __ne__(self, other):
return not self.__eq__(other)
def __repr__(self):
return "%s()" % self.__class__.__name__
__reduce__ = object.__reduce__
class _ttinfo(object):
__slots__ = ["offset", "delta", "isdst", "abbr", "isstd", "isgmt"]
def __init__(self):
for attr in self.__slots__:
setattr(self, attr, None)
def __repr__(self):
l = []
for attr in self.__slots__:
value = getattr(self, attr)
if value is not None:
l.append("%s=%s" % (attr, `value`))
return "%s(%s)" % (self.__class__.__name__, ", ".join(l))
def __eq__(self, other):
if not isinstance(other, _ttinfo):
return False
return (self.offset == other.offset and
self.delta == other.delta and
self.isdst == other.isdst and
self.abbr == other.abbr and
self.isstd == other.isstd and
self.isgmt == other.isgmt)
def __ne__(self, other):
return not self.__eq__(other)
def __getstate__(self):
state = {}
for name in self.__slots__:
state[name] = getattr(self, name, None)
return state
def __setstate__(self, state):
for name in self.__slots__:
if name in state:
setattr(self, name, state[name])
class tzfile(datetime.tzinfo):
# http://www.twinsun.com/tz/tz-link.htm
# ftp://elsie.nci.nih.gov/pub/tz*.tar.gz
def __init__(self, fileobj):
if isinstance(fileobj, basestring):
self._filename = fileobj
fileobj = open(fileobj)
elif hasattr(fileobj, "name"):
self._filename = fileobj.name
else:
self._filename = `fileobj`
# From tzfile(5):
#
# The time zone information files used by tzset(3)
# begin with the magic characters "TZif" to identify
# them as time zone information files, followed by
# sixteen bytes reserved for future use, followed by
# six four-byte values of type long, written in a
# ``standard'' byte order (the high-order byte
# of the value is written first).
if fileobj.read(4) != "TZif":
raise ValueError, "magic not found"
fileobj.read(16)
(
# The number of UTC/local indicators stored in the file.
ttisgmtcnt,
# The number of standard/wall indicators stored in the file.
ttisstdcnt,
# The number of leap seconds for which data is
# stored in the file.
leapcnt,
# The number of "transition times" for which data
# is stored in the file.
timecnt,
# The number of "local time types" for which data
# is stored in the file (must not be zero).
typecnt,
# The number of characters of "time zone
# abbreviation strings" stored in the file.
charcnt,
) = struct.unpack(">6l", fileobj.read(24))
# The above header is followed by tzh_timecnt four-byte
# values of type long, sorted in ascending order.
# These values are written in ``standard'' byte order.
# Each is used as a transition time (as returned by
# time(2)) at which the rules for computing local time
# change.
if timecnt:
self._trans_list = struct.unpack(">%dl" % timecnt,
fileobj.read(timecnt*4))
else:
self._trans_list = []
# Next come tzh_timecnt one-byte values of type unsigned
# char; each one tells which of the different types of
# ``local time'' types described in the file is associated
# with the same-indexed transition time. These values
# serve as indices into an array of ttinfo structures that
# appears next in the file.
if timecnt:
self._trans_idx = struct.unpack(">%dB" % timecnt,
fileobj.read(timecnt))
else:
self._trans_idx = []
# Each ttinfo structure is written as a four-byte value
# for tt_gmtoff of type long, in a standard byte
# order, followed by a one-byte value for tt_isdst
# and a one-byte value for tt_abbrind. In each
# structure, tt_gmtoff gives the number of
# seconds to be added to UTC, tt_isdst tells whether
# tm_isdst should be set by localtime(3), and
# tt_abbrind serves as an index into the array of
# time zone abbreviation characters that follow the
# ttinfo structure(s) in the file.
ttinfo = []
for i in range(typecnt):
ttinfo.append(struct.unpack(">lbb", fileobj.read(6)))
abbr = fileobj.read(charcnt)
# Then there are tzh_leapcnt pairs of four-byte
# values, written in standard byte order; the
# first value of each pair gives the time (as
# returned by time(2)) at which a leap second
# occurs; the second gives the total number of
# leap seconds to be applied after the given time.
# The pairs of values are sorted in ascending order
# by time.
# Not used, for now
if leapcnt:
leap = struct.unpack(">%dl" % (leapcnt*2),
fileobj.read(leapcnt*8))
# Then there are tzh_ttisstdcnt standard/wall
# indicators, each stored as a one-byte value;
# they tell whether the transition times associated
# with local time types were specified as standard
# time or wall clock time, and are used when
# a time zone file is used in handling POSIX-style
# time zone environment variables.
if ttisstdcnt:
isstd = struct.unpack(">%db" % ttisstdcnt,
fileobj.read(ttisstdcnt))
# Finally, there are tzh_ttisgmtcnt UTC/local
# indicators, each stored as a one-byte value;
# they tell whether the transition times associated
# with local time types were specified as UTC or
# local time, and are used when a time zone file
# is used in handling POSIX-style time zone envi-
# ronment variables.
if ttisgmtcnt:
isgmt = struct.unpack(">%db" % ttisgmtcnt,
fileobj.read(ttisgmtcnt))
# ** Everything has been read **
# Build ttinfo list
self._ttinfo_list = []
for i in range(typecnt):
gmtoff, isdst, abbrind = ttinfo[i]
# Round to full-minutes if that's not the case. Python's
# datetime doesn't accept sub-minute timezones. Check
# http://python.org/sf/1447945 for some information.
gmtoff = (gmtoff+30)//60*60
tti = _ttinfo()
tti.offset = gmtoff
tti.delta = datetime.timedelta(seconds=gmtoff)
tti.isdst = isdst
tti.abbr = abbr[abbrind:abbr.find('\x00', abbrind)]
tti.isstd = (ttisstdcnt > i and isstd[i] != 0)
tti.isgmt = (ttisgmtcnt > i and isgmt[i] != 0)
self._ttinfo_list.append(tti)
# Replace ttinfo indexes for ttinfo objects.
trans_idx = []
for idx in self._trans_idx:
trans_idx.append(self._ttinfo_list[idx])
self._trans_idx = tuple(trans_idx)
# Set standard, dst, and before ttinfos. before will be
# used when a given time is before any transitions,
# and will be set to the first non-dst ttinfo, or to
# the first dst, if all of them are dst.
self._ttinfo_std = None
self._ttinfo_dst = None
self._ttinfo_before = None
if self._ttinfo_list:
if not self._trans_list:
self._ttinfo_std = self._ttinfo_first = self._ttinfo_list[0]
else:
for i in range(timecnt-1,-1,-1):
tti = self._trans_idx[i]
if not self._ttinfo_std and not tti.isdst:
self._ttinfo_std = tti
elif not self._ttinfo_dst and tti.isdst:
self._ttinfo_dst = tti
if self._ttinfo_std and self._ttinfo_dst:
break
else:
if self._ttinfo_dst and not self._ttinfo_std:
self._ttinfo_std = self._ttinfo_dst
for tti in self._ttinfo_list:
if not tti.isdst:
self._ttinfo_before = tti
break
else:
self._ttinfo_before = self._ttinfo_list[0]
# Now fix transition times to become relative to wall time.
#
# I'm not sure about this. In my tests, the tz source file
# is setup to wall time, and in the binary file isstd and
# isgmt are off, so it should be in wall time. OTOH, it's
# always in gmt time. Let me know if you have comments
# about this.
laststdoffset = 0
self._trans_list = list(self._trans_list)
for i in range(len(self._trans_list)):
tti = self._trans_idx[i]
if not tti.isdst:
# This is std time.
self._trans_list[i] += tti.offset
laststdoffset = tti.offset
else:
# This is dst time. Convert to std.
self._trans_list[i] += laststdoffset
self._trans_list = tuple(self._trans_list)
def _find_ttinfo(self, dt, laststd=0):
timestamp = ((dt.toordinal() - EPOCHORDINAL) * 86400
+ dt.hour * 3600
+ dt.minute * 60
+ dt.second)
idx = 0
for trans in self._trans_list:
if timestamp < trans:
break
idx += 1
else:
return self._ttinfo_std
if idx == 0:
return self._ttinfo_before
if laststd:
while idx > 0:
tti = self._trans_idx[idx-1]
if not tti.isdst:
return tti
idx -= 1
else:
return self._ttinfo_std
else:
return self._trans_idx[idx-1]
def utcoffset(self, dt):
if not self._ttinfo_std:
return ZERO
return self._find_ttinfo(dt).delta
def dst(self, dt):
if not self._ttinfo_dst:
return ZERO
tti = self._find_ttinfo(dt)
if not tti.isdst:
return ZERO
# The documentation says that utcoffset()-dst() must
# be constant for every dt.
return tti.delta-self._find_ttinfo(dt, laststd=1).delta
# An alternative for that would be:
#
# return self._ttinfo_dst.offset-self._ttinfo_std.offset
#
# However, this class stores historical changes in the
# dst offset, so I belive that this wouldn't be the right
# way to implement this.
def tzname(self, dt):
if not self._ttinfo_std:
return None
return self._find_ttinfo(dt).abbr
def __eq__(self, other):
if not isinstance(other, tzfile):
return False
return (self._trans_list == other._trans_list and
self._trans_idx == other._trans_idx and
self._ttinfo_list == other._ttinfo_list)
def __ne__(self, other):
return not self.__eq__(other)
def __repr__(self):
return "%s(%s)" % (self.__class__.__name__, `self._filename`)
def __reduce__(self):
if not os.path.isfile(self._filename):
raise ValueError, "Unpickable %s class" % self.__class__.__name__
return (self.__class__, (self._filename,))
class tzrange(datetime.tzinfo):
def __init__(self, stdabbr, stdoffset=None,
dstabbr=None, dstoffset=None,
start=None, end=None):
global relativedelta
if not relativedelta:
from dateutil import relativedelta
self._std_abbr = stdabbr
self._dst_abbr = dstabbr
if stdoffset is not None:
self._std_offset = datetime.timedelta(seconds=stdoffset)
else:
self._std_offset = ZERO
if dstoffset is not None:
self._dst_offset = datetime.timedelta(seconds=dstoffset)
elif dstabbr and stdoffset is not None:
self._dst_offset = self._std_offset+datetime.timedelta(hours=+1)
else:
self._dst_offset = ZERO
if dstabbr and start is None:
self._start_delta = relativedelta.relativedelta(
hours=+2, month=4, day=1, weekday=relativedelta.SU(+1))
else:
self._start_delta = start
if dstabbr and end is None:
self._end_delta = relativedelta.relativedelta(
hours=+1, month=10, day=31, weekday=relativedelta.SU(-1))
else:
self._end_delta = end
def utcoffset(self, dt):
if self._isdst(dt):
return self._dst_offset
else:
return self._std_offset
def dst(self, dt):
if self._isdst(dt):
return self._dst_offset-self._std_offset
else:
return ZERO
def tzname(self, dt):
if self._isdst(dt):
return self._dst_abbr
else:
return self._std_abbr
def _isdst(self, dt):
if not self._start_delta:
return False
year = datetime.datetime(dt.year,1,1)
start = year+self._start_delta
end = year+self._end_delta
dt = dt.replace(tzinfo=None)
if start < end:
return dt >= start and dt < end
else:
return dt >= start or dt < end
def __eq__(self, other):
if not isinstance(other, tzrange):
return False
return (self._std_abbr == other._std_abbr and
self._dst_abbr == other._dst_abbr and
self._std_offset == other._std_offset and
self._dst_offset == other._dst_offset and
self._start_delta == other._start_delta and
self._end_delta == other._end_delta)
def __ne__(self, other):
return not self.__eq__(other)
def __repr__(self):
return "%s(...)" % self.__class__.__name__
__reduce__ = object.__reduce__
class tzstr(tzrange):
def __init__(self, s):
global parser
if not parser:
from dateutil import parser
self._s = s
res = parser._parsetz(s)
if res is None:
raise ValueError, "unknown string format"
# Here we break the compatibility with the TZ variable handling.
# GMT-3 actually *means* the timezone -3.
if res.stdabbr in ("GMT", "UTC"):
res.stdoffset *= -1
# We must initialize it first, since _delta() needs
# _std_offset and _dst_offset set. Use False in start/end
# to avoid building it two times.
tzrange.__init__(self, res.stdabbr, res.stdoffset,
res.dstabbr, res.dstoffset,
start=False, end=False)
if not res.dstabbr:
self._start_delta = None
self._end_delta = None
else:
self._start_delta = self._delta(res.start)
if self._start_delta:
self._end_delta = self._delta(res.end, isend=1)
def _delta(self, x, isend=0):
kwargs = {}
if x.month is not None:
kwargs["month"] = x.month
if x.weekday is not None:
kwargs["weekday"] = relativedelta.weekday(x.weekday, x.week)
if x.week > 0:
kwargs["day"] = 1
else:
kwargs["day"] = 31
elif x.day:
kwargs["day"] = x.day
elif x.yday is not None:
kwargs["yearday"] = x.yday
elif x.jyday is not None:
kwargs["nlyearday"] = x.jyday
if not kwargs:
# Default is to start on first sunday of april, and end
# on last sunday of october.
if not isend:
kwargs["month"] = 4
kwargs["day"] = 1
kwargs["weekday"] = relativedelta.SU(+1)
else:
kwargs["month"] = 10
kwargs["day"] = 31
kwargs["weekday"] = relativedelta.SU(-1)
if x.time is not None:
kwargs["seconds"] = x.time
else:
# Default is 2AM.
kwargs["seconds"] = 7200
if isend:
# Convert to standard time, to follow the documented way
# of working with the extra hour. See the documentation
# of the tzinfo class.
delta = self._dst_offset-self._std_offset
kwargs["seconds"] -= delta.seconds+delta.days*86400
return relativedelta.relativedelta(**kwargs)
def __repr__(self):
return "%s(%s)" % (self.__class__.__name__, `self._s`)
class _tzicalvtzcomp:
def __init__(self, tzoffsetfrom, tzoffsetto, isdst,
tzname=None, rrule=None):
self.tzoffsetfrom = datetime.timedelta(seconds=tzoffsetfrom)
self.tzoffsetto = datetime.timedelta(seconds=tzoffsetto)
self.tzoffsetdiff = self.tzoffsetto-self.tzoffsetfrom
self.isdst = isdst
self.tzname = tzname
self.rrule = rrule
class _tzicalvtz(datetime.tzinfo):
def __init__(self, tzid, comps=[]):
self._tzid = tzid
self._comps = comps
self._cachedate = []
self._cachecomp = []
def _find_comp(self, dt):
if len(self._comps) == 1:
return self._comps[0]
dt = dt.replace(tzinfo=None)
try:
return self._cachecomp[self._cachedate.index(dt)]
except ValueError:
pass
lastcomp = None
lastcompdt = None
for comp in self._comps:
if not comp.isdst:
# Handle the extra hour in DST -> STD
compdt = comp.rrule.before(dt-comp.tzoffsetdiff, inc=True)
else:
compdt = comp.rrule.before(dt, inc=True)
if compdt and (not lastcompdt or lastcompdt < compdt):
lastcompdt = compdt
lastcomp = comp
if not lastcomp:
# RFC says nothing about what to do when a given
# time is before the first onset date. We'll look for the
# first standard component, or the first component, if
# none is found.
for comp in self._comps:
if not comp.isdst:
lastcomp = comp
break
else:
lastcomp = comp[0]
self._cachedate.insert(0, dt)
self._cachecomp.insert(0, lastcomp)
if len(self._cachedate) > 10:
self._cachedate.pop()
self._cachecomp.pop()
return lastcomp
def utcoffset(self, dt):
return self._find_comp(dt).tzoffsetto
def dst(self, dt):
comp = self._find_comp(dt)
if comp.isdst:
return comp.tzoffsetdiff
else:
return ZERO
def tzname(self, dt):
return self._find_comp(dt).tzname
def __repr__(self):
return "<tzicalvtz %s>" % `self._tzid`
__reduce__ = object.__reduce__
class tzical:
def __init__(self, fileobj):
global rrule
if not rrule:
from dateutil import rrule
if isinstance(fileobj, basestring):
self._s = fileobj
fileobj = open(fileobj)
elif hasattr(fileobj, "name"):
self._s = fileobj.name
else:
self._s = `fileobj`
self._vtz = {}
self._parse_rfc(fileobj.read())
def keys(self):
return self._vtz.keys()
def get(self, tzid=None):
if tzid is None:
keys = self._vtz.keys()
if len(keys) == 0:
raise ValueError, "no timezones defined"
elif len(keys) > 1:
raise ValueError, "more than one timezone available"
tzid = keys[0]
return self._vtz.get(tzid)
def _parse_offset(self, s):
s = s.strip()
if not s:
raise ValueError, "empty offset"
if s[0] in ('+', '-'):
signal = (-1,+1)[s[0]=='+']
s = s[1:]
else:
signal = +1
if len(s) == 4:
return (int(s[:2])*3600+int(s[2:])*60)*signal
elif len(s) == 6:
return (int(s[:2])*3600+int(s[2:4])*60+int(s[4:]))*signal
else:
raise ValueError, "invalid offset: "+s
def _parse_rfc(self, s):
lines = s.splitlines()
if not lines:
raise ValueError, "empty string"
# Unfold
i = 0
while i < len(lines):
line = lines[i].rstrip()
if not line:
del lines[i]
elif i > 0 and line[0] in (" ", "\t"):
lines[i-1] += line[1:]
del lines[i]
else:
i += 1
tzid = None
comps = []
invtz = False
comptype = None
for line in lines:
if not line:
continue
name, value = line.split(':', 1)
parms = name.split(';')
if not parms:
raise ValueError, "empty property name"
name = parms[0].upper()
parms = parms[1:]
if invtz:
if name == "BEGIN":
if value in ("STANDARD", "DAYLIGHT"):
# Process component
pass
else:
raise ValueError, "unknown component: "+value
comptype = value
founddtstart = False
tzoffsetfrom = None
tzoffsetto = None
rrulelines = []
tzname = None
elif name == "END":
if value == "VTIMEZONE":
if comptype:
raise ValueError, \
"component not closed: "+comptype
if not tzid:
raise ValueError, \
"mandatory TZID not found"
if not comps:
raise ValueError, \
"at least one component is needed"
# Process vtimezone
self._vtz[tzid] = _tzicalvtz(tzid, comps)
invtz = False
elif value == comptype:
if not founddtstart:
raise ValueError, \
"mandatory DTSTART not found"
if tzoffsetfrom is None:
raise ValueError, \
"mandatory TZOFFSETFROM not found"
if tzoffsetto is None:
raise ValueError, \
"mandatory TZOFFSETFROM not found"
# Process component
rr = None
if rrulelines:
rr = rrule.rrulestr("\n".join(rrulelines),
compatible=True,
ignoretz=True,
cache=True)
comp = _tzicalvtzcomp(tzoffsetfrom, tzoffsetto,
(comptype == "DAYLIGHT"),
tzname, rr)
comps.append(comp)
comptype = None
else:
raise ValueError, \
"invalid component end: "+value
elif comptype:
if name == "DTSTART":
rrulelines.append(line)
founddtstart = True
elif name in ("RRULE", "RDATE", "EXRULE", "EXDATE"):
rrulelines.append(line)
elif name == "TZOFFSETFROM":
if parms:
raise ValueError, \
"unsupported %s parm: %s "%(name, parms[0])
tzoffsetfrom = self._parse_offset(value)
elif name == "TZOFFSETTO":
if parms:
raise ValueError, \
"unsupported TZOFFSETTO parm: "+parms[0]
tzoffsetto = self._parse_offset(value)
elif name == "TZNAME":
if parms:
raise ValueError, \
"unsupported TZNAME parm: "+parms[0]
tzname = value
elif name == "COMMENT":
pass
elif name.upper().startswith('X-'):
# Ignore experimental properties.
pass
else:
raise ValueError, "unsupported property: "+name
else:
if name == "TZID":
for p in parms:
if not p.upper().startswith('X-'):
raise ValueError, \
"unsupported TZID parm: "+p
tzid = value
elif name in ("TZURL", "LAST-MODIFIED", "COMMENT"):
pass
elif name.upper().startswith('X-'):
# Ignore experimental properties.
pass
else:
raise ValueError, "unsupported property: "+name
elif name == "BEGIN" and value == "VTIMEZONE":
tzid = None
comps = []
invtz = True
def __repr__(self):
return "%s(%s)" % (self.__class__.__name__, `self._s`)
if sys.platform != "win32":
TZFILES = ["/etc/localtime", "localtime"]
TZPATHS = ["/usr/share/zoneinfo", "/usr/lib/zoneinfo", "/etc/zoneinfo"]
else:
TZFILES = []
TZPATHS = []
def gettz(name=None):
tz = None
if not name:
try:
name = os.environ["TZ"]
except KeyError:
pass
if name is None or name == ":":
for filepath in TZFILES:
if not os.path.isabs(filepath):
filename = filepath
for path in TZPATHS:
filepath = os.path.join(path, filename)
if os.path.isfile(filepath):
break
else:
continue
if os.path.isfile(filepath):
try:
tz = tzfile(filepath)
break
except (IOError, OSError, ValueError):
pass
else:
tz = tzlocal()
else:
if name.startswith(":"):
name = name[:-1]
if os.path.isabs(name):
if os.path.isfile(name):
tz = tzfile(name)
else:
tz = None
else:
for path in TZPATHS:
filepath = os.path.join(path, name)
if not os.path.isfile(filepath):
filepath = filepath.replace(' ','_')
if not os.path.isfile(filepath):
continue
try:
tz = tzfile(filepath)
break
except (IOError, OSError, ValueError):
pass
else:
tz = None
if tzwin:
try:
tz = tzwin(name)
except OSError:
pass
if not tz:
from dateutil.zoneinfo import gettz
tz = gettz(name)
if not tz:
for c in name:
# name must have at least one offset to be a tzstr
if c in "0123456789":
try:
tz = tzstr(name)
except ValueError:
pass
break
else:
if name in ("GMT", "UTC"):
tz = tzutc()
elif name in time.tzname:
tz = tzlocal()
return tz
# vim:ts=4:sw=4:et
+12
View File
@@ -0,0 +1,12 @@
# -*- coding: utf-8 -*-
from .tz import *
from .tz import __doc__
__all__ = ["tzutc", "tzoffset", "tzlocal", "tzfile", "tzrange",
"tzstr", "tzical", "tzwin", "tzwinlocal", "gettz",
"enfold", "datetime_ambiguous", "datetime_exists",
"resolve_imaginary", "UTC", "DeprecatedTzFormatWarning"]
class DeprecatedTzFormatWarning(Warning):
"""Warning raised when time zones are parsed from deprecated formats."""
+419
View File
@@ -0,0 +1,419 @@
from six import PY2
from functools import wraps
from datetime import datetime, timedelta, tzinfo
ZERO = timedelta(0)
__all__ = ['tzname_in_python2', 'enfold']
def tzname_in_python2(namefunc):
"""Change unicode output into bytestrings in Python 2
tzname() API changed in Python 3. It used to return bytes, but was changed
to unicode strings
"""
if PY2:
@wraps(namefunc)
def adjust_encoding(*args, **kwargs):
name = namefunc(*args, **kwargs)
if name is not None:
name = name.encode()
return name
return adjust_encoding
else:
return namefunc
# The following is adapted from Alexander Belopolsky's tz library
# https://github.com/abalkin/tz
if hasattr(datetime, 'fold'):
# This is the pre-python 3.6 fold situation
def enfold(dt, fold=1):
"""
Provides a unified interface for assigning the ``fold`` attribute to
datetimes both before and after the implementation of PEP-495.
:param fold:
The value for the ``fold`` attribute in the returned datetime. This
should be either 0 or 1.
:return:
Returns an object for which ``getattr(dt, 'fold', 0)`` returns
``fold`` for all versions of Python. In versions prior to
Python 3.6, this is a ``_DatetimeWithFold`` object, which is a
subclass of :py:class:`datetime.datetime` with the ``fold``
attribute added, if ``fold`` is 1.
.. versionadded:: 2.6.0
"""
return dt.replace(fold=fold)
else:
class _DatetimeWithFold(datetime):
"""
This is a class designed to provide a PEP 495-compliant interface for
Python versions before 3.6. It is used only for dates in a fold, so
the ``fold`` attribute is fixed at ``1``.
.. versionadded:: 2.6.0
"""
__slots__ = ()
def replace(self, *args, **kwargs):
"""
Return a datetime with the same attributes, except for those
attributes given new values by whichever keyword arguments are
specified. Note that tzinfo=None can be specified to create a naive
datetime from an aware datetime with no conversion of date and time
data.
This is reimplemented in ``_DatetimeWithFold`` because pypy3 will
return a ``datetime.datetime`` even if ``fold`` is unchanged.
"""
argnames = (
'year', 'month', 'day', 'hour', 'minute', 'second',
'microsecond', 'tzinfo'
)
for arg, argname in zip(args, argnames):
if argname in kwargs:
raise TypeError('Duplicate argument: {}'.format(argname))
kwargs[argname] = arg
for argname in argnames:
if argname not in kwargs:
kwargs[argname] = getattr(self, argname)
dt_class = self.__class__ if kwargs.get('fold', 1) else datetime
return dt_class(**kwargs)
@property
def fold(self):
return 1
def enfold(dt, fold=1):
"""
Provides a unified interface for assigning the ``fold`` attribute to
datetimes both before and after the implementation of PEP-495.
:param fold:
The value for the ``fold`` attribute in the returned datetime. This
should be either 0 or 1.
:return:
Returns an object for which ``getattr(dt, 'fold', 0)`` returns
``fold`` for all versions of Python. In versions prior to
Python 3.6, this is a ``_DatetimeWithFold`` object, which is a
subclass of :py:class:`datetime.datetime` with the ``fold``
attribute added, if ``fold`` is 1.
.. versionadded:: 2.6.0
"""
if getattr(dt, 'fold', 0) == fold:
return dt
args = dt.timetuple()[:6]
args += (dt.microsecond, dt.tzinfo)
if fold:
return _DatetimeWithFold(*args)
else:
return datetime(*args)
def _validate_fromutc_inputs(f):
"""
The CPython version of ``fromutc`` checks that the input is a ``datetime``
object and that ``self`` is attached as its ``tzinfo``.
"""
@wraps(f)
def fromutc(self, dt):
if not isinstance(dt, datetime):
raise TypeError("fromutc() requires a datetime argument")
if dt.tzinfo is not self:
raise ValueError("dt.tzinfo is not self")
return f(self, dt)
return fromutc
class _tzinfo(tzinfo):
"""
Base class for all ``dateutil`` ``tzinfo`` objects.
"""
def is_ambiguous(self, dt):
"""
Whether or not the "wall time" of a given datetime is ambiguous in this
zone.
:param dt:
A :py:class:`datetime.datetime`, naive or time zone aware.
:return:
Returns ``True`` if ambiguous, ``False`` otherwise.
.. versionadded:: 2.6.0
"""
dt = dt.replace(tzinfo=self)
wall_0 = enfold(dt, fold=0)
wall_1 = enfold(dt, fold=1)
same_offset = wall_0.utcoffset() == wall_1.utcoffset()
same_dt = wall_0.replace(tzinfo=None) == wall_1.replace(tzinfo=None)
return same_dt and not same_offset
def _fold_status(self, dt_utc, dt_wall):
"""
Determine the fold status of a "wall" datetime, given a representation
of the same datetime as a (naive) UTC datetime. This is calculated based
on the assumption that ``dt.utcoffset() - dt.dst()`` is constant for all
datetimes, and that this offset is the actual number of hours separating
``dt_utc`` and ``dt_wall``.
:param dt_utc:
Representation of the datetime as UTC
:param dt_wall:
Representation of the datetime as "wall time". This parameter must
either have a `fold` attribute or have a fold-naive
:class:`datetime.tzinfo` attached, otherwise the calculation may
fail.
"""
if self.is_ambiguous(dt_wall):
delta_wall = dt_wall - dt_utc
_fold = int(delta_wall == (dt_utc.utcoffset() - dt_utc.dst()))
else:
_fold = 0
return _fold
def _fold(self, dt):
return getattr(dt, 'fold', 0)
def _fromutc(self, dt):
"""
Given a timezone-aware datetime in a given timezone, calculates a
timezone-aware datetime in a new timezone.
Since this is the one time that we *know* we have an unambiguous
datetime object, we take this opportunity to determine whether the
datetime is ambiguous and in a "fold" state (e.g. if it's the first
occurrence, chronologically, of the ambiguous datetime).
:param dt:
A timezone-aware :class:`datetime.datetime` object.
"""
# Re-implement the algorithm from Python's datetime.py
dtoff = dt.utcoffset()
if dtoff is None:
raise ValueError("fromutc() requires a non-None utcoffset() "
"result")
# The original datetime.py code assumes that `dst()` defaults to
# zero during ambiguous times. PEP 495 inverts this presumption, so
# for pre-PEP 495 versions of python, we need to tweak the algorithm.
dtdst = dt.dst()
if dtdst is None:
raise ValueError("fromutc() requires a non-None dst() result")
delta = dtoff - dtdst
dt += delta
# Set fold=1 so we can default to being in the fold for
# ambiguous dates.
dtdst = enfold(dt, fold=1).dst()
if dtdst is None:
raise ValueError("fromutc(): dt.dst gave inconsistent "
"results; cannot convert")
return dt + dtdst
@_validate_fromutc_inputs
def fromutc(self, dt):
"""
Given a timezone-aware datetime in a given timezone, calculates a
timezone-aware datetime in a new timezone.
Since this is the one time that we *know* we have an unambiguous
datetime object, we take this opportunity to determine whether the
datetime is ambiguous and in a "fold" state (e.g. if it's the first
occurrence, chronologically, of the ambiguous datetime).
:param dt:
A timezone-aware :class:`datetime.datetime` object.
"""
dt_wall = self._fromutc(dt)
# Calculate the fold status given the two datetimes.
_fold = self._fold_status(dt, dt_wall)
# Set the default fold value for ambiguous dates
return enfold(dt_wall, fold=_fold)
class tzrangebase(_tzinfo):
"""
This is an abstract base class for time zones represented by an annual
transition into and out of DST. Child classes should implement the following
methods:
* ``__init__(self, *args, **kwargs)``
* ``transitions(self, year)`` - this is expected to return a tuple of
datetimes representing the DST on and off transitions in standard
time.
A fully initialized ``tzrangebase`` subclass should also provide the
following attributes:
* ``hasdst``: Boolean whether or not the zone uses DST.
* ``_dst_offset`` / ``_std_offset``: :class:`datetime.timedelta` objects
representing the respective UTC offsets.
* ``_dst_abbr`` / ``_std_abbr``: Strings representing the timezone short
abbreviations in DST and STD, respectively.
* ``_hasdst``: Whether or not the zone has DST.
.. versionadded:: 2.6.0
"""
def __init__(self):
raise NotImplementedError('tzrangebase is an abstract base class')
def utcoffset(self, dt):
isdst = self._isdst(dt)
if isdst is None:
return None
elif isdst:
return self._dst_offset
else:
return self._std_offset
def dst(self, dt):
isdst = self._isdst(dt)
if isdst is None:
return None
elif isdst:
return self._dst_base_offset
else:
return ZERO
@tzname_in_python2
def tzname(self, dt):
if self._isdst(dt):
return self._dst_abbr
else:
return self._std_abbr
def fromutc(self, dt):
""" Given a datetime in UTC, return local time """
if not isinstance(dt, datetime):
raise TypeError("fromutc() requires a datetime argument")
if dt.tzinfo is not self:
raise ValueError("dt.tzinfo is not self")
# Get transitions - if there are none, fixed offset
transitions = self.transitions(dt.year)
if transitions is None:
return dt + self.utcoffset(dt)
# Get the transition times in UTC
dston, dstoff = transitions
dston -= self._std_offset
dstoff -= self._std_offset
utc_transitions = (dston, dstoff)
dt_utc = dt.replace(tzinfo=None)
isdst = self._naive_isdst(dt_utc, utc_transitions)
if isdst:
dt_wall = dt + self._dst_offset
else:
dt_wall = dt + self._std_offset
_fold = int(not isdst and self.is_ambiguous(dt_wall))
return enfold(dt_wall, fold=_fold)
def is_ambiguous(self, dt):
"""
Whether or not the "wall time" of a given datetime is ambiguous in this
zone.
:param dt:
A :py:class:`datetime.datetime`, naive or time zone aware.
:return:
Returns ``True`` if ambiguous, ``False`` otherwise.
.. versionadded:: 2.6.0
"""
if not self.hasdst:
return False
start, end = self.transitions(dt.year)
dt = dt.replace(tzinfo=None)
return (end <= dt < end + self._dst_base_offset)
def _isdst(self, dt):
if not self.hasdst:
return False
elif dt is None:
return None
transitions = self.transitions(dt.year)
if transitions is None:
return False
dt = dt.replace(tzinfo=None)
isdst = self._naive_isdst(dt, transitions)
# Handle ambiguous dates
if not isdst and self.is_ambiguous(dt):
return not self._fold(dt)
else:
return isdst
def _naive_isdst(self, dt, transitions):
dston, dstoff = transitions
dt = dt.replace(tzinfo=None)
if dston < dstoff:
isdst = dston <= dt < dstoff
else:
isdst = not dstoff <= dt < dston
return isdst
@property
def _dst_base_offset(self):
return self._dst_offset - self._std_offset
__hash__ = None
def __ne__(self, other):
return not (self == other)
def __repr__(self):
return "%s(...)" % self.__class__.__name__
__reduce__ = object.__reduce__
+80
View File
@@ -0,0 +1,80 @@
from datetime import timedelta
import weakref
from collections import OrderedDict
from six.moves import _thread
class _TzSingleton(type):
def __init__(cls, *args, **kwargs):
cls.__instance = None
super(_TzSingleton, cls).__init__(*args, **kwargs)
def __call__(cls):
if cls.__instance is None:
cls.__instance = super(_TzSingleton, cls).__call__()
return cls.__instance
class _TzFactory(type):
def instance(cls, *args, **kwargs):
"""Alternate constructor that returns a fresh instance"""
return type.__call__(cls, *args, **kwargs)
class _TzOffsetFactory(_TzFactory):
def __init__(cls, *args, **kwargs):
cls.__instances = weakref.WeakValueDictionary()
cls.__strong_cache = OrderedDict()
cls.__strong_cache_size = 8
cls._cache_lock = _thread.allocate_lock()
def __call__(cls, name, offset):
if isinstance(offset, timedelta):
key = (name, offset.total_seconds())
else:
key = (name, offset)
instance = cls.__instances.get(key, None)
if instance is None:
instance = cls.__instances.setdefault(key,
cls.instance(name, offset))
# This lock may not be necessary in Python 3. See GH issue #901
with cls._cache_lock:
cls.__strong_cache[key] = cls.__strong_cache.pop(key, instance)
# Remove an item if the strong cache is overpopulated
if len(cls.__strong_cache) > cls.__strong_cache_size:
cls.__strong_cache.popitem(last=False)
return instance
class _TzStrFactory(_TzFactory):
def __init__(cls, *args, **kwargs):
cls.__instances = weakref.WeakValueDictionary()
cls.__strong_cache = OrderedDict()
cls.__strong_cache_size = 8
cls.__cache_lock = _thread.allocate_lock()
def __call__(cls, s, posix_offset=False):
key = (s, posix_offset)
instance = cls.__instances.get(key, None)
if instance is None:
instance = cls.__instances.setdefault(key,
cls.instance(s, posix_offset))
# This lock may not be necessary in Python 3. See GH issue #901
with cls.__cache_lock:
cls.__strong_cache[key] = cls.__strong_cache.pop(key, instance)
# Remove an item if the strong cache is overpopulated
if len(cls.__strong_cache) > cls.__strong_cache_size:
cls.__strong_cache.popitem(last=False)
return instance

Some files were not shown because too many files have changed in this diff Show More