depau/youtube-dl
1
0
Fork 0
mirror of https://github.com/ytdl-org/youtube-dl.git synced 2024-06-02 10:29:28 +00:00
Code Issues Releases Wiki Activity

Be smarter about downloading pages of user video ids - now no extra page is downloaded when we have all ids.

Browse source
This commit is contained in:
Paweł Paprota 2010-12-23 09:08:43 +01:00
parent 1951dabf46
commit a51c00f70c
1 changed files with 7 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
youtube-dl
View file

@ -33,7 +33,7 @@ std_headers = {
'User-Agent': 'Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.12) Gecko/20101028 Firefox/3.6.12',
'Accept-Charset': 'ISO-8859-1,utf-8;q=0.7,*;q=0.7',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Encoding': 'gzip, deflate',
# 'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'en-us,en;q=0.5',
}
@ -2086,12 +2086,15 @@ class YoutubeUserIE(InfoExtractor):
if mobj.group(1) not in ids_in_page:
ids_in_page.append(mobj.group(1))
if len(ids_in_page) == 0:
# Got them all, now download!
video_ids.extend(ids_in_page)
# A little optimization - if current page is not "full", ie. does not contain PAGE_SIZE video ids then we can assume
# that this page is the last one - there are no more ids on further pages - no need to query again.
if len(ids_in_page) < self._GDATA_PAGE_SIZE:
break
pagenum += 1
video_ids.extend(ids_in_page)
self._downloader.to_screen("[youtube] user %s: Collected %d video ids" % (username, len(video_ids)))