yle-dl toiminta (Ratkaisu )

[nm]

Käytössäni on Ubuntu 14.04 ja yle-dl versio 2.10.1.

Ikivanha versio. Ihme että lataa enää mitään. Poista se ja asenna versio 2.30: https://github.com/aajanki/yle-dl/blob/master/OS-install-instructions.md
Eli Ubuntu 14.04:ään näin:

Koodia: [Valitse]

sudo apt-get install rtmpdump wget libav-tools python-dev python-setuptools \
     python-pip python-crypto python-requests python-lxml python-socks \
     php5-cli php5-curl php5-mcrypt
sudo php5enmod mcrypt
sudo pip install pyOpenSSL ndg-httpsclient pyasn1
sudo pip install yle-dl
Päivitys jatkossa:

Koodia: [Valitse]

sudo pip install --upgrade yle-dl

Videot kyllä latautuu (hyvin hitaasti) mutta koko on ihan järkyttävän iso. Kokopitkä leffa kymmeniä gigatavuja 1080 -laadulla. Kuuluuko tuo koko olla noin valtava?

Kahden tunnin leffasta pitäisi tulla Areenan 4 Mbps maksimi-bitratella 3,5 Gt:n kokoinen tiedosto.

[rale]

Koitin postaa vanhan version mutta edelleen se näyttää siellä olevan.

Koodia: [Valitse]

rauno@2015-RAULEP:~$ yle-dl
yle-dl 2.10.1: Download media files from Yle Areena and Elävä Arkisto
Copyright (C) 2009-2016 Antti Ajanki <antti.ajanki@iki.fi>, license: GPLv3

/usr/local/bin/yle-dl [options] URL

options:

-o filename             Save stream to the named file
--latestepisode         Download the latest episode
--showurl               Print URL, don't download
--showtitle             Print stream title, don't download
--showepisodepage       Print web page for each episode
--vfat                  Create Windows-compatible filenames
--sublang lang          Download subtitles, lang = fin, swe, smi, none or all
--hardsubs              Download stream with hard subs if available
--maxbitrate br         Maximum bitrate stream to download, integer in kB/s
                        or "best" or "worst". Not exact on HDS streams.
--rtmpdump path         Set path to rtmpdump binary
--adobehds cmd          Set command for executing AdobeHDS.php script
                        Default: "php /usr/local/share/yle-dl/AdobeHDS.php"
--proxy uri             Proxy for downloading stream manifests
                        Example: --proxy socks5://localhost:7777
--destdir dir           Save files to dir
--backend be            Downloaders that are tried until one of them
                        succeeds (a comma-separated list). Possible values:
                          adobehdsphp - AdobeHDS.php
                          youtubedl - youtube-dl (HDS stream)
--pipe                  Dump stream to stdout for piping to media player
                        E.g. "yle-dl --pipe URL | vlc -"
--resume                Resume a partial download
-V, --verbose           Show verbose debug output
rauno@2015-RAULEP:~$



Tuon uuden asentamisessa  tuo eka asennusrimpsu antoi tällaisen tuloksen:

Koodia: [Valitse]

>     
python-pip python-crypto python-requests python-lxml python-socks \
>      php5-cli php5-curl php5-mcrypt
[sudo] password for rauno:
Luetaan pakettiluetteloita... Valmis
Muodostetaan riippuvuussuhteiden puu       
Luetaan tilatiedot... Valmis       
N: Ignoring file 'google-chrome.list.save.3' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension
N: Ignoring file 'google-chrome.list.save.4' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension
N: Ignoring file 'google-chrome.list.save.1' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension
N: Ignoring file 'google-chrome.list.save.2' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension
N: Ignoring file 'google-chrome.list.save.3' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension
N: Ignoring file 'google-chrome.list.save.4' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension
N: Ignoring file 'google-chrome.list.save.1' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension
N: Ignoring file 'google-chrome.list.save.2' in directory '/etc/apt/sources.list.d/' as it has an invalid filename extension
E: Pakettia python-socks ei löydy
rauno@2015-RAULEP:~$


[nm]

Koitin postaa vanhan version mutta edelleen se näyttää siellä olevan.

Koodia: [Valitse]

rauno@2015-RAULEP:~$ yle-dl
yle-dl 2.10.1: Download media files from Yle Areena and Elävä Arkisto
Copyright (C) 2009-2016 Antti Ajanki <antti.ajanki@iki.fi>, license: GPLv3

/usr/local/bin/yle-dl [options] URL

Näin:

Koodia: [Valitse]

sudo rm /usr/local/bin/yle-dl
Python-paketteja jää varmaankin /usr/localin alle, mutta niiden ei pitäisi häiritä. Uusi versio yle-dl-skriptistä asentunee myös samaan sijaintiin.

E: Pakettia python-socks ei löydy

python-socks-paketti näemmä puuttuu 14.04:stä, mutta se ei ole kriittinen yle-dl:n toiminnan kannalta. (Siitä on apua, jos pystytät oman välityspalvelimen esimerkiksi sitä varten että haluat katsoa Areenan ohjelmia ulkomailla.)

Eli jätä tuo puuttuva paketti pois komentoriviltä:

Koodia: [Valitse]

sudo apt-get install rtmpdump wget libav-tools python-dev python-setuptools \
     python-pip python-crypto python-requests python-lxml \
     php5-cli php5-curl php5-mcrypt
sudo php5enmod mcrypt
sudo pip install pyOpenSSL ndg-httpsclient pyasn1
sudo pip install yle-dl

[rale]

mielestäni asennus meni ok mutta herjaa näin:

Koodia: [Valitse]

rauno@2015-RAULEP:~$ yle-dl https://areena.yle.fi/1-3913222
yle-dl: komentoa ei löytynyt
rauno@2015-RAULEP:~$


[nm]

Näkyykö tiedosto yle-dl hakemistossa /usr/local/bin:

Koodia: [Valitse]

ls -l /usr/local/bin/yle-dl
Mitä pip install --upgrade listaa päätteeseen:

Koodia: [Valitse]

sudo pip install --upgrade yle-dl

[rale]

yle-dl hakemistossa /usr/local/bin:

Koodia: [Valitse]

rauno@2015-RAULEP:~$ ls -l /usr/local/bin/yle-dl
ls: tiedostoa /usr/local/bin/yle-dl ei voi käsitellä: Tiedostoa tai hakemistoa ei ole
rauno@2015-RAULEP:~$


pip install --upgrade

Koodia: [Valitse]

rauno@2015-RAULEP:~$ sudo pip install --upgrade yle-dl
[sudo] password for rauno:
Requirement already up-to-date: yle-dl in /usr/local/lib/python2.7/dist-packages
Cleaning up...
rauno@2015-RAULEP:~$


[nm]

OK. Pakota uudelleenasennus:

Koodia: [Valitse]

sudo pip install --upgrade --force-reinstall yle-dl

[rale]

Tuonne loppuun tuli punaisella nuo viimeisen kappaleen tekstit. Kokeilin kuitenkin ladata videota asennuksen jälkeen mutta ei ladannut.
Laitoin vaan tuon tulosteen lopun koska muuten olisi ollut liikaa merkkejä.

Koodia: [Valitse]

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
  Can't roll back lxml; was not uninstalled
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-yQjlTw-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml
Storing debug log for failure in /home/rauno/.pip/pip.log
rauno@2015-RAULEP:~$



[nm]

Varmaan sama ongelma libxml2:n ja libxslt1:n headerien kanssa kuin täällä: https://forum.ubuntu-fi.org/index.php?topic=20696.msg401959#msg401959

Asenna puuttuvat paketit:

Koodia: [Valitse]

sudo apt-get install libxml2-dev libxslt1-dev
Ja sitten uusi yritys:

Koodia: [Valitse]

sudo pip install --upgrade --force-reinstall yle-dl

[rale]

Vieä tuli reinstallissa tällainen punateksti. Pitää vissiin yrittää poistaa edellinen asennus ja yrittää taas?

Koodia: [Valitse]

pip can't proceed with requirement 'Py3AMF (from yle-dl)' due to a pre-existing build directory.
 location: /tmp/pip_build_root/Py3AMF
This is likely due to a previous installation that failed.
pip is being responsible and not assuming it can delete this.
Please delete it and try again.

Storing debug log for failure in /home/rauno/.pip/pip.log
rauno@2015-RAULEP:~$



[nm]

Poista kesken jääneen Pu3AMF-asennuksen buildaushakemisto:

Koodia: [Valitse]

sudo rm /tmp/pip_build_root/Py3AMF -rf
Sitten taas uusi yritys.

Koodia: [Valitse]

sudo pip install --upgrade --force-reinstall yle-dl

[rale]

Ei tee mitään kun tuon Pu3AMF-asennuksen buildaushakemiston komennon laittaa. Tulee vaan heti tämä   rauno@2015-RAULEP:~$

[nm]

Ei tee mitään kun tuon Pu3AMF-asennuksen buildaushakemiston komennon laittaa. Tulee vaan heti tämä   rauno@2015-RAULEP:~$

Kyllä rm poisti sen hakemiston. Perinteisesti Unixissa komennot eivät listaa mitään ylimääräistä silloin kun operaatio onnistuu.

[rale]

Punaisella kaikki Downloading/unpacking future (from yle-dl) alaspäin

Koodia: [Valitse]



rauno@2015-RAULEP:~$ sudo pip install --upgrade --force-reinstall yle-dl
[sudo] password for rauno:
Yritä uudelleen.
[sudo] password for rauno:
Yritä uudelleen.
[sudo] password for rauno:
Downloading/unpacking yle-dl
  Downloading yle_dl-2.30-py2.py3-none-any.whl (45kB): 45kB downloaded
Downloading/unpacking requests (from yle-dl)
  Downloading requests-2.18.4-py2.py3-none-any.whl (88kB): 88kB downloaded
Downloading/unpacking Py3AMF (from yle-dl)
  Downloading Py3AMF-0.8.7.tar.gz (1.6MB): 1.6MB downloaded
  Running setup.py (path:/tmp/pip_build_root/Py3AMF/setup.py) egg_info for package Py3AMF
   
    no previously-included directories found matching 'doc/build'
    no previously-included directories found matching 'doc/_build'
    warning: no previously-included files matching '*.swf' found anywhere in distribution
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
Downloading/unpacking future (from yle-dl)

pip can't proceed with requirement 'future (from yle-dl)' due to a pre-existing build directory.
 location: /tmp/pip_build_root/future
This is likely due to a previous installation that failed.
pip is being responsible and not assuming it can delete this.
Please delete it and try again.

Storing debug log for failure in /home/rauno/.pip/pip.log
rauno@2015-RAULEP:~$



[nm]

Koodia: [Valitse]

pip can't proceed with requirement 'future (from yle-dl)' due to a pre-existing build directory.
 location: /tmp/pip_build_root/future
This is likely due to a previous installation that failed.
pip is being responsible and not assuming it can delete this.
Please delete it and try again.


Edelleen kesken jääneitä buildeja -- nyt future-moduulia koskien. Poista vaikka koko /tmp/pip_build_root:

Koodia: [Valitse]

sudo rm /tmp/pip_build_root -rf

[rale]

Koville tämä ottaa. Asennettaessa punaisella oli tuo alla oleva paitsi Cleaning up...

Koodia: [Valitse]

Can't roll back lxml; was not uninstalled
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-B6oWe7-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml


Tässä vielä videon latausyrityksen jälkeinen tieto:

Koodia: [Valitse]


rauno@2015-RAULEP:~$ yle-dl https://areena.yle.fi/1-4226382
/usr/local/lib/python2.7/dist-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.7.1) or chardet (2.0.1) doesn't match a supported version!
  RequestsDependencyWarning)
Traceback (most recent call last):
  File "/usr/local/bin/yle-dl", line 7, in <module>
    from yledl.yledl import main
  File "/usr/local/lib/python2.7/dist-packages/yledl/__init__.py", line 2, in <module>
    from .yledl import download, StreamAction
  File "/usr/local/lib/python2.7/dist-packages/yledl/yledl.py", line 37, in <module>
    from .downloaders import downloader_factory, StreamFilters, IOContext, \
  File "/usr/local/lib/python2.7/dist-packages/yledl/downloaders.py", line 21, in <module>
    import requests
  File "/usr/local/lib/python2.7/dist-packages/requests/__init__.py", line 90, in <module>
    from urllib3.exceptions import DependencyWarning
ImportError: cannot import name DependencyWarning
rauno@2015-RAULEP:~$


[nm]

Koville tämä ottaa. Asennettaessa punaisella oli tuo alla oleva paitsi Cleaning up...

Koodia: [Valitse]

Can't roll back lxml; was not uninstalled
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-B6oWe7-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml


Listauksen aiemmista virhekohdista voisi paremmin päätellä mitä vielä puuttuu. Kaikki virheet eivät välttämättä erotu värin perusteella. Google-haun perusteella puuttuva paketti on ehkä zlib1g-dev.

Koodia: [Valitse]

sudo apt-get install zlib1g-dev

/usr/local/lib/python2.7/dist-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.7.1) or chardet (2.0.1) doesn't match a supported version!
  RequestsDependencyWarning)

Jos saat tämän virheen edelleen zlib1g-devin asennuksen ja uuden yle-dl-asennusyrityksen jälkeen, voit kokeilla asentaa moduulit urllib3 ja chardet itse pipillä:

Koodia: [Valitse]

sudo pip install --upgrade urllib3
sudo pip install --upgrade chardet

[rale]

Tein em. toimenpiteet.
Asennuksen koodirimpsua en voi kokonaisuudessaan laittaa 20.000 merkin rajoituksen takia. Mitään punaista siinä ei enää ollut mutta
Videon latausyrityksessä tulee tällainen:

Koodia: [Valitse]

rauno@2015-RAULEP:~$ yle-dl https://areena.yle.fi/1-4353410
yle-dl 2.30: Download media files from Yle Areena and Elävä Arkisto
Copyright (C) 2009-2017 Antti Ajanki <antti.ajanki@iki.fi>, license: GPLv3
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:339: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
ERROR: Can't read https://areena.yle.fi/1-4353410
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yledl/downloaders.py", line 101, in http_get
    r = cached_requests_session.get(url, headers=headers, timeout=20)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 506, in send
    raise SSLError(e, request=request)
SSLError: HTTPSConnectionPool(host='areena.yle.fi', port=443): Max retries exceeded with url: /1-4353410 (Caused by SSLError(SSLError(1, '_ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure'),))
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecurePlatformWarning
ERROR: Can't read https://player.yle.fi/api/v1/programs.jsonp?id=1-4353410&callback=yleEmbed.programJsonpCallback
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/yledl/downloaders.py", line 101, in http_get
    r = cached_requests_session.get(url, headers=headers, timeout=20)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 506, in send
    raise SSLError(e, request=request)
SSLError: HTTPSConnectionPool(host='player.yle.fi', port=443): Max retries exceeded with url: /api/v1/programs.jsonp?id=1-4353410&callback=yleEmbed.programJsonpCallback (Caused by SSLError(SSLError(1, '_ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure'),))
ERROR: Unsupported stream: Failed to download program data
rauno@2015-RAULEP:~$



[nm]

En onnistu toistamaan ongelmaa omissa Ubuntu 14.04 -asennuksissani, mutta voit vielä yrittää korjata sitä näin:

Koodia: [Valitse]

sudo apt-get install python-dev libssl-dev libffi-dev

Koodia: [Valitse]

sudo pip install --upgrade --force-reinstall pyOpenSSL ndg-httpsclient pyasn1
Lähde: https://www.franzoni.eu/python-requests-ssl-and-insecureplatformwarning/


Ubuntu 16.04:ssä näitä viimeisiä ongelmia ei olisi tullut vastaan.

[rale]

Ei tuottanut tulosta nuokaan. Ehkä laitan yle-dl:n vasta sitten kun olen laittanut koneeseen tulevan 18.04 julkaisun joskus puolen vuoden päästä.