[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

wget fails in Debian Jessie

I had a script running just fine under Debian Squeeze, but that server is now offline for repair and upgrade to Jessie, so I am now running the script under Jessie, and the script is failing when attempting to scrape data off a website using wget.  Under Jessie, wget produces an SSL error when it tries to log in to the website.  There is an open bug report #785016, to which I added a comment, but so far no response has come from the developers.  Needing to get this script fully working, I am attempting to use curl for the purpose, but I'm not having any luck there, either.  I don't get the error, but I don't get the correct web page, either.  These are the two lines using wget:

wget --save-cookies cookies.txt --keep-session-cookies --no-check-certificate --post-data 'timeOffset=300&UserName=xxxxx%40mygrande.net&Password=yyyyy&RememberMe=false' https://mytotalconnectcomfort.com/portal

wget --load-cookies cookies.txt --no-check-certificate https://mytotalconnectcomfort.com/portal/188049/Zones/page2

I tried the following using curl:

curl -c cookies.txt -d 'timeOffset=300&UserName=xxxxxxx%40mygrande.net&Password=yyyyyyy&RememberMe=false' https://mytotalconnectcomfort.com/portal

Instead of returning the main page, it just returns:

<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="/portal/">here</a>.</h2>
</body></html>

What it should return is this:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd";>



<!--[if IE 8 ]>    <html class="ie8" xmlns="http://www.w3.org/1999/xhtml";> <![endif]-->
<!--[if IE 9 ]>    <html class="ie9" xmlns="http://www.w3.org/1999/xhtml";> <![endif]-->
<!--[if (gt IE 9)|!(IE)]><!-->
<html xmlns="http://www.w3.org/1999/xhtml";>
<!--<![endif]-->
<head>
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta http-equiv="content-type" content="text/html; charset=UTF-8" />

    <title>Thermostat(s) in Home</title>

...

data-url="/portal/Device/Control/43935?page=1" data-clickenabled="True">
                    <td class="leftendcap" />
                    <td class="location-zone-title">
                        <div class="location-name">Diningroom</div>
                    </td>
                    <td class="zone-temperature">

                        <span class="tempValue" style="">
                            66&deg;
                        </span>
                    </td>

...

data-url="/portal/Device/Control/220861?page=1" data-clickenabled="True">
                    <td class="leftendcap" />
                    <td class="location-zone-title">
                        <div class="location-name">Guest</div>
                    </td>
                    <td class="zone-temperature">

                        <span class="tempValue" style="">
                            78&deg;
                        </span>
                    </td>

...

data-url="/portal/Device/Control/219808?page=1" data-clickenabled="True">
                    <td class="leftendcap" />
                    <td class="location-zone-title">
                        <div class="location-name">Leslie</div>
                    </td>
                    <td class="zone-temperature">

                        <span class="tempValue" style="">
                            73&deg;
                        </span>
                    </td>

... etc.

from which I would be able to scrape the temperatures.  Does anyone have any ideas how I could get curl to handle the task, since wget is failing?  Some other utility?
Reply to: