PDA

View Full Version : bboschecker - check for new blackberry device OS versions (open source, php)

rivviepop
11-07-2007, 11:17 PM
Update = version 0.2 released, new thread here:
http://www.blackberryforums.com/general-blackberry-discussion/129283-bboschecker-0-2-released-blackberry-os-download-checker.html

===

Today I whipped up some PHP code that will scrape the OS downloads page from the FAQ (BlackBerry Operating System Downloads - BlackBerryFAQ (http://blackberryfaq.com/index.php/BlackBerry_Operating_System_Downloads)) and do all the hard work of finding OS releases for your device. The code is released under the GPLv2, fixes and additions welcome. PM me, post here, whatever - so long as you abide by the spirit and law of the GPL license. :) Open Source for the win.

Download: Index of /code/ (http://rivviepop.freehostia.com/code/) (click on ZIP file to download)

NOTE: an online, working version of this is NOT provided, it might lead to abuse of the RIM download servers if a kabillion people try and use it at once. You need to download and run on your own webserver (which isn't hard, it's one whole PHP file with nothing to configure - it either works or it doesn't). If you want to run this on Windows you will need to change one setting at the top (location of a temp file to store web cookies).

The quick one-two punch: simply load the PHP script in your Firefox from your PHP webserver and you're greeted with this:

http://i172.photobucket.com/albums/w21/rivviepop/boc_1.jpg

Put in your device model desired and click Search, let it crank and it looks like this:

http://i172.photobucket.com/albums/w21/rivviepop/boc_2.jpg

See an OS you'd like to read more about, just click on the provided carrier link and you're off and running! It couldn't be more simple to use.

If it feels "slow" loading each carrier page, this is on purpose! I built in a one second (configurable) pause between each page fetch from the RIM servers. It is in our best interest to fly under the radar and not abuse RIM's servers with a tool like this. It is highly suggested you do not lower this value - in fact, increase it if possible on your server.

TODO: add the ability to specify the network type (i.e. search CDMA only), currently the way the line breaks are in the FAQ page code would make this harder than the 10 minutes I want to spend coding it. :)

Blog post: bboschecker.php - check BlackBerry device OS releases « rivviepop phantom (http://rivviepop.wordpress.com/2007/11/07/bboscheckerphp-check-blackberry-device-os-releases/)
rivviepop
11-07-2007, 11:23 PM
Here's the code in a standard 'code' block here on the forum, in case you just wanna read it and stuff. ph34r my PHP skillz. :)

bboschecker.php

<?php
/*
bboschecker 0.1 - check for new BB device OS releases
Copyright (C) 2007 rivviepop (blackberryforums.com)

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

http://www.gnu.org/licenses/gpl-2.0.txt
*/

// where is the list of carriers?
$BBF_LIST = "http://blackberryfaq.com/index.php/BlackBerry_Operating_System_Downloads";

// we will delay this many seconds when retrieving pages from the
// RIM server, we don't want to be a bad netizen.
$RIM_SLEEP = 1;

// the common download entry point (saves on useless parsing)
$RIM_DLURL = "https://www.blackberry.com/Downloads/browseSoftware.do";

// the user agent we'll masquerade as
$SCRIPT_UA = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";

// where to store the session cookiejar
$SCRIPT_CJ = "/tmp/bbos_cookies";
?>

<html>
<head>
<title>BlackBerry Device OS Checker</title>
</head>
<body>

<h3>BlackBerry Device OS Checker</h3>

<?php

// we need to set this globally and pass it around so that the session
// and cookie jar work as a cohesive unit
$curl_handle = curl_init();
curl_setopt($curl_handle, CURLOPT_USERAGENT, $SCRIPT_UA);
curl_setopt($curl_handle, CURLOPT_HEADER, FALSE);
curl_setopt($curl_handle, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($curl_handle, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl_handle, CURLOPT_COOKIEFILE, $SCRIPT_CJ);
curl_setopt($curl_handle, CURLOPT_COOKIEJAR, $SCRIPT_CJ);

function getPage($url, $method="GET", $postfields="") {
global $curl_handle;
if (!is_string($url)) {
return "";
} else {
curl_setopt($curl_handle, CURLOPT_URL, $url);
if ($method == "POST") {
curl_setopt($curl_handle, CURLOPT_POST, TRUE);
curl_setopt($curl_handle, CURLOPT_POSTFIELDS, $postfields);
} else {
curl_setopt($curl_handle, CURLOPT_POST, FALSE);
curl_setopt($curl_handle, CURLOPT_POSTFIELDS, "");
}

$output = "";
$output = @<hidden>($curl_handle);

if (curl_errno($curl_handle)) {
print curl_error($curl_handle);
return "";
}

if (!is_string($output) || !strlen($output)) {
echo "Failure Contacting Server";
return "";
}

return $output;
}
}

if (isset($_REQUEST['device']) && is_numeric($_REQUEST['device'])) {

$device = $_REQUEST['device'];
// IE hack - it buffers and won't display unless 256b
print(str_repeat(" ", 300) . "\n");

echo "Downloading carrier list from BlackBerryFAQ... ";
flush();
$carriers = getPage($BBF_LIST);
echo "<b>done</b>.<br>\n";
flush();

echo "Parsing carrier list for links... ";
flush();
$link_array = array();
$cnt_array = explode("\n", $carriers);
foreach ($cnt_array as $line) {
if (stristr($line, "www.blackberry.com")) {
// http://www.the-art-of-web.com/php/parse-links/
$regexp = "<a\s[^>]*href=(\"??)([^\" >]*?)\\1[^>]*>(.*)<\/a>";
if (preg_match_all("/$regexp/siU", $line, $matches, PREG_SET_ORDER)) {
foreach ($matches as $match) {
# $match[2] = link address
# $match[3] = link text
$link_array[$match[2]] = $match[3];
}
}
}
}
echo "<b>done</b>.<br><br>\n";
flush();

echo "Searching for OS releases for the BlackBerry <b>$device</b><br><br>\n";
flush();
reset($link_array);
while (list($link, $title) = each($link_array)) {
echo "Checking <a href=\"$link\"><b>$title</b></a>...<br>\n";
flush();
$rim_page = getPage($link);
$rim_array = explode("\n", $rim_page);
$got_v = 0;
$got_c = 0;
$value = "";
$code = "";
foreach ($rim_array as $line) {
// get the value
if (stristr($line, $device)) {
$regexp = "<option\svalue=\"([^\"]*)\">(.*)<\/option>";
if (preg_match("/$regexp/siU", $line, $match)) {
$got_v = 1;
$value = $match[1];
}
}
// get the code
if (stristr($line, "type=\"hidden\" name=\"code\"")) {
$regexp = "value=\"([^\"]*)\"(.*)";
if (preg_match("/$regexp/siU", $line, $match)) {
$got_c = 1;
$code = $match[1];
}
}
}
if ($got_v && $got_c) {
echo "&nbsp;&nbsp;Found one, retrieving product page...<br>\n";
flush();
$postvars = "productName=$value&code=$code";
$os_page = getPage($RIM_DLURL, "POST", $postvars);
$os_array = explode("\n", $os_page);
foreach ($os_array as $osline) {
if (stristr($osline, "Applications:")) {
$result = preg_replace("/<(.*)>/siU", "", $osline);
echo "&nbsp;&nbsp;&nbsp;&nbsp;<b>$result</b><br>\n";
}
}
}
sleep($RIM_SLEEP);
}

echo "<br><b>All done!</b><br>\n";

} else {
?>

Enter the model number ONLY of the device.<br>
(7290, 8100, 8320, 8800, etc.)<br>
<form name="deviceselect" method="GET" action="<?php $_PHP_SELF; ?>">
<br>
<input type="text" size="10" name="device">
<input type="submit" value="Search Carriers">
</form>

<b>Note</b>: This tool will only search carriers who<br>
host their downloads on www.blackberry.com<br>

<?php
} // device
curl_close($curl_handle);
?>

<br>
Code by <a href="http://www.blackberryforums.com/member.php?u=82101">rivviepop on BlackBerryForums</a><br>
Released under the <a href="http://www.gnu.org/licenses/gpl-2.0.txt">GPLv2 license</a><br>
</body>
</html>
mrking
11-08-2007, 12:44 AM
The download link on your blog takes me here....

Free Hostia: domain hosting and web hosting service (http://www.freehostia.com/)

I just copy pasted code from your second post.
rivviepop
11-08-2007, 12:58 AM
The download link on your blog takes me here....
Free Hostia: domain hosting and web hosting service (http://www.freehostia.com/)
I just copy pasted code from your second post.

darn them! thanks a lot, I just edited the post to show how I did it and it works well enough. apparently Freehostia intercepts direct ZIP links from an offsite server (referer) but doesn't when you click on one from the directory browse.
Malkier
11-08-2007, 01:10 AM
Excellent idea, looks great.
Glad to see you placed a pause between page fetches. Respect is always good.

Thanks Rivviepop (y)
dietcoke
11-22-2007, 08:26 PM
cant this find os 4.3?
ezrunner
11-22-2007, 09:13 PM
Wirelessly posted (BlackBerry8700/4.2.1 Profile/MIDP-2.0 Configuration/CLDC-1.1 VendorID/100)

Sweeet rivvie

Nice work!
John Clark
11-22-2007, 09:27 PM
since I don't know much about php and/or setting up web pages I never figured out how to run this....yep, I'm a newbie in this dept.
paulbblc
11-22-2007, 09:58 PM
cant this find os 4.3?

Please tell me you're just kidding... please.

Nice work Rivvie!
rivviepop
11-23-2007, 12:18 AM
since I don't know much about php and/or setting up web pages I never figured out how to run this....yep, I'm a newbie in this dept.

As far as PHP goes, getting this to run is super simple as long as your Apache/PHP setup is of the more standard variety. For instance a Fedora/Red Hat/CentOS based install - out of the box, no modifications - will run it perfectly, it's what I use.

The one feature which it uses is the 'cURL' stuff; this *might* not be pre-compiled in on all instances which could be a stumbling block. It's part of PHP that hooks in to an external library, you'll have to look at your server (or even desktop, this could run under a basic setup) to find out yes or no. cURL is an advanced HTTP program/library that offers features which I use.
John Clark
11-23-2007, 12:23 AM
Oh, dear....I'm going to have to go back to school.

You're so far over my head I've got Athlete's Scalp!
rivviepop
11-23-2007, 12:34 AM
Oh, dear....I'm going to have to go back to school.
You're so far over my head I've got Athlete's Scalp!

Put me in the c.ockpit of one of your planes and our roles are reversed. "oh look, shiny!" *push* *flip*

:)
Keebler
11-23-2007, 08:53 AM
Nice work rivvie, I was wondering when someone might do this
NJBlackBerry
11-23-2007, 08:56 AM
Put me in the c.ockpit of one of your planes and our roles are reversed. "oh look, shiny!" *push* *flip*

:)

The difference Rivvie, is that you would try to write a device driver for one of the engines :-) :-) :-)

Nice work, BTW.
guyheavensent
11-27-2007, 01:49 PM
i use this quite frequently. i appreciate your time making it. if you dont mind i have a few questions/problems i was wondering if you could help me out with.

when i run it and type 8830 or 8130 or 7103 it shows a whole list of vendors and their OS, but it misses out on Verizon Wireless. Why is that? Something overlooked someplace? :(
rivviepop
11-27-2007, 02:48 PM
when i run it and type 8830 or 8130 or 7103 it shows a whole list of vendors and their OS, but it misses out on Verizon Wireless. Why is that? Something overlooked someplace? :(

The tool only supports downloads which are hosted by the RIM servers (blackberry.com) as noted above, as they have a common web page format that can be parsed identically. Carriers who do not host their downloads via RIM aren't supported, Verizon is one of them (as is T-Mobile USA) since a) it's a completely different format, and b) they require you to enter your phone number to see the downloads.

Look at this page:
BlackBerry Operating System Downloads - BlackBerryFAQ (http://blackberryfaq.com/index.php/BlackBerry_Operating_System_Downloads)

All the ones with a yellow lock icon are hosted on RIM, all the ones with a little blue box and up-right arrow are hosted on a unique carrier server, not supported. You could, with some work, take my code and extend it to know the format of Verizon's site and include entry fields for the phone number; I however have no desire to work on this featureset.
jungleland
11-28-2007, 01:50 PM
what if we don't have a web server..


Can't this program operate as a stand alone app on ones PC or mac? or the handheld itself?
dietcoke
11-28-2007, 01:56 PM
what if we don't have a web server..


Can't this program operate as a stand alone app on ones PC or mac? or the handheld itself?


I don't have apache or windoze server 2003 to install php on.

How many people would really abuse this?
rivviepop
11-28-2007, 04:05 PM
what if we don't have a web server..
Can't this program operate as a stand alone app on ones PC or mac? or the handheld itself?

You can install Apache & PHP on a desktop without any problems, many of the engineers at my place (including me, but I use linux) have this as a standard chest of tools. I've not done it with my code, but as long as the Win32 PHP supports cURL then it'll be fine.
rivviepop
11-28-2007, 04:08 PM
I don't have apache or windoze server 2003 to install php on.

You can do it on XP or 2000 as well, and Apache/PHP are free. Nothing is stopping you but time.

How many people would really abuse this?

You can't possibly claim to know the hearts & minds of every BBF member, and neither can I. As a matter of respect for RIM's servers I choose not to make it available and just avoid the abuse issue completely, I am a good netizen and do my part.