25 ways to insecurity

The 2009 CWE/SANS Top 25 Most Dangerous Programming Errors was recently released by CWE/SANS.

Most of the items are old news but I think it is a good checklist that should be on the boiler plate for web application design documents. By putting security requirements in the software specification and design documents, the project manager can then allocate time and resources to security aspects of development. In addition, it reminds developers to ask themselves if the software is meeting those requirements throughout the development process. This is opposed to thinking about security after the entire application has been written and discovering a fundamental design flaw that will require re-writing a good portion of the application.

I particularly appreciate that each item on the CWE/SANS list is weighted including weakness prevalence, remediation cost, attack frequency, attacker awareness, etc. No project has an unlimited budget but you can prioritize on where to focus your resources to achieve the most secure solution. Generally it is a good idea to ensure that the cost of defeating an application’s security far outweighs any benefits to be gained from doing so. The cost of defeating an application might include labor time, computing resources, fines, and threat of jail time with a cell mate named Bubba, etc.

It is quite a challenge to develop secure web applications because generally by their nature they need to accept user input. I believe that it is typically much more difficult develop a secure system than it is to break in to the system given the same number of hours so there is often more burden on the developer. It might take only two or three days to develop a working database driven web application but many additional weeks to harden it against attacks and make it reliable, scalable, and highly available. Including security requirements in the software specification and design is essential to planning and allocating resources.

Ideally automated tests should be included to continuously test vulnerabilities throughout the life of an application. This way security vulnerabilities introduced by code changes will be detected early in the development process instead of later in production. Automated tests could attempt buffer overflows, sql injections, etc. and could be executed prior to a developer’s check-in or on a nightly cron job that automatically checks out the code and runs the tests against it. Although costly to implement initially, automated security testing will likely pay for itself many times over the course of an application’s life. I plan to talk more about automated testing in future posts.

PHP HttpRequest class options and notes

In a recent post I talked about the PECL_HTTP extension for PHP. In this post I will cover a few of the options you can set for the HttpRequest object and related functions.

The PECL_HTTP extension allows you to set a number of options when you make a request. Usually you put the options in a key=>value array and pass the array as an argument to the request functions (I.e. http_get(), http_post_data(), http_request(), etc.) or assign the array to the HttpRequest object using the setOptions() method. Here is a code example using the HttpRequest object:

$http_req = new HttpRequest("http://www.example.com");
$http_req->setOptions(array(timeout=>10,useragent=>"MyScript"));

In the above code the timeout was set to 10 seconds and the user agent, which is a request header that identifies the browser to the server, is set to “MyScript”. I am going to cover just a few of the options but a full list of all the request options can be found here.

timeout

The timeout option specifies the maximum amount of time in seconds that the request may take to complete. Set this too high and your HTTPD process that PHP is running in could be stalled for quite a bit waiting for a request that may never complete. If you set it too low you might have problems with sites that are just slow to respond. This may require some tweaking depending on what you are doing. If you are making requests to a web server in Taiwan you might want to set the timeout a bit higher. The default timeout does not appear to be documented in the HttpRequest options page on PHP.NET (that I could find) but if you look at the http_request_api.c file in the HTTP_PECL source code, it looks like it is 0L AKA nothing:

HTTP_CURL_OPT(CURLOPT_TIMEOUT, 0L);

This indicates it will wait forever unless you explicitly set a timeout so it might be a good idea to set one! I put together a page that makes an HTTP request and then another one that will sleep for some number of seconds that I can test against.

Here is the code for the page that will sleep:

<?php

echo "Sleeping.";
sleep(30);

?>

Here is the code for the page that will make the HTTP request:

<?php
$http_req = new HttpRequest("http://localhost/alongtime.php");
$http_req->setOptions(array(timeout=>10, useragent=>"MyScript"));
try {
    $http_req->send();
} catch (HttpException $ex) {
    if (isset($ex->innerException)){
        echo $ex->innerException->getMessage();
        exit;
    } else {
        echo $ex;
        exit;
    }
} //end try block
echo $http_req->getResponseBody();
?>

When I pull up the page that makes the HTTP request in my browser I get the following error:

Timeout was reached; Operation timed out after 10000 milliseconds with 0 bytes received (http://localhost/alongtime.php)

If I don’t set the timeout option at all, the page responds 30 seconds later since it will wait forever or at least the 30 second sleep time on the target page.

connecttimeout

The connecttimeout option indicates the maximum amount of time in seconds that the request may take just connecting to the server. This does not include the time it takes for the server to process and return the data for the request. This option will have the same considerations as above although the number should be considerably lower since it is only the connection timeout and not the timeout for the whole request. Again, the default value is not documented but if you look at the http_request_api.c file in the HTTP_PECL source code, it looks like it is 3 seconds:

HTTP_CURL_OPT(CURLOPT_CONNECTTIMEOUT, 3);

dns_cache_timeout

One of the interesting features of the HTTP_PECL extension is that it will cache DNS lookups. Some of the Windows operating systems do this but many of the Linux distributions do not by default. By the way, if you want to clear your cached DNS lookup entries on a Windows box use the command “ipconfig /flushdns”. If you are making multiple requests to the same site, DNS lookup caching should provide a significant performance advantage because a round trip to the DNS server isn’t required for every request. The dns_cache_timeout option sets the number of seconds that will pass before the cached DNS lookup results will expire and a new DNS lookup will be performed. Again, the default value is not documented but if you look at the http_request_api.c file in the HTTP_PECL source code, it looks like it is 60 seconds which is probably fine for most applications:

HTTP_CURL_OPT(CURLOPT_DNS_CACHE_TIMEOUT, 60L);

redirect

The redirect option determines how many redirects the request will follow before it returns with a response. The default is 0 (this IS documented), which may not work in many situations because some applications respond with one or two redirects for authentication, etc. If you set this too high your application may get bounced around too many times and never return. I have not tried it but you could probably put someone in a redirect loop. Anyway, a value of around 4 or 5 should be adequate for most applications I would imagine.

useragent

The useragent option allows you to specify a different User-Agent request header to send to the server than the default which is “PECL::HTTP/x.y.z (PHP/x.y.z)” where x.y.z are the versions.

I made a little one-liner test page that returns the user agent info sent to the server:

<?php

echo $_SERVER['HTTP_USER_AGENT'];

?>

If I make an HTTP request to this page using the HttpRequest object without setting the useragent I get:

PECL::HTTP/1.6.2 (PHP/5.2.6-2ubuntu4)

If I do something like this:

$http_req->setOptions(array(timeout=>10, useragent=>"Mark’s Browser"));

I will get:

Mark’s Browser

The reason I bring this up is because some applications that you might make a request to may respond different depending on your user agent setting. In some cases you may need to spoof a specific browser to get what you are after.

Conclusion

As mentioned before, there are many more HttpRequest options. I just covered a few notable ones that I have some limited experience with.

Review: Lego 6211 Star Wars Imperial Star Destroyer

5 out of 5 stars
Jen gave me the Lego 6211 Star Wars Imperial Star Destroyer for Christmas and I finished building it over the following week.

The build is pretty easy (even for a 35 year old) and a lot fun. The kit comes in over 1300 parts so you really feel like your building something and not just putting a couple halves together. The parts come in numbered bags that correspond to the numbered sections in the instructions. There are often multiple bags with the same number for a single section. Lego included a few extras of the tiniest parts that tend to get lost in the carpet.

There are only a couple downsides to the kit. The two “laser blasters” that launch projectiles work by you just quickly depressing the launch button that drives a wedge piece behind the projectile to push it out. This is kind of silly to me and I think the kit would have been better if they were just static elements in my opinion.

The other negative was the capsule that goes inside. It is assembled by connecting two specially molded pieces together. I don’t like this kind of Lego construction and would have preferred it if they just left it out or designed it to build from smaller, standard Lego pieces.

I put together a list of pros and cons:

Pros

  • Over 1300 parts so it will keep you building for a bit.
  • Numbered bags.
  • Spare tiny pieces.
  • Looks great!
  • It’s big. About 23 inches long.

Cons

  • The laser blasters that shoot projectiles are kind of cheesy.
  • The capsule that goes inside is mostly just two halves that go together.
  • Doesn’t shoot real bolts of green light. ;)

Build gallery

I took some pictures while my Imperial Star Destroyer was under construction:

Embellishment

I felt the kit would have been better if they included lights and a small universe. Fortunately, I was able to accomplish this with a small string of Christmas lights, a telescope photo of the stars, black velvet, and a little Photoshop magic:

Conclusion

I easily give this kit 5 out of 5 stars. It was fun to build and looks great.

How to: PECL HTTP request exception and error handling

In a previous post, we created a simple PHP page that told us if http://www.example.com is up or down by using the PECL HTTP extension to make an HTTP request to the site and look for the string “example” in the response. Here is the code for our test page, httptest.php:

<?php

$http_req = new HttpRequest("http://www.example.com");
$http_req->setOptions(array(timeout=>10,useragent=>"MyScript"));
$http_req->send();

if (stripos($http_req->getResponseBody(), "example") === false){
    echo "The page is down!";
} else {
    echo "The page is up!";
}

?>

The problem with this code is that there is no error handling. Below are a few examples of what can go wrong and the resulting errors:

DNS Lookup Failure

If the DNS lookup fails the page will return the following error:

Fatal error: Uncaught exception 'HttpInvalidParamException' with message 'Empty or too short HTTP message: ''' in /var/www/httptest.php:12 inner exception 'HttpRequestException' with message 'Couldn’t resolve host name; Couldn’t resolve host 'www.somewebsitethatdoesnotexist.com'
(http://www.somewebsitethatdoesnotexist.com/)' in /var/www/httptest.php:4 Stack trace: #0 /var/www/httptest.php(12): HttpRequest->send() #1 {main} thrown in /var/www/httptest.php on line 12

Since www.example.com is a valid DNS name I used “www.somewebsitethatdoesnotexist.com” instead to demonstrate what happens with an invalid name or failed DNS lookup. Note the “inner exception” that says “Couldn’t resolve host name”. More on “inner exceptions” in a bit. This is not very pretty for a diagnostic page.

Connection Failure

In this example I again used “www.somewebsitethatdoesnotexist.com” but I added the following entry to the /etc/hosts file on the server:

10.10.10.10 www.somewebsitethatdoesnotexist.com

Now the DNS entry will resolve using the /etc/hosts file but this is not a valid IP for any machine on my neetwork so I see this error:

Fatal error: Uncaught exception ‘HttpInvalidParamException’ with message ‘Empty or too short HTTP message: ''' in /var/www/httptest.php:12 inner exception ‘HttpRequestException’ with message ‘Timeout was reached; connect() timed out! (http://www.somewebsitethatdoesnotexist.com/)’ in /var/www/httptest.php:4 Stack trace: #0 /var/www/httptest.php(12): HttpRequest->send() #1 {main} thrown in /var/www/httptest.php on line 12

Again we have a inner exception buried in all of that telling me that the connection time out.

404 Error

In this example I put in “http://localhost/notarealpage.php” for the URL. This will connect to the local Apache server but that page doesn’t exist so the server will return a 404 file not found error. The server responded but since we are not checking the response code from the server our code just tells us the page is down and that is true but it would be useful to know that it is because the page is missing!

The page is down!

If the server responds OK we will get a 200 status code. We should handle any other response appropriately.

Handle the exceptions

The first thing we can do is put a try catch block around our code and try catching the HttpException as shown in example section of the documentation for the HttpRequest::send method:

<?php

$http_req = new HttpRequest("http://www.example.com");
$http_req->setOptions(array(timeout=>10,useragent=>"MyScript"));

try {
    $http_req->send();
} catch (HttpException $ex) {
    echo $ex->getMessage();
}

if (stripos($http_req->getResponseBody(), "example") === false){
    echo "The page is down!";
} else {
    echo "The page is up!";
}

?>

If there is a time out or connection failure the HttpException is caught and we see this:

Empty or too short HTTP message: ''The page is down!

Hmm… that is not very informative and the same error is displayed for both a name lookup failure and a connection timeout. We can also try changing:

echo $ex->getMessage();
to
echo $ex;

Now we get this:

exception 'HttpInvalidParamException' with message 'Empty or too short HTTP message: ''' in /var/www/httptest.php:16 inner exception 'HttpRequestException' with message 'Couldn’t resolve host name; Couldn’t resolve host 'www.ssomewebsitethatdoesnotexist.com'
(http://www.ssomewebsitethatdoesnotexist.com/)' in /var/www/httptest.php:5 Stack trace: #0 /var/www/httptest.php(16): HttpRequest->send() #1 {main}The page is down!

That is painfully ugly but at least get the “Couldn’t resolve host name” message in there so we know what went wrong. Still, we can do better.

In addition to putting a try-catch around the send() method you probably should surround all of your HttpRequest code with a try-catch that eventually catches “Exception” to be safe.

The undocumented inner exception

The HttpException object, which is not really documented all that well as of this writing, has something completely undocumented called an inner exception. The inner exception is a more detailed exception that is nested inside the HttpException object. We can check if an inner exception is set and if so, display that instead:

<?php

$http_req = new HttpRequest("http://www.example.com");
$http_req->setOptions(array(timeout=>10,useragent=>"MyScript"));

try {
    $http_req->send();
} catch (HttpException $ex) {
    if (isset($ex->innerException)){
        echo $ex->innerException->getMessage();
        exit;
    } else {
        echo $ex;
        exit;
    }
}

if (stripos($http_req->getResponseBody(), "example") === false){
    echo "The page is down!";
} else {
    echo "The page is up!";
}

?>

Now we get just the part we are interested in:

Couldn’t resolve host name; Couldn’t resolve host 'www.ssomewebsitethatdoesnotexist.com'
(http://www.ssomewebsitethatdoesnotexist.com/)

If the lookup is OK but we get a connection timeout we now see this:

Timeout was reached; connect() timed out! (http://www.somewebsitethatdoesnotexist.com/)

If no inner exception is detected the HttpException is echoed.

Check status codes

Sometimes the server may be responding but we do not get a 200 status. This could because of a redirect, security error, missing page, or a 500 server error. The HttpRequest object provides a getResponseCode() method so we can check what the response was and handle it appropriately. If redirects are followed the last received response is used. For this example we will simply echo out some of the common status codes if we don’t get a 200:

<?php

$http_req = new HttpRequest("http://www.example.com/blah");
$http_req->setOptions(array(timeout=>10,useragent=>"MyScript"));

try {
    $http_req->send();
} catch (HttpException $ex) {
    if (isset($ex->innerException)){
        echo $ex->innerException->getMessage();
        exit;
    } else {
        echo $ex;
        exit;
    }
}

$response_code = $http_req->getResponseCode();

switch ($response_code){
    case 200:
    break;
    case 301:
    echo "301 Moved Permanently";
    exit;
    case 401:
    echo "401 Unauthorized";
    exit;
    case 404:
    echo "404 File Not Found";
    exit;
    case 500:
    echo "500 Internal Server Error";
    exit;
    default:
    echo "Did not receive a 200 response code from the server";
    exit;
}

if (stripos($http_req->getResponseBody(), "example") === false){
    echo "The page is down!";
} else {
    echo "The page is up!";
}

?>

This handles a few of the more common response/status codes. To test the code we can put in a valid URL but a bogus page (I.e. http://www.example.com/blah) If everything works right we now see the following response from our diagnostic page:

404 File Not Found

Final Notes

Our little diagnostic page can now handle most of the errors it will likely encounter when it attempts to test our example site, example.com. If we wanted to take this a bit further we could add a database back-end that maintains a list of multiple sites we would like to test. To take things a step further we could execute this PHP script from the command line via a cron job that runs every 5 minutes. We could then have the script send an e-mail when a site was down with the problem indicated in the message. If we wanted to maintain some up-times stats would could log the outages to a database and generate uptime/SLA reports daily, weekly, yearly, etc.

In reality, I would just use something like IPSentry or Nagios to conserve effort for future generations but it was nice to think about. ;) Happy coding!

How to: Find your php.ini file and enable PHP error reporting

On some distributions PHP error reporting or display of errors is disabled by default as a security precaution. This is a good idea for production systems because errors may reveal useful information to undesirables. In a development environment on the other hand, it is generally useful to see your errors. ;) If error display is disabled you may just see a blank browser window/empty page when you expect an error. To enable errors and error display, find the your php.ini file and make sure the following lines are set:

;Show all errors, except for notices and coding standards warnings
error_reporting = E_ALL & ~E_NOTICE

display_errors = On

On Ubuntu you can find the php.ini file here:
/etc/php5/apache2/php.ini

On other distributions try:
/etc/php.ini

On Windows you might find it here:
c:\windows\php.ini

If you are running XAMPP it will be in the php folder off the XAMPP root.

You will need to restart Apache (or IIS as the case may be) so your changes will take effect:

On Ubuntu:

sudo /etc/init.d/apache2 restart

On other distributions you might try:

sudo /etc/init.d/httpd restart