Mon Dec 5 18:08:48 2011 Sorry, ‘Auth’ password cannot be read from a file

Another one I’ve seen is:

Failed to enable OpenVPN. Please check the configuration file

These errors occur when your VPN provider requires password authentication. Some providers include the password in a file so you don’t have to type it in; others require you to type it in whenever you log on.

Synology compiled their build of OpenVPN with password file support disabled, as is standard for OpenVPN builds. Fortunately, there is a one-byte patch to /usr/local/synovpn/sbin/openvpn which reverses that: just change the byte at 0x21113 from 0A to EA. This modifies the control flow within OpenVPN to permit reading from password files. (Note that this will only work on OpenVPN 2.1.4 for ARM, built by Synology on 24/08/2011, with MD5 hash d6305ae735b3e49c360fc1be5eccb539.)

I’ve prepared a pre-patched copy, which is available for download here. Follow these instructions to get it working:

1. (Skip this if you aren’t prompted for a username/password when you connect to your VPN provider.) Create a file named pass.txt in /usr/syno/etc/synovpn/openvpn – the first line should be your username; the second should be your password. Then add auth-user-pass pass.txt to openvpn.conf.user.

2. Replace /usr/local/synovpn/sbin/openvpn with the one provided above.

3. Make the replacement executable: chmod +x /usr/local/synovpn/sbin/openvpn

Please let me know whether this worked for you by leaving a note in the comments.

Update: download this instead if your Synology device has an x86 CPU. You can check here if you’re not sure.

The front door to our house is augmented by a motorised door opener, which is controlled by a Locca Access receiver. We have a bunch of remote keys that trigger the Locca receiver to open the door, but I started thinking: wouldn’t it be great to be able to trigger the door opener from an iOS or Android device?

Since the ‘KeeLoq’ code hopping algorithm (the cornerstone of the Locca system) was broken back in 2009, my first thought was to reverse-engineer the remote keys and attempt to build a clone with a supplementary Wi-Fi chip. Further investigation indicated that the specialist equipment necessary to execute the required attacks against KeeLoq would be prohibitively expensive.

My next idea was to modify a genuine Locca remote key so it could be triggered artificially. The artificial trigger could either be mechanical, i.e. use a motor to push the button, or electronic, i.e. simulate a button push by providing power to the HCS200 KeeLoq encoder chip and RF transmitter. It’s cleaner and more reliable to trigger the remote electronically, if possible.

Modifying the Locca Remote Key

Upon opening the casing of the remote – which is surprisingly difficult – it appears to be a simple circuit with a tiny push button situated between the power source (2x CR2032 3V batteries) and the rest of the components. When you push the button, the circuit is completed, the components are powered up, and the KeeLoq chip gets to work.

In theory, if we bypass the push button by completing the circuit around it, the above chain of events should occur and the door will open. I tested this theory by soldering two wires to each side of the button. As expected, the door opens when the wires are crossed. I got out my Arduino Uno and added a transistor, thus completing the remote key circuit when the base terminal is pulled high (achieved by a digitalWrite on the Arduino).

Controlling the Arduino from afar

Of course, we still need a way to tell the Arduino to perform that digitalWrite from an iPhone. There are two viable modes of communication: Bluetooth or Wi-Fi. Bluetooth doesn’t carry many advantages, other than the fact that Bluetooth shields for Arduino boards are fairly common and well documented. However, we’d need to design a protocol and deal with the intricacies of Bluetooth on iOS – no thanks.

For this prototype, I decided to connect the Arduino Uno to an external server via USB, and to use serial-over-USB to signal it to trigger the remote key. The Arduino sketch looks like this:

When a byte is received via the serial line, triggerRemote is called, and the door should open. Clearly the iPhone can’t directly write to the serial line, so I wrote a small HTTP interface in Python which listens on the LAN and writes a byte to /dev/ttyACM0 (the Arduino Uno’s serial character device on Linux) when a suitable request is received. This runs on the machine that the Uno is connected to. Now any device on the LAN can now open the door with a single HTTP request.

Putting it all together

All that’s left to do is create a simple app to serve as a wrapper around the HTTP interface:

When you tap the yellow button, a HTTP request is dispatched in the background, and the door opener springs to life. Here it is in action:

Downloads

This is free and unencumbered software released into the public domain. Please see the Unlicense for further details.

• Download the Arduino sketch
• Download the Python server
• Download the iPhone app
• Download the Android app
• Browse the Git repository

Of course, one can use Django’s built-in cache framework with or without Varnish. Django is great at caching parts of generated pages, but in my opinion it’s better to use a dedicated HTTP accelerator if you want to cache whole pages.

Configuring Varnish for use with Django

Varnish uses VCL files for configuration. These give precise control over every aspect of the request pipeline; VCL is essentially a mini language for specifying how an incoming request should be handled.

I use the following VCL for production Django sites, most of which is borrowed from Chase Seibert:

It makes the following adjustments to Varnish’s behaviour:

• All cookies except for csrftoken and sessionid are ignored. Varnish will stop serving cached pages when cookies are set; this includes cookies set by Google Analytics or other client-side JavaScript code. By ignoring these cookies, Varnish will still serve cached content to unauthenticated users. Authenticated users will never see a cached page.

• Accept-Encoding is normalized to ensure that Varnish doesn’t cache multiple copies of the same page. Browsers tend to differ slightly in how they specify Accept-Encoding, and Varnish uses it as a differentiator when calculating the hash key for a given request.

• URLs beginning with /static are always cached. Static content is likely to remain the same whether a user is logged in or not, so we remove cookies from these requests to ensure that they get cached.

Caveats

Even with these adjustments in place, there are still two major caveats:

1. If an object changes after being cached, there will be an unspecified delay before those changes become visible to unauthenticated users.

2. Once a user logs in or accesses a form with CSRF protection, they will never see a cached page again unless they erase the csrftoken and/or sessionid cookies, even after logging out. Django doesn’t delete the sessionid cookie upon logout; instead, the session is regenerated and a new sessionid is set.

python-varnish and django-varnish

To solve caveat #1, we must purge an object’s URL from the Varnish cache after it has been changed. django-varnish does just that: it monitors certain models and purges an object’s get_absolute_url when the object is updated. I use a fork which can also purge the entire cache when certain models are updated.

django-varnish depends on python-varnish, which provides a simple Python API for management of Varnish servers via their management ports.

Installation

Unfortunately the original maintainer’s code no longer works with the current version of Varnish, but alternative up-to-date forks are available. These can installed via pip:

1
2

pip install -e git://github.com/kennu/python-varnish#egg=varnish
pip install -e git://github.com/kennu/django-varnish#egg=django-varnish

Configuration

Once installed, just add django.contrib.humanize and varnishapp to INSTALLED_APPS, add (r'^admin/varnish/', include('varnishapp.urls')) to your URLconf, and add some settings:

• VARNISH_WATCHED_MODELS is a list of installed models whose get_absolute_urls you want to purge from the Varnish cache upon saving. Example: ('auth.user','profiles.profile')

• VARNISH_GLOBAL_WATCHED_MODELS is a list of installed models who will purge the entire Varnish cache upon saving.

• VARNISH_MANAGEMENT_ADDRS is a list of Varnish cache addresses with their management ports. Example: ('server1:6082','server2:6082')

• VARNISH_SECRET is the shared secret used to authenticated with the Varnish server. This can be found at /etc/varnish/secret on an Ubuntu/Debian installation of Varnish.

Once you’ve set all of that up, changes to an object will result in the object’s absolute URL being purged from your Varnish cache.

Delete Django session cookies upon logout

Solving caveat #2 requires a behavioural change to Django: we need to delete the csrftoken and sessionid cookies after the django.contrib.auth.views.logout view is called. We could try to monkey patch this view, or we can write a small piece of middleware to do the job. Monkey patching is a terrible, terrible crime, so I went for the middleware option instead.

Create a file named middleware.py in your project’s root directory:

Then add this new middleware to the beginning of MIDDLEWARE_CLASSES in your project’s settings file.

Results

I ran ab against the homepage of a project I’ve been working on recently, testing against both Varnish and Apache. At the time of testing, each direct request for the page resulted in a total of 50 database queries. I tested from the machine where the site is hosted.

I used the following arguments: -n 1000 -c 50 -k. This instructs it to perform 1000 requests with a concurrency level of 50, with HTTP persistent connections enabled.

The results (below) highlight why it’s absolutely necessary to employ a proper caching strategy for your production Django deployments. Whether you use Varnish, or another HTTP accelerator, or even Django’s built-in whole page cache, the implications are clear: if you allow users to hit Apache directly on every page load, your site won’t stay online for long.

(Update: It was suggested within the comments that I run ab with the -k argument to enable HTTP persistent connections, so I repeated the test and amended the results above.)

One of its most useful features is the so-called Download Station. This allows you to upload torrents, NZBs and HTTP links to the NAS and it does the rest; downloads are completed automatically in the background. It is lacking in a few areas, however, and is superseded by dedicated software such as Transmission and SABnzbd. The good news is that both of these applications (and many others) can also be installed onto any compatible Synology NAS via ‘Synology package files’ (SPKs).

Zebulon’s SPK Repository

Zebulon maintains an SPK repository with a selection of third-party applications; one just needs to add that repository to the Package Center to start installing applications from it.

After logging in to DSM, open up the Package Center application and click Settings. Then select Package Sources and click Add. Enter the following details:

1
2

Name: Zebulon's SPK Repository
Location: http://synopkg.superzebulon.org/spkrepo/packages

Once you’ve added the repository, select the Other Sources tab in Package Center. You’ll now see a list of the application packages maintained by Zebulon. From here, installing Transmission or SABnzbd is simply a matter of clicking Install. Once an application is installed, visit the Installed tab to activate it.

A word of warning: it’s important that you create a new, separate shared folder for each application to store its downloaded content. Otherwise, you’ll run into permission errors.

So now life is good: your Synology NAS can perform download tasks all by itself. But what if you wanted to connect through an encrypted VPN service for additional privacy? Or perhaps you’re forced to because your ISP throttles BitTorrent and Usenet traffic. Fortunately, there is a solution.

Installing the OpenVPN client

Under the Available tab in Package Center, there is a list of official Synology packages. One of these is VPN Server, which is described as offering ‘an easy VPN solution that turns your DiskStation into a VPN server’. With a bit of configuration tweaking, however, it’s possible to turn this into a VPN client instead. Install the package, but don’t activate it yet.

Your VPN provider should have provided sets of OpenVPN config files. If you use Viscosity on a Mac, these can be found in ~/Library/Application Support/Viscosity/OpenVPN. You’ll need two .crt files and a .key file, plus the .conf configuration file. SFTP into the DiskStation and drop these files into /usr/syno/etc/synovpn/openvpn, renaming the .conf file to openvpn.conf.user.

Now activate VPN Server in Package Center. These configuration changes effectively transform the VPN ‘server’ into a client; now, while this package is active, an OpenVPN daemon will be kept alive in the background and all outgoing connections from the NAS will be routed through the VPN. Now, any Transmission and SABnzbd traffic coming from your NAS is SSL encrypted and protected against traffic shaping.

As an iPhone developer with an app on Cydia, I can see great potential within the PS3. It’s crying out for a decent package manager, but you need OS-level access to do that effectively. Unlocking the PS3 in this way has other benefits too; the system can be modified in any way you wish. The gates are effectively thrown open.

So today I’m releasing three tools which open the PS3’s hypervisor (lv1) and GameOS kernel (lv2) to full read/write access from packaged userland applications. These tools can be used to create and test kernel patches in RAM, which negates the risk of bricking your PS3 by flashing it with an incorrectly patched binary. You can also use the tools to create a patched lv1 or lv2 binary, if you wish, although I suggest thoroughly testing your patches in RAM first.

Disclaimer

I don’t condone piracy and these tools do not enable copied games to run on the PS3. To rephrase: these tools will not allow backup managers to suddenly start working. The tools are packaged in source code form and do not include any Sony code or other Sony assets such as encryption keys. If you’re not a developer, these tools will be useless to you, so please do not try to use them.

Tools I’m releasing today

• resign_self.py. This allows you to automatically replace any segment within a self and re-sign the self so the signature and hashes are all valid again. Similar to makeself, but it is more suited towards patching lv1 and lv2 (and has already been used for this purpose).

• insert_lv1_lv2.py. This is just a convenience script I made to take a modified, re-signed lv1.self and lv2_kernel.self, and automatically create a PUP which is identical to an original PUP except for those two files.

• lv1dumper. This is an application which runs on the PS3 that you can compile and package using PSL1GHT and geohot’s tools. After running it, lv1 will be mapped at 0x8000000014000000 with read/write access, and you will be able to poke lv2 without the system shutting down to protect itself. It disables the new lv2 memory hashing feature Sony added to 3.55 (probably to stop future USB jailbreaks). It can also be used to dump lv1.

lv1dumper requires that some patches to lv1 and lv2 are already in place. I’ll now describe how to add these patches. They have been tested but I cannot guarantee that they won’t brick your console.

Adding necessary patches for lv1dumper

You need to extract and decrypt code segments from lv1.self and lv2_kernel.self (just use unself and copy them directly out of the ELF), and make the following changes to to them, assuming you’re using 3.55:

(Update: the code segment is not the entire ELF file, it’s inside the ELF file. Use readelf to find out where it is. You need to copy it out of the ELF into a separate file.)

• lv1_undocumented_function_114 in lv1 must be patched so that it can be used to map any area of real memory. graf_chokolo found this trick months ago, but it still applies here. Patch the byte at D5A47 from 00 to 01 (it’s at 2D5A47 in IDA) in segment 1.

• You then need to add peek and poke to lv2. Patch 1933C to E8 63 00 00 60 00 00 00 and 19348 to F8 83 00 00 60 00 00 00 in segment 0.

You can then use resign_self.py to re-insert your patched code segment back into the self. You’ll firstly need to change a few bytes in some useless strings because of the way zlib deflate works; the script will tell you what to do. I found that changing strings was the easiest way to do this, it just takes a bit of trial and error.

Finally, use insert_lv1_lv2.py to create your modified PUP. You’ll need to update to the PUP, then install geohot’s jailbreak PUP over the top of it. If you’ve done everything right, lv1dumper should just exit after you run it and you’ll have r/w access to lv1 and lv2 (peek and poke). The lv1_peek, lv1_poke, lv2_peek and lv2_poke functions in lv1dumper should illustrate how that access can be used.

I’m hoping that some interesting and innovative stuff can come out of this; perhaps we can start to see unofficial apps enjoying the same success on the PS3 that they do on the iPhone.

Download from Github

Work on Wi-Fi Sync 2.0 is progressing well. The new release will fix the outstanding issues reported by a small number of users, in addition to some new features such as sync-over-3G support. You’ll also be able to disable automatic backups for wirelessly connected devices only, rather than disabling backups for all devices (even those connected via USB) as with the current version. 2.0 will also be far friendlier to those who like to manage their own startup items; you’ll be able to start and exit Wi-Fi Sync whenever you like, and you can choose whether to run it at startup. I don’t even plan to ship an installer - instead you’ll be able to copy the app into Applications, like many other Mac apps. It will require iTunes 10. I’ll have more news shortly.

I’ve had a lot of questions about iPad and Windows support. I think I’ve had a breakthrough with the iPad (largely thanks to comex, author of the Spirit jailbreak), so expect more news on that very soon.

Images of XP SP3 and Vista are available. They’re currently set to expire on 01/01/2010.

Class ‘ZipArchive’ not found in /home/…

MAMP doesn’t come with PHP’s zip extension, so you either have to compile it yourself or drop a precompiled copy into your extensions directory. Compiling is a pain and precompiled extensions are hard to come by, so to save time, download it here and copy to /Applications/MAMP/bin/php5/lib/php/extensions/no-debug-non-zts-20050922. (Update: Link no longer available.)

Don’t forget to enable it within php.ini. You’ll also need to make the downloaded file executable: chmod +x zip.so

I’ve modified the installer so it skips the CPU check, and will thus install on PPC-based Macs and OSx86 machines. You can download it here: Silverlight.3.0.pkg. (Update: Link no longer available.)