Today’s post is a rant followed by a technical solution that hopefully some will find useful. You can skip past the rant by clicking here

The Rant: Cloud Storage and Sync

I tend to use a lot of open source software. It’s free, and it’s usually as well or better built than commercial software. It does sometimes require a bit of technical knowledge and troubleshooting though.

I also own an iPhone. In some ways, I really love it. I can check on all my websites, servers, social streams, emails, and calls from one compact gadget. From anywhere. However, the iPhone is unfortunately a very closed sort of platform. Apple controls all of the software available for the device, and chooses not to provide some seemingly basic features – like accessing files stored on the phone from your computer.

Now almost anybody you ask will say that you should just sync all your data to the cloud. Then you can access it from your computer easily (or from anywhere else). But I have a problem with that.

When you sync your data to the cloud, you’re sending it to a server that’s controlled by a third party. Any of the following things can then happen:

1. Some nosy admin on the other end can inadvertently or purposely start looking through your data. He could get fired, but only if his boss finds out.
2. The third party could have its servers compromised, by hackers for instance. Oops! now bl4sterz.ru knows your every personal detail.
3. By storing your data, the third party in a sense “owns” it. If they decide to start embedding ads in your data, or charging a monthly fee to use their service, what can you do about it?

For these sorts of reasons, I never send information to third parties on the web unless I’m basically comfortable with that information being publicly available to all. Yes, that means you, Facebook. And Twitter. Et cetera.

The Problem: iPhone/Lightning Calendar Sync, without the Cloud

OK, enough rant about cloud storage services. Here’s why this all came up:

As I said before, I have an iPhone. I also use Mozilla’s Thunderbird email browser with a calendar plugin called Lightning to manage my appointments and such. I would like to have the calendar on the iPhone sync with Lightning. (Mozilla Sunbird is basically the same as Lightning for Thunderbird – it’s made by the same folks and has the same functionality. Sunbird just lacks the email client.)

If you Google how to do this, everyone says that you should sync both Lightning and the iPhone with Google calendar. But that’s a cloud solution: I don’t want Google to know all my appointments!

The Solution: Set Up Your Own CalDAV server

Faced with this problem, I’ve worked out a nice solution: install your own CalDAV server. You can do this on either your own local network (more secure), or any webserver you have access to remotely (easier to share calendars with many people).

CalDAV is an implementation of a web standard called WebDAV which makes it easy to store files, calendars, addressbooks, and other information in a format that is easily readable by many devices and platforms. CalDAV specifically can identify users and store events – including recurring events and alerts. You can set up shared calendars, configure read-write access for different users to different calendars. In short, CalDAV can provide enterprise-level calendaring functionality much like Microsoft’s Exchange server.

There are a number of CalDAV servers available, both free and commercial. After assessing several, I went with DAViCal, an open source CalDAV server implementation built on top of Apache, Perl, PHP5, and PostgreSQL server. If you need to install this suite, you can grab a prepackaged distribution like the Bitnami stack, or install each package from the Linux shell if you’re comfortable doing so.

The details of how to install DAViCal depend on the operation system distribution you’re running on your server box. See the Distribution Specifics section of the DAViCal Installation Wiki. When I installed DAViCal, I ran into the following problems. I’m posting them here, as well as some tips, in hopes that they may save you some time:

• If you’re installing in a shared server environment, you’ll need to change the hard-coded Postgre database name and username in davical/dba/create-database.sh. You may also need to manually specify the location of the AWL library in this script if it’s not in /usr/share/awl.
• When running davical/dba/create-database.sh, pay attention to the output. Near the end, it will say something like “created user user with password password” – save these, as you will need them to log into DAViCal’s web administration interface later.
• I ran into a problem where some of DAViCal’s installation Perl scripts wouldn’t run because I was missing some required Perl modules. When I tried installing them (yum install perl-DBD-Pg), I got “package not found”… until I checked /etc/yum.conf which had exclude= … perl* …. Removing the perl* fixed this issue
• The Installation Wiki gives two methods to install DAViCal: as a directory on your server (example.com/davical) or as a VHost subdomain (davical.example.com). If you want this to work with iPhone, go the subdomain route – when I tried to tack on a directory in the iPhone CalDAV settings, it kept removing it.
   

  If you’re using an automated system like cPanel to manage your server VHosts, and hence don’t want to edit httpd.conf manually, just follow the directions for installing DAViCal in a directory (creating the symlink to davical/htdocs), and then create the subdomain in cPanel, but giving the symlink for the directory.

Once installed, visit davical.example.com in your web browser. You should see a login screen soliciting a username and password, or you may see a diagnostic screen with some details if your installation isn’t complete.

Log into DAViCal with the username and password the dba/create-database.sh script generated. In the menu up top, choose User Functions -> View My Details. At the bottom of this page, under Principal Collections, click Create Collection. This will basically create a new calendar you can subscribe to with a CalDAV client. Fill in the DAV Path and Display Name. Click Create.

Now, to subscribe to your new CalDAV, you’ll use the URL http://davical.example.com/caldav.php/username/DAVPath, where username is the username you used to log into the administrative interface for DAViCal, and DAVPath is what you set when creating the collection (previous paragraph).

On the iPhone, you can subscribe to this calendar by chosing Settings -> Mail, Contacts, Calendars -> Accounts -> Add Account -> Other -> Calendars -> Add CalDAV Account. Server is your davical.example.com, User Name and Password are from your administrative account, and Description is any descriptive name you want appearing in your iPhone Calendar list. (Maybe the Display Name you chose in DAViCal admin?). Click Next: the iPhone will try to autodetect other settings. If it works, great! your calendar has been added. If not, click the Advanced Settings button that appears, then copy http://davical.example.com/caldav.php/username/DAVPath (see previous paragraph) into the Account URL.

In Lightning or Sunbird, right-click in the calendar list and choose New Calendar. Then, On the Network and Next. Choose CalDAV for the format, and enter http://davical.example.com/caldav.php/username/DAVPath (see two paragraphs ago) into Location. Hit Next. Enter your username and password. Now you should be subscribed to your CalDAV calendar!

Summary

The iPhone can sync many things very well indeed – music, email, even web browser bookmarks. But for open source calendar software, the only easy way is to use Google’s cloud service. If you’re unwilling to do that, however, rejoice: there is another way!

Running a CalDAV server like DAViCal is a great way to sync calendars across computers and mobile devices, and even to share calendars with friends. And if you’re still worried about the security of your data on a public server (even one that you own or control), you can run a CalDAV server on a local network. It just won’t be accessible for remote synchronization.

I hope this guide was useful to you. Feel free to leave your comments below.

Twitter has a URL http://www.twitter.com/statuses/update.xml where you can POST a twitter update. The following HTML form will update your twitter status after asking you for your username and password:

<form action="http://twitter.com/statuses/update.xml" method="POST">
	<input type="text" name="status" />
	<input type="submit" value=""Tweet!" />
</form>

This is cool, but it would be nice to circumvent the password protection in a web application where you probably already have a user logged in and validated? Fortunately this is possible using HTTP headers: (please make sure you read my SECURITY NOTE below)

<form action='twitter.php' method='post'>
	<input type='text' name='status' />
	<input type='submit' value='tweet' />
</form>
<?php
	if($_POST['status']!="") {

		$data = "status=".stripslashes($_POST['status']);
		$fp = fsockopen("www.twitter.com", 80);
		$user = "twitter_username";
		$pass = "twitter_password";

		fputs($fp, "POST /statuses/update.xml HTTP/1.1\r\n");
		fputs($fp, "Host: www.twitter.com\r\n");
		fputs($fp, "Referer: None\r\n");
		fputs($fp, "Authorization: Basic ".
			base64_encode($user.":".$pass)."\r\n");
		fputs($fp, "Content-type: ".
			"application/x-www-form-urlencoded\r\n");
		fputs($fp, "Content-length: ". strlen($data) ."\r\n");
		fputs($fp, "Connection: close\r\n\r\n");
		fputs($fp, $data);

		while(!feof($fp))$str .= fgets($fp, 128);
		if(!strstr(" OK ",$str))
			echo "There was a problem posting your twitter update.";
		else echo "twitter update posted successfully!";
	}
?>

SECURITY NOTE:

Please notice that the code above sends the username and password hash unencrypted over a plaintext HTTP pipe. The password hash is thus vulnerable to rainbow table attacks.

A better solution uses HTTPS with cURL. This way, the username & password hash info don’t have to be sent plaintext:

<form action='twitter.php' method='post'>
	<input type='text' name='status' /> <input type='submit' value='tweet' />
</form>
<?php
	if($_POST['status']!="") {
		$url = "https://twitter.com/statuses/update.xml";
		$user = "twitter_username";
		$pass = "twitter_password";

		$ch = curl_init();
		curl_setopt($ch, CURLOPT_URL, $url);
		curl_setopt($ch, CURLOPT_POST, true);
		curl_setopt($ch, CURLOPT_HEADER, true);
		curl_setopt($ch, CURLOPT_USERPWD, $user.":".$pass);
		curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
		curl_setopt($ch, CURLOPT_POSTFIELDS,
			"status=".stripslashes($_POST['status']));

		ob_start();
		curl_exec($ch);
		$str = ob_get_contents();
		ob_end_clean();
		curl_close($ch);

		if($str=="" || strstr($str,"<error>"))
			echo "There was a problem posting your twitter update.";
		else echo "twitter update posted successfully!";
	}
?>

Since I’ve started using twitter I’ve found that it’s very useful for companies and organizations that want to get messages out to lots of people quickly. You can even get twitter updates by phone using SMS!

Happy tweeting!