Thursday, January 7, 2010
Monday, July 13, 2009
Google-App-Engine-Django Tutorial, Part 2 - Set up
In this post, we'll setup the environment needed to create and run a Google App Engine (GAE) application using Django and google-app-engine-django helper.
The procedures here are mostly covered in the exellent getting started article for GAE-D, which will likely be kept more up to date than this one. I will try to provide the exact setup process as I ran it, if anyone might find it useful.
I'm going to use Ubuntu Linux for this tutorial, because, well, that's what I use. If there any difference refer to documentation of the different softare, which i will try to provide.
I won't go through the details of this. All the following procedures, except uploading the application to GAE can be achieved without it, but in the end of the day, you'd want to use some of google's cpu cycles, no?
Basically you have to sign up to GAE, Activate the account with an SMS activation code, and then create a new application. All this is done using very simple wizards. I have to note that when I named the application, almost any combination of number words was taken. Strange, but I don't think it matters that much. An Application name like dobo_hellodjango would be good enough.
You will want (and need) to run python 2.5, as this is the runtime that GAE uses. While I managed to experiment locally with python 2.6, it throw outs lots of errors, and it's not compatible with what is going to run eventually.
Install python any way you like, i did:
Next, let's create a folder for our various packages and expiriments. I'm going to use /home/amitay/dev/appengine.
Keep in mind that the following terminal commands assume that you're in that directory.
now, download the GAE python SDK (the zip version if you're using linux) to that folder.
Back to the terminal, unzip it:
Next thing, let's get google-app-engine-django (GAE-D) directly from the repository using SVN. Yes, we're doing bleeding edge development here. Wicked.
If you need SVN on ubuntu, you can simply run:
When SVN is installed, check out GAE-D trunk:
And now, we'll export it to a new folder which will be our little secret project home.
"Export" means we're creating an unversioned copy of GAE-D project, to be used as a basis for our project.
Note: I'm not going to deal, and i'm not sure myself, with how to update the GAE-D in your project with newer versions of it, I'm just following the GAE-D instructions. It seems to me a bit problematic that the trunk itself, and not something it generates is the template for your project.
Anyway, we were at exporting GAE-D, to a new folder for our project, which we'll call "hellodjango":
We'll need to create a link to in our project, on ".google_appengine" to the SDK we've previously downloaded and extracted:
(your absolute path may vary). Didn't test it, but if the GAE SDK is installed in windows using the installer, this step can be skipped.
Now, we need to copy django 1.0 to a subfolder named "django" of our project. While the GAE-D readme states it works with "Django 1.0beta_1 or greater", 1.1 versions from the django SVN trunk are not supported. So we'll download the 1.0 stable version instead.
The following lines download django, unzips, and copy the django project directory to "hellodjango\django" directory". You can do it manually too. If you're a sucker.
The project base should be ready now, So we'll simply move its folder, and start up the development server to test it:
manage.py is the management utility for Django. We're running its "runserver" command, to start the development server on port 8009 (you can omit the port number, which will use the default 8080 port, which is simply used in my computer). notice i use "python2.5" to interpret manage.py, this is simply because i have python 2.6 installed as well, and i want to use the supported version.
Ignore the WARNING messages you get regarding the datastore, they'll disappear once we create some data.
If everything worked (crossing my fingers for you), you should now be able to browse to http://localhost:8009, and see the development server running.
Next Post I'll go through writing some code in our application, and uploading it to Google App Engine, So it will forever live.
The procedures here are mostly covered in the exellent getting started article for GAE-D, which will likely be kept more up to date than this one. I will try to provide the exact setup process as I ran it, if anyone might find it useful.
I'm going to use Ubuntu Linux for this tutorial, because, well, that's what I use. If there any difference refer to documentation of the different softare, which i will try to provide.
Set up an App Engine Application
I won't go through the details of this. All the following procedures, except uploading the application to GAE can be achieved without it, but in the end of the day, you'd want to use some of google's cpu cycles, no?
Basically you have to sign up to GAE, Activate the account with an SMS activation code, and then create a new application. All this is done using very simple wizards. I have to note that when I named the application, almost any combination of number words was taken. Strange, but I don't think it matters that much. An Application name like dobo_hellodjango would be good enough.
Setting up the django + GAE project
You will want (and need) to run python 2.5, as this is the runtime that GAE uses. While I managed to experiment locally with python 2.6, it throw outs lots of errors, and it's not compatible with what is going to run eventually.
Install python any way you like, i did:
$ sudo apt-get install python2.5
Next, let's create a folder for our various packages and expiriments. I'm going to use /home/amitay/dev/appengine.
cd /home/amitay/dev
mkdir appengine
cd appengine
Keep in mind that the following terminal commands assume that you're in that directory.
now, download the GAE python SDK (the zip version if you're using linux) to that folder.
Back to the terminal, unzip it:
$ unzip google_appengine_1.2.3.zip
Next thing, let's get google-app-engine-django (GAE-D) directly from the repository using SVN. Yes, we're doing bleeding edge development here. Wicked.
If you need SVN on ubuntu, you can simply run:
$ sudo apt-get install subversion
When SVN is installed, check out GAE-D trunk:
$ svn checkout http://google-app-engine-django.googlecode.com/svn/trunk/ google-app-engine-django
And now, we'll export it to a new folder which will be our little secret project home.
"Export" means we're creating an unversioned copy of GAE-D project, to be used as a basis for our project.
Note: I'm not going to deal, and i'm not sure myself, with how to update the GAE-D in your project with newer versions of it, I'm just following the GAE-D instructions. It seems to me a bit problematic that the trunk itself, and not something it generates is the template for your project.
Anyway, we were at exporting GAE-D, to a new folder for our project, which we'll call "hellodjango":
$ svn export google-app-engine-django hellodjango
We'll need to create a link to in our project, on ".google_appengine" to the SDK we've previously downloaded and extracted:
$ ln -s /home/amitay/dev/appengine/google_appengine hellodjango/.google_appengine
(your absolute path may vary). Didn't test it, but if the GAE SDK is installed in windows using the installer, this step can be skipped.
Now, we need to copy django 1.0 to a subfolder named "django" of our project. While the GAE-D readme states it works with "Django 1.0beta_1 or greater", 1.1 versions from the django SVN trunk are not supported. So we'll download the 1.0 stable version instead.
The following lines download django, unzips, and copy the django project directory to "hellodjango\django" directory". You can do it manually too. If you're a sucker.
$ wget http://www.djangoproject.com/download/1.0.2/tarball/
$ tar xzvf Django-1.0.2-final.tar.gz
$ cp -r Django-1.0.2-final/django/hellodjango/django
The project base should be ready now, So we'll simply move its folder, and start up the development server to test it:
$ cd hellodjango/
$ python2.5 manage.py runserver 8009
manage.py is the management utility for Django. We're running its "runserver" command, to start the development server on port 8009 (you can omit the port number, which will use the default 8080 port, which is simply used in my computer). notice i use "python2.5" to interpret manage.py, this is simply because i have python 2.6 installed as well, and i want to use the supported version.
Ignore the WARNING messages you get regarding the datastore, they'll disappear once we create some data.
If everything worked (crossing my fingers for you), you should now be able to browse to http://localhost:8009, and see the development server running.
Next Post I'll go through writing some code in our application, and uploading it to Google App Engine, So it will forever live.
Google-App-Engine-Django Tutorial, Part 1 - Background
So, you decided to join the latest buzz train going to the cloud, and check out Google App Engine (GAE from now on). In this post I'll try to demonstrate how to develop an GAE application using Python, the Django framework (or a subset of), Google App Engine SDK, And Google App engine Django helper.
I did the development for the write-up on an Ubuntu Linux machine so some of the instructions are specific to that environment. Most of them are easily translatable to other platforms.
Google App engine allows you to run web applications written in selected languages on Google's servers cloud. You are also provided with some API for using other google infrastructure: A Data Store (which is NOT a relational database, but on most applications will play the database role), Mailer, MemCache, etc.
It's important to understand that GAE is not like setting up you own physical or virtual server, be it your own servers, or a virtualization service like Amazone EC2. With programming to use GAE off the shelf scalability, you're being limited to:
I'll try not to get into the scope of the GAE, there is plenty of information about it on the web.
Python is a dynamic programming language. It's elegant, and it's supported by GAE.
The Python GAE SDK is a set of python libraries and programs that lets you develop your GAE application locally, using a GAE compatible local web server, and manage your application (i.e. uploading it to GAE)
Django is a web framework written in python (and used by writing python code). It's considered an MVC framework, though they changed the common terminology for the Controller and View.
Typically Django runs on a web server (i.e. Apache), and uses a single relational database server (PostgresSQL, MySql).
Many parts of django can't run on GAE, since they rely on Django model, which relies on a relational DB backend, which isn't provided by GAE. Most applications written for django (for example django.contrib.auth, django-registration) won't and can't work on GAE without some very serious modifications to the model.
Still, many parts of it can be used: Its template system, Url Front controller, forms processing, etc. And indeed, GAE currently included and supports out of the box some django components or ports of them: namely the templates and forms.
However, it can be quite complicated to try and run an application based on Django in GAE, even if avoiding the unsupported componenets. And there comes...
google-app-engine-django (let's call it GAE-D, ok?) provides a helper which comes to help with the complication listed below. It provides the following thing:
1) A template for quickly starting a django based GAE application.
2) Classes and Wrappers for some core Django componenets that allows using them on GAE.
3) Modified manage.py, the Django Management utility, adopting it to GAE an extending g it to handle GAE operations which are not part of Django.
Those are the different parts we're going to fit together in order to create a first GAE application. I didn't plan on giving this introduction about GAE, Django, etc, but it happened so. The original plan was just detailing the
On the next post I'll try provide the actual HOW-TO and steps for creating a very basic application. Feeling hungry?
I did the development for the write-up on an Ubuntu Linux machine so some of the instructions are specific to that environment. Most of them are easily translatable to other platforms.
The Players
Googe App Engine
Google App engine allows you to run web applications written in selected languages on Google's servers cloud. You are also provided with some API for using other google infrastructure: A Data Store (which is NOT a relational database, but on most applications will play the database role), Mailer, MemCache, etc.
It's important to understand that GAE is not like setting up you own physical or virtual server, be it your own servers, or a virtualization service like Amazone EC2. With programming to use GAE off the shelf scalability, you're being limited to:
- Using the targeted runtimes: For now it is Python 2.5 or Java bytecode (in Beta)
- Not installing any software, just upload code (yours and libraries). If your code or libraries have dependencies for things that are not supported by GAE, tough luck. As a result, you have to use GAE stack provided by their API to replace usual components.
- Not usingg file system writes, opening sockets, spawning and multi-threading, others...
I'll try not to get into the scope of the GAE, there is plenty of information about it on the web.
Python
Python is a dynamic programming language. It's elegant, and it's supported by GAE.
Google App Engine SDK for python
The Python GAE SDK is a set of python libraries and programs that lets you develop your GAE application locally, using a GAE compatible local web server, and manage your application (i.e. uploading it to GAE)
Django
Django is a web framework written in python (and used by writing python code). It's considered an MVC framework, though they changed the common terminology for the Controller and View.
Typically Django runs on a web server (i.e. Apache), and uses a single relational database server (PostgresSQL, MySql).
Many parts of django can't run on GAE, since they rely on Django model, which relies on a relational DB backend, which isn't provided by GAE. Most applications written for django (for example django.contrib.auth, django-registration) won't and can't work on GAE without some very serious modifications to the model.
Still, many parts of it can be used: Its template system, Url Front controller, forms processing, etc. And indeed, GAE currently included and supports out of the box some django components or ports of them: namely the templates and forms.
However, it can be quite complicated to try and run an application based on Django in GAE, even if avoiding the unsupported componenets. And there comes...
google-app-engine-django
google-app-engine-django (let's call it GAE-D, ok?) provides a helper which comes to help with the complication listed below. It provides the following thing:
1) A template for quickly starting a django based GAE application.
2) Classes and Wrappers for some core Django componenets that allows using them on GAE.
3) Modified manage.py, the Django Management utility, adopting it to GAE an extending g it to handle GAE operations which are not part of Django.
Those are the different parts we're going to fit together in order to create a first GAE application. I didn't plan on giving this introduction about GAE, Django, etc, but it happened so. The original plan was just detailing the
On the next post I'll try provide the actual HOW-TO and steps for creating a very basic application. Feeling hungry?
Monday, June 1, 2009
99 Design patterns on the wall
Found this on a web site that contain programs for singing (couting, printlning, Console.Outing...) "99 bottles of beer on the wall" in many (manier than I knew existed) programming languages.
It demonstrates the true power of enterprise-ish programming: Take a simple concept and make it robust by harnessing the power of design patterns, modular degin, defensive programming, etc...
(Only found 1 design pattern I recognize - Observer - but perhaps a savvier programmer would spot a few extra.
It demonstrates the true power of enterprise-ish programming: Take a simple concept and make it robust by harnessing the power of design patterns, modular degin, defensive programming, etc...
(Only found 1 design pattern I recognize - Observer - but perhaps a savvier programmer would spot a few extra.
Slax - linux on a key
A few days ago I had to take out my main hard disk to be replaced. I was terrified with the thought of being left out without an operable computer. I wanted to find something that will leave me with some web access and help me prepare to computer set up the operating systems of choice when i get back a replacement hard drive.
After some googling, and through the live cd list I got myself introduced to Slax - a linux you can install as a live cd, or on a USB stick, and to do almost anything with.
I'll begin this short review with the conclusion - It's good! I recommend anyone making himself a ready to use copy of Slax to be used in the time of need.
I'll go through with how I installed and ran Slax. Of course it might be different in your environment. I wanted to install Slax on a USB stick, as I always have bad expirience with fucked up CD medias, and I hate moving parts in my computer. I leave the to my bicycle.
Installation
My installation process began from Windows XP. I downloaded slax as a tar file from http://www.slax.org/get_slax.php. The tar file (which is something like a zip archive file) is completely identical in its contents to the ISO image, which is simply easier to to be write to a CD.
I connected a small 500Mb usb stick which Justin left here a year ago, muahaha slax basic installation is about 250mb., and formatted it. Then i simply untarred the tar file using 7zip (which is btw ugly as hell but works like the devil) to the root of the fresh usb.
The Usb drive was then formatted by running /slax/boot/bootinst.bat on the disk. Why not every distribution provides it? So much better than CDs.
Slax supports what it calls Modules, which are small packages of software, which can be added to the Slax version you make and be part of it when it loads, or be downloaded and added when slax runs. I chose to add "Firefox 3.0.10 + Flash 10 + JRE 6u13", and "Transmission" - a bit torrent client. To add packages to be loaded as part of Slax you simply copy them to /slax/modules on your disk. Beautifuly simple, and that's it.
Booting
I never tried to but from a Usb drive in my computer, but i knew the BIOS was suppose to support it (Award bios of such and such version). It has a nice feature to open a "Boot from..." menu when it loads by hitting F12. I did it, and then tried all the options - "USB-FDD", "USB-HD", "USB-CD". Nada, Zilch. I entered the Bios setup and started poking at things. Then I stumble upon something like "Generic Usb flash drive" as part of the hard disks list! WTF? I also disabled USB keyboard as some post said it might help.
Well, booted again and chose "Hard disk" on the boot menu, and viola! there it is! listed with my other non-too-solid-state disks!
Later on i discovered that i can boot to it with USB keyboard enabled, but only sometimes. First time i can't boot to usb as described, and then on the boot menu i ctrl-alt-del, and on the second time it's there. Strangeness, but one that doesn't bother me too much. It will if i ever wanted to permenently booth to a usb disk.
Anyway, then Slax starts to boot, printing out all kind of nice kernels and daemons message, and opens KDE, its window manager. And i think that calls for another header:
Running
Well, Slax started, automatically detected my hardware: Network, sound card, dvd drive. Sweet. It also recognized all my hard disks (i did it before disconnected on of them), and automatically mounted them so i can access them (you can see them under "Storage Media" in Konqueror - the File manager/ default web browser).
Later on i managed to browse the interweb, burn a cd*, rsize the hard disks partitions with the installed GParted.
It feels really fast and stable compared to windows and Ubuntu (which i later installed, and so far i have a less positive experience with), and the modules are simple to use and install. Makes me think of installing it for a more permenent use, or trying out Slackware which it is based on.
So, get some slax today.
*CD burning using k3b - It gives an error message when trying to burn a cd image on a DVD media. For it to work (for example, buring Ubuntu iso image on a dvd instead of on a cd), choose from the menu "Tools/Burn DVD Iso Image..." and then it will treat the image right. Thank god i had a web browser working to find that out.
After some googling, and through the live cd list I got myself introduced to Slax - a linux you can install as a live cd, or on a USB stick, and to do almost anything with.
I'll begin this short review with the conclusion - It's good! I recommend anyone making himself a ready to use copy of Slax to be used in the time of need.
I'll go through with how I installed and ran Slax. Of course it might be different in your environment. I wanted to install Slax on a USB stick, as I always have bad expirience with fucked up CD medias, and I hate moving parts in my computer. I leave the to my bicycle.
Installation
My installation process began from Windows XP. I downloaded slax as a tar file from http://www.slax.org/get_slax.php. The tar file (which is something like a zip archive file) is completely identical in its contents to the ISO image, which is simply easier to to be write to a CD.
I connected a small 500Mb usb stick which Justin left here a year ago, muahaha slax basic installation is about 250mb., and formatted it. Then i simply untarred the tar file using 7zip (which is btw ugly as hell but works like the devil) to the root of the fresh usb.
The Usb drive was then formatted by running /slax/boot/bootinst.bat on the disk. Why not every distribution provides it? So much better than CDs.
Slax supports what it calls Modules, which are small packages of software, which can be added to the Slax version you make and be part of it when it loads, or be downloaded and added when slax runs. I chose to add "Firefox 3.0.10 + Flash 10 + JRE 6u13", and "Transmission" - a bit torrent client. To add packages to be loaded as part of Slax you simply copy them to /slax/modules on your disk. Beautifuly simple, and that's it.
Booting
I never tried to but from a Usb drive in my computer, but i knew the BIOS was suppose to support it (Award bios of such and such version). It has a nice feature to open a "Boot from..." menu when it loads by hitting F12. I did it, and then tried all the options - "USB-FDD", "USB-HD", "USB-CD". Nada, Zilch. I entered the Bios setup and started poking at things. Then I stumble upon something like "Generic Usb flash drive" as part of the hard disks list! WTF? I also disabled USB keyboard as some post said it might help.
Well, booted again and chose "Hard disk" on the boot menu, and viola! there it is! listed with my other non-too-solid-state disks!
Later on i discovered that i can boot to it with USB keyboard enabled, but only sometimes. First time i can't boot to usb as described, and then on the boot menu i ctrl-alt-del, and on the second time it's there. Strangeness, but one that doesn't bother me too much. It will if i ever wanted to permenently booth to a usb disk.
Anyway, then Slax starts to boot, printing out all kind of nice kernels and daemons message, and opens KDE, its window manager. And i think that calls for another header:
Running
Well, Slax started, automatically detected my hardware: Network, sound card, dvd drive. Sweet. It also recognized all my hard disks (i did it before disconnected on of them), and automatically mounted them so i can access them (you can see them under "Storage Media" in Konqueror - the File manager/ default web browser).
Later on i managed to browse the interweb, burn a cd*, rsize the hard disks partitions with the installed GParted.
It feels really fast and stable compared to windows and Ubuntu (which i later installed, and so far i have a less positive experience with), and the modules are simple to use and install. Makes me think of installing it for a more permenent use, or trying out Slackware which it is based on.
So, get some slax today.
*CD burning using k3b - It gives an error message when trying to burn a cd image on a DVD media. For it to work (for example, buring Ubuntu iso image on a dvd instead of on a cd), choose from the menu "Tools/Burn DVD Iso Image..." and then it will treat the image right. Thank god i had a web browser working to find that out.
Stupid Idea of the Day (SIOTD): Backup flying balloons
There's no need to stress the importance of off-site backup. Perhaps one of the blockers for the wide implementation of that is the speed. But I think, that backup simply isn't cool enough.
I propose: Backup by wireless equipped flying balloons!
The balloons will have routes going above houses. Computer software in those houses will determine the presence of the Wireless Flying back up device ("Look mommy, a balloon!"), connect to it, and send the data waiting to be backed up.
Similar to an old-school milkman bringing bottles door to door. Only it's taking, and not bringing. And data, not milk. And roof to roof.
My current Internet upload speed is 128kbps. On 802.11g wireless connection the maximum speed is 54mbps - about 420 times faster. That means that the balloon will have to stay over my roof only 1/420 of the entire day to achieve the backup quota per day i currently have over the internet,. This is about 3 and half minutes! But of course, we can program the balloon stay as long as needed, or the get along to other friends if you're wasting to much if it time.
Ok, i think i wasted enough of your time.
One last thing: Peer to Balloon to Peer file sharing!
I propose: Backup by wireless equipped flying balloons!
The balloons will have routes going above houses. Computer software in those houses will determine the presence of the Wireless Flying back up device ("Look mommy, a balloon!"), connect to it, and send the data waiting to be backed up.
Similar to an old-school milkman bringing bottles door to door. Only it's taking, and not bringing. And data, not milk. And roof to roof.
My current Internet upload speed is 128kbps. On 802.11g wireless connection the maximum speed is 54mbps - about 420 times faster. That means that the balloon will have to stay over my roof only 1/420 of the entire day to achieve the backup quota per day i currently have over the internet,. This is about 3 and half minutes! But of course, we can program the balloon stay as long as needed, or the get along to other friends if you're wasting to much if it time.
Ok, i think i wasted enough of your time.
One last thing: Peer to Balloon to Peer file sharing!
Subscribe to:
Posts (Atom)