Search terms:

This post is my record of the process, and I’ll keep it updated as I change my mind or add new things. If you have any suggestions I’d love to hear them; I’m new to Django and haven’t used most of these tools — virtualenv, mkvirtualenv, pip, git, github — a huge amount before. Some of this is very basic, but I wanted to document everything I need to do.

So far this doesn’t include getting a live server working. NOTE: This was written in 2010 and should be accurate for Django 1.2 and 1.3 but I think some of the project structure, and maybe settings, have changed with Django 1.4.

For further reading, here are some articles I found useful while working on this:

• Django Project Conventions, Revisited by Zachary Voase
• Working with virtualenv by Arthur Koziel
• Notes on using pip and virtualenv with Django at SaltyCrane
• Using virtualenv, pip and django-site-gen to quickly start new Django projects by Charles Leifer

---

File structure

I want to keep a similar structure for all my projects. For example, this is the location and structure I have for django-hines:

phil/
    .virtualenvs/
        django-hines/
    Projects/
        personal/
            django-hines/
                .gitignore
                hines/
                    __init__.py
                    aggregator/
                        __init__.py
                        admin.py
                        managers.py
                        migrations/
                        models.py
                        templates/
                            aggregator/
                                day.html
                                index.html
                            base_aggregator.html
                        templatetags/
                            __init__.py
                            aggregator_tags.py
                        test.py
                        urls.py
                        views.py
                    context_processors.py
                    manage.py
                    settings_default.py
                    settings.py
                    settings.py.template
                    static/
                        css/
                        img/
                        js/
                    templates/
                        404.html
                        500.html
                        base.html
                        comments/
                        flatpages/
                    weblog/
                logs/
                scripts/
                README.mdown
                requirements.txt

aggregator and weblog are two apps within the hines project (I haven’t expanded the weblog directory, but it’s the same as the aggregator one). Templates specific to an app are within their own projectname/appname/templates/ directory. Common project templates, or changes to third-party templates are in the projectname/templates/ directory. Everything within django-hines is checked in to git, except for hines/settings.py which are the settings local to my environment.

---

Setting up the environment

To install virtualenv (which includes ):

$ easy_install virtualenv

That might install pip, but at least once I’ve found it didn’t. In which case you need to install it yourself:

$ easy_install pip

Then either install virtualenvwrapper:

$ sudo pip install virtualenvwrapper

or maybe upgrade an existing version:

$ sudo pip install --upgrade virtualenvwrapper

(The sudo might or might not be required or allowed, depending on your server.)

Then, if it doesn’t already exist:

$ mkdir ~/.virtualenvs

And put this in ~/.bashrc:

export WORKON_HOME=$HOME/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh
export PIP_VIRTUALENV_BASE=$WORKON_HOME # Tell pip to create its virtualenvs in $WORKON_HOME.
export PIP_RESPECT_VIRTUALENV=true # Tell pip to automatically use the currently active virtualenv.

You may need to replace /usr/local/bin/ with a different path. For example, on one shared server I use /home/philgyford/bin/.

Those instructions will be loaded each subsequent time you log in. To make them take effect for this session, do this:

$ source ~/.bashrc

Then:

$ mkvirtualenv --no-site-packages --distribute django-projectname

That creates and starts you working on django-projectname. Install other things with pip:

(django-projectname)$ pip install yolk
(django-projectname)$ yolk -l
(django-projectname)$ pip install Django
(django-projectname)$ pip install MySQL-python
(django-projectname)$ pip install django-debug-toolbar
(django-projectname)$ pip install south

Instead of installing MYSQL-python you might want to install psycopg2 instead, if you’re using Postgres.

Note that we’ve installed South for managing database migrations there. More on that in a moment.

Add the directory where we’ll keep files:

(django-projectname)$ mkdir ~/Projects/subdir/django-projectname
(django-projectname)$ cd ~/Projects/subdir/django-projectname
(django-projectname)$ mkdir logs
(django-projectname)$ django-admin.py startproject projectname
(django-projectname)$ mkdir projectname/templates

Now we’ll have ~/Projects/subdir/django-projectname/projectname/ with a templates/ directory within it.

Set up a database:

$ mysql -u root -p
mysql> CREATE DATABASE projectname DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
mysql> GRANT ALL ON projectname.* TO username@localhost IDENTIFIED BY 'password';

Set the DATABASES settings in ~/Projects/subdir/django-projectname/projectname/settings.py. Then:

(django-projectname)$ ./manage.py runserver

And then you should be able to go to http://127.0.0.1:8000/. (You might need to do something like chmod u+x manage.py to get ./manage.py... to work.)

---

Settings

Rename settings.py to settings\_default.py.

At the beginning of settings\_default.py add this:

import os,sys
PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__))

Also, further down, set the templates directory (which we created earlier):

TEMPLATE_DIRS = (
    os.path.join(PROJECT_ROOT, 'templates'),
)

If you’re going to use South (which we installed with pip, earlier) then add it to the list of INSTALLED_APPS:

INSTALLED_APPS = (
    ...
    'south',
)

Then, create a local settings file that will override defaults:

(django-projectname)$ cd ~/Projects/subdir/django-projectname/projectname
(django-projectname)$ vi settings.py

And move any settings that are server-specific from settings\_default.py to settings.py. Suggested local settings:

• DEBUG
• TEMPLATE_DEBUG
• ADMINS
• MANAGERS
• DATABASES
• SECRET_KEY

At the start of settings.py, add this to import the default settings:

try:
    from settings_default import *
except ImportError:
    pass

If your project is going to be checked in to public version control, then the settings.py file should not be checked in, while the settings\_default.py can be. If your version control is private you could probably do a separate settings file for each server and check them all in.

Next, create a template for settings.py which will be checked in to version control, and used by anyone checking the project out:

(django-projectname)$ cp settings.py settings.py.template

And remove any specific settings in there to make an anonymous example.

(It seems more common to do this the other way round — have settings.py as the default settings, and create a settings\_local.py for the local overrides. But, it seems that then it’s hard to do things like INSTALLED_APPS += ('debug\_toolbar',), which we do in a moment. It generated errors, possibly something to do with scope…?)

---

Django-debug-toolbar

We already installed it with pip.

Add this in settings.py and settings.py.template:

INTERNAL_IPS = ('127.0.0.1',)
MIDDLEWARE_CLASSES += ('debug_toolbar.middleware.DebugToolbarMiddleware',)
INSTALLED_APPS += ('debug_toolbar',)

That IP address is correct if you’re working locally.

---

Git

Make the ignore file:

(django-projectname)$ cd ~/Projects/subdir/django-projectname
(django-projectname)$ vi .gitignore

And in .gitignore have something like this:

# File types #
##############
*.pyc
*.swo
*.swp
*.swn

# Directories #
###############
logs/

# Specific files #
##################
projectname/settings.py

# OS generated files #
######################
.DS_Store?
ehthumbs.db
Icon?
Thumbs.db
*~

Alternatively, having the File types and OS generated files in your own ~/.gitignore file that will apply to all projects is probably better.

---

Adding to Github

Make a README.mdown (or .txt or whatever) in ~/Projects/subdir/django-projectname.

Create a new project on Github. Then, assuming git is already set up on your machine:

(django-projectname)$ cd ~/Projects/subdir/django-projectname
(django-projectname)$ pip freeze > requirements.txt
(django-projectname)$ git init
(django-projectname)$ git add .
(django-projectname)$ git status
(django-projectname)$ git commit -m 'Initial commit'
(django-projectname)$ git remote add origin git@github.com:username/django-projectname.git
(django-projectname)$ git push origin master

---

Starting work

Probably uncomment django.contrib.admin in INSTALLED\_APPS, and the three admin-related lines in urls.py.

Set up database etc:

(django-projectname)$ cd ~/Projects/subdir/django-projectname/projectname
(django-projectname)$ ./manage.py syncdb

Create initial app:

(django-projectname)$ cd ~/Projects/subdir/django-projectname/projectname
(django-projectname)$ ./manage.py startapp appname

Once you’ve added the first model(s) to your app, you’ll need to either repeat this:

(django-projectname)$ ./manage.py syncdb

or, if you’re using South, then do this:

(django-projectname)$ ./manage.py schemamigration appname --initial
(django-projectname)$ ./manage.py migrate appname

The first line creates the migration file for the initial database schema (in appname/migrations/), and the second line applies it to the database. See “Ongoing work”, below, for what you need to do if you make subsequent changes to the app’s model(s).

---

Tests

Tests are done on a per-app basis, rather than per-project. After setting a site up with data you might want to test (eg, dummy weblogs and entries, flatpages, comments added through the front end, etc.)

(django-projectname)$ mkdir ~/Projects/subdir/django-projectname/projectname/appname/fixtures
(django-projectname)$ cd ~/Projects/subdir/django-projectname/projectname/appname/fixtures
(django-projectname)$ ./manage.py dumpdata appname --indent > test_data.json

This will dump the data for the specified app. You can dump the entire database by omitting the appname. This file (and those from other apps, if required) can then be loaded when you run tests. appname/tests.py can be something like:

from django.test.client import Client
from django.test import TestCase

class WeblogClientTests(TestCase):

    fixtures = [
            '../fixtures/test_data.json', 
            '../../otherappname/fixtures/test_data.json',
        ]

    def test_HomePage(self):
        '''Test if the homepage renders.'''
        c = Client()
        response = c.get('/')
        self.failUnlessEqual(response.status_code, 200)

Then run all tests on an app using:

(django-projectname)$ ./manage.py test appname

I’m not sure of the best way to update test\_data.json files as you develop the site, other than manually, or by replacing them with a fresh file.

---

Other tips

ContextProcessors

There are probably variables that you’ll want to appear on every page. Create a context\_processors.py file, probably within an app (although it works at project level too). Within it have function(s) like this:

def date_formats(request):
    from django.conf import settings
    return {
        'date_format_long': 'l j F Y',
    }

That would, as an example, make a date\_format\_long variable available in all of your templates.

Then in settings\_default.py add this:

import django.conf.global_settings as DEFAULT_SETTINGS
TEMPLATE_CONTEXT_PROCESSORS = DEFAULT_SETTINGS.TEMPLATE_CONTEXT_PROCESSORS + (
    'appname.context_processors.date_formats',
)

with your own appname and method name.

Then be sure to use the custom RequestContext in your views. eg:

from django.template import RequestContext

def weblog_entry_detail(request, year, month, day, slug):
    ...
    return render_to_response('weblog/entry_detail.html', {
        'entry': entry,
    }, context_instance=RequestContext(request))

Error templates

Don’t forget to make error templates for 404 and 500 errors (as mentioned here). These should be in the top level of your templates, named 404.html and 500.html. Not having them isn’t a problem while working on the site with DEBUG=True set, but as soon as you switch to “live” you’ll get reports of missing templates whenever one of those errors occurs.

---

Ongoing work

Virtualenv

When returning in the future, start working on a virtualenv by:

$ workon django-projectname

And stop working on it with:

(django-projectname)$ deactivate

To remove a virtual environment:

$ rmvirtualenv django-projectname

South

If you’re using South, then when you make changes to your app’s model(s) you’ll need to do this to update the database:

(django-projectname)$ ./manage.py schemamigration appname --auto
(django-projectname)$ ./manage.py migrate appname

If your model change isn’t simple you may get a prompt for further action. See Advanced Changes. If change something that means existing data needs to change, see Data Migrations.

Git

Adding new files:

(django-projectname)$ git add file_or_folder_name

Committing more work:

(django-projectname)$ git commit -a

And this again whenever you want to push to Github:

(django-projectname)$ git push origin master

Pip

To upgrade something already installed with pip, and record the change:

(django-projectname)$ cd ~/Projects/subdir/django-projectname
(django-projectname)$ pip install --upgrade django-debug-toolbar
(django-projectname)$ pip freeze > requirements.txt

Updates:

• 16 Nov 2010: Added instructions for South.
• 27 Jun 2011: Tweaked pip/virtualenv installation.
• 2 May 2012: A few little tweaks.

Commenting is disabled on posts once they’re 30 days old.