A Fairly Low Ceremony Method for Modularizing Django Settings
Updates
- No longer wrapping modules in
def require - Using
pkgutilandinspectto get module names and constants
Every time I need to add anything to settings.py in Django, I find myself spending a good minute trying to find a logical place to put it. But it’s always the same. I give up, put it somewhere arbitrary, and move on feeling cheapened, dirty, and defeated. There must be a better way…
This wiki page provides a ton of examples that get very close. They let you override settings per environment, but you’re still arbitrarily placing unrelated settings together. I’d like something more modular. Today I worked out a method that is very modular and includes all the perks of the above examples at the cost of a small bit of ceremony.
For reference, take a look at the updated example project settings. The gist of it is moving settings into it’s own package, importing all settings in __init__.py, and placing specific settings into aptly named modules.
Version 2.0
import inspect
import pkgutil
for _, module_name, _ in pkgutil.walk_packages(__path__):
module = __import__(module_name, globals(), locals(), [])
for var_name, val in inspect.getmembers(module):
if var_name.isupper():
locals().update({var_name: val})
This version improves on the first by getting module names using pkgutil.walk_packages which means you can put settings in packages now too.
Also, instead of setting up the secrets, paths, and environment in here and passing them via require I let the settings modules import the settings they need.
import os
import env
import paths
import secrets
if env.DEV_ENV:
database_dir = os.path.join(paths.BASE_PATH, 'databases')
if not os.path.exists(database_dir):
os.mkdir(database_dir)
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(database_dir, 'sqlite3'),
}
}
else:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'USER': secrets.DATABASE_USER,
'PASSWORD': secrets.DATABASE_PASSWORD,
'HOST': secrets.DATABASE_HOST,
'PORT': secrets.DATABASE_PORT,
}
}
Instead of import from I just import the modules directly and grab the settings from them. This makes things a bit more explicit and ensures the imported settings don’t get added twice in __init__.py.
Extending settings is exactly the same as version 1.0:
import env
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
# Uncomment the next line to enable the admin:
# 'django.contrib.admin',
# Uncomment the next line to enable admin documentation:
# 'django.contrib.admindocs',
)
if env.DEV_ENV:
INSTALLED_APPS += (
'debug_toolbar',
)
So in this version we’ve cleaned up __init__.py quite a bit and removed the ceremony from the settings modules. This is exactly what I wanted from the get go and I’m pretty stinkin happy with it.
Version 1.0
import os
from secrets import *
SETTINGS_PATH = os.path.abspath(os.path.dirname(__file__))
PROJECT_PATH = os.path.dirname(SETTINGS_PATH)
BASE_PATH = os.path.dirname(PROJECT_PATH)
ENV = os.getenv('DJANGO_ENVIRONMENT', 'development')
DEV_ENV = ENV == 'development'
TEST_ENV = ENV == 'staging'
PROD_ENV = ENV == 'production'
def _constants(d):
return dict((key, val) for key, val in d.items() if key.isupper())
for settings_file in [filename.rstrip('.py') for filename in os.listdir(SETTINGS_PATH)
if not filename.endswith('.pyc') and filename not in ('__init__.py', 'secrets.py')]:
module = __import__(settings_file, globals(), locals(), [])
settings = module.require(**_constants(locals()))
locals().update(_constants(settings))
The first thing we do is import secrets.py. This is where you’ll put the SECRET_KEY, database credentials, etc, and is the standard of the wiki examples. Next, we setup our project paths and the environment variable. But from here I stray from the norm a bit. Instead of importing a module based on the environment, I go ahead and import everything in the settings package (listdir may not be the most elegant way to get each filename, let me know if there’s a better approach).
I quickly ran into a problem though. I want to use the secrets, project paths, and environment settings inside the imported modules. This is where the ceremony comes in. To get around this I wrap each settings module in a function I can call in __init__.py, passing in the settings it needs like so:
import os
def require(DEV_ENV, BASE_PATH, DATABASE_USER, DATABASE_PASSWORD,
DATABASE_HOST, DATABASE_PORT, **kwargs):
if DEV_ENV:
database_dir = os.path.join(BASE_PATH, 'databases')
if not os.path.exists(database_dir):
os.mkdir(database_dir)
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(database_dir, 'sqlite3'),
}
}
else:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'USER': DATABASE_USER,
'PASSWORD': DATABASE_PASSWORD,
'HOST': DATABASE_HOST,
'PORT': DATABASE_PORT,
}
}
return locals()
So each module must def require (hat tip to RequireJS) with **kwargs and return locals(). Not too bad. Also, the above example shows another way in which I diverge from the wiki examples. Instead of overriding per environment, I just check the environment setting and drop in the correct setting. To extend settings per environment you would do the following:
def require(DEV_ENV, **kwargs):
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
# Uncomment the next line to enable the admin:
# 'django.contrib.admin',
# Uncomment the next line to enable admin documentation:
# 'django.contrib.admindocs',
)
if DEV_ENV:
INSTALLED_APPS += (
'debug_toolbar',
)
return locals()
All in all, I feel like this is a step in the right direction. If the settings package crammed full of sparse modules makes you queazy, than stick with one of the wiki examples. But for me, the ability to drop in a widgery.py module when I want to configure my widgery app is a breath of fresh air. Everything finally has a place.
Let me know what you think!
