Module configure

source code

Simple configuration system.

Allow applications and libraries to record configuration keys which can
be set by the user in the configuration file before startup, or specified
on the command line.

When running a program there are multiple sources of information that
need to be considered.  When developing the code the programmer will
usually supply default values which the user can override.

The configuration file is the traditional method for storing this information.
Historically these were sets of key:value pairs, but this has evolved over
the years to Windows [section]key:value in the .ini format.  The python
library ConfigParser understands the .ini format.  More recent systems have
moved to a hierarchical key scheme with typed values.

Generally applications have system configuration shared by all users and
specific configuration for the individual users. Information shared from
run to run, such as recent activities also needs to be considered.

On unix configuration is stored in ad hoc text files::

  /etc/apprc or /etc/app/apprc for system configuration
  ~/.apprc or ~/.app/apprc for user configuration

On OS X configuration info is stored in xml-based plist files::

  /Library/Preferences/com.company.app.plist for system configuration
  ~/Library/Preferences/com.company.app.plist for user configuration

On Windows configuration is stored in registry keys::

  HKEY_LOCAL_MACHINE\Software\companypp for system configuration
  HKEY_CURRENT_USER\Software\companypp for user configuration

All three systems support environment variables, which can be used to
affect the runtime configuration.  While not used much for new applications,
existing packages such as mysql may require the user to set the environment.
On OS X, environment variables are set in ~/.MacOSX/environment.plist.
See <http://developer.apple.com/qa/qa2001/qa1067.html> for details.
On Windows environment variables are set in >My Computer >Properties
>Advanced >Environment variables.  On unix environment variables are
usually set in /etc/profile and ~/.profile for Bourne shell and derivatives.
Some config variables may be initialized from the environment.

Command-line flags are still common, particularly for backend software.
Individual configuration options can be set on the command line using
--key=value.  These values take precedence over any other configuration
information.  Shortcut flags, such as -v for --park.logging.verbose=True
can be configured on an application by application basis.

Note: libraries need to control their own configuration.  The user does
not want to configure park separately for every app that uses it.  Conversely,
some apps may want specialized park configuration parameters.  We can
use 'CCS' (Cascading Configuration Sheets) to handle this, so long as
libraries and apps use a unified configuration environment, with config
keys using package namespace conventions.

Note: we are not trying to solve the problem of connecting and configuring
components in an enterprise framework.  For tasks like this you will be
much better served by instantiating the components yourself from a Python
script, where you have direct access to objects and their attributes (or at
least to object proxies).  This system is better suited to setting values
which don't change much from run to run.

Example
=======

First define the contents of the registry for the package.  For documentation
purposes this is probably best done in a separate file config.py at the
root of your package directory.  That way you can easily point your
users to the list of all registry keys that your application cares about.
If your application is plugin-based, each plugin can have its own
registry entries.

To use the configured values, simply reference them from config.  Note that
config values cannot be used at the module level since the config file may
not be loaded at the time the module is imported, and since the config value
may change while the program is running.  Similarly, config values should
not be used as defaults values for keyword parameters since these are
usually evaluated at load time.

For cascading configs, the package should trigger loading of the config
file when the module is first loaded.  The application should be sure to
import all packages before loading its own configuration file.  That
way, if the user overrides some package defaults for the particular
application (e.g., the choice of plotting package), it will not affect
other applications which use the same library.

mylib/configure.py::
    '''
    Package configuration parameters.
    '''
    __all__ = []
    from park.util.configure import registry
    registry.int('mylib.key','5',tip='number of keys')
    ...

    # Finally, load the configuration file from disk, if there is a
    # package specific configuration file.
    registry.load('mylib.danse.us')

mylib/module.py::
    from park.util.configure import config
    ...
    def myfun(self, key=None):
        # Grab the current default value of the key from config if it
        # is not passed as a parameter.
        if not key: key = config.mylib.key.value
    @config.mylib.key.on_modify
    def update_key(self, old=None, new=None, key=None):
        # Track configuration updates
        print key,"modified from",old,"to",new

mylib/__init__.py file::

    ...
    import .configure
    ...


myapp/configure.py::
    '''
    Application configuration parameters.
    '''
    __all__ = []
    from park.util.configure import registry
    registry.int('myapp.key','5',tip='number of keys')
    ...

    # Finally, load the configuration file from disk, if there is a
    # package specific configuration file.
    registry.load('myapp.danse.us')


myapp/myapp.py::


    import mylib      # Force load of package registry first
    import configure  # Force load of application registry last

    configure.registry.check()  #  Check the configuration values