1
2 """
3 Simple configuration system.
4
5 Allow applications and libraries to record configuration keys which can
6 be set by the user in the configuration file before startup, or specified
7 on the command line.
8
9 When running a program there are multiple sources of information that
10 need to be considered. When developing the code the programmer will
11 usually supply default values which the user can override.
12
13 The configuration file is the traditional method for storing this information.
14 Historically these were sets of key:value pairs, but this has evolved over
15 the years to Windows [section]key:value in the .ini format. The python
16 library ConfigParser understands the .ini format. More recent systems have
17 moved to a hierarchical key scheme with typed values.
18
19 Generally applications have system configuration shared by all users and
20 specific configuration for the individual users. Information shared from
21 run to run, such as recent activities also needs to be considered.
22
23 On unix configuration is stored in ad hoc text files::
24
25 /etc/apprc or /etc/app/apprc for system configuration
26 ~/.apprc or ~/.app/apprc for user configuration
27
28 On OS X configuration info is stored in xml-based plist files::
29
30 /Library/Preferences/com.company.app.plist for system configuration
31 ~/Library/Preferences/com.company.app.plist for user configuration
32
33 On Windows configuration is stored in registry keys::
34
35 HKEY_LOCAL_MACHINE\Software\company\app for system configuration
36 HKEY_CURRENT_USER\Software\company\app for user configuration
37
38 All three systems support environment variables, which can be used to
39 affect the runtime configuration. While not used much for new applications,
40 existing packages such as mysql may require the user to set the environment.
41 On OS X, environment variables are set in ~/.MacOSX/environment.plist.
42 See <http://developer.apple.com/qa/qa2001/qa1067.html> for details.
43 On Windows environment variables are set in >My Computer >Properties
44 >Advanced >Environment variables. On unix environment variables are
45 usually set in /etc/profile and ~/.profile for Bourne shell and derivatives.
46 Some config variables may be initialized from the environment.
47
48 Command-line flags are still common, particularly for backend software.
49 Individual configuration options can be set on the command line using
50 --key=value. These values take precedence over any other configuration
51 information. Shortcut flags, such as -v for --park.logging.verbose=True
52 can be configured on an application by application basis.
53
54 Note: libraries need to control their own configuration. The user does
55 not want to configure park separately for every app that uses it. Conversely,
56 some apps may want specialized park configuration parameters. We can
57 use 'CCS' (Cascading Configuration Sheets) to handle this, so long as
58 libraries and apps use a unified configuration environment, with config
59 keys using package namespace conventions.
60
61 Note: we are not trying to solve the problem of connecting and configuring
62 components in an enterprise framework. For tasks like this you will be
63 much better served by instantiating the components yourself from a Python
64 script, where you have direct access to objects and their attributes (or at
65 least to object proxies). This system is better suited to setting values
66 which don't change much from run to run.
67
68 Example
69 =======
70
71 First define the contents of the registry for the package. For documentation
72 purposes this is probably best done in a separate file config.py at the
73 root of your package directory. That way you can easily point your
74 users to the list of all registry keys that your application cares about.
75 If your application is plugin-based, each plugin can have its own
76 registry entries.
77
78 To use the configured values, simply reference them from config. Note that
79 config values cannot be used at the module level since the config file may
80 not be loaded at the time the module is imported, and since the config value
81 may change while the program is running. Similarly, config values should
82 not be used as defaults values for keyword parameters since these are
83 usually evaluated at load time.
84
85 For cascading configs, the package should trigger loading of the config
86 file when the module is first loaded. The application should be sure to
87 import all packages before loading its own configuration file. That
88 way, if the user overrides some package defaults for the particular
89 application (e.g., the choice of plotting package), it will not affect
90 other applications which use the same library.
91
92 mylib/configure.py::
93 '''
94 Package configuration parameters.
95 '''
96 __all__ = []
97 from park.util.configure import registry
98 registry.int('mylib.key','5',tip='number of keys')
99 ...
100
101 # Finally, load the configuration file from disk, if there is a
102 # package specific configuration file.
103 registry.load('mylib.danse.us')
104
105 mylib/module.py::
106 from park.util.configure import config
107 ...
108 def myfun(self, key=None):
109 # Grab the current default value of the key from config if it
110 # is not passed as a parameter.
111 if not key: key = config.mylib.key.value
112 @config.mylib.key.on_modify
113 def update_key(self, old=None, new=None, key=None):
114 # Track configuration updates
115 print key,"modified from",old,"to",new
116
117 mylib/__init__.py file::
118
119 ...
120 import .configure
121 ...
122
123
124 myapp/configure.py::
125 '''
126 Application configuration parameters.
127 '''
128 __all__ = []
129 from park.util.configure import registry
130 registry.int('myapp.key','5',tip='number of keys')
131 ...
132
133 # Finally, load the configuration file from disk, if there is a
134 # package specific configuration file.
135 registry.load('myapp.danse.us')
136
137
138 myapp/myapp.py::
139
140
141 import mylib # Force load of package registry first
142 import configure # Force load of application registry last
143
144 configure.registry.check() # Check the configuration values
145
146 """
147
148
149
150 import os
151
152 from park.util.host import Host
153 from park.util import magnitude
154
155
156 -class Entry(object):
157 - def __init__(self, key, value, parser=None, **kw):
158 self.meta = kw
159 self.key = key
160 self.parser = parser
161 self.__value = self.parse(value)
162 self.__save = []
163 self.__callstack = []
164 - def parse(self, value):
165 try:
166 return self.parser(value)
167 except Exception,err:
168 raise ValueError("When parsing %s='%s', %s"%(self.key,value,err))
169 - def _set_value(self, value):
170 old = self.__value
171 self.__value = self.parse(value)
172 if old == new: return
173 for fn in self.__callstack:
174 fn(key=self.key, old=old, new=new)
175 - def _get_value(self):
177 value = property(_get_value, _set_value)
179 return "%s:%s"%(self.key, str(self.__value))
180 - def __enter__(self, value):
181 self.__save.push(self.__value)
182 self.__value = value
183 - def __exit__(self):
184 self.__value = self.__save.pop()
185 - def on_modify(self, fn):
186 """
187 Register fn to be called when the config value is modified.
188 """
189 self.__callstack.push(fn)
190 return fn
194 return key in self.__dict__
196 return self.__dict__[key]
198 self.__dict__[key] = node
200 del self.__dict__[key]
206
207 - def add(self, key, value, parser=None, **kw):
208 """
209 Add an entry into the registry.
210
211 key is a dotted name forming a hierarchical namespace
212 value is the default
213 If 'after' is given, the configured value only applies after
214 """
215 if parser is None: raise TypeError('parser not specified for '+key)
216
217
218 parts = key.split('.')
219 node = self.config
220 for sub in parts[:-1]:
221 if not sub in node: node[sub] = Node()
222 node = node[sub]
223 node[parts[-1]] = Entry(key, value, parser=parser, **kw)
224
225 - def load(self, key):
226 """
227 Parse the config files, setting the defaults.
228
229 Key should be 'app.org.blah' where app is the name of the
230 application, org is the name of the organization and app.org.blah
231 is your primary application URL.
232
233 Configuration will be loaded from /etc/org/apprc and ~/.org/apprc
234 """
235
236
237 parts = key.split('.')
238 app,org = parts[:2]
239 system = os.path.join('etc', org, app+'rc')
240 user = os.path.join(os.environ['HOME'], '.'+org, app+'rc')
241 for filename in [system, user]:
242 config = loadconfig(filename)
243 for key,value in config.items():
244 self.set(key, value)
245
246 - def save(self, key):
247 """
248 Save current settings to a config file.
249 """
250
251 parts = key.split('.')
252 app,org = parts[:2]
253 system = os.path.join('etc', org, app+'rc')
254 user = os.path.join(os.environ['HOME'], '.'+org, app+'rc')
255
256 file = open(user,'wt')
257 keys = self.__items.keys()
258 keys.sort()
259 for k in keys:
260 entry = self.__items[k]
261 file.write('# '+entry.meta['tip'])
262 file.write(entry.key + ':' + str(entry.value))
263
264
265 - def host(self, key, value, **kw):
266 """
267 Specify user:password@host:port
268 """
269 self.add(key, value, parser=Host, **kw)
270
271 - def url(self, key, value, *kw):
272 """
273 Specify a URL. Returns a urlparse.urlparse object.
274 """
275 import urlparse
276 register(key, value, parser=urlparse.urlparse, **kw)
277
278 - def set(self, key, value, **kw):
279 """
280 Parse a set of values separated by spaces.
281 """
282 self.add(key, value, parser=_parse_set, **kw)
283
284 - def str(self, key, value, **kw):
285 """
286 Parse a string.
287 """
288 self.add(key, value, parser=_parse_str, **kw)
289
290 - def int(self, key, value, **kw):
291 """
292 Parse an integer.
293 """
294 self.add(key, value, parser=int, **kw)
295
296 - def item(self, key, value, choices=None, **kw):
297 """
298 Item chosen from a closed set.
299 """
300 self.add(key, value, parser=Choice(choices), **kw)
301
302 - def list(self, key, value, **kw):
303 """
304 Parse a list.
305 """
306 self.add(key, value, parser=_parse_list, **kw)
307
308 - def expr(self, key, value, **kw):
309 """
310 Parse a value which may be an expression.
311 """
312 self.add(key, value, parser=park.util.math.eval, **kw)
313
314 - def units(self, key, value, dim=None, **kw):
315 """
316 Parse a value with units.
317 """
318 self.add(key, value, parser=Units(dim), **kw)
319
320
321 config = Node()
322 registry = Registry(config)
323
325 content = {}
326 if not os.path.exists(filename): return content
327 fd = open(filename, 'rt')
328 for line in fd:
329 line = line.strip()
330 if line == '' or line.startswith('#'): continue
331 idx = line.index(':')
332 key = line[:idx]
333 value = line[idx+1:]
334 content[key] = value
335
337 """
338 Retrieve value in specified units.
339 """
347 """
348 Convert the given string 'val units' into a value in the target units.
349 """
350 parts = value.split()
351 number = float(parts[0])
352 units = " ".join(parts[1:])
353 mag = magnitude.mg(number,units)
354 self.zero + mag
355 return mag.toval(self.units)
356
359 self.choices = choices
361 if not value in self.choices:
362 raise ValueError("Item not in choices")
363 return value
366 return __builtins__.set(value.split())
367
370