1
2 """
3 Reflectometry reduction file formats.
4
5 === File formats ===
6
7 Supported formats are:
8
9 ICP on NCNR NG-1 and NG-7
10 NeXus on SNS Liquids and Magnetic
11
12 The list of available formats can be found at runtime using
13 reflectometry.reduction.formats()
14
15 Sample data for some of these formats is available in datadir. In ipython
16 type the following:
17
18 import reflectometry.reduction as red
19 ls $red.datadir
20
21 === Loading files ===
22
23 Data files are loaded using:
24
25 data = red.load('path/to/file')
26
27 This creates a reflectometry data object in memory whose fields can be
28 accessed directly (see below). Note that some data formats can store
29 multiple measurements in the file, so the returned value may be a list
30 of data objects.
31
32 Once loaded, data fields can be accessed using data.field for general
33 data, or data.component.field for specific components such as detectors
34 or slits. Within ipython, type data.<Tab> to see the fields available.
35 See help(refldata) for a complete description of all the fields.
36
37 Some datasets are huge and can take substantial time to load. Instead
38 of loading the entire dataset, you can use:
39
40 data = red.loadmeta('path/to/file')
41
42 to load just the metadata, and later use:
43
44 data.load()
45
46 to load the data itself. Again, loadmeta() returns a list of data
47 objects if there are multiple datasets in the file. Note that the
48 metadata can be wrong if for example an NCNR ICP run was aborted
49 before all the measurements were complete. data.load() will return
50 the points which were actually measured.
51
52 === Saving files === [Not implemented]
53
54 Saving files is the inverse of loading:
55
56 red.save(data, 'filename.ext')
57 red.save(data, 'filename') # .ext defaults to .dat
58 red.save(data, 'arbitraryfilename', format='.ext') #
59
60 This saves the contents of data into the file of type '.ext'. Alternatively,
61 the data can be saved to an arbitrary filename if the format='.ext'
62 keyword is given. If the extension is missing, '.dat' is used.
63
64 If no filename is given, then
65
66 The save function can be used to convert from one file format to
67 another. This is can be useful for comparing the results of reduction
68 from different reduction programs.
69
70 After normalizing by monitor, the data may be in various states of reduction:
71
72 * refl - specular reflectivity, Q dQ R dR wavelength
73 * spec - specular intensity, not yet corrected by slits
74 * back - background estimate
75 * slit - intensity measurement for slit corrections
76 * rock - slice through the Qx-Qz plane
77 * qxqz - 2-D data
78
79 It is up to the individual formats to determine how they will store this
80 information.
81
82 With no file extension, an ascii format is used with a .dat extension.
83 This is a multicolumn formation with a header defined by:
84
85 # field value
86 # field value
87 ...
88 # columns Q dQ R dR wavelength
89 0.001 0.000121 0.98596 0.00212 4.75
90 ...
91 0.02 0.00215 1.2356e-7 2.195e-8 4.75
92
93 The columns included will be different for the different states of
94 reduction, in particular, enough information needs to be preserved
95 so that intensity scans can be aligned with the corresponding specular
96 intensity.
97
98 With no filename, the file is saved to a file of the same name as the
99 original, but with an extension of e.g., .refl.dat if it is reflectivity
100 data.
101
102 Other common output formats include .xml for an xml version of the
103 multicolumn ascii format, following the standards for reduced SANS
104 data, and .nxs for an NeXus/HDF5 versions of the same information.
105
106 Note that the reduction process combines many files into one. Storing
107 details such as the sample description from all these files is impractical,
108 and so only one 'head' file will be chosen. This will be the file on
109 which all the corrections have been applied, or the file with the lowest
110 sequence number if multiple files have been combined. When saving to
111 the NeXus format, all the metadata from the head file will be preserved.
112
113 === Registering new formats ===
114
115 New formats can be created and register using
116
117 red.formats.register(loader)
118
119 See the formats.register documentation for a description of the loader
120 function interface.
121
122 Currently available formats are returned from::
123
124 red.formats.available()
125
126 """
127
128 import os.path
129
130 from registry import ExtensionRegistry
131 __all__ = ['loadmeta','load','datadir']
132
133 datadir = os.path.join(os.path.dirname(__file__),'examples')
134
135
136
137 registry = ExtensionRegistry()
150
151 -def load(file, format=None):
152 """
153 Load the reflectometry measurement description and the data.
154
155
156 Returns a single measurement if there is only one measurement in
157 the file, otherwise it returns a list of measurements.
158
159 Use formats() to list available file formats.
160 """
161 measurements = registry.load(file, format=format)
162 for data in measurements: data.load()
163 return measurements[0] if len(measurements)==1 else measurements
164
166 """
167 Return a list of available file formats.
168 """
169 return registry.formats()
170
172 """
173 Register loader for a file extension.
174
175 For each normal file extension for the format, call
176 register('.ext',loader)
177 You should also register the format name as
178 register('name',loader)
179 This allows the user to recover the specific loader using:
180 load('path',format='name')
181
182 The loader has the following signature:
183
184 [data1, data2, ...] = loader('path/to/file.ext')
185
186 The loader should raise an exception if file is not of the correct
187 format. When encountering an exception, load will try another loader
188 in reverse order in which the they were registered. If all loaders
189 fail, the exception raised by the first loader will be forwarded to
190 the application.
191
192 The returned objects should support the ReflData interface and
193 include enough metadata so that guess_intent() can guess the
194 kind and extent of the measurement it contains. The metadata need
195 not be correct, if for example the length and the actual values of
196 the motors are not known until the file is completely read in.
197
198 After initialization, the application will make a call to data.load()
199 to read in the complete metadata. In order to support large datasets,
200 data.detector.counts can use weak references. In that case the
201 file format should set data.detector.loadcounts to a method which
202 can load the counts from the file. If load() has already loaded
203 the counts in it can set data.detector.counts = weakref.ref(counts)
204 for the weak reference behaviour, or simply data.detector.counts = counts
205 if the data is small.
206
207 Both loader() and data.load() should call the self.resetQ() before
208 returning in order to set the Qx-Qz values from the instrument geometry.
209
210 File formats should provide a save() class method. This method
211 will take a ReflData object plus a filename and save it to the file.
212
213 See source in refldata.py for a description of the ReflData format.
214
215 See source in ncnr_ng1.py for a complete example.
216
217 """
218 registry[ext] = loader
219
220
225
230
235
236
237 register('.ng7', icp_ng7)
238 register('.ng7.gz', icp_ng7)
239 register('NCNR NG-7',icp_ng7)
240
241 register('.nxs', nexus)
242 register('NeXus', nexus)
243
244 register('NCNR NG-1', icp_ng1)
245 for ext in ['.na1', '.nb1', '.nc1', '.nd1', '.ng1']:
246 register(ext, icp_ng1)
247 register(ext+'.gz', icp_ng1)
248
249 for ext in ['.ca1', '.cb1', '.cc1', '.cd1', '.cg1']:
250 register(ext, icp_ng1)
251 register(ext+'.gz', icp_ng1)
252
254
255
256 import os.path
257 root = os.path.dirname(__file__)
258 ng7file = os.path.join(root,'examples','ng7','jul04032.ng7')
259 ng1file = os.path.join(root,'examples','ng1','psih1001.ng1')
260 cg1file = os.path.join(root,'examples','cg1area','psdca022.cg1.gz')
261 assert load(ng7file).detector.wavelength == 4.76
262 assert load(ng1file).name == 'gsip4007.ng1'
263 assert loadmeta(cg1file).name == 'psdca022.cg1'
264 assert available() == ['NCNR NG-1','NCNR NG-7','NeXus'],available()
265
266 if __name__ == "__main__": test()
267