File Formats
============
Unfortunately ImageD11 has evolved with some very poor file formats. These should be
replaced as soon as
possible with something more powerful, but in the meantime, here is some documentation
about what is
actually there right now.
Mostly the files are ascii following a simplistic model of:
# name = value
or
# name value
or
# name1 name2 name3
value1 value2 value3
Image Formats
=============
FIXME - it uses fabio.open()
Input/Output : 2D Peak search results
=====================================
The peaksearch.py script saves results in an ascii file which is read in again in the
graphical interface (peaksearch menu). For each image processed, the file contains the
header information on lines beginning with '#' a series of threshold levels and peaks.
The file is typically named peaks.out
Example with most of the header information snipped::
# File plat_a0050.edf
# Processed on Wed Nov 30 17:09:34 2005
# Spatial correction from /data/opid11/inhouse/Frelon2K/spatial2k.spline
# SPLINE X-PIXEL-SIZE 46.776480
# SPLINE Y-PIXEL-SIZE 48.081500
# HeaderID = EH:000001:000000:000000
# y1 = -3.88889e-05
# Instron strain = 0.002
[... lots of header information snipped ...]
# Omega = -39
#Threshold level 1200.000000
# Number_of_pixels Average_counts x y xc yc sig_x sig_y cov_xy
119 2371.050420 377.755365 1467.049632 372.961306 1468.529932 3.122391 2.406393 -0.363226
44
The Omega header entry is currently the only thing which is relied on. If your raw
images did not contain Omega header entries then a helper script recoveromegas.py is
available to help to insert these values. The numerical values for each peak found in
the image which are recorded are:
Number_of_pixels - the number of pixels in the blob (units: pure integer)
Average_counts - the average pixel intensity in the blob (units : ADU)
x - the centre of mass peak position in the raw image in the fast/slow (CHECK) array direction (units: pixels)
y - the centre of mass peak position in the raw image in the fast/slow (CHECK) array direction (units: pixels)
xc - the x value corrected for spatial distortion, using the spline file mentioned in the header (units: pixels)
yc - the x value corrected for spatial distortion, using the spline file mentioned in the header (units: pixels)
sig_x - the second moment of the intensity distribution in the x direction (eg: Gaussian sigma, related to the peak width) (units: pixels)
sig_y - the second moment of the intensity distribution in the y direction (eg: Gaussian sigma, related to the peak width) (units: pixels)
cov_xy - the x/y covariance of the intensity distribution. Ranges from -1 to 1 with zero being a circular peak and extreme values representing ellipses. (units: pure float)
Written by : scripts/peaksearch.py scripts/recoveromega.py
Read by : ImageD11/guipeaksearch.py
#Input.2FOutput_:_Merged_peaks?
Input/Output : Merged peaks
===========================
Versions > 1.0
Many columns, they are:
1: sc distortion corrected centre of mass, slow index direction
2: fc distortion corrected centre of mass, fast index direction
3: omega omega (rotation sequence) centre of mass
4: Number_of_pixels ... in the 3D connected object
5: avg_intensity average intensity of the pixels in the object
6: s_raw raw position (no spatial), slow direction
7: f_raw raw position (no spatial), fast direction
8: sigs second moment, slow direction
9: sigf second moment, fast direction
10: covsf covariance for fast/slow
11: sigo second moment, omega direction (out of plane of image)
12: covso covariance for slow/omega
13: covfo covariance for fast/omega
14: sum_intensity total intensity for pixels above threshold
15: sum_intensity^2 summed intensity squared [not useful?]
16: IMax_int maximum pixel
17: IMax_s array index slow direction for maximum pixel
18: IMax_f array index fast direction for maximum pixel
19: IMax_o array index omega direction for maximum pixel
20: Min_s minimum pixel position in slow direction
21: Max_s maximum pixel position in slow direction
22: Min_f minimum pixel position in fast direction
23: Max_f maximum pixel position in fast direction
24: Min_o minimum pixel position in omega direction
25: Max_o maximum pixel position in omega direction
26: dety unlikely to be correct flipped direction
27: detz unlikely to be correct flipped direction
28: onfirst blob is present on first image
29: onlast blob is present on last image
30: spot3d_id line number - 1 (titles) in the file on creation by peaksearch.py
The peaksearching menu in the gui reads in the previous peaksearch output and merges peaks which have adjacent Omega values (and centre of mass within some tolerance) to give a new file which contains only the peaks. Typically named peaks.flt
# xc yc omega npixels avg_intensity x_raw y_raw sigx sigy covxy
1020.355425 1458.043233 -37.802212 170.000000 2696.852941 1024.230029 1458.756529 2.060744 2.430557 -0.124729
1060.621433 1456.483538 -37.750000 4.000000 1227.250000 1064.493991 1457.499083 0.499964 0.499999 -0.003485
665.542804 728.847852 -37.750000 6.000000 1243.166667 672.161282 732.332082 1.060509 0.743424 0.767337
Column labels are as before but with the addition of an omega column giving the centre
of mass of the blob in omega too.
Written by : ImageD11/guipeaksearch.py
Read by : ImageD11/guitransformer.py ImageD11/refinegrains.py
gvectorfile
============
Scattering vectors computed by the transformation module, normally via the
guitransformer module...
The first line of the file contains the unit cell parameters and lattice centering (one
of P,A,B,C,I,F). The wavelength and wedge angle are expected on the next lines (needed
to compute ideal two theta, omega and azimuth angles for computed orientations). A list
of computed d* values expected h,k,l values for the unit cell follow, I don't think the
program uses them anymore. Note that d* is 1/d-spacing in Angstrom. The actual
scattering vectors follow::
78.712000 78.712000 78.712000 90.000000 90.000000 90.000000 I
# wavelength = 0.939500
# wedge = -0.051189
# axis 0.000000 0.000000 1.000000
[... the remaining part of the parameter information is snipped ...]
# ds h k l
0.0179669 -1 -1 0
0.0179669 -1 0 -1
0.0179669 -1 0 1
0.0179669 -1 1 0
0.0179669 0 -1 -1
0.0179669 0 -1 1
0.0179669 0 1 -1
0.0179669 0 1 1
0.0179669 1 -1 0
0.0179669 1 0 -1
0.0179669 1 0 1
0.0179669 1 1 0
0.0254091 -2 0 0
0.0254091 0 -2 0
0.0254091 0 0 -2
[ lots of hkl's snipped ]
# gx gy gz xc yc ds eta omega spot3d_id xl yl zl
-0.006049 -0.006821 -0.011411 1553.000000 1506.534500 0.014605 141.377789 41.000000 35325 211440.500505 -1811.094736 -2266.916293
-0.012846 0.008203 -0.009504 1564.869800 1510.194700 0.017962 121.948198 122.022400 105544 211439.925254 -3027.977786 -1888.286052
0.013156 0.009757 0.007648 1503.726500 1543.564400 0.018077 -64.969351 54.000300 46271 211436.535556 3254.006490 1519.487824
0.000474 0.018005 -0.001647 1500.521000 1525.574600 0.018086 -95.223069 1.993700 3308 211438.759075 3578.198430 -327.094270
-0.003736 0.010752 0.014069 1557.512700 1555.861300 0.018097 38.972120 160.000000 155486 211434.450605 -2261.233048 2795.170225
-0.013099 -0.009687 -0.007903 1566.912500 1513.290200 0.018108 115.878145 53.000000 44665 211439.526323 -3236.750899 -1570.156341
0.017023 -0.001041 -0.006196 1502.350000 1516.761000 0.018145 -109.965181 93.999200 78237 211439.811196 3388.245269 -1230.889139
...
gx x-component of scattering vector (along the beam) with all angles at
zero (units 1/Angstrom)
gy y-component of scattering vector (toward the door) with all angles at
zero (units 1/Angstrom)
gz z-component of scattering vector (roughly up) with all angles at zero
(units 1/Angstrom)
xc spatially corrected peak x-position on detector (units: pixels)
yc spatially corrected peak y-position on detector (units: pixels)
ds 1/d-spacing - modulus of scattering vector - <math> ds = \frac{\lambda}{ 2 \sin \theta}</math>
eta - azimuthal angle.
omega rotation angle of scan
spotid is an spot identifier to follow the individual spot at any process step
xl x-component of scattering vector in the laboratory coordinate system (along the beam) with all angles at zero (units microns)
yl y-component of scattering vector in the laboratory coordinate system (toward the door) with all angles at zero (units microns)
zl z-component of scattering vector in the laboratory coordinate system (roughly up) with all angles at zero (units microns)
Written by: ImageD11/guitransformer.py
Read by: ImageD11/indexing.py and GrainSpotter and possibly other programs..
Input/Output : UBI matrices
===========================
After successful(?) completion of some indexing a set of UBI matrices can be saved from
the indexing.py script or guiindexing interface. These are just 3x3 matrices separated
by blank lines::
0.743414 5.539980 -1.573009
5.484442 -1.159425 -1.505417
-1.758983 -1.292534 -5.361137
-3.422333 4.030841 -2.331791
-3.311710 -4.157822 -2.286774
-3.261669 -0.019438 4.770287
It should be the case that (CHECK/TESTCASE)
<math> \begin{matrix} h & k & l \end{matrix} \begin{matrix} UBI_{11} & UBI_{12} &
UBI_{13} \\ UBI_{21} & UBI_{22} & UBI_{23} \\ UBI_{31} & UBI_{32} & UBI_{33}
\end{matrix} \begin{matrix} xr & yr & zr \end{matrix} </math>
Written by: ImageD11/indexing.py ImageD11/guiindexer.py
Read by: ImageD11/refinegrains.py
Parameter files
===============
This files should all have the format name value.
For the transformation module the parameters are:
Unit cell
cell_a
cell_b
cell_c
cell_alpha
cell_beta
cell_gamma
cell_lattice_[P,A,B,C,I,F,R] F
Diffractometer angles and geometry:
chi : rotation of detector around beam (not tested, probably only in CVS)
wedge : tilt of axis around y
distance : sample detector distance (units: millimetres)
tilt-y : first detector tilt
tilt-z : second detector tilt
wavelength : of the incoming x-rays
y-center : beam centre on the detector (units: pixels)
y-size : pixel size in the y direction (units: microns)
z-center : beam centre on the detector (units: pixels)
z-size : pixel size in the z direction (units: microns)
CHECK that y/z are the right way around and also the hidden detector flip matrix.
fit_tolerance is used to decide which spots to assign to hkl rings in fitting the
geometrical parameters (units: degrees).
The indexing parameters are:
FIXME