File Formats¶
Unfortunately ImageD11 has evolved with some very poor file formats. These should be replaced as soon as possible with something more powerful, but in the meantime, here is some documentation about what is actually there right now. Mostly the files are ascii following a simplistic model of:
# name = value
or
# name value
or
# name1 name2 name3 value1 value2 value3
Image Formats¶
FIXME - it uses fabio.open()
Input/Output : 2D Peak search results¶
The peaksearch.py script saves results in an ascii file which is read in again in the graphical interface (peaksearch menu). For each image processed, the file contains the header information on lines beginning with ‘#’ a series of threshold levels and peaks. The file is typically named peaks.out Example with most of the header information snipped:
# File plat_a0050.edf
# Processed on Wed Nov 30 17:09:34 2005
# Spatial correction from /data/opid11/inhouse/Frelon2K/spatial2k.spline
# SPLINE X-PIXEL-SIZE 46.776480
# SPLINE Y-PIXEL-SIZE 48.081500
# HeaderID = EH:000001:000000:000000
# y1 = -3.88889e-05
# Instron strain = 0.002
[... lots of header information snipped ...]
# Omega = -39
#Threshold level 1200.000000
# Number_of_pixels Average_counts x y xc yc sig_x sig_y cov_xy
119 2371.050420 377.755365 1467.049632 372.961306 1468.529932 3.122391 2.406393 -0.363226
44
The Omega header entry is currently the only thing which is relied on. If your raw images did not contain Omega header entries then a helper script recoveromegas.py is available to help to insert these values. The numerical values for each peak found in the image which are recorded are:
Number_of_pixels - the number of pixels in the blob (units: pure integer) Average_counts - the average pixel intensity in the blob (units : ADU) x - the centre of mass peak position in the raw image in the fast/slow (CHECK) array direction (units: pixels) y - the centre of mass peak position in the raw image in the fast/slow (CHECK) array direction (units: pixels) xc - the x value corrected for spatial distortion, using the spline file mentioned in the header (units: pixels) yc - the x value corrected for spatial distortion, using the spline file mentioned in the header (units: pixels) sig_x - the second moment of the intensity distribution in the x direction (eg: Gaussian sigma, related to the peak width) (units: pixels) sig_y - the second moment of the intensity distribution in the y direction (eg: Gaussian sigma, related to the peak width) (units: pixels) cov_xy - the x/y covariance of the intensity distribution. Ranges from -1 to 1 with zero being a circular peak and extreme values representing ellipses. (units: pure float) Written by : scripts/peaksearch.py scripts/recoveromega.py Read by : ImageD11/guipeaksearch.py #Input.2FOutput_:_Merged_peaks?
Input/Output : Merged peaks¶
Versions > 1.0
Many columns, they are:
1: sc distortion corrected centre of mass, slow index direction 2: fc distortion corrected centre of mass, fast index direction 3: omega omega (rotation sequence) centre of mass 4: Number_of_pixels ... in the 3D connected object 5: avg_intensity average intensity of the pixels in the object 6: s_raw raw position (no spatial), slow direction 7: f_raw raw position (no spatial), fast direction 8: sigs second moment, slow direction 9: sigf second moment, fast direction 10: covsf covariance for fast/slow 11: sigo second moment, omega direction (out of plane of image) 12: covso covariance for slow/omega 13: covfo covariance for fast/omega 14: sum_intensity total intensity for pixels above threshold 15: sum_intensity^2 summed intensity squared [not useful?] 16: IMax_int maximum pixel 17: IMax_s array index slow direction for maximum pixel 18: IMax_f array index fast direction for maximum pixel 19: IMax_o array index omega direction for maximum pixel 20: Min_s minimum pixel position in slow direction 21: Max_s maximum pixel position in slow direction 22: Min_f minimum pixel position in fast direction 23: Max_f maximum pixel position in fast direction 24: Min_o minimum pixel position in omega direction 25: Max_o maximum pixel position in omega direction 26: dety unlikely to be correct flipped direction 27: detz unlikely to be correct flipped direction 28: onfirst blob is present on first image 29: onlast blob is present on last image 30: spot3d_id line number - 1 (titles) in the file on creation by peaksearch.py
The peaksearching menu in the gui reads in the previous peaksearch output and merges peaks which have adjacent Omega values (and centre of mass within some tolerance) to give a new file which contains only the peaks. Typically named peaks.flt
# xc yc omega npixels avg_intensity x_raw y_raw sigx sigy covxy 1020.355425 1458.043233 -37.802212 170.000000 2696.852941 1024.230029 1458.756529 2.060744 2.430557 -0.124729 1060.621433 1456.483538 -37.750000 4.000000 1227.250000 1064.493991 1457.499083 0.499964 0.499999 -0.003485 665.542804 728.847852 -37.750000 6.000000 1243.166667 672.161282 732.332082 1.060509 0.743424 0.767337
Column labels are as before but with the addition of an omega column giving the centre of mass of the blob in omega too.
Written by : ImageD11/guipeaksearch.py Read by : ImageD11/guitransformer.py ImageD11/refinegrains.py
gvectorfile¶
Scattering vectors computed by the transformation module, normally via the guitransformer module... The first line of the file contains the unit cell parameters and lattice centering (one of P,A,B,C,I,F). The wavelength and wedge angle are expected on the next lines (needed to compute ideal two theta, omega and azimuth angles for computed orientations). A list of computed d* values expected h,k,l values for the unit cell follow, I don’t think the program uses them anymore. Note that d* is 1/d-spacing in Angstrom. The actual scattering vectors follow:
78.712000 78.712000 78.712000 90.000000 90.000000 90.000000 I
# wavelength = 0.939500
# wedge = -0.051189
# axis 0.000000 0.000000 1.000000
[... the remaining part of the parameter information is snipped ...]
# ds h k l
0.0179669 -1 -1 0
0.0179669 -1 0 -1
0.0179669 -1 0 1
0.0179669 -1 1 0
0.0179669 0 -1 -1
0.0179669 0 -1 1
0.0179669 0 1 -1
0.0179669 0 1 1
0.0179669 1 -1 0
0.0179669 1 0 -1
0.0179669 1 0 1
0.0179669 1 1 0
0.0254091 -2 0 0
0.0254091 0 -2 0
0.0254091 0 0 -2
[ lots of hkl's snipped ]
# gx gy gz xc yc ds eta omega spot3d_id xl yl zl
-0.006049 -0.006821 -0.011411 1553.000000 1506.534500 0.014605 141.377789 41.000000 35325 211440.500505 -1811.094736 -2266.916293
-0.012846 0.008203 -0.009504 1564.869800 1510.194700 0.017962 121.948198 122.022400 105544 211439.925254 -3027.977786 -1888.286052
0.013156 0.009757 0.007648 1503.726500 1543.564400 0.018077 -64.969351 54.000300 46271 211436.535556 3254.006490 1519.487824
0.000474 0.018005 -0.001647 1500.521000 1525.574600 0.018086 -95.223069 1.993700 3308 211438.759075 3578.198430 -327.094270
-0.003736 0.010752 0.014069 1557.512700 1555.861300 0.018097 38.972120 160.000000 155486 211434.450605 -2261.233048 2795.170225
-0.013099 -0.009687 -0.007903 1566.912500 1513.290200 0.018108 115.878145 53.000000 44665 211439.526323 -3236.750899 -1570.156341
0.017023 -0.001041 -0.006196 1502.350000 1516.761000 0.018145 -109.965181 93.999200 78237 211439.811196 3388.245269 -1230.889139
...
gx x-component of scattering vector (along the beam) with all angles at zero (units 1/Angstrom) gy y-component of scattering vector (toward the door) with all angles at zero (units 1/Angstrom) gz z-component of scattering vector (roughly up) with all angles at zero (units 1/Angstrom) xc spatially corrected peak x-position on detector (units: pixels) yc spatially corrected peak y-position on detector (units: pixels) ds 1/d-spacing - modulus of scattering vector - <math> ds = frac{lambda}{ 2 sin theta}</math> eta - azimuthal angle. omega rotation angle of scan spotid is an spot identifier to follow the individual spot at any process step xl x-component of scattering vector in the laboratory coordinate system (along the beam) with all angles at zero (units microns) yl y-component of scattering vector in the laboratory coordinate system (toward the door) with all angles at zero (units microns) zl z-component of scattering vector in the laboratory coordinate system (roughly up) with all angles at zero (units microns) Written by: ImageD11/guitransformer.py Read by: ImageD11/indexing.py and GrainSpotter and possibly other programs..
Input/Output : UBI matrices¶
After successful(?) completion of some indexing a set of UBI matrices can be saved from the indexing.py script or guiindexing interface. These are just 3x3 matrices separated by blank lines:
0.743414 5.539980 -1.573009
5.484442 -1.159425 -1.505417
-1.758983 -1.292534 -5.361137
-3.422333 4.030841 -2.331791
-3.311710 -4.157822 -2.286774
-3.261669 -0.019438 4.770287
It should be the case that (CHECK/TESTCASE) <math> begin{matrix} h & k & l end{matrix} begin{matrix} UBI_{11} & UBI_{12} & UBI_{13} \ UBI_{21} & UBI_{22} & UBI_{23} \ UBI_{31} & UBI_{32} & UBI_{33} end{matrix} begin{matrix} xr & yr & zr end{matrix} </math> Written by: ImageD11/indexing.py ImageD11/guiindexer.py Read by: ImageD11/refinegrains.py
Parameter files¶
This files should all have the format name value. For the transformation module the parameters are: Unit cell
cell_a cell_b cell_c cell_alpha cell_beta cell_gamma cell_lattice_[P,A,B,C,I,F,R] F Diffractometer angles and geometry:
chi : rotation of detector around beam (not tested, probably only in CVS) wedge : tilt of axis around y distance : sample detector distance (units: millimetres) tilt-y : first detector tilt tilt-z : second detector tilt wavelength : of the incoming x-rays y-center : beam centre on the detector (units: pixels) y-size : pixel size in the y direction (units: microns) z-center : beam centre on the detector (units: pixels) z-size : pixel size in the z direction (units: microns)
CHECK that y/z are the right way around and also the hidden detector flip matrix. fit_tolerance is used to decide which spots to assign to hkl rings in fitting the geometrical parameters (units: degrees).
The indexing parameters are: FIXME