Welcome to AGT’s documentation!¶
Introduction¶
This plugin is a toolbox for processing electrical resistivity (Geoscan Research RM15/RM85), magnetic (Sensys MXPDA / Bartington Grad601) and electromagnetic induction (Geonics EM31) data with shapefiles creation.
Electrical data processing module (Geoscan Research RM15/RM85) - grid survey¶
This module enables basic processing of resistivity data collected with a regular grid. For the moment, only the Pole-Pole configuration is available.
Input file
The input file is an ascii file (.dat) that contains the data and header that gives all the information concerning the configuration of the resistance meter and the survey mode:
Column file containing the data
- RM85 Name of the resistance meter
- 2 Number of grids
- 30 Grid length (m)
- 30 Grid width (m)
- 0.5 Probe spacing (m)
- 3 Number of channels
- 4 Number of probes
- 1 Line step (m)
- Pole-Pole Probe configuration
- 10 Current intensity
- 1 Number of the first grid
- 0 Coordinates of grid’s first point (bottom-left point)
- 0
- 2 Number of the second grid
- 0 coordinates of grid’s first point (bottom-left point)
- 30
- 20.95 Resistivity measurement
- 25
- 20.5
- 8.55
- 8.9
- 5.26
- .
- .
- .
The value of dummy log is 999.
Header has two different roles, so must be accurately completed because it is used:
- for data processing (automatically read by the software)
- for storing metadata of the survey
Processing
The basic processing consists of reorganizing the raw data in order to separate each channel and assign the right X,Y position of each measurement.. The measured resistance values are transformed into apparent resistivity, depending on the chosen probe configuration (for the moment only the pole-pole configuration is available). Data can be exported in shapefiles (.shp) or ascii files (.dat). Two sub-modules are subsequently proposed:
Median filtering
This function allows noise removal using a median filter algorithm. The user has to define the kernel size and the threshold defined as a percentage tolerance with the median value.
Georeferencing
This function allows grid georeferencing using geometric translations and rotations. The method uses two control points provided by the user.
Magnetic data processing module (Sensys MXPDA / Bartington Grad601) - grid survey¶
This module allows basic processing of magnetic survey data, collected with a Sensys MXPDA or Bartington Grad601 magnetometer.
Input file
The input file is an .dat file, generated by Sensys Magneto-Arch software or a .dat file generated by Bartington software. File configurations are as follows:
Sensys MXPDA
- X Y Value
- 0 0 2.8
- 0 0.1 2.87
- 0 0.2 3.08
- 0 0.3 2.59
- 0 0.4 1.89
- .
- .
- .
Bartington Grad601
- Time = 09:16:55
- Date = 06/10/2017
- Grid Number = 1
- Number of Sensors = 2
- Grid Size = 40 x 40
- Method of collection = ZigZag
- Starting Direction = West
- Data Range = 100 nT
- Line Spacing = 1.00 m
- Sampling = 4 samples / m
- Sensor Spacing = 1.0 m
- Mean = 0.50
- Max = 21.40
- Min = -4.01
- 0.125 0.5 0.87 Y, X, value
- 0.375 0.5 0.86
- 0.625 0.5 1.21
- 0.875 0.5 1.78
- 2.875 0.5 1.16
- .
- .
- .
Processing
The basic processing consists of reorganizing the data in order to separate each profile (a profile is defined with a fixed X coordinate). Data can be exported in shapefiles (.shp) and ascii files (.dat).
Three processing modules are proposed:
Median removal
This function removes the median value of each profile, in order to eliminate the shift due to electronic components of the sensors and constant magnetic disturbances. It is possible to limit the number of points used to determine the median value by using a percentile. Using a percentile threshold excludes strong magnetic anomalies from the median value computing.
Trend removal
This function allows a first-, second- or third-order polynomial removal on each profile.
Georeferencing
This function allows grid georeferencing using geometric translations and rotations. The method uses two control points provided by the user.
Magnetic data processing module (Sensys MXPDA) - GNSS survey¶
This module allows basic processing of magnetic survey data, collected with a Sensys MXPDA magnetometer coupled with a GNSS.
Input file
The input file is an .asc file, generated by Sensys Magneto-Arch software. Its configuration is as follows:
X, Y, difference of the vertical component of the magnetic field, profile name, the number of the probe.
- 30694328.591 5432511.556 5.5 “20161010-110332_GZP.prm” 1
- 30694328.717 5432511.772 31.2 “20161010-110332_GZP.prm” 2
- 30694328.844 5432511.987 -21.6 “20161010-110332_GZP.prm” 3
- 30694328.971 5432512.203 -8.3 “20161010-110332_GZP.prm” 4
- 30694329.098 5432512.418 -12.3 “20161010-110332_GZP.prm” 5
- …
Collected data are georeferenced in UTM coordinates. The two first digits of the X coordinate correspond to the UTM zone (UTM-30 in this example).
Processing
he basic processing consists of reorganizing the data in order to separate each profile (a profile is defined as a set of data collected with one probe along one line). Points are then georeferenced in the chosen mapping system. Data can be exported in shapefiles (.shp) and ascii files (.dat).
Four processing modules are proposed:
Decimation
This function reduces the number of collected points by keeping a fraction of them (1 / n, n is chosen by the user). The user can use raw data or median filtered data (computed with a n data point moving median).
Median removal
This function removes the median value of each profile, in order to eliminate the shift due to electronic components of the sensors and constant magnetic disturbances. It is possible to limit the number of points used to determine the median value by using a percentile. Using a percentile threshold excludes strong magnetic anomalies from the median value computing.
Trend removal
This function allows a first-, second- or third-order polynomial removal on each profile.
Stationary point removal
This function eliminates stationary data points collected while the magnetometer is immobile.
EMI data processing module (EM31 from Geonics)¶
The module transforms the electrical conductivity values provided by the EM31 (based on McNeil, 1980). This processing overpasses the boundaries of the linear approximation which is only valid in first approximation for an instrument held on the ground (Z=0) and respecting the low induction number condition (i.e. low electrical conductivity). Data is transformed into a .shp file and can be uploaded into GIS.
Input file
Input file is an ascii file format (.dat) as delivered by DAT31W (Geoncis software). It contains the X, and Y position of each measurement, as well as the quadrature (QV1 in mS / m), in-phase part of the electromagnetic signal (IV1 in ppt) and a time stamp:
- / EAST, NORTH, QV1, IV1, TIME /
- 642039.43420000 7097622.22880000 30.10 1.03 15: 32: 39.555
- 642039.43560000 7097622.22740000 30.25 1.03 15: 32: 39.904
- 642039.43548000 7097622.22934000 30.18 1.03 15: 32: 40.262
- 642039.43478000 7097622.23249000 30.13 1.01 15: 32: 40.614
- 642039.43402000 7097622.23591000 30.02 1.00 15: 32: 40.991
- 642039.50925000 7097622.14500000 29.95 1.00 15: 32: 41.353
- 642039.58235000 7097622.05660000 29.98 1.00 15: 32: 41.699
- 642039.67784000 7097621.93750000 30.18 1.02 15: 32: 42.071
The user has to specify the coordinate system used during the survey.
Processing
The processing module estimates the apparent electrical conductivity values based on the solution of the integrals and the Hankel transform (Thiesson et al., 2014). This solution takes into account the height of the device and the coils configuration. It can therefore be applied regardless of the type of soil studied even valid in salty soil contexts.
McNeill J.D., 1980 - Electromagnetic terrain conductivity measurement at low induction number, technical note TN6, Geonics Ltd, Toronto, 15p.
Thiesson J., Kessouri P., Schamper C., Tabbagh A., 2014 - Calibration of frequency-domain electromagnetic devices used in near-surface surveying. Near Surface Geophysics, 12, 481-491.
EMI multi-frequency data processing module (GEM2 from Geophex, EMP400 from GSSI)¶
This module enables different processing methods to be performed on the data collected with the GEM2 (Geophex) or the EMP400 (GSSI).
- Merging of GNSS and GEM2/EMP400 datasets (with clock offset correction)
- Positionning shift correction between the GNSS antenna and the GEM2/EMP400 device
- Data decimation
- Median filtering by profile
- 1D moving window filter (median or average method)
- Electrical conductivity computation
- Magnetic susceptibility computation with correction of the induction effect
- Device calibration based on vertical resistivity sounding or mean resistivity
GEM2 file input
The input file is an ascii file (.csv) in export format proposed by the software EMExport of the Geophex series. The module allows three different acquisition protocols (without GNSS, with a GNSS connected to the GEM2 and with a GNSS not connected to the GEM2) and it contains a variable data series based on an acquisition with GNSS or acquisition without GNSS.
GEM2 survey with GNSS
Line,Sample,X,Y,Mark,Status,GPSStat,GPSalt,Time[ms],Time[hhmmss.sss],PowerLn,I_5025Hz,Q_5025Hz, I_10325Hz,Q_10325Hz,I_21275Hz,Q_21275Hz,I_43725Hz,Q_43725Hz,I_89925Hz,Q_89925Hz,QSum
0, 21, 509779.22, 5061220.33, 0,0,4, 575.22, 35967000.100,095927.0001, 0.3,-1.22403e+003, 6.32362e+000,-1.55238e+003,-1.78986e+002,-1.99524e+003, 1.17386e+003,-2.12842e+003, 4.58693e+003, 3.66881e+003, 3.00984e+003, 8.59797e+003
GEM2 survey without GNSS
Line,Sample,X,Y,Mark,Status,Time[ms],Time[hhmmss.sss],PowerLn,I_5025Hz,Q_5025Hz,I_10325Hz, Q_10325Hz,I_21275Hz,Q_21275Hz,I_43725Hz,Q_43725Hz,I_89925Hz,Q_89925Hz,QSum
0,0, 0.00, 0.00, 0, 0, 52275802.200, 143115.8022, 0.2, 1.47162e+003, -3.85928e+002,9.69091e+002,1.29074e+001,4.03300e+002,1.14524e+003,6.69531e+001,5.15901e+003, 6.22617e+003, 5.16867e+003, 1.10741e+004,0,1, 0.00,1.06,0,0, 52275842.200,143115.8422, 0.1, 1.67830e+003, 1.36072e+001, 1.14406e+003,-1.37615e+002, 5.27930e+002, 1.16965e+003,-2.98461e+001, 5.38998e+003, 5.90062e+003, 5.46728e+003, 1.19029e+004
EMP400 file input The input file is an ascii file (.EMI) in export format proposed by the EMP400. The module allows two survey modes (without GNSS and with separate GNSS) and it contains a variable data series based on the survey procedure. Note that the EMP400 has a GNSS but, due to the low precision, the plugin doesn’t consider these data.
EMP400 survey
Header of 36 lines
- Record#, XCoord, YCoord, Time, InPhase[15000], Quad[15000], Conductivity[15000], InPhase[8000], Quad[8000], Conductivity[8000], InPhase[5000], Quad[5000], Conductivity[5000], Remark, Mark, Lat, Long, Alt, Tilt, Errors
- 31, 0.500, 18.000,07:19:22.385,-21373 ,1129 ,25.706 ,-6046 ,684 ,29.208 ,-2588 ,438 ,29.882 ,,, 45.6537417, 3.1515333,376.0000000,,NO ERRORS
The plugin automatically detects the number of frequencies employed for the data collection and the value of each frequency. Then, it detects whether the acquisition was performed with a GNSS or not. These specifications are then used in the different processing methods applied to the dataset.
The user has to specify the reference coordinate system (RCS) used during the survey. By default it is the WGS84 UTM31 North system. This RCS can be changed in the tab Parameters and saved as the new reference system default.
The input GNSS file is an ascii file (.dat). It must contain basic information, X and Y position, altitude and time of acquisition (hh:mm:ss.ss). Care must be taken to specify the RCS used for the GNSS data acquisition.
Input GNSS file format
- /X, Y, Z, Time/
- 1709059.979, 6946271.346, 232.25,12:11:05
- 1709059.975, 6946271.352, 232.22,12:11:06
- 1709009.729, 6946415.921, 232.15,12:42:52
Processing
Merging GNSS and GEM2/EMP400 data This function is dedicated to merging the GEM2/EMP400 input file and the GNSS input file based on the time of acquisition of each data. By default the GEM2/EMP400 input file is a file without GNSS data. Timing is based on the time stamp of both devices. We can specify the offset existing between the two clocks in order to optimize the data positioning. Data decimation
This function reduces the number of data by keeping a fraction of them (1/n, n is chosen by the user). Final data is based on the result of a median calculated with a moving window on the n decimated points.
Spatial GNSS/GEM2/EMP400 shift
In the case of a GEM2/EMP400-GNSS acquisition, for practical reasons, it is possible to have a spatial shift between the GNSS antenna and the center of the GEM2/EMP400. Shift correction is calculated in accordance with the direction of the surveyor and the device.
Median filtering by profile (In-phase/Out-of-phase)
This function allows the removal of the median value for each profile in order to reduce the effects due to the difference in ground clearance, or an inaccurate horizontal orientation. Median calculation by profile can be applied on in-phase and/or out-of-phase datasets.
Moving window filter
This function applies a 1D moving window filter along each profile. The user can select the method (median or average method), the in-phase or out-of-phase component and the window size.
Geophysical parameter calculation
Calibration
This processing enables a device calibration based on the up and down measurements collected at the location of a vertical electrical sounding or a known mean electrical resistivity value. In order to do the calibration, click on the button Calibration parameter. You will then have to fill in the EMI calibration file path. This file contains the up and down measurements (Thiesson et al. 2014) needed for the calibration. It must contain 6 measurement points, alternating between the up and down measurement, starting with a down measurement. As the GEM2/EMP400 is unable to acquire discrete points, each measurement corresponds to a static continuous profile to determine the value of each point (with an average calculation). The user has to specify the height of the device from the ground for both up and down measurements (when the GEM2/EMP400 is lying on the ground the respective height value is 0.02 m due to the frame of the GEM2/EMP400). It is then possible to choose for the reference value, an average resistivity value or a 5 layers model. If you only have a 3 or 4 layers, fill the last one(s) with the same value (Example: for a 3 layers model, rau3=rau4=rau5).
Electrical conductivity and magnetic susceptibility computation
The processing module estimates the apparent electrical conductivity values and the apparent magnetic susceptibility values based on the solution of the integrals and the Hankel transform (Thiesson et al. 2014). This solution takes into account the device’s height and the coils’ configuration. It can therefore be applied regardless of the type of soil and is even valid in high conductivity soil medium.
For the apparent magnetic susceptibility computation, the process enables the removal of the conductivity effect on the in-phase signal. Warning: This option implies a relatively long processing time. It is also necessary to firstly check that the validity of the previously calculated conductivity (necessary for this calculation) is not negative in which case the estimation of the susceptibility will be incorrect.
The processed points are saved in a shapefile with the user-defined RCS. It contains X and Y positions, in-phase and out-of-phase processed values and the computed conductivity and susceptibility values for each frequency. In addition, the shapefile contains the profile number, the time stamp, the GNSS quality and the altitude.
Thiesson J., Kessouri P., Schamper C., Tabbagh A. 2014 - Calibration of frequency-domain electromagnetic devices used in near-surface surveying. Near Surface Geophysics, 12, 481-491.
Future developments¶
- RM15/RM85 Download module
- Advanced processing module