normalizer module¶

`normalizer.NormalizeKnightRuizOriginal`(ccMapObj)	Original Knight-Ruiz algorithm for matrix balancing
`normalizer.normalizeCCMapByKR`(ccMap[, ...])	Normalize a ccmap using Knight-Ruiz matrix balancing method.
`normalizer.normalizeGCMapByKR`(...[, ...])	Normalize a gcmap using Knight-Ruiz matrix balancing method.
`normalizer.normalizeCCMapByIC`(ccMap[, tol, ...])	Normalize a ccmap by Iterative correction method
`normalizer.normalizeGCMapByIC`(...[, vmin, ...])	Normalize a gcmap using Iterative Correction.
`normalizer.normalizeCCMapByMCFS`(ccMap[, ...])	Scale ccmap using Median Contact Frequency
`normalizer.normalizeGCMapByMCFS`(...[, ...])	Scale all maps in gcmap using Median Contact Frequency

NormalizeKnightRuizOriginal(ccMapObj, tol=1e-12, x0=None, delta=0.1, Delta=3, fl=0)¶

Original Knight-Ruiz algorithm for matrix balancing

Ported from a matlab script given in the supporting information of the following paper:

P.A. Knight and D. Ruiz (2013). A fast algorithm for matrix balancing (2013). IMA Journal of Numerical Analysis, 33, 1029-1047”
Matrix must be symmetric and non-negative
For input matrix A, this function find a vector X such that diag(X)*A*diag(X) is close to doubly stochastic.

Warning

This is original ported code and kept here for comparison and testing.
Do not use it because for large matrix it may end up with consuming all the memory for large matrix.

Parameters:	ccMapObj (`gcMapExplorer.lib.ccmap.CCMAP`) – A CCMAP object containing observed contact frequency
Returns:	normCCMap – Normalized Contact map.
Return type:	`CCMAP`

normalizeCCMapByKR(ccMap, memory='RAM', tol=1e-12, outFile=None, vmin=None, vmax=None, percentile_thershold_no_data=None, thershold_data_occup=None, workDir=None)¶

Normalize a ccmap using Knight-Ruiz matrix balancing method.

Note

This function uses a modified version of orginal ported code given in NormalizeKnightRuizOriginal().
Please refer to: P.A. Knight and D. Ruiz (2013). A fast algorithm for matrix balancing (2013). IMA Journal of Numerical Analysis, 33, 1029-1047

Parameters:	ccMap (`gcMapExplorer.lib.ccmap.CCMAP` or ccmap file) – A CCMAP object containing observed contact frequency or a ccmap file. memory (str) – Accepted keywords are `RAM` and `HDD`: `RAM`: All intermediate arrays are generated in memory(RAM). This version is faster, however, it requires RAM depending on the input matrix size. `HDD`: All intermediate arrays are generated as memory mapped array files on hard-disk. tol (float) – Tolerance for matrix balancing. Smaller tolreance increases accuracy in sums of rows and columns. outFile (str) – Name of output ccmap file, to save directly the normalized map as a ccmap file. In case of this option, `None` will return. vmin (float) – Minimum thershold value for normalization. If contact frequency is less than or equal to this thershold value, this value is discarded during normalization. vmax (float) – Maximum thershold value for normalization. If contact frequency is greater than or equal to this thershold value, this value is discarded during normalization. percentile_thershold_no_data (int) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. `percentile_thershold_no_data` should be between 1 and 100. This options discard the rows and columns which are above this percentile. For example: if this value is 99, those row or columns will be discarded which contains larger than number of zeros (missing data) at 99 percentile. To calculate percentile, all blank rows are removed, then in all rows, number of zeros are counted. Afterwards, number of zeros at percentile_thershold_no_data percentile is obtained. In next step, if a row contain number of zeros larger than this percentile value, the whole row and column is assigned to have missing data. This percentile indicates highest numbers of zeros (missing data) in given rows/columns. thershold_data_occup (float) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. This ratio is (number of bins with data) / (total number of bins in the given row/column). For example: if thershold_data_occup = 0.8, then all rows containing more than 20% of missing data will be discarded. Note that this parameter is suitable for low resolution data because maps are likely to be much less sparse. workDir (str) – Path to the directory where temporary intermediate files are generated. If `None`, files are generated in the temporary directory according to the main configuration.
Returns:	ccMapObj – Normalized Contact map. When `outFile` is provided, `None` is returned. In case of any other error, `None` is returned.
Return type:	`gcMapExplorer.lib.ccmap.CCMAP` or `None`

normalizeGCMapByKR(gcMapInputFile, gcMapOutFile, mapSizeCeilingForMemory=20000, vmin=None, vmax=None, tol=1e-12, percentile_thershold_no_data=None, thershold_data_occup=None, compression='lzf', workDir=None, logHandler=None)¶

Normalize a gcmap using Knight-Ruiz matrix balancing method.

Parameters:	gcMapInputFile (str) – Name of input gcmap file. gcMapOutFile (str) – Name of output gcmap file. mapSizeCeilingForMemory (int) – Maximum size of contact map allowed for calculation using RAM. If map size or shape is larger than this value, normalization will be performed using disk (HDD). vmin (float) – Minimum thershold value for normalization. If contact frequency is less than or equal to this thershold value, this value is discarded during normalization. vmax (float) – Maximum thershold value for normalization. If contact frequency is greater than or equal to this thershold value, this value is discarded during normalization. tol (float) – Tolerance for matrix balancing. Smaller tolreance increases accuracy in sums of rows and columns. percentile_thershold_no_data (int) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. `percentile_thershold_no_data` should be between 1 and 100. This options discard the rows and columns which are above this percentile. For example: if this value is 99, those row or columns will be discarded which contains larger than number of zeros (missing data) at 99 percentile. To calculate percentile, all blank rows are removed, then in all rows, number of zeros are counted. Afterwards, number of zeros at percentile_thershold_no_data percentile is obtained. In next step, if a row contain number of zeros larger than this percentile value, the whole row and column is assigned to have missing data. This percentile indicates highest numbers of zeros (missing data) in given rows/columns. thershold_data_occup (float) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. This ratio is (number of bins with data) / (total number of bins in the given row/column). For example: if thershold_data_occup = 0.8, then all rows containing more than 20% of missing data will be discarded. Note that this parameter is suitable for low resolution data because maps are likely to be much less sparse. compression (str) – Compression method in output gcmap file. Presently allowed : `lzf` for LZF compression and `gzip` for GZIP compression. workDir (str) – Path to the directory where temporary intermediate files are generated. If `None`, files are generated in the temporary directory according to the main configuration.
Returns:
Return type:	None

normalizeCCMapByIC(ccMap, tol=0.0001, vmin=None, vmax=None, outFile=None, iteration=500, percentile_thershold_no_data=None, thershold_data_occup=None, workDir=None)¶

Normalize a ccmap by Iterative correction method

This method normalize the raw contact map by removing biases from experimental procedure. For more details, see this publication.

Parameters:	ccMap (`gcMapExplorer.lib.ccmap.CCMAP` or ccmap file.) – A CCMAP object containing observed contact frequency or a ccmap file tol (float) – Tolerance value. If variance of Delta-B is less than tolerance, stop the iterative process. vmin (float) – Minimum thershold value for normalization. If contact frequency is less than or equal to this thershold value, this value is discarded during normalization. vmax (float) – Maximum thershold value for normalization. If contact frequency is greater than or equal to this thershold value, this value is discarded during normalization. outFile (str) – Name of output ccmap file, to save directly the normalized map as a ccmap file. In case of this option, `None` will return. iteration (int) – Number of iteration to stop the normalization. percentile_thershold_no_data (int) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. `percentile_thershold_no_data` should be between 1 and 100. This options discard the rows and columns which are above this percentile. For example: if this value is 99, those row or columns will be discarded which contains larger than number of zeros (missing data) at 99 percentile. To calculate percentile, all blank rows are removed, then in all rows, number of zeros are counted. Afterwards, number of zeros at percentile_thershold_no_data percentile is obtained. In next step, if a row contain number of zeros larger than this percentile value, the whole row and column is assigned to have missing data. This percentile indicates highest numbers of zeros (missing data) in given rows/columns. thershold_data_occup (float) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. This ratio is (number of bins with data) / (total number of bins in the given row/column). For example: if thershold_data_occup = 0.8, then all rows containing more than 20% of missing data will be discarded. Note that this parameter is suitable for low resolution data because maps are likely to be much less sparse. workDir (str) – Path to the directory where temporary intermediate files are generated. If `None`, files are generated in the temporary directory according to the main configuration.
Returns:	normCCMap – Normalized Contact map. When `outFile` is provided, `None` is returned. In case of any other error, `None` is returned.
Return type:	`gcMapExplorer.lib.ccmap.CCMAP` or `None`

normalizeGCMapByIC(gcMapInputFile, gcMapOutFile, vmin=None, vmax=None, tol=1e-12, iteration=500, percentile_thershold_no_data=None, thershold_data_occup=None, compression='lzf', workDir=None, logHandler=None)¶

Normalize a gcmap using Iterative Correction.

This method normalize the raw contact map by removing biases from experimental procedure. For more details, see this publication.

Parameters:	gcMapInputFile (str) – Name of input gcmap file. gcMapOutFile (str) – Name of output gcmap file. vmin (float) – Minimum thershold value for normalization. If contact frequency is less than or equal to this thershold value, this value is discarded during normalization. vmax (float) – Maximum thershold value for normalization. If contact frequency is greater than or equal to this thershold value, this value is discarded during normalization. tol (float) – Tolerance value. If variance of Delta-B is less than tolerance, stop the iterative process. iteration (int) – Number of iteration to stop the normalization. percentile_thershold_no_data (int) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. `percentile_thershold_no_data` should be between 1 and 100. This options discard the rows and columns which are above this percentile. For example: if this value is 99, those row or columns will be discarded which contains larger than number of zeros (missing data) at 99 percentile. To calculate percentile, all blank rows are removed, then in all rows, number of zeros are counted. Afterwards, number of zeros at percentile_thershold_no_data percentile is obtained. In next step, if a row contain number of zeros larger than this percentile value, the whole row and column is assigned to have missing data. This percentile indicates highest numbers of zeros (missing data) in given rows/columns. thershold_data_occup (float) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. This ratio is (number of bins with data) / (total number of bins in the given row/column). For example: if thershold_data_occup = 0.8, then all rows containing more than 20% of missing data will be discarded. Note that this parameter is suitable for low resolution data because maps are likely to be much less sparse. compression (str) – Compression method in output gcmap file. Presently allowed : `lzf` for LZF compression and `gzip` for GZIP compression. workDir (str) – Path to the directory where temporary intermediate files are generated. If `None`, files are generated in the temporary directory according to the main configuration.
Returns:
Return type:	None

normalizeCCMapByMCFS(ccMap, stats='median', vmin=None, vmax=None, outFile=None, percentile_thershold_no_data=None, thershold_data_occup=None, workDir=None)¶

Scale ccmap using Median Contact Frequency

This method can be used to normalize contact map using Median contact values for particular distance between two locations/coordinates. At first, Median distance contact frequency for each distance is calculated. Subsequently, the observed contact frequency is divided by median contact frequency obtained for distance between the two locations.

Parameters:	ccMapObj (`gcMapExplorer.lib.ccmap.CCMAP` or ccmap file) – A CCMAP object containing observed contact frequency or a ccmap file stats (str) – Statistics to be calculated along diagonals: It may be either “mean” or “median”. By default, it is “median”. vmin (float) – Minimum thershold value for normalization. If contact frequency is less than or equal to this thershold value, this value is discarded during normalization. vmax (float) – Maximum thershold value for normalization. If contact frequency is greater than or equal to this thershold value, this value is discarded during normalization. outFile (str) – Name of output ccmap file, to save directly the normalized map as a ccmap file. In case of this option, `None` will return. percentile_thershold_no_data (int) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. `percentile_thershold_no_data` should be between 1 and 100. This options discard the rows and columns which are above this percentile. For example: if this value is 99, those row or columns will be discarded which contains larger than number of zeros (missing data) at 99 percentile. To calculate percentile, all blank rows are removed, then in all rows, number of zeros are counted. Afterwards, number of zeros at percentile_thershold_no_data percentile is obtained. In next step, if a row contain number of zeros larger than this percentile value, the whole row and column is assigned to have missing data. This percentile indicates highest numbers of zeros (missing data) in given rows/columns. thershold_data_occup (float) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. This ratio is (number of bins with data) / (total number of bins in the given row/column). For example: if thershold_data_occup = 0.8, then all rows containing more than 20% of missing data will be discarded. Note that this parameter is suitable for low resolution data because maps are likely to be much less sparse. workDir (str) – Path to the directory where temporary intermediate files are generated. If `None`, files are generated in the temporary directory according to the main configuration.
Returns:	ccMapObj – Normalized Contact map. When `outFile` is provided, `None` is returned. In case of any other error, `None` is returned.
Return type:	`gcMapExplorer.lib.ccmap.CCMAP` or `None`

normalizeGCMapByMCFS(gcMapInputFile, gcMapOutFile, stats='median', vmin=None, vmax=None, percentile_thershold_no_data=None, thershold_data_occup=None, compression='lzf', workDir=None, logHandler=None)¶

Scale all maps in gcmap using Median Contact Frequency

This method can be used to normalize contact map using Median contact values for particular distance between two locations/coordinates. At first, Median distance contact frequency for each distance is calculated. Subsequently, the observed contact frequency is divided by median contact frequency obtained for distance between the two locations.

Parameters:	gcMapInputFile (str) – Name of input gcmap file. gcMapOutFile (str) – Name of output gcmap file. stats (str) – Statistics to be calculated along diagonals: It may be either “mean” or “median”. By default, it is “median”. vmin (float) – Minimum thershold value for normalization. If contact frequency is less than or equal to this thershold value, this value is discarded during normalization. vmax (float) – Maximum thershold value for normalization. If contact frequency is greater than or equal to this thershold value, this value is discarded during normalization. percentile_thershold_no_data (int) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. `percentile_thershold_no_data` should be between 1 and 100. This options discard the rows and columns which are above this percentile. For example: if this value is 99, those row or columns will be discarded which contains larger than number of zeros (missing data) at 99 percentile. To calculate percentile, all blank rows are removed, then in all rows, number of zeros are counted. Afterwards, number of zeros at percentile_thershold_no_data percentile is obtained. In next step, if a row contain number of zeros larger than this percentile value, the whole row and column is assigned to have missing data. This percentile indicates highest numbers of zeros (missing data) in given rows/columns. thershold_data_occup (float) – It can be used to filter the map, where rows/columns with largest numbers of missing data can be discarded. This ratio is (number of bins with data) / (total number of bins in the given row/column). For example: if thershold_data_occup = 0.8, then all rows containing more than 20% of missing data will be discarded. Note that this parameter is suitable for low resolution data because maps are likely to be much less sparse. compression (str) – Compression method in output gcmap file. Presently allowed : `lzf` for LZF compression and `gzip` for GZIP compression. workDir (str) – Path to the directory where temporary intermediate files are generated. If `None`, files are generated in the temporary directory according to the main configuration.
Returns:
Return type:	None