JCIM Metalprot2019

Subscriber access provided by Macquarie University
Computational Biochemistry
Comparative Assessment of 7 Docking Programs on a Non-
Redundant Metalloprotein Subset of the PDBbind Refined
Suleyman Selim Cinaroglu, and Emel Timucin
J. Chem. Inf. Model., Just Accepted Manuscript • DOI: 10.1021/acs.jcim.9b00346 • Publication Date (Web): 28 Aug 2019
Downloaded from pubs.acs.org on August 29, 2019
Just Accepted
“Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted
online prior to technical editing, formatting for publication and author proofing. The American Chemical
Society provides “Just Accepted” as a service to the research community to expedite the dissemination
of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in
full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully
peer reviewed, but should not be considered the official version of record. They are citable by the
Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore,
the “Just Accepted” Web site may not include all articles that will be published in the journal. After
a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web
site and published as an ASAP article. Note that technical editing may introduce minor changes
to the manuscript text and/or graphics which could affect content, and all legal disclaimers and
ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or
consequences arising from the use of information contained in these “Just Accepted” manuscripts.
is published by the American Chemical Society. 1155 Sixteenth Street N.W.,

Washington, DC 20036
Published by American Chemical Society. Copyright © American Chemical Society.
However, no copyright claim is made to original U.S. Government works, or works
produced by employees of any Commonwealth realm Crown government in the course
of their duties.
Page 1 of 30 Journal of Chemical Information and Modeling
1
2
3 Comparative Assessment of 7 Docking Programs on a Non-Redundant Metalloprotein
4
5 Subset of the PDBbind Refined
6
7 Süleyman Selim Çınaroğlu, Emel Timuçin
8
9 Department of Biostatistics and Medical Informatics, School of Medicine, Acibadem Mehmet Ali
10
11 Aydinlar University, Istanbul 34752, Turkey
12
13 ABSTRACT
14
15 Extensive usage of molecular docking for computer-aided drug discovery resulted in development
16
17 of numerous programs with versatile scoring and posing algorithms. Selection of the docking
18
program among these vast number of options is central to the outcome of drug discovery. To this
19
20 end, comparative assessment studies of docking offer valuable insights into the selection of the
21
22 optimal tool. Despite the availability of various docking assessment studies, the performance
23 difference of docking programs has not been well addressed on metalloproteins which comprise
24
25 a substantial portion of the human proteome and have been increasingly targeted for treatment of
26
a wide variety of diseases. This study reports comparative assessment of 7 docking programs on
27
28 a diverse metalloprotein set which was compiled for this study. The refined set of the PDBbind
29
30 (2017) was screened to gather 710 complexes with metal ion(s) closely located to the ligands (<4
31 Å). The redundancy was eliminated by clustering and overall 213 complexes were compiled as
32
33 the non-redundant metalloprotein subset of the PDBbind refined. The scoring, ranking and posing
34 powers of 7 non-commercial docking programs, namely AutoDock4, AutoDock4Zn, AutoDock Vina,
35
36 Quick Vina 2, LeDock, PLANTS and UCSF DOCK6 were comprehensively evaluated on this non-
37
redundant set. Results indicated that PLANTS (80%) followed by LeDock (77%), QVina (76%)
38
39 and Vina (73%) had the most accurate posing algorithms while AutoDock4 (48%) and DOCK6
40
41 (56%) were the least successful in posing. Contrary to their moderate-to-high level of posing
42 success, none of the programs was successful in scoring or ranking of the binding affinities (r2 ≈
43
44 0). Screening power was further evaluated by using active-decoy ligand sets for a large
45 compilation of metalloprotein targets. PLANTS stood out among other programs to be able to
46
47 enrich the active ligand for every target, underscoring its robustness for screening of
48
49
metalloproteins inhibitors. This study provides useful information for drug discovery studies
50 targeting metalloproteins.
51
52
53
54
55
56
57
58
59 1
60 ACS Paragon Plus Environment
Journal of Chemical Information and Modeling Page 2 of 30
1
2
3 INTRODUCTION
4
5
Computational methodologies offer effective and low-cost solutions to the drug discovery
6
7 problems, representing alternative approaches to traditional experiments for hit identification and
8
9 lead optimization.1-3 Particularly when the structural information is available for the target protein,
10 docking becomes the routinely visited tool. In line with its extensive usage, currently more than 70
11
12 docking programs are available. Among these numerous options, some of the docking programs
13 can accurately predict the high-affinity compounds for a given receptor or ligand class while other
14
15 programs may fail to perform at the same level.4-6 Given this warning, selection of the docking
16
program holds primal importance for the structure-based drug design studies. To fully benefit from
17
18 a docking application, the user should be aware of the leverages and limitations of the current
19
20 docking methodologies.7 From this perspective, comparative assessment studies of docking
21 provide a useful guideline to the modelers and also play a pivotal role in the perfection of docking
22
23 methodologies.
24
25 Metalloproteins which refer to proteins with one or more metal ions coordinated within its structure
26
27 comprise almost half of the human proteome. Metalloproteins can carry out critical functions in
28 human body ranging from catalyzing the reactions in respiration to scavenging free radicals.8, 9
29
30 Their extensive involvement in versatile biological processes also causes them to appear in
31 various pathological conditions including neurodegeneration, cancer, inflammation, HIV infection
32
33 and hypertension.10-12 Thus, discovery of novel inhibitors of metalloproteins carry paramount
34
35
importance for treatment of these various pathologies. Undoubtedly, the modeler who aims to
36 identify novel and/or more potent metalloprotein inhibitors needs to select the correct docking
37
38 program which can account for the metal interactions. Although numerous studies have compared
39 the performance of docking programs on a given benchmark which contains some
40
41 metalloproteins,5, 13-18 none of them has exclusively focused on metalloprotein sets which
42
possesses metal(s) in close proximity to the ligand. Thereof, insights garnered by comparative
43
44 assessment studies of docking on metalloprotein sets can greatly contribute to the metalloprotein-
45
46 targeted drug discovery, unraveling the capabilities and limitations of the current approaches.
47
48 To this end, this study comparatively assessed the performances of 7 freely available docking
49
programs on a non-redundant diverse metalloprotein set of 213 complexes obtained from the
50
51 refined set of PDBbind (2017). Scoring, ranking and posing powers were extensively evaluated
52
53 for 7 programs. Further to this evaluation, screening powers of all programs were also assessed
54 by using 12 different metalloprotein targets. The insights revealed by this assessment study overall
55
56 reflect the power of PLANTS particularly for docking and screening of metalloprotein inhibitors.
57
58
59 2
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27 Fig. 1. Receptors of the metalloprotein sets. (A) Composition of the set before and after
28
clustering according to the metals. (B) PANTHER classification of the non-redundant set (after
29
30 clustering) according to molecular function.
31
32
33 MATERIALS and METHODS
34
35
Construction of the Dataset. The refined PDBbind set (2017)19, 20 composed of 4154 structures
36
37 was scanned to collect a total of 710 complexes with at least one metal ion. The metalloprotein
38
39 complexes that accommodate the metal ion(s) close in the proximity of the ligand (<4 Å) were
40 selected by an in-house script. The complexes that contain modified and/or non-standard amino
41
42 acids were filtered out. Ultimately, a total of 710 metalloprotein complexes were collected (Table
43 S1). To conduct a reliable statistical assessment, the redundancy in this set was eliminated by
44
45 clustering. The protein sequences were clustered by the cd-hit algorithm21 with the sequence
46
identity threshold of 90%. This threshold level ensured that the clusters are formed basically by
47
48 the same proteins. If the binding sites were formed by two or more polypeptide chains, the protein
49
50 which harbors the metal coordination site was included in the clustering. The protein with the
51 longest sequence which holds all domains of the cluster was selected as the cluster
52
53 representative. The metalloproteins were classified by using the PANTHER1422, 23 server
54
according to protein class and molecular function. The ligands were analyzed by Filter-it™ in
55
56 Silicos-it (http://silicos-it.be.s3-website-eu-west-1.amazonaws.com/) according to distributions of
57
58
59 3
1
2
3 binding free energy and some important physicochemical properties such as molecular weight
4
5 (MW), octanol–water partitioning coefficient (lipophilicity) (logP), solubility (logS), number of
6 rotatable bonds.
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52 Fig. 2. Summary of the ligands after clustering. (A) Distribution of the ligand binding affinity as a
53 function of logKD before and after clustering. (B) 4 other properties of ligands including solubility
54 (logS) molecular weight (MW-g/mol), partition constant (logP) and number of rotatable bonds
55
56 (torsions). Fig. S1illustrates the same 4 properties before clustering.
57
58
59 4
1
2
3 Structure preparation. Metalloproteins were initially prepared by removing the crystal water
4
5 molecules and processed by the DockPrep tool of the UCSF Chimera24 to repair/model the
6 truncated sidechains, add hydrogens and assign partial charges according the AMBER ff14SB.25
7
8 The ligands were processed by the OpenBabel26 using the MMFF94 parameters to generate their
9
low-energy conformations which were used as the input coordinates that sufficiently differ from
10
11 the original coordinates of the corresponding ligand.27 The partial charges of each ligand were
12
13 calculated by the AM1-BCC charge method implemented in the Antechamber.28-30 Further
14 processing of the ligands is explained in the following section.
15
16
Molecular docking. 7 docking programs including AutoDock431 (referred to AutoDock from
17
18 hereon), AutoDock4Zn32 (referred to ADZn), UCSF DOCK 6.833 (referred to DOCK6), LeDock,34
19
20 PLANTS,35-37 Quick Vina 238, 39 (referred to QVina), and AutoDock Vina40 (referred to Vina) were
21 tested by re-docking of the non-redundant set. These programs are freely available to the
22
23 academic users and can be run by command-line. The source codes of AutoDock, DOCK6, Vina
24
and QVina are available.
25
26
27 For AutoDock and ADZn, protein and ligand files were prepared by using the AutoDockTools. 31
28 Partial charges were calculated using the Gasteiger’s method.41 During both docking runs,
29
30 Lamarckian Genetic Algorithm (LGA) method was used for pose sampling in which the population
31 size and number of generations were set to 150 and 27,000 respectively. The number of energy
32
33 evaluations was set to 2,500,000. AutoDock scores were calculated by using the default scoring
34
35
scheme with the default van der Waals parameters for each metal. Specifically, the radius (r, Å)
36 and well-depth (ε, kcal.mol-1) were set as rCa=1.98, rMg=1.30, rMn=1.30, rZn=1.48 and εCa=0.550,
37
38 εMg=0.875, εMn=0.875 and εZn=0.550, respectively. While a formal charge of +2e was assigned to
39 all metals. While, ADZn scores were calculated by the special force field previously described.32
40
41 For zinc, a charge-independent term accounting for the coordination potential was added to the
42
default AutoDock scoring function. For other metals, the standard AutoDock force field was
43
44 applied.
45
46 For Vina and QVina, the receptor and ligand files were prepared as described for AutoDock.
47
48 During docking runs, the exhaustiveness was set to 8 while the energy range was kept 3 kcal/mol.
49
The metal charges were manually set to +2e for both programs. The same knowledge-based
50
51 scoring function40 was used for calculation of both docking scores.
52
53 For UCSF DOCK6, the receptor and ligand files were prepared by the DOCK6 accessory tools.
54
55 After removing the hydrogen atoms, DMS program was used to calculate the solvent accessible
56
57
58
59 5
1
2
3 surface of each receptor with a probe radius of 1.4 Å. The negative binding site space was
4
5 generated by sphgen cpp, and the spheres within 8.0 Å of the crystal ligand were retained for
6 docking. The grid files were generated with the grid spacing of 0.3 Å. Docking scores were
7
8 calculated by using the Grid score which consists of non-bonded van der Waals and Coulomb
9
potentials between the ligand and receptor.42 The van der Waals parameters of radius (r, Å) and
10
11 well-depth (ε, kcal.mol-1), rCa=1.33, rMg=0.787, rMn=1.69, rZn=1.10 and εCa=0.450 εMg=0.875,
12
13 εMn=0.014 and εZn=0.013 were used respectively. A formal charge of +2e was assigned to all
14 metals.
15
16
For PLANTS, the receptor and the ligand files were processed by the SPORES 1.3 tool with the
17
18 application of the complete mode.43, 44 During docking, the search speed was set to 1 and the
19
20 scoring function was selected as chemplp36, 37 which is a newer scoring function using the
21 intermolecular score from the older plp force field45, 46 and the additional term to account for
22
23 hydrogen bonding and metal-acceptor interactions between protein and ligand similar to the
24
chemscore of GOLD.47 Metal parameters were used as the default which corresponds to a formal
25
26 charge of +2e for zinc, while zero for the rest of the metal.
27
28 For LeDock, the receptor files were processed by the LePro tool. During docking, the clustering
29
30 RMSD was set to 1.0 Å. All other parameters were set to default for sampling by a combination of
31 simulated annealing and evolutionary optimization. Docking scores were calculated by the default
32
33 scoring function. Metal parameters were used as the default.
34
35 Search space for each docking run was individually determined by the eBoxSize tool.48 This tool
36
37 calculates an optimal box size for every ligand and was reported to maximize docking accuracy.
38 For all docking runs, the number of the predicted poses were set to 10. Unless otherwise stated,
39
40 default parameters were used for each program. The ligands were treated as flexible while the
41
42
proteins were kept rigid during all of the docking runs. Ligand flexibility was handled by using the
43 default setting of each program.
44
45 Construction and screening of the decoy ligand sets. To test the screening power, ligand sets
46
47 containing active and decoy ligands were constructed for 3 different target proteins for each metal
48
49
ion. Overall a total of 12 different ligand sets were constructed by the Directory of Useful Decoys-
50 Enhanced (DUD-E)49, 50 and screened by each docking program. Among 102 targets in DUD-E, 4
51
52 targets were automatically extracted from the DUD-E as they contain at least one metal ion (1 with
53 Ca2+, 1 with Mg2+ and 2 with Zn2+). For the rest of the targets which were not listed in the DUD-E
54
55 were manually selected from protein databank (PDB). The active ligands were obtained from
56
57
58
59 6
1
2
3 PubChem51, 52 and the decoy ligands were generated by DUD-E. In each of the ligand set the
4
5 active-to-decoy ligand ratio was set as 1:50. All of the ligands in the sets were processed by the
6 obgen module of OpenBabel26 by which the ionization of the ligands was carried out at pH 7.0.
7
8 Screening was conducted by using the same docking parameters described for re-docking.
9
Overall a total of 46,414 ligands were screened by each docking program.
10
11
12 Performance analysis. Performance of the docking programs was assessed by 4 criteria namely
13 scoring, ranking, posing and screening.53 Each criteria was analyzed for both of the lowest energy
14
15 (best score) and lowest RMSD (best pose) poses. RMSD between the experimental conformation
16
and each docking pose was calculated by the obrms tool in OpenBabel which corrects the
17
18 symmetry between structures. Scoring power was assessed by the Pearson correlation coefficient
19
20 between the docking scores of re-docked complexes and the experimental affinities (𝑙𝑜𝑔𝐾𝐷). Re-
21
22 scoring power for which the complexes were directly re-scored without any minimizations was also
23 evaluated for all programs except from LeDock for which re-scoring was not possible and for
24
25 QVina which has the same scoring function with Vina. Further to these scoring functions, an
26 additional scoring function ΔvinaRF20 was also included in the assessment. Ranking power which
27
28 assesses the capacity of a scoring function to accurately rank the ligands based on their affinities
29
30
was also assessed by the Spearman correlation coefficient.
31
32 Posing power was tested by analyzing the overall ligand and metal coordination site. Ligand
33 posing was analyzed by the rate of the correctly predicted poses which was calculated by the
34
35 RMSD between the docked and the crystal pose. When RMSD was equal or lower than 2 Å, ligand
36
37
posing was considered accurate. Posing of metal coordination site was tested by analyzing the
38 closest 3 metal ligating atoms in the ligand. If the docked pose correctly predicts all 3 metal ligating
39
40 atoms in the native ligand pose, metal coordination posing is considered a “high-level” success. If
41 docking correctly predicts only 2 of the metal ligating atoms, metal coordination posing is
42
43 considered a “medium-level” success. If docking predicts only a single metal ligating atom in the
44
ligand, metal coordination posing is considered a “low-level” success.
45
46
47 Screening power was analyzed by receiver operating characteristic (ROC) curve and the
48 enrichment factor (EF) which is the concentration of the active ligands among the top scoring hits
49
50 compared with their concentration in the entire database. ROC curves were plotted by the
51
ROCKER tool.54 AUC and EF distributions across docking tools were compared by the Kruskal-
52
53 Wallis test.
54
55
56
57
58
59 7
1
2
3 RESULTS
4
5 Construction of the Non-redundant Metalloprotein Set. Metalloprotein subset of PDBbind
6 refined that was composed of 710 complexes contains 4 different metal ions; 523 of Zn2+, 101 of
7
8 Mg2+, 45 of Mn2+and 41 of Ca2+ and a large redundancy. The redundancy was eliminated by
9
clustering of 710 complexes into 213 clusters for which a representative member was selected for
10
11 the non-redundant set (Table S1). The largest cluster in the non-redundant set was composed of
12
13 217 members of carbonic anhydrase enzymes and the second largest cluster was formed by
14 matrix metalloproteinases and had 23 members. While a large fraction of the clusters (90%) has
15
16 less than 5 members. Although the main set composed of largely Zn2+ containing receptors (74%),
17
the non-redundant set has only 46% of Zn2+ receptors. The ratio of other metals were also
18
19 increased, leading to a more uniform representation of metals after clustering (Fig. 1A). Overall,
20
21 the non-redundant set was composed of 213 metalloproteins with diverse structure and functions
22 (Fig. 1B).
23
24 Ligand properties were analyzed before (Fig. S1) and after clustering (Fig. 2). Distributions of the
25 binding affinity was similar before (1.2 pM—7mM) and after clustering (4.8 pM—6mM) (Fig. 2A).
26
27 Distributions of other ligand features such as torsion, molecular weight, lipophilicity and water
28
solubility were also analyzed for the overall (Fig. S1) and the non-redundant set (Fig. 2B).
29
30 Essentially a large fraction of the ligands in both sets has lower molecular weights than 500 Da.
31
32 Similarly, distribution of the number of rotatable bonds in ligands indicated that most of the ligands
33 in both sets have less than 10 number of rotatable bonds. Clustering led to a collection of slightly
34
35 more soluble ligands in the non-redundant set (Fig. 2B). This analysis overall reflected the
36 potential of the ligands in the non-redundant set as drug candidates.
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59 8
1
2
3 Table 1. Scoring and ranking powers.
4
5 AutoDock ADZn DOCK6 LeDock PLANTS Qvina Vina
6
7 Overall 0.06b 0.07 0.14* 0.14* 0.04 0.11 0.11
8 (213)a (0.02)c (0.02) (0.12) (0.14)* (0.03) (0.13) (0.13)
9
Ca2+ 0.31 0.27 0.12 0.46* 0.37 0.57** 0.59**
10
(24) (0.22) (0.24) (0.03) (0.41)* (0.34) (0.55)** (0.55)**
11
12 Mg2+ -0.03 -0.06 0.09 0.16 0.09 0.12 0.14
13 (68) (-0.07) (-0.08) (0.06) (0.16) (0.08) (0.17) (0.17)
14
15 Mn2+ 0.25 0.31 0.23 -0.48* -0.42* -0.43* -0.44*
16 (24) (-0.06) (0.05) (-0.16) (-0.42)* (-0.41)* (-0.37) (-0.38)
17
18 Zn2+ 0.04 0.21* 0.10 0.10 0.02 0.09 0.08
19 (97) (0.01) (0.17) (0.11) (0.16) (0.01) (0.12) (0.11)
20
21
*p<0.05, **p<0.01
aNumber of complexes were given in parentheses.
22 bPearson correlation coefficients between the docking scores and logK .
23 d
cSpearman ranking coefficients between the docking scores and logK .
24 d
25
26
27 Assessment of Scoring and Ranking. Scoring and ranking powers of 7 docking programs were
28
29 listed in the Table 1 and illustrated by the scatter plots (Fig. 3). None of the programs was
30 successful in scoring or ranking, only the scores of DOCK6 and LeDock explained a very small
31
32 portion of the variance in the experimental affinities (r2=0.02). Despite their absence of
33
scoring/ranking performance, all docking programs produced negative scores for the entire set
34
35 (Fig. 3). Docking performance was further investigated by analyzing the lowest RMSD poses, yet
36
37 the scoring/ranking power was still zero for all programs (data not shown). However, when scoring
38 power was analyzed based on metal types, docking scores of Vina (r=0.59), QVina (r=0.57) and
39
40 LeDock (r=0.46) were found correlated with the affinities of the Ca2+ containing subset, while ADZn
41
scores were weakly but significantly correlated with the affinities of the Zn2+ containing subset
42
43 (Table 1). Otherwise, scoring power of ADZn was similar to the that of AutoDock for the rest of the
44
45 metal types. Similar to the comparison of AutoDock and ADZn, the scoring and ranking
46 performances of QVina and Vina were similar to each other. Additionally, only AutoDock, DOCK6
47
48 and ADZn yielded scores that were positively correlated with the 𝑙𝑜𝑔𝐾𝐷 values of the Mn2+ subset,
49
50 while the rest of the programs showed a negative relation (Fig. 3). None of the programs were
51 successful in scoring or ranking of the Mg2+ and Mn2+ subsets.
52
53
54
55
56
57
58
59 9
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22 Fig. 3. Scoring power was illustrated by scatter-plots of the docking scores and the experimental
23 binding free energies (logKD). The rightmost plot at the bottom panel shows the performance on
24 the overall set of 213 diverse complexes by using normalized scores for each docking program.
25
26 Interestingly, scoring functions of AutoDock, ADZn and Vina were more successful in re-scoring if
27
28 the crystal poses than re-docking (Table 2). On the other hand, PLANTS’ showed no improvement
29
in scoring when directly crystal poses were scored. Re-scoring powers based on individual metal
30
31 ions were strikingly different from that of re-docking. As such, PLANTS and DOCK6 successfully
32
33 re-scored the Mn2+ containing subset and Vina re-scored the Ca2+ containing subset.
34
35 Table 2. Re-scoring power.
36
Overall Ca2+ Mg2+ Mn2+ Zn2+
37
38 AutoDock 0.16* 0.33 0.22 0.35 0.06
39
40 ADZN 0.18* 0.24 0.20 0.33 0.17
41
42 DOCK6 0.14* -0.32 0.07 0.41* 0.23*
43
PLANTS -0.06 -0.46* -0.09 0.62** 0.01
44
45 Vina 0.14* 0.47* 0.14 -0.33 0.16
46
47 Δvina 0.15* 0.47* 0.15 -0.20 0.15
48
49 *p<0.05, **p<0.01
50
51
52 To test the contribution of the metal ion to the scoring power, re-docking of the non-redundant set
53
54 was re-performed by using the metal free receptors. The scoring power was similarly assessed
55 by the scatter plots (Fig. S2). The scoring powers of LeDock, PLANTS, Vina, QVina and ADZn
56
57
58
59 10
1
2
3 were not affected by removal of the metal atoms. However, when metal atoms were ignored, the
4
5 scoring power of AutoDock was increased (r=0.14, p=0.039), compared with the condition when
6 metals were kept (r=0.062, p=0.37). Contrary to this observation, scoring power of DOCK6’s was
7
8 worsened in the absence of metals (r=0.071, p=0.30) than it was in the presence (r=0.140,
9
p=0.042).
10
11
12 Table 3. Scoring and ranking powers on the largest cluster of Zn2+ containing 217 carbonic
13 anhydrase enzymes.
14
15 r
16
AutoDock 0.12 (0.16)*
17
18 ADZn **0.61 (0.34)**
19
20 DOCK6 **0.29 (0.32)**
21
22 LeDock **0.59 (0.60)**
23 PLANTS **0.31 (0.31)**
24
25 Qvina **0.39 (0.36)**
26
27 Vina **0.40 (0.37)**
28
*p<0.05, **p<0.01
29
30
Spearman rank correlation
31 coefficients were given in
32 parentheses.
33
34
35 Scoring and ranking powers of docking programs on the largest cluster composed of 217 Zn2+
36
37 harboring carbonic anhydrase enzymes were also assessed (Table 3). Strikingly, all programs
38 showed higher scoring and ranking powers in this set. Particularly, ADZn and LeDock had the
39
40 highest scoring powers, while LeDock was also successful in ranking. Slightly less than QVina
41
and Vina, PLANTS and DOCK6 had similar level of scoring and ranking power while AutoDock
42
43 was the least successful program.
44
45 The duration of re-docking of the entire non-redundant set by each program was measured (Table
46
47 4). All dockings were run on the same machine equipped with an 8-core Intel 6700HQ processor
48
clocked at 2.6 GHz, 12 GB RAM running on Lubuntu linux system. PLANTS, QVina and LeDock
49
50 were the fastest programs while docking of the entire set was much slower with the rest of the
51
52 tools. Particularly, AutoDock and ADZn were the slowest programs. No failed docking runs were
53 noted.
54
55
56
57
58
59 11
1
2
3 Table 4. Duration of docking for the entire non-redundant set.
4
5 hr:min:sec
6
7
AutoDock 27:15:39
8 ADZN 26:33:02
9
10 DOCK6 14:26:26
11
12 LeDock 5:47:26
13
PLANTS 3:41:53
14
15 QVina 4:19:34
16
17 Vina 12:23:56
18
19
20 Assessment of Posing. Posing power which shows the capacity of a docking program to
21
22 correctly find the native binding pose (<2 Å RMSD from the crystal pose) was assessed (Fig. 4).
23
PLANTS and LeDock stood out among other docking programs to accurately predict 51% of the
24
25 crystal poses as the lowest energy pose. Secondly, Vina and QVina have the prediction accuracy
26
27 of 43% and 40% respectively. These tools were followed by DOCK6 (38%) while AutoDock and
28 ADZn performed the worst with the prediction accuracies of 23% and %18 respectively (Fig. 4A).
29
30 When the lowest RMSD pose was analyzed, the posing power of the docking tools was increased
31
(Fig. 4B). For instance, Qvina and Vina showed ~33% increase in the posing accuracy, resulting
32
33 in 76% and 73% posing accuracies respectively. Similarly, the LeDock, PLANTS and AutoDock
34
35 had around 25-29% of higher success rates when the best pose was considered. Strikingly,
36 PLANTS showed the highest success rate (80%) which surpassed the LeDock’s rate (77%) when
37
38 the lowest RMSD poses were analyzed. Despite its moderate performance with the lowest energy
39 poses, DOCK6’s success rate was increased the least (18%) compared with the other docking
40
41 programs. Posing power of AutoDock (48%) and ADZn (46%) were reported to be the least
42
successful regardless of the lowest energy or lowest RMSD poses.
43
44
45 When posing accuracy was analyzed according to metal types; LeDock, PLANTS or Vina was
46 found to be the most accurate program for the top score pose assessment (Fig. 5A). For Ca2+ and
47
48 Mn2+ containing subsets, Vina (71%) and PLANTS (67%) accurately predicted the crystal poses.
49
For Mg2+ and Zn2+ containing subsets, LeDock produced the highest success rates of 59% and
50
51 52% respectively. AutoDock and ADZn failed to reach the posing success marked by the 5 other
52
53 programs. For the lowest RMSD pose; PLANTS, LeDock, Vina and QVina were distinguished
54 amongst others by their relatively higher success than DOCK6, AutoDock and ADZn (Fig. 5B). For
55
56 instance, PLANTS reached 92% (22/24) success rate for the Mn2+ containing complexes.
57
58
59 12
1
2
3 Similarly, LeDock was the most successful program with the accuracy of 88% (21/24) for the Ca2+
4
5 containing complexes. For the Mg2+ subset, PLANTS and Vina had the highest success rate of
6 74% (50/68) and lastly for Zn2+ subset again PLANTS was the most successful by predicting 81%
7
8 (79/97) of the crystal poses as the lowest RMSD pose. On the other hand, AutoDock and ADZn
9
were consistently the least successful program having the lowest success rates for every metal
10
11 ion regardless of the assessment of lowest energy (Fig. 5A) or lowest RMSD poses (Fig. 5B).
12
13 When the best pose was considered, AutoDock’s success rate was found to be 54% (13/24) for
14 Ca2+, 40% (27/68) for Mg2+, 54% (13/24) for Mn2+ and 52% (50/97) for Zn2+ subset. DOCK6 was
15
16 the third least successful program particularly for posing of the ligands from the Mg2+ and Mn2+
17
containing subsets.
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40 Fig. 4. Pose prediction of 7 docking programs for the non-redundant set. The cumulative
41 proportions of RMSD measurements for (A) the top-scored (lowest energy) and (B) best poses
42 (lowest RMSD) were shown. The poses that were considered accurate were marked with the
43
44
RMSD = 2 Å line.
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59 13
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54 Fig. 5. Pose prediction for each metal ion containing subset for (A) the top-scored and (B) best
55 poses.
56
57
58
59 14
1
2
3 Posing power was also assessed by analyzing the metal coordination site (Fig. 6). For both lowest
4
5 energy and RMSD poses; PLANTS, LeDock and DOCK6 were the most successful programs that
6 most accurately predicted the closest 3 ligand atoms in the metal coordination site (Fig. 6A).
7
8 Particularly when the lowest RMSD pose considered, LeDock achieved high, medium and low
9
success rates of 28.0, 55.4 and 75.2% while PLANTS obtained higher rates of 33.2, 75.7 and
10
11 92.1%. On the other hand, high-level success rates of AutoDock, ADZn, Vina and QVina were
12
13 lower than 10% regardless of the analyzed pose. When metals were individually analyzed, posing
14 power of PLANTS and LeDock was selective toward Ca2+ complexes, while AutoDock, Vina and
15
16 QVina gave higher success rate for Mn2+ complexes (Fig. 6B).
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53 Fig. 6. Posing accuracy of the metal coordination. (A) High, medium and low success rates were
54 shown for (A) the top-scored (lowest energy) and (B) best poses (lowest RMSD).
55
56
57
58
59 15
1
2
3 Assessment of posing in the absence of metals showed that only the accuracy of LeDock and
4
5 AutoDock was slightly affected by metal removal when the lowest energy pose was considered
6 while others were not affected (Fig. S3). However, when the lowest RMSD pose was considered,
7
8 metal removal did not have any impact on the posing powers of the programs. When the posing
9
power was assessed on the carbonic anhydrase set based on the lowest energy pose, posing
10
11 accuracy of all programs except from AutoDock and ADZn was decreased compared with their
12
13 powers in the non-redundant set (Fig. S4). When the lowest RMSD pose was considered, this
14 difference was less noticeable. Taken together these results propose that PLANTS consistently
15
16 produced the highest posing power regardless of the analyzed pose, dataset or metal presence
17
(Figs. S3-S4).
18
19
20 Table 5. Screening power.
21 Ca2+ Mg2+ Mn2+ Zn2+
22
23 AUC 0.66 ± 0.09 0.79 ± 0.10 0.74 ± 0.04 0.79 ± 0.07
24 AutoDock
25 EF1% 3.39 ± 2.66 4.48 ± 5.38 10.18 ± 17.63 6.03 ± 1.07
26
27 AUC 0.66 ± 0.10 0.78 ± 0.10 0.76 ± 0.73 0.76 ± 0.09
ADZn
28 EF1% 1.59 ± 2.75 6.27 ± 6.41 10.18 ± 17.64 7.11 ± 10.53
29
30 AUC 0.63 ± 0.06 0.80 ± 0.05 0.70 ± 0.02 0.83 ± 0.03
31 DOCK6
32 EF1% 8.91 ± 3.64 2.11 ± 1.84 2.26 ± 3.91 17.75 ± 3.02
33
34
AUC 0.66 ± 0.09 0.76 ± 0.10 0.72 ± 0.11 0.83 ± 0.03
LeDock
35 EF1% 3.13 ± 2.43 3.13 ± 3.00 1.13 ± 1.96 2.83 ± 2.60
36
37 AUC 0.72 ± 0.14 0.85 ± 0.07 0.82 ± 0.02 0.87 ± 0.06
38 PLANTS
39 EF1% 13.56 ± 6.58 21.94 ± 15.31 15.53 ± 11.28 16.32 ± 7.59
40
AUC 0.56 ± 0.15 0.76 ± 0.10 0.61 ± 0.09 0.84 ± 0.08
41 Qvina
42 EF1% 3.61 ± 4.42 5.46 ± 7.01 1.69 ± 2.93 14.37 ± 6.99
43
44 AUC 0.56 ± 0.16 0.76 ± 0.10 0.62 ± 0.10 0.84 ± 0.08
45 Vina
46 EF1% 3.62 ± 4.43 5.54 ± 7.13 1.7 ± 2.94 14.79 ± 7.42
47
AUC and EF values were given as “mean±SD” of 3 different targets for each metal.
48
49
None of the comparisons were significant (p>0.05)
50
51
52 Assessment of Screening. Screening power was assessed by the ability of the docking
53
54 programs to distinguish an active ligand from an inactive (decoy) ligand(s). Decoy ligands should
55 possess similar physiochemical properties with the active ligand and yet dissimilar 2D and 3D
56
57
58
59 16
1
2
3 topologies.49, 50 Here the DUD-E server was exhaustively scanned and only 4 metalloprotein
4
5 targets were identified to be already deposited in the server, meeting the <4 Å distance criteria.
6 For the remaining targets, the active-decoys sets were manually generated by DUD-E which
7
8 produces 50 decoys for a given active ligand. In order to have at least 1000 decoy ligands for a
9
given target metalloprotein, the remaining targets were selected (pla2g2e, spla2x, folk, glcb,
10
11 kdm5a, pH1N1, map, and lkha4) to contain at least 20 active inhibitors listed in the PubChem.
12
13 Further these metalloprotein targets were confirmed to contain the metal(s) that are not separated
14 more than 4 Å from the ligand. This way ensured that the metalloprotein targets selected from the
15
16 PDBbind or DUD-E or PDB contain metal-ligand interactions. Ultimately, the targets and the
17
active-decoy ligand compositions (Table S2) were screened by all docking programs. The
18
19 screening power was assessed by the analyses of ROC curve and ligand enrichment. The mean
20
21 values for the area under ROC curve (AUC) and enrichment factors were listed in the Tables 5
22 and S3 and the mean ROC curves were shown in the Fig. 7.
23
24
None of the AUC or EF values for a given program are statistically significant (p>0.05). On
25
26 average, PLANTS had the highest AUC values for every metal (Table 5). Further PLANTS
27
28 consistently scored the active ligands with more negative values than those of the decoy ligands
29 (Fig. S5). On the other hand, the rest of the 6 docking programs showed EF values ranging from
30
31 1.13-to-10.18 for Ca2+, Mg2+ and Mn2+ containing subsets. For Zn2+ containing targets, the
32 screening powers of DOCK6, Vina, QVina, ADZn and PLANTS were almost comparable (Table 5).
33
34 AutoDock, although its posing power was not satisfactory, had a moderate screening power on
35
the Mn2+ and Zn2+ containing targets (Table 5 and Fig 7). Contrary to this observation, LeDock
36
37 despite being promising for posing (Figs. 4-6), it failed to enrich the active ligands in the top
38
39 rankings (Table 5 and Fig. 7). Lastly, in line with the correlation analysis of the QVina and Vina
40 docking scores (Table 1), these two tools had very similar screening performance such that even
41
42 their score distributions are very similar (Table 5).
43
44 DISCUSSION
45
46
47 Versatile metals are often encountered in the active sites and/or binding pockets of proteins.
48 Despite their significant contributions to the protein-ligand interactions, during docking the user
49
50 may underestimate the importance of the metals in the binding pocket and can even neglect
51 metals in the binding pockets. However, metals not only greatly contribute to the binding and/or
52
53 catalysis but also should be carefully handled as they form distinct bonds which cannot be treated
54
55
neither covalently nor non-covalently. To date, despite the extensive use of versatile docking
56 programs for metalloprotein receptors,55-59 there are not any studies properly addressing the
57
58
59 17
1
2
3 performance difference of versatile docking programs on a diverse metalloprotein set. Specifically,
4
5 the examples of such studies include either assessment docking program(s) on single type of
6 metalloprotein receptors55-57, 59 or assessment of a single docking program on a diverse
7
8 metalloenzyme set.58 Here in this study we present a comprehensive assessment of scoring,
9
ranking, posing and screening powers of 7 freely available docking programs on the diverse non-
10
11 redundant metalloprotein subset of the refined PDBbind (2017).
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37 Fig. 7. Average ROC curve was calculated for each metal ion from 3 different active-decoy
38 ligand sets and represented by red lines.
39
40
41
42 Rationale behind the selection of docking programs. AutoDock and DOCK are among the
43 most widely cited and used docking programs.5, 60 Although these tools deserve laudation for their
44
45 contributions to drug discovery studies,6, 61 it is important to note that sometimes the user may
46
randomly choose them due to high number of citations, whilst high citation can be a mere result
47
48 of their early discovery. Another program selected for this study Vina is also widely used owing to
49
50 its particularly higher speed and accuracy than AutoDock.40 While QVina, a relatively young
51 docking program, was derived from Vina and reported to further enhance the speed and accuracy
52
53 of Vina upon being tested on a 195 protein-ligand set38 and thus it was also selected for this study.
54 Higher speed of Qvina was further confirmed here as it was almost 3 times faster than Vina (Table
55
56 4). Furthermore, QVina was shown to be more accurate than GOLD, but slightly less accurate
57
58
59 18
1
2
3 than DOCK6.38 Among the selected programs, LeDock and PLANTS contrary to AutoDock and
4
5 DOCK6, are less frequently cited, probably due to their relatively young age. Nevertheless, recent
6 studies have underscored the particular success of LeDock, in scoring, posing and screening of
7
8 novel inhibitors.18, 62, 63 Similarly, PLANTS was also shown to accurately predict the poses 298
9
complexes with a higher accuracy rate than GOLD.37 Hence, despite being less frequently used
10
11 by the community than AutoDock and DOCK6, we valued these two programs to be assessed
12
13 owing to their recently promoted successes. Overall, the selected docking programs in this
14 assessment study represent a spectrum of docking programs which are freely available to
15
16 academic users and carry a higher chance to be randomly chosen by particularly first-time
17
modelers than their costly commercial alternatives.
18
19
20 Compilation and Quality of the Non-redundant Metalloprotein Set. Comparative assessment
21 studies provide useful information to the modelers.16, 53, 64-67 Nevertheless, unless certain
22
23 standards were met, their guidance could be counterproductive. Quality of the test sets is
24
undoubtedly one of the critical standards playing a significant role in the outcome of comparative
25
26 assessment studies.66 Essentially when a certain type of receptor was unfairly populated in the
27
28 set, the overall conclusions might be inclined towards that receptor type. Hence, the redundancy
29 in the sets should be well-addressed and avoided prior to reach any conclusions.53, 66, 68
30
31 To ensure a reliable statistical analysis, the redundancy in the metalloprotein subset of the refined
32
33 PDBbind was eliminated, resulting in a 213 membered non-redundant set (Table S1) which was
34
35
used to draw conclusions about the performance of the docking programs. Although none of the
36 docking programs was successful in neither scoring nor ranking of the non-redundant
37
38 metalloprotein set (Table 1), the scoring and ranking powers of all programs were notably
39 enhanced for the carbonic anhydrase set (Table 2). Intriguingly, although both LeDock and ADZn
40
41 was significantly successful in the carbonic anhydrase set (Table 3), their performance was worse
42
for the diverse Zn2+ containing subset (Table 1). These discrepancies in the performances of
43
44 docking programs between single-receptor and diverse non-redundant sets indeed underline a
45
46 tendency of LeDock and ADZn to better score carbonic anhydrase inhibitors. To avoid inconsistent
47 conclusions about the performance of docking programs, we underline the necessity of compiling
48
49 non-redundant sets for comparative evaluation studies.
50
51 The non-redundant set constructed here could be of interest for those who aim to assess the
52
53 performance of other structure-based tools on metalloproteins. Nevertheless, certain
54 improvements can be still made to further increase the quality of this set. First, metal types can
55
56 be increased by inclusion of other abundant metals in biomolecules such as Fe2+ and Co2+.
57
58
59 19
1
2
3 Evidently, inclusion of metalloproteins with diverse metal types would assist in a better
4
5 representation of the human metalloproteome.69 Second, the non-redundant set was compiled by
6 representing a single member of each of the 213 cluster (Table S1). According to the rules of the
7
8 “Comparative Assessment of Scoring Functions” (CASF) the clusters with less than 5 members
9
are eliminated while 3 representatives from each cluster are selected to span a wide-affinity
10
11 range.53, 66 However, clustering of the metalloprotein subset of the refined PDBbind indicated that
12
13 only 22 clusters out of 213 have 5 or more members while a large fraction of the clusters (191)
14 has less than 5 members. Hence, whilst compiling this non-redundant metalloprotein set, if we
15
16 complied with this CASF rule, our test set would comprise only 22 different metalloproteins. In
17
order to ensure a high level of metalloprotein diversity in the test set (Fig. 1B), all of the clusters
18
19 were represented by a single member leading to a total of 213 distinct metalloproteins (Table S1).
20
21 From Most-To-Least Successful Docking Programs for Metalloprotein Targets. Based on
22
23 our comparative assessment of 7 docking programs, we could not find a simple solution to
24
metalloprotein docking. Essentially, none of the programs were successful in all of the four criteria
25
26 i.e. scoring, ranking, posing and screening assessed here. However, we note that certain
27
28 programs are particularly powerful for a given criterion and/or for a given metal type.
29
30 Notwithstanding the failure of all programs in scoring and ranking of the re-docked set (Tables 1-
31 2, Fig. 3), PLANTS despite its young age is acknowledged as being the most successful program
32
33 for posing (Figs. 4-6) and the fastest program of this assessment study (Table 4). Particularly,
34
35
PLANTS produced the highest posing power regardless of the analyzed pose, lowest energy or
36 RMSD; dataset, non-redundant or carbonic anhydrase; or metal presence. This program uses a
37
38 special optimization algorithm to address the ligand binding problem in which they apply an ant
39 colony optimization inspired by the behaviors of ants finding the shortest route from nest to food
40
41 source.36 To improve the scoring power of PLANTS, its chemplp scoring function which was tested
42
here, can be optimized or replaced by more accurate scoring functions.35 Nevertheless, accurate
43
44 scoring of metal-ligand interactions can be a challenging task, simple non-covalent scoring
45
46 functions were previously reported to have screening power for metalloprotein inhibitors.58 In line
47 with this paradigm, we note here that PLANTS produced the highest screening power for every
48
49 metal except from Zn2+ (Table 5) which is another important metric to assess the docking programs
50 is to test them against a decoy ligand database.50 Nevertheless the screening results were not
51
52 statistically significant screening, we note that the raw EF values indicate that every program
53
54
except from PLANTS were failed to enrich the active ligands for at least one metalloprotein target
55 (Table S3).
56
57
58
59 20
1
2
3 Similar to the PLANTS’ case, LeDock another young program was particularly successful in posing
4
5 (Figs. 4-6) and slightly in scoring/ranking (Tables 1-2), and it performed the third fastest docking
6 run (Table 4). Parallel to this outcome, among other well-known programs previously LeDock’s
7
8 success was reported for general purpose test sets.18, 70 However, LeDock failed to re-produce its
9
pose prediction power for screening (Table 5). Despite it can successfully predict the binding pose
10
11 (and slightly the scores) of the metalloprotein inhibitors, it should be approached cautiously for hit
12
13 identification.
14
15 Followed by PLANTS and LeDock, DOCK6 was the most successful third program in posing
16
particularly for Ca2+ containing subset (Figs. 4-6) and enriched the active inhibitors of particularly
17
18 Zn2+ and Ca2+ containing subsets (Table 5). QVina and Vina showed a significant scoring ability
19
20 of the Ca2+ containing subset (Table 1) while Vina and QVina were moderately performed in pose
21 prediction and screening. The most notable point for Vina and QVina is that they produced parallel
22
23 results even in the numerical values of the scoring and screening powers (Tables 1 and Fig. 6).
24
These parallel results were highly likely to be due to the fact that both of the programs used the
25
26 same scoring scheme. However, given the higher speed of QVina than Vina (Table 4), it can be
27
28 alternatively used for Vina essentially when a large dataset was considered.
29
30 Among 7 programs, AutoDock was the least successful program for almost all criteria except from
31 screening. Despite its poor power for scoring, ranking and posing, it was not the least successful
32
33 program in screening (Fig. 7 and Table 5). One of the important finding is the assessment of ADZn
34
35
which has a special scoring function for Zn2+-ligand interactions.32 As anticipated, compared with
36 AutoDock, ADZn was successful particularly in scoring of the Zn2+ containing subset. However, our
37
38 assessment did not report any advantage of using this scoring function for the diverse non-
39 redundant metalloprotein set, especially considering the fact that AutoDock and ADZn were the
40
41 slowest programs tested here (Table 4).
42
43 The impact of the metal presence on the success of docking was also assessed and we observed
44
45 that DOCK6 is the only program whose scoring success is dependent on the metal presence,
46 while none of the other programs’ performance was negatively affected by metal removal (Fig.
47
48 S2). On the contrary, the scoring performance of AutoDock was increased when metals were
49
removed (Fig. S2), compared with the condition when metals were kept. Overall these results
50
51 highlight the sensitivity of the scoring function of DOCK6 to the metal presence.
52
53 When the crystal poses were directly re-scored by the scoring functions, we observed only slight
54
55 increases in the performances of scoring functions of AutoDock, ADZn and Vina (Table 2).
56
57
58
59 21
1
2
3 Essentially, their re-scoring performance were comparable with that of Δvina which was ranked
4
5 the highest among 25 scoring functions tested in the CASF-16 study.53 Although only slight
6 increases in the performances of scoring functions were observed, we stress that a short
7
8 minimization of the complexes prior to re-scoring might ameliorate the re-scoring power.
9
10 Posing assessments showed that all programs consistently had increased their posing accuracy
11
12 when the best pose was evaluated (Figs. 4-6). Particularly for the Zn2+ and Mn2+ containing
13 subsets, the prediction accuracies of the top-scored pose assessments were almost doubled for
14
15 the best pose assessments (Fig. 5). These observations basically reinforce the significance of
16
inspecting all of the docked poses rather than directly selecting the top scored pose which may
17
18 not necessarily represent the native binding conformation.
19
20 Given the fewer number of Ca2+ and Mn2+ containing complexes (24) than Zn2+ set (97), whether
21
22 there was any biased scoring or posing power of the docking programs with respect to metal types
23
was checked. But no significant differences in the performances according to the size of the
24
25 metalloprotein subset were noted. As such, the scoring powers of LeDock, PLANTS, QVina and
26
27 Vina was the highest for the Ca2+ containing subset, while they had the lowest power for the other
28 smaller subset of Mn2+ (Table 1), suggesting that scoring power is not affected by the size of
29
30 dataset. Similarly, posing power was not reported to be affected by the size of dataset. (Fig. 5).
31 We note that QVina and Vina tend to give similar results because they use the same scoring
32
33 function. Otherwise, no other explanations were made for such inclined results toward better
34
35
correlation for Ca2+ and worse for Mn2+ subsets.
36
37 Critical Considerations During Metalloprotein Docking. A typical docking run for a given
38 protein-ligand pair is realized in two steps (i) sampling of the ligand conformations, (ii) scoring of
39
40 the conformations according to the formed protein-ligand interactions. For these two steps, (i)
41
42
posing and (ii) scoring functions are implemented. During development of docking, these functions
43 are typically optimized to reproduce the experimental binding pose, but not the experimental
44
45 affinities. Therefore, a strong correlation between docking scores and experimental affinities may
46 not be necessarily observed for every docking run.50, 71, 72 In any case, docking programs are
47
48 generally assessed by their scoring and ranking powers. Metalloenzymes that were previously
49
targeted by structure-based approaches were often analyzed by commonly used docking
50
51 programs.58, 59 However, the scoring functions of such common docking programs do not ensure
52
53 accurate computation of metal-ligand interactions. The poor scoring and ranking powers of the
54 tested programs are likely to arise from their scoring approaches, failing to accurately address
55
56 metal-ligand interaction energies. For example, the scoring function of AutoDock which is a
57
58
59 22
1
2
3 physics-based semi-empirical function indeed addresses ligand and metal interactions.73 But the
4
5 metal-ligand interactions were treated as any other pairwise interactions which are described
6 mainly van der Waals and Coulomb electrostatic and to a lesser degree desolvation terms.32 Due
7
8 to certain characteristics of the metal bond, these terms may fail to accurately determine the
9
binding potential of metal interactions. One example may be the distance of the Lennard-Jones
10
11 potential which would be higher than the metal coordination distance, and as a result this potential
12
13 may be underestimated by this scoring function. Given these limitations in the scoring function,
14 AutoDock developers have recently introduced a special function for Zn2+ containing
15
16 metalloproteins, ADZn which more accurately accounts for the Zn2+-ligand interactions.32 Together
17
with ADZn, there are also other efforts to establish a general scoring and docking scheme for
18
19 metalloproteins.74 Notwithstanding these more accurate and specialized scoring functions for
20
21 metalloproteins, the selected 7 programs still carry a high chance to be randomly selected for a
22 given metalloprotein docking. Furthermore, there are a number of reports for the virtual screening
23
24 successes of simple non-covalent scoring functions in the identification of metalloprotein
25 inhibitors.58 Hence, although the implementation of scoring functions which accurately address
26
27 metal-ligand interactions would be the ideal condition for metalloproteins, general scoring
28
functions can still be opted due to their frequent usage and promoted success. From this
29
30 perspective, this study providing a comprehensive assessment of the relative performances of
31
32 common docking approaches is relevant for understanding of the advantages and limitations of
33 common scoring approaches for metalloprotein docking.
34
35
One of the important parameters for metalloprotein docking is evidently the parameters that
36
37 account for metal-ligand interactions in the scoring functions. Because the docking tools tested
38
39 here use different default parameters for metal ions, we did not have any chance to evaluate these
40 factors. However, this limitation partly exists because of the fact that the scoring function of
41
42 LeDock cannot be modified and could be overcome by evaluating the performance of each
43 docking program by using the same metal parameters. According to the tested metal parameters
44
45 here, the same formal charge of +2e was used for zinc for AutoDock, DOCK6 and PLANTS.
46
47
Previously formal charge of +2e was applied to zinc for docking simulations of AutoDock and
48 DOCK356 Although the use of same formal charge is an advantage for reliable assessment of
49
50 these 3 programs, we stress that the formal charge of +2e for zinc might also need an optimization.
51 Particularly, the use of optimized zinc charge of +0.95e led to an improved posing and scoring
52
53 accuracies of AutoDock3 for matrix metalloproteinase inhibitors in a previous study.57 Overall,
54
recognizing the cases in which optimization of metal charges ameliorated the docking
55
56 performance for Zn2+-containing metalloenzymes,57 we note that the application of the default
57
58
59 23
1
2
3 parameters for all metal ions might be related to the random affinity correlations. Taken together
4
5 with the use of different metal parameters in scoring functions, we underscore that extensive
6 optimization of the metal parameters might be necessary for each program to reach its highest
7
8 performance. Lastly, the use of polarized force fields, although they may critically slowdown
9
docking calculations, could better address metal-ligand interactions.75, 76
10
11
12 CONCLUSIONS
13
14 A docking program should ideally address the needs of a modeler/biochemist which ranges from
15
16 predicting the binding pose and/or marking the key binding interactions for a given protein-ligand
17 pair or identification of novel drug hits for a given target. But, the capabilities of current approaches
18
19 may not be equal and fall behind from the ideal. Thus, comparative assessment studies would
20 have a lot to offer, especially in the overall understanding of the limitations and also the
21
22 competency of the docking programs. This study reporting the comprehensive assessment of
23
docking programs for scoring, posing and screening of metalloprotein inhibitors would aid in the
24
25 metalloprotein research by guiding and alerting the docking user. Owing to its success both in
26
27 pose prediction and virtual screening we stress that PLANTS could be the choice of docking when
28 a metalloprotein docking study is concerned except from one aiming to predict experimental
29
30 binding affinities.
31
32 ASSOCIATED CONTENT
33
34 The Supporting Information is available free of charge on the ACS Publications website: 4 figures
35
36 for distribution of molecular properties of ligand in the metalloprotein subset of the PDBbind
37
refined, scatter plots for scoring power in the absence of metals, pose prediction accuracy in the
38
39 absence of metals and pose prediction accuracy for the largest cluster composed of 217 carbonic
40
41 anhydrase sequences. Docking scores of active (green) and decoy (red) ligands obtained for
42 virtual screening assessments. 3 tables for the cluster list, decoy ligand sets for screening
43
44 assessment and raw AUC, EF values against 12 different decoy ligand sets.
45
46 AUTHOR INFORMATION
47
48 Corresponding Authors
49
50 E-mail: suleyman.selim@mail.com
51
E-mail: emel.timucin@acibadem.edu.tr
52
53 ORCID
54
55 Süleyman Selim Çınaroğlu: 0000-0001-7120-3540
56 Emel Timuçin: 0000-0003-0048-0668
57
58
59 24
1
2
3 Notes
4
5 The authors declare no competing financial interest.
6
7 ACKNOWLEDGEMENT
8
9
10 Authors thankfully acknowledge the support from TUBITAK ULAKBIM, High Performance and
11 Grid Computing Center (TRUBA resources) for the computational resources.
12
13
REFERENCES
14
15 1 Ferreira, L.; dos Santos, R.; Oliva, G.; Andricopulo, A., Molecular Docking and Structure-
16
Based Drug Design Strategies. Molecules 2015, 20, 13384-13421.
17
18 2 Jorgensen, W. L., Science 2004, 303, 1813-1818.
19 3 Kuntz, I. D., Structure-Based Strategies for Drug Design and Discovery. Science 1992,
20 257, 1078-1082.
21 4 Chang, M. W.; Ayeni, C.; Breuer, S.; Torbett, B. E., Virtual Screening for Hiv Protease
22
23 Inhibitors: A Comparison of Autodock 4 and Vina. PLoS ONE 2010, 5, e11955.
24 5 Park, M. S.; Dessal, A. L.; Smrcka, A. V.; Stern, H. A., Evaluating Docking Methods for
25 Prediction of Binding Affinities of Small Molecules to the G Protein Βγ Subunits. J. Chem.
26 Inf. Model. 2009, 49, 437-443.
27
6 Schames, J. R.; Henchman, R. H.; Siegel, J. S.; Sotriffer, C. A.; Ni, H.; McCammon, J. A.,
28
29 Discovery of a Novel Binding Trench in Hiv Integrase. J. Med. Chem. 2004, 47, 1879-1881.
30 7 Chen, Y. C., Beware of Docking! Trends Pharmacol. Sci. 2015, 36, 78-95.
31 8 Waldron, K. J.; Rutherford, J. C.; Ford, D.; Robinson, N. J., Nature 2009, 460, 823-830.
32 9 Yannone, S. M.; Hartung, S.; Menon, A. L.; Adams, M. W. W.; Tainer, J. A., Curr. Opin.
33
Biotechnol. 2012, 23, 89-95.
34
35 10 Brown, D. R., Metallomics 2010, 2, 186-194.
36 11 Martin, D. P.; Puerta, D. T.; Cohen, S. M., Metalloprotein inhibitors. Ligand Des. Med.
37 Inorg. Chem. 2014, 375.
38 12 Roberts, B. R.; Ryan, T. M.; Bush, A. I.; Masters, C. L.; Duce, J. A., The role of
39
40
metallobiology and amyloid‐β peptides in Alzheimer's disease. J. Neurochem. 2012, 120,
41 149-166.
42 13 Bursulaya, B. D.; Totrov, M.; Abagyan, R.; Brooks, C. L., Comparative Study of Several
43 Algorithms for Flexible Ligand Docking. J. Comput.-Aided Mol. Des. 2003, 17, 755-763.
44
14 Cheng, T.; Li, X.; Li, Y.; Liu, Z.; Wang, R., Comparative Assessment of Scoring Functions
45
46 on a Diverse Test Set. J. Chem. Inf. Model. 2009, 49, 1079-1093.
47 15 Gaillard, T., Evaluation of Autodock and Autodock Vina on the Casf-2013 Benchmark. J.
48 Chem. Inf. Model. 2018, 58, 1697-1706.
49 16 Kontoyianni, M.; McClellan, L. M.; Sokol, G. S., Evaluation of Docking Performance:
50
Comparative Data on Docking Algorithms. J. Med. Chem. 2004, 47, 558-565.
51
52 17 Wang, R.; Lu, Y.; Wang, S., Comparative Evaluation of 11 Scoring Functions for Molecular
53 Docking. J. Med. Chem. 2003, 46, 2287-2303.
54 18 Wang, Z.; Sun, H.; Yao, X.; Li, D.; Xu, L.; Li, Y.; Tian, S.; Hou, T., Comprehensive
55 Evaluation of Ten Docking Programs on a Diverse Set of Protein–Ligand Complexes: The
56
57
58
59 25
1
2
3 Prediction Accuracy of Sampling Power and Scoring Power. Phys. Chem. Chem. Phys.
4 2016, 18, 12964-12975.
5
6 19 Wang, R.; Fang, X.; Lu, Y.; Wang, S., The Pdbbind Database: Collection of Binding
7 Affinities for Protein-Ligand Complexes with Known Three-Dimensional Structures. J. Med.
8 Chem. 2004, 12, 2977-2980.
9 20 Wang, R.; Fang, X.; Lu, Y.; Yang, C. Y.; Wang, S., The Pdbbind Database: Methodologies
10
and Updates. J. Med. Chem. 2005, 12, 4111-4119.
11
12 21 Huang, Y.; Niu, B.; Gao, Y.; Fu, L.; Li, W., Cd-Hit Suite: A Web Server for Clustering and
13 Comparing Biological Sequences. Bioinformatics 2010, 5, 680-682.
14 22 Mi, H.; Muruganujan, A.; Thomas, P. D., Panther in 2013: Modeling the Evolution of Gene
15 Function, and Other Gene Attributes, in the Context of Phylogenetic Trees. Nucleic Acids
16
Res. 2012, 41, D377-D386.
17
18 23 Thomas, P. D., Panther: A Library of Protein Families and Subfamilies Indexed by
19 Function. Genome Res. 2003, 13, 2129-2141.
20 24 Pettersen, E. F.; Goddard, T. D.; Huang, C. C.; Couch, G. S.; Greenblatt, D. M.; Meng, E.
21 C.; Ferrin, T. E., Ucsf Chimera - a Visualization System for Exploratory Research and
22
23 Analysis. J. Comput. Chem. 2004, 25, 1605-1612.
24 25 Maier, J. A.; Martinez, C.; Kasavajhala, K.; Wickstrom, L.; Hauser, K. E.; Simmerling, C.,
25 Ff14sb: Improving the Accuracy of Protein Side Chain and Backbone Parameters from
26 Ff99sb. J. Chem. Theory Comput. 2015, 8, 3696-3713.
27
26 O'Boyle, N. M.; Banck, M.; James, C. A.; Morley, C.; Vandermeersch, T.; Hutchison, G.
28
29 R., Open Babel: An Open Chemical Toolbox. J. Cheminf. 2011, 1, 33.
30 27 Halgren, T. A., Merck Molecular Force Field. I. Basis, Form, Scope, Parameterization, and
31 Performance of Mmff94. J. Comput. Chem. 1996, 17, 490-519.
32 28 Jakalian, A.; Bush, B. L.; Jack, D. B.; Bayly, C. I., Fast, Efficient Generation of High-Quality
33
Atomic Charges. Am1-Bcc Model: I. Method. J. Comput. Chem. 2000, 2, 132-146.
34
35 29 Jakalian, A.; Jack, D. B.; Bayly, C. I., Fast, Efficient Generation of High-Quality Atomic
36 Charges. Am1-Bcc Model: Ii. Parameterization and Validation. J. Comput. Chem. 2002,
37 16, 1623-1641.
38 30 Wang, J.; Wang, W.; Kollman, P. a.; Case, D. a., Antechamber, an Accessory Software
39
40
Package for Molecular Mechanical Calculations. J. Am. Chem. Soc 2001, 222, U403.
41 31 Morris, G.; Huey, R., Autodock4 and Autodocktools4: Automated Docking with Selective
42 Receptor Flexibility. J. Comput. Chem. 2009, 30, 2785-2791.
43 32 Santos-Martins, D.; Forli, S.; Ramos, M. J.; Olson, A. J., Autodock4zn: An Improved
44 Autodock Force Field for Small-Molecule Docking to Zinc Metalloproteins. J. Chem. Inf.
45
46 Model. 2014, 8, 2371-2379.
47 33 Allen, W. J.; Balius, T. E.; Mukherjee, S.; Brozell, S. R.; Moustakas, D. T.; Lang, P. T.;
48 Case, D. A.; Kuntz, I. D.; Rizzo, R. C., Dock 6: Impact of New Features and Current
49 Docking Performance. J. Comput. Chem. 2015, 36, 1132-1156.
50
34 Zhang, N.; Zhao, H., Enriching Screening Libraries with Bioactive Fragment Space. Bioorg.
51
52 Med. Chem. Lett. 2016, 26, 3594-3597.
53 35 Korb, O.; Stützle, T.; Exner, T. E., Plants: Application of Ant Colony Optimization to
54 Structure-Based Drug Design. Notes Comput. Sci. 2006, 4150, 247-258.
55
56
57
58
59 26
1
2
3 36 Korb, O.; Stützle, T.; Exner, T. E., An Ant Colony Optimization Approach to Flexible
4 Protein–Ligand Docking. Swarm Intel. 2007, 1, 115-134.
5
6 37 Korb, O.; Stützle, T.; Exner, T. E., Empirical Scoring Functions for Advanced Protein-
7 Ligand Docking with Plants. J. Chem. Inf. Model. 2009, 49, 84-96.
8 38 Alhossary, A.; Handoko, S. D.; Mu, Y.; Kwoh, C. K., Fast, Accurate, and Reliable Molecular
9 Docking with Quickvina 2. Bioinformatics 2015, 31, 2214-2216.
10
39 Handoko, S. D.; Ouyang, X.; Su, C. T. T.; Kwoh, C. K.; Ong, Y. S., Quickvina: Accelerating
11
12 Autodock Vina Using Gradient-Based Heuristics for Global Optimization. IEEE/ACM
13 Trans. Comput. Biol. Bioinf. 2012, 5, 1266-1272.
14 40 Trott, O.; Olson, A. J., Autodock Vina: Improving the Speed and Accuracy of Docking with
15 a New Scoring Function, Efficientoptimization, and Multithreading. J. Comput. Chem.
16
2010, 31, 455-461.
17
18 41 Gasteiger, J.; Marsili, M., Iterative Partial Equalization of Orbital Electronegativity-a Rapid
19 Access to Atomic Charges. Tetrahedron 1980, 22, 3219-3228.
20 42 Meng, E. C.; Shoichet, B. K.; Kuntz, I. D., Automated Docking with Grid‐Based Energy
21 Evaluation. J. Comput. Chem. 1992, 4, 505-524.
22
23 43 Brink, T. T.; Exner, T. E., Influence of Protonation, Tautomeric, and Stereoisomeric States
24 on Protein-Ligand Docking Results. J. Chem. Inf. Model. 2009, 49, 1535-1546.
25 44 Ten Brink, T.; Exner, T. E., Pkabased Protonation States and Microspecies for Protein-
26 Ligand Docking. J. Comput.-Aided Mol. Des. 2010, 24, 935-942.
27
45 Gehlhaar, D. K.; Verkhivker, G. M.; Rejto, P. A.; Sherman, C. J.; Fogel, D. R.; Fogel, L. J.;
28
29 Freer, S. T., Molecular Recognition of the Inhibitor Ag-1343 by Hiv-1 Protease:
30 Conformationally Flexible Docking by Evolutionary Programming. Chem. Biol. 1995, 5,
31 317-324.
32 46 Verkhivker, G. M., Computational analysis of ligand binding dynamics at the intermolecular
33
hot spots with the aid of simulated tempering and binding free energy calculations. J. Mol.
34
35 Graphics Modell. 2004, 5, 335-348.
36 47 Verdonk, M. L.; Cole, J. C.; Hartshorn, M. J.; Murray, C. W.; Taylor, R. D., Improved
37 Protein-Ligand Docking Using Gold. Proteins: Struct., Funct., Genet. 2003, 4, 609-623.
38 48 Feinstein, W. P.; Brylinski, M., Calculating an Optimal Box Size for Ligand Docking and
39
40 Virtual Screening against Experimental and Predicted Binding Pockets. J. Cheminf. 2015,
41 1, 18.
42 49 Huang, N.; Shoichet, B. K.; Irwin, J. J., Benchmarking Sets for Molecular Docking. J. Med.
43 Chem. 2006, 49, 6789-6801.
44
50 Mysinger, M. M.; Carchia, M.; Irwin, J. J.; Shoichet, B. K., Directory of Useful Decoys,
45
46 Enhanced (Dud-E): Better Ligands and Decoys for Better Benchmarking. J. Med. Chem.
47 2012, 55, 6582-6594.
48 51 Bolton, E. E.; Wang, Y.; Thiessen, P. A.; Bryant, S. H., Chapter 12 Pubchem: Integrated
49 Platform of Small Molecules and Biological Activities. Annu. Rep. Comput. Chem. 2008, 4,
50
217-241.
51
52 52 Kim, S.; Thiessen, P. A.; Bolton, E. E.; Chen, J.; Fu, G.; Gindulyte, A.; Han, L.; He, J.; He,
53 S.; Shoemaker, B. A.; Wang, J.; Yu, B.; Zhang, J.; Bryant, S. H., Pubchem Substance and
54 Compound Databases. Nucleic Acids Res. 2016, 44, D1202-D1213.
55
56
57
58
59 27
1
2
3 53 Su, M.; Du, Y.; Yang, Q.; Wang, R.; Liu, Z.; Feng, G.; Li, Y., Comparative Assessment of
4 Scoring Functions: The Casf-2016 Update. J. Chem. Inf. Model. 2018, 2, 895-913.
5
6 54 Laetti, S.; Niinivehmas, S.; Pentikaeinen, O. T., Rocker: Open Source, Easy-to-Use Tool
7 for Auc and Enrichment Calculations and Roc Visualization. J. Cheminf. 2016, 1, 45.
8 55 Esposito, E. X.; Baran, K.; Kelly, K.; Madura, J. D., Docking of Sulfonamides to Carbonic
9 Anhydrase Ii and Iv. J. Mol. Graphics Modell. 2000, 18, 283-289.
10
56 Hu, X.; Balaz, S.; Shelver, W. H., A Practical Approach to Docking of Zinc
11
12 Metalloproteinase Inhibitors. J. Mol. Graphics Modell. 2004, 22, 293-307.
13 57 Hu, X.; Shelver, W. H., Docking Studies of Matrix Metalloproteinase Inhibitors: Zinc
14 Parameter Optimization to Improve the Binding Free Energy Prediction. J. Mol. Graphics
15 Modell. 2003, 22, 115-126.
16
58 Irwin, J. J.; Raushel, F. M.; Shoichet, B. K., Virtual Screening against Metalloenzymes for
17
18 Inhibitors and Substrates. Biochemistry 2005, 37, 12316-12328.
19 59 Shamsara, J., Evaluation of 11 Scoring Functions Performance on Matrix
20 Metalloproteinases. Int. J. Med. Chem. 2014.
21 60 Sousa, S. F.; Fernandes, P. A.; Ramos, M. J., In Proteins: Struct., Funct., Genet.; 2006;
22
23 Vol. 65, pp 15-26.
24 61 Amaro, R. E.; Schnaufer, A.; Interthal, H.; Hol, W.; Stuart, K. D.; McCammon, J. A.,
25 Discovery of Drug-Like Inhibitors of an Essential Rna-Editing Ligase in Trypanosoma
26 Brucei. Proc. Natl. Acad. Sci. 2008, 105, 17278-17283.
27
62 Çınaroğlu, S. S.; Timuçin, E., In Silico Identification of Inhibitors Targeting N-Terminal
28
29 Domain of Human Replication Protein A. J. Mol. Graphics Modell. 2019, 86, 149-159.
30 63 Er, M.; Abounakhla, A. M.; Tahtaci, H.; Bawah, A. H.; Çınaroğlu, S. S.; Onaran, A.; Ece,
31 A., An Integrated Approach Towards the Development of Novel Antifungal Agents
32 Containing Thiadiazole: Synthesis and a Combined Similarity Search, Homology
33
Modelling, Molecular Dynamics and Molecular Docking Study. Chem. Cent. J. 2018, 12,
34
35 121.
36 64 Kellenberger, E.; Rodrigo, J.; Muller, P.; Rognan, D., Comparative Evaluation of Eight
37 Docking Tools for Docking and Virtual Screening Accuracy. Proteins: Struct., Funct.,
38 Genet. 2004, 57, 225-242.
39
40
65 Li, Y.; Han, L.; Liu, Z.; Wang, R., Comparative Assessment of Scoring Functions on an
41 Updated Benchmark: 2. Evaluation Methods and General Results. J. Chem. Inf. Model.
42 2014, 6, 1717-1736.
43 66 Li, Y.; Liu, Z.; Li, J.; Han, L.; Liu, J.; Zhao, Z.; Wang, R., Comparative Assessment of
44 Scoring Functions on an Updated Benchmark: 1. Compilation of the Test Set. J. Chem.
45
46 Inf. Model. 2014, 6, 1700-1716
47 67 Zhou, Z.; Felts, A. K.; Friesner, R. A.; Levy, R. M., Comparative Performance of Several
48 Flexible Docking Programs and Scoring Functions: Enrichment Studies for a Diverse Set
49 of Pharmaceutically Relevant Targets. J. Chem. Inf. Model. 2007, 4, 1599-1608.
50
68 Li, Y.; Su, M.; Liu, Z.; Li, J.; Liu, J.; Han, L.; Wang, R., Assessing Protein–Ligand Interaction
51
52 Scoring Functions with the Casf-2013 Benchmark. Nat. Protoc. 2018, 13, 666-680.
53 69 Liu, J.; Chakraborty, S.; Hosseinzadeh, P.; Yu, Y.; Tian, S.; Petrik, I.; Bhagi, A.; Lu, Y.,
54 Metalloproteins Containing Cytochrome, Iron–Sulfur, or Copper Redox Centers. Chem.
55 Rev. 2014, 8, 4366-4469.
56
57
58
59 28
1
2
3 70 Çınaroğlu, S. S.; Timuçin, E., Insights into an Alternative Benzofuran Binding Mode and
4 Novel Scaffolds of Polyketide Synthase 13 Inhibitors. J. Mol. Model. 2019, 25, 130.
5
6 71 Enyedy, I. J.; Egan, W. J., Can We Use Docking and Scoring for Hit-to-Lead Optimization?
7 J. Comput.-Aided Mol. Des. 2008, 22, 161-168.
8 72 Gohlke, H.; Klebe, G., In Angew. Chem., Int. Ed.; 2002; Vol. 41, pp 2644-2676.
9 73 Huey, R.; Morris, G. M.; Olson, A. J.; Goodsell, D. S., A Semiempirical Free Energy Force
10
Field with Charge-Based Desolvation. J. Comput. Chem. 2007, 28, 1145-1152.
11
12 74 Bai, F.; Liao, S.; Gu, J.; Jiang, H.; Wang, X.; Li, H., An Accurate Metalloprotein-Specific
13 Scoring Function and Molecular Docking Program Devised by a Dynamic Sampling and
14 Iteration Optimization Strategy. J. Chem. Inf. Model. 2015, 55, 833-847.
15 75 Sakharov, D. V.; Lim, C., Zn Protein Simulations Including Charge Transfer and Local
16
Polarization Effects. J. Am. Chem. Soc. 2005, 127, 4921-4929.
17
18 76 Zhu, T.; Xiao, X.; Ji, C.; Zhang, J. Z. H., A New Quantum Calibrated Force Field for Zinc–
19 Protein Complex. J. Chem. Theory Comput. 2013, 9, 1788-1798.
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59 29
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
ACS Paragon Plus Environment
24 Overall Docking Success Success with Considering Metals
25
26

JCIM Metalprot2019

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

JCIM Metalprot2019

Uploaded by

Copyright:

Available Formats

Subscriber access provided by Macquarie University

is published by the American Chemical Society. 1155 Sixteenth Street N.W.,

You might also like