- Open Access
An integrated method for cancer classification and rule extraction from microarray data
© Huang; licensee BioMed Central Ltd. 2009
- Received: 08 September 2008
- Accepted: 24 February 2009
- Published: 24 February 2009
Different microarray techniques recently have been successfully used to investigate useful information for cancer diagnosis at the gene expression level due to their ability to measure thousands of gene expression levels in a massively parallel way. One important issue is to improve classification performance of microarray data. However, it would be ideal that influential genes and even interpretable rules can be explored at the same time to offer biological insight.
Introducing the concepts of system design in software engineering, this paper has presented an integrated and effective method (named X-AI) for accurate cancer classification and the acquisition of knowledge from DNA microarray data. This method included a feature selector to systematically extract the relative important genes so as to reduce the dimension and retain as much as possible of the class discriminatory information. Next, diagonal quadratic discriminant analysis (DQDA) was combined to classify tumors, and generalized rule induction (GRI) was integrated to establish association rules which can give an understanding of the relationships between cancer classes and related genes.
Two non-redundant datasets of acute leukemia were used to validate the proposed X-AI, showing significantly high accuracy for discriminating different classes. On the other hand, I have presented the abilities of X-AI to extract relevant genes, as well as to develop interpretable rules. Further, a web server has been established for cancer classification and it is freely available at http://bioinformatics.myweb.hinet.net/xai.htm.
- Acute Myeloid Leukemia
- Acute Lymphoblastic Leukemia
- Association Rule
- Rule Extraction
- Microarray Gene Expression Data
The challenge of cancer treatment is to develop specific therapies based on distinct tumor types, to maximize efficacy and minimize toxicity. Hence, improvements in cancer classification have been paid more and more attention. Recently, microarray gene expression data has been successfully used to investigate useful information for cancer classification at the gene expression level. One of the earliest methods for cancer classification is the weighted voting machine which is based on a linear model . Other methods includes hierarchical clustering , machining learning [3, 4], compound covariate , shrunken centroids , partial least square , principal component analysis disjoint models , factor mixture models , consensus analysis of multiple classifiers using non-repetitive variables  etc. On the whole, these methods are mostly concentrated in the improvement of accuracy rather than other issues.
In addition to classification, another challenge is to extract relevant genes, even creditable and interpretable rules from microarray gene expression data to offer biological insight between genes. Several kinds of rules have been successfully developed in different subjects of molecular biology. In our earlier studies, decision rules based on decision tree algorithms have been effectively extracted from the thermodynamic database of proteins and mutants to explore potential knowledge of protein stability prediction [11–13]. On the other hand, association rule techniques can also reveal relevant associations between different items. Borgelt and Berthold  presented an algorithm to find fragments in a set of molecules that help to discriminate between different classes of activity in a drug discovery context. Oyama et al.  proposed a data mining method to discover association rules related to protein-protein interactions. Moreover, association rules which demonstrate diverse mutations and chemical treatments have been reported from 300 gene expression profiles of yeast . Carmona-Saez et al.  have offered an approach which integrates gene annotations and expression data to discover intrinsic associations.
Typically, a classification system may achieve high accuracy by non-linear models, but these models are hard to provide rules. In contrast, a rule extraction system is necessary to consider the model interpretability which can provide a pathway to explore underlying relationships among data; however, this restriction often affects the system performance in classification. Hence, a learning model which can provide accurate classification, as well as useful rules, would be ideal. Even so, a relatively few attempts have been made to integrate the two types of systems on microarray gene expression data. In earlier reports, Li et al.  has proposed a classifier named PCL (prediction by collective likelihoods) which is based on the concept of emerging patterns and can provide the rules describing the microarray gene expression data. Tan et al.  have introduced a new classifier named TSP (top scoring pair) which is based on relative expression reversals and can generate accurate decision rules. These studies also revealed the phenomenon of trade-off between credibility and comprehensibility in such a hybrid system. For that reason, I have made attempts to design an integrated and effective framework with less interaction between cancer classification and rule extraction functions.
In this paper, I have presented an integrated method (named X-AI) which is based on a three-tiered architecture from the viewpoint of system design of software engineering. Different tests have been carried out on two leukemia datasets for evaluating the performance of X-AI. The obtained results indicated that X-AI is able to perform well on both functions of classification and rule extraction in microarray analysis.
Datasets and pre-processing
I used two different leukemia datasets for the following reasons: (i) both datasets have been analyzed and discussed in many literatures, which is helpful to compare with their results; (ii) the rules extracted from the similar cancer type of datasets could be compared to each other; (iii) the robustness of classification system could be observed by the datasets that are obtained from different experiments; and (iv) the two datasets represent the nature of the binary classification and multi-class problems, which is useful to evaluate the effectiveness of the proposed method for different classification problems.
The first acute leukemia data (named L1) of Golub et al.  is composed of 72 samples from two different types of acute leukemia, acute lymphoblastic leukemia (ALL) and acute myeloid leukemia (AML). The training set has 38 bone marrow samples (27 ALL and 11 AML) and the test set consists of 24 bone marrow and 10 peripheral blood samples (20 ALL and 14 AML). Bone marrow mononuclear cells were collected by Ficoll sedimentation in the training set and RNA was hybridized to Affymetrix oligonucleotide microarrays, by which each sample has expression patterns of 7129 probes measured. The second acute leukemia data (named L2) of Armstrong et al.  includes 12582 gene expression values for 57 peripheral blood or bone marrow samples. The training set contains 57 leukemia samples (20 ALL, 17 MLL (mixed lineage leukemia) and 20 AML) and the test set contains 15 samples (4 ALL, 3 MLL and 8 AML). For microarray data, pre-processing is of critical importance in downstream analyses. In order to equalize expression values for each sample and avoid the bias against samples, all values in a sample have been re-scaled by a multiplicative factor which is determined by linear regression of genes with present calls. All multiplicative factors are available on the established web server. Duoit et al.  applied thresholding, filtering and logarithmic transformation steps before analyzing the leukemia dataset. Accordingly, the expression values were limited by both upper and lower bounds. Since it could be easy to neglect information leakage effects during pre-processing of the proteomic profiling on mass spectrometry data as well as the microarray expression data , the upper bound is lifted to 24000 and the lower bound -800, which can increase the changes of finding relevant genes due to a larger search space. Further, I tried to perform the feature selection function instead of a simple filter to systematically reduce the number of genes. The mechanism is described in the following section.
More details of datasets can be found on the web server and in Broad Institute http://www.broad.mit.edu which evolved from research collaborations in the MIT and Harvard communities and made the generated data available to the scientific community.
The three-tiered architecture integrates the tasks of microarray data analysis from the pre-processing to the data mining including classification and rule extraction. Each function layer with independency can be changed internally without affecting other layers. Therefore, this architecture can provide the consistency of data to different components of the same layer, and reduce the interaction between layers as well as between the components of the same layer.
where R i is the number of samples in the i-th interval, C j the number of samples in the j-th class, n the total number of samples. The algorithm mainly consists of two phases, named Phase I and II. Phase I comprises the calculation of the chi-squared value for adjacent intervals, and the merge of adjacent intervals under a chi-squared threshold which will be decrementing until an inconsistency rate of data is exceeded; Phase II includes the finer process of Phase I for each feature, and the evaluation of the merge degree which reveals the relevant feature to data. For example, a feature is regarded as an irrelevance for data if it is merged to only one value at the end of Phase II.
In this work, I have applied the algorithm to two different datasets to analyze the relative importance of genes for the discrimination of tumor classes. And it was chiefly carried out from a suit of free open-source software , which provides numerous machine learning algorithms from various learning paradigms.
Diaquadratic discriminant analysis (DQDA)
where μ i is the mean of x for the ω i class, Σ i the l by l covariance matrix. When the covariance matrices are diagonal, , the ML discriminate rule can be written as , which is a special case of diagonal quadratic discriminant analysis (DQDA). In practice, μ i and Σ i are estimated by corresponding sample quantities. we have effectively utilized it for the analysis of discriminating two- and three-state proteins . In this study, the combination of selected genes was used as the feature vector to discriminate tumor classes.
Generalized rule induction (GRI)
where p(a) represents the probability of the observed attribute value of a, as a measure of the coverage of the antecedent; p(c) represents the prior probability of the value of c, as a measure of the common of the observed attribute value of c in the consequent; p(c|a) represents an modified probability of observing this value of c after taking into account the additional information of the value of a. For rules with more than one antecedent, p(a) is regarded as the probability of the conjunction of the variable values in the antecedent. Accordingly, a set of optimal rules was then generated by ITRULE algorithm, which calculates J-measures of rules by employing depth-first search over possible left-hand sides.
Here, the genes selected by Chi2 algorithm were considered as the attributes of the antecedent. And the tumor class was the only attribute of the consequent.
Performance evaluation and test procedure
I considered the classification of the leukemia datasets L1 and L2 as the two-class and three-class problems, respectively. To evaluate the performance of the classification problems, both classification accuracy and misclassified number were calculated along with corresponding number of selected genes.
Support and confidence
This measure reveals the comprehensiveness of the rule to the dataset.
Holdout validation and leave-one-out cross-validation tests
The present method was validated by both holdout validation and leave-one-out cross-validation (LOOCV) tests. Holdout validation derives a predictor from the training set, and uses the blind or independent test set to evaluate the predictor. LOOCV is simple n-fold cross-validation, where n is the number of samples in the dataset. Each sample is left out in turn, and the predictor is trained on all the remaining ones. The procedure is repeated for n times to obtain a mean score.
Analysis of important genes
X-AI provides a feature selection function to systematically extract the relative important genes for discriminating different classes. In Table 1, the top ten genes for each training set of two datasets are listed according to the order of the chi-squared statistic. The selected genes provide input information to both subsequent functions of classification and rule development, and the small number of selected genes has a low data dimension, as well as low calculation complexity. Nevertheless, the decision of the nmber is flexible and largely depends on the analysis requirement.
Top ten genes selected by feature selection function of X-AI for two datasets
CST3 Cystatin C (amyloid angiopathy and cerebral hemorrhage)
PTX3 Pentaxin-related gene, rapidly induced by IL-1 beta
CHRNA7 Cholinergic receptor, nicotinic, alpha polypeptide 7
GLUTATHIONE S-TRANSFERASE, MICROSOMAL
APLP2 Amyloid beta (A4) precursor-like protein 2
ALKALINE PHOSPHATASE, INTESTINAL PRECURSOR
CD33 CD33 antigen (differentiation antigen)
H. sapiens mRNA for oct-binding factor
Homo sapiens mRNA for KIAA0959 protein, partial cds
Homo sapiens mRNA for KIAA0867 protein, complete cds
Homo sapiens myosin light chain kinase (MLCK) mRNA, complete cds
Homo sapiens transmembrane protein (WFS1) mRNA, complete cds
Homo sapiens TWIK-related acid-sensitive K+ channel (TASK) mRNA, complete cds
wg66h09.x1 Homo sapiens cDNA, 3' end
H. sapiens mRNA for Tcell leukemia
Human transcriptional activator (BRG1) mRNA, complete cds
H. sapiens mRNA for phosphoinositide 3-kinase
Prediction performance of system
The results show that the classification function performs well in discriminating these different classes when the input information is provided by the feature selector function of X-AI. Namely, the integration of the both functions can be feasible and effective for the binary classification and three-class problems.
Comparison with other methods
Association rule development
The function of feature selection did not only reduce the number of input genes, but also improve the efficiency of rule development. It also results in a rational and acceptable number of rules. Based on the genes of Table 1, X-AI included all the samples for each dataset to establish association rules.
Two different classes of rules generated from dataset L1
L09209_s > 1056.5 & M23197 > 326.0
M23197 > 401.5
M27891 > 2096.5
X95735 > 994.0 & M55150 > 1250.5
X95735 > 994.0
U46499 < 154.5
L09209_s < 992.5
X95735 < 994.0
Three different classes of rules generated from dataset L2
32847_at > 147.0
36239_at > 2201.0
39318_at < 1063.0 & 32579_at < 2285.0
1325_at < 1501.5, 39318_at < 1063.0 & 32579_at < 2285.0
1325_at < 1501.5, 36239_at < 214.0 & 40191_s_at < 508.5
36239_at < 214.0 & 40191_s_at < 508.5
39318_at < 1063.0 & 35164_at < -794.5
40191_s_at < 519.0 & 36239_at < 167.0
1325_at < 1501.5, 39318_at < 1063.0 & 35164_at < -794.5
1325_at < 1501.5, 40191_s_at < 519.0 & 36239_at < 167.0
1325_at < 1501.5, 36239_at < 214.0 & 37539_at < -362.0
36239_at < 214.0 & 37539_at < -362.0
37539_at < -725.5
32579_at < 2285.0
1325_at < 1501.5 & 32579_at < 2285.0
36239_at < 214.0
1325_at < 201.0, 35260_at > 794.5 & 40191_s_at > 1107.5
1325_at < 201.0 & 36239_at > 214.0
1325_at < 201.0
Web server for cancer classification
Because X-AI selected two different sets of input genes from two datasets for training the classifiers, it results in two classifiers with different sets of input genes. Users can optionally assign one of both to predict cancer classes. In addition to the cancer classification page, the web server has provided help and reference pages for interested researchers.
In this study, I have proposed an integrated method for accurate cancer classification, relevant gene selection, and the associate rule development from DNA microarray data. Applying the concepts of system design, the modules in the present architecture are tight cohesion and loose coupling.
Through different tests, the method shows high classification accuracy on two leukemia datasets. In addition, the selected genes and the generated rules are in accord with recent studies. The results suggest that the method can effectively integrate these related functions for the analysis of microarray data.
This work was supported by grant no. NSC97-2221-E-451-013 from the National Science Council, Taiwan, ROC. I would like to thank Dr. Chang-Sheng Wang for critical reading, and reviewers for providing valuable comments to improve the manuscript.
- Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA, Bloomfield CD, Lander ES: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science. 1999, 286 (5439): 531-537. 10.1126/science.286.5439.531.View ArticlePubMedGoogle Scholar
- Alon U, Barkai N, Notterman DA, Gish K, Ybarra S, Mack D, Levine AJ: Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc Natl Acad Sci USA. 1999, 96 (12): 6745-6750. 10.1073/pnas.96.12.6745.PubMed CentralView ArticlePubMedGoogle Scholar
- Zhang H, Yu CY, Singer B, Xiong M: Recursive partitioning for tumor classification with gene expression microarray data. Proc Natl Acad Sci USA. 2001, 98 (12): 6730-6735. 10.1073/pnas.111153698.PubMed CentralView ArticlePubMedGoogle Scholar
- Olshen AB, Jain AN: Deriving quantitative conclusions from microarray expression data. Bioinformatics. 2002, 18 (7): 961-970. 10.1093/bioinformatics/18.7.961.View ArticlePubMedGoogle Scholar
- Hedenfalk I, Duggan D, Chen Y, Radmacher M, Bittner M, Simon R, Meltzer P, Gusterson B, Esteller M, Kallioniemi OP, Wilfond B, Borg A, Trent J, Raffeld M, Yakhini Z, Ben-Dor A, Dougherty E, Kononen J, Bubendorf L, Fehrle W, Pittaluga S, Gruvberger S, Loman N, Johannsson O, Olsson H, Sauter G: Gene-expression profiles in hereditary breast cancer. N Engl J Med. 2001, 344 (8): 539-548. 10.1056/NEJM200102223440801.View ArticlePubMedGoogle Scholar
- Tibshirani R, Hastie T, Narasimhan B, Chu G: Diagnosis of multiple cancer types by shrunken centroids of gene expression. Proc Natl Acad Sci USA. 2002, 99 (10): 6567-6572. 10.1073/pnas.082099299.PubMed CentralView ArticlePubMedGoogle Scholar
- Huang X, Pan W: Linear regression and two-class classification with gene expression data. Bioinformatics. 2003, 19 (16): 2072-2078. 10.1093/bioinformatics/btg283.View ArticlePubMedGoogle Scholar
- Bicciato S, Luchini A, Di Bello C: PCA disjoint models for multiclass cancer analysis using gene expression data. Bioinformatics. 2003, 19 (5): 571-578. 10.1093/bioinformatics/btg051.View ArticlePubMedGoogle Scholar
- Martella F: Classification of microarray data with factor mixture models. Bioinformatics. 2006, 22 (2): 202-208. 10.1093/bioinformatics/bti779.View ArticlePubMedGoogle Scholar
- Su Z, Hong H, Perkins R, Shao X, Cai W, Tong W: Consensus analysis of multiple classifiers using non-repetitive variables: diagnostic application to microarray gene expression data. Comput Biol Chem. 2007, 31 (1): 48-56. 10.1016/j.compbiolchem.2007.01.001.View ArticlePubMedGoogle Scholar
- Huang LT, Gromiha MM, Hwang SF, Ho SY: Knowledge acquisition and development of accurate rules for predicting protein stability changes. Comput Biol Chem. 2006, 30 (6): 408-415. 10.1016/j.compbiolchem.2006.06.004.View ArticlePubMedGoogle Scholar
- Huang LT, Gromiha MM, Ho SY: Sequence analysis and rule development of predicting protein stability change upon mutation using decision tree model. Journal of Molecular Modeling. 2007, 13 (8): 879-890. 10.1007/s00894-007-0197-4.View ArticlePubMedGoogle Scholar
- Huang LT, Gromiha MM, Ho SY: iPTREE-STAB: interpretable decision tree based method for predicting protein stability changes upon mutations. Bioinformatics. 2007, 23 (10): 1292-1293. 10.1093/bioinformatics/btm100.View ArticlePubMedGoogle Scholar
- Borgelt C, Berthold MR: Mining molecular fragments: finding relevant substructures of molecules. 2002, The 2002 IEEE international Conference on Data Mining, Washington, DC, 51-58.Google Scholar
- Oyama T, Kitano K, Satou K, Ito T: Extraction of knowledge on protein-protein interaction by association rule discovery. Bioinformatics. 2002, 18 (5): 705-714. 10.1093/bioinformatics/18.5.705.View ArticlePubMedGoogle Scholar
- Creighton C, Hanash S: Mining gene expression databases for association rules. Bioinformatics. 2003, 19 (1): 79-86. 10.1093/bioinformatics/19.1.79.View ArticlePubMedGoogle Scholar
- Carmona-Saez P, Chagoyen M, Rodriguez A, Trelles O, Carazo JM, Pascual-Montano A: Integrated analysis of gene expression by Association Rules Discovery. BMC Bioinformatics. 2006, 7: 54-10.1186/1471-2105-7-54.PubMed CentralView ArticlePubMedGoogle Scholar
- Li J, Liu H, Downing JR, Yeoh AE, Wong L: Simple rules underlying gene expression profiles of more than six subtypes of acute lymphoblastic leukemia (ALL) patients. Bioinformatics. 2003, 19 (1): 71-78. 10.1093/bioinformatics/19.1.71.View ArticlePubMedGoogle Scholar
- Tan AC, Naiman DQ, Xu L, Winslow RL, Geman D: Simple decision rules for classifying human cancers from gene expression profiles. Bioinformatics. 2005, 21 (20): 3896-3904. 10.1093/bioinformatics/bti631.PubMed CentralView ArticlePubMedGoogle Scholar
- Armstrong SA, Staunton JE, Silverman LB, Pieters R, den Boer ML, Minden MD, Sallan SE, Lander ES, Golub TR, Korsmeyer SJ: MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia. Nat Genet. 2002, 30 (1): 41-47. 10.1038/ng765.View ArticlePubMedGoogle Scholar
- Dudoit S, Fridlyand J, Speed T: Comparison of discrimination methods for the classification of tumors using gene expression data. Technical Report 576, Statistics Dept, UC Berkeley. 2000Google Scholar
- Barla A, Jurman G, Riccadonna S, Merler S, Chierici M, Furlanello C: Machine learning methods for predictive proteomics. Brief Bioinform. 2008, 9 (2): 119-128. 10.1093/bib/bbn008.View ArticlePubMedGoogle Scholar
- Yourdon E, Constantine LL: Structured design: fundamentals of a discipline of computer program and systems design. 1979, Englewood Cliffs, N.J., Prentice HallGoogle Scholar
- Berrar DP, Dubitzky W, Granzow M: A practical approach to microarray data analysis. 2003, Boston, MA, Kluwer Academic PublishersView ArticleGoogle Scholar
- Huan L, Rudy S: Chi2: Feature Selection and Discretization of Numeric Attributes. Seventh International Conference on Tools with Artificial Intelligence (ICTAI). 1995, 388-Google Scholar
- Witten IH, Frank E: Data Mining: Practical machine learning tools and techniques. 2005, San Francisco, Morgan Kaufmann, 2Google Scholar
- Theodoridis S, Koutroumbas K: Pattern recognition. 2006, Amsterdam; Boston, Elsevier/Academic Press, 3Google Scholar
- Huang LT, Gromiha MM: Analysis and prediction of protein folding rates using quadratic response surface models. Journal of Computational Chemistry. 2008, 29 (10): 1675-1683. 10.1002/jcc.20925.View ArticlePubMedGoogle Scholar
- Smyth P, Goodman RM: An information theoretic approach to rule induction from databases. Knowledge and Data Engineering, IEEE Transactions on. 1992, 4 (4): 301-316. 10.1109/69.149926.View ArticleGoogle Scholar
- Wang Y, Tetko IV, Hall MA, Frank E, Facius A, Mayer KF, Mewes HW: Gene selection from microarray data for cancer classification – a machine learning approach. Comput Biol Chem. 2005, 29 (1): 37-46. 10.1016/j.compbiolchem.2004.11.001.View ArticlePubMedGoogle Scholar
- Furey TS, Cristianini N, Duffy N, Bednarski DW, Schummer M, Haussler D: Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics. 2000, 16 (10): 906-914. 10.1093/bioinformatics/16.10.906.View ArticlePubMedGoogle Scholar
- Li J, Wong L: Identifying good diagnostic gene groups from gene expression profiles using the concept of emerging patterns. Bioinformatics. 2002, 18 (5): 725-734. 10.1093/bioinformatics/18.5.725.View ArticlePubMedGoogle Scholar
- Antonov AV, Tetko IV, Mader MT, Budczies J, Mewes HW: Optimization models for cancer classification: extracting gene interaction information from microarray expression data. Bioinformatics. 2004, 20 (5): 644-652. 10.1093/bioinformatics/btg462.View ArticlePubMedGoogle Scholar
- Fort G, Lambert-Lacroix S: Classification using partial least squares with penalized logistic regression. Bioinformatics. 2005, 21 (7): 1104-1111. 10.1093/bioinformatics/bti114.View ArticlePubMedGoogle Scholar
- Liu CC, Chen WS, Lin CC, Liu HC, Chen HY, Yang PC, Chang PC, Chen JJ: Topology-based cancer classification and related pathway mining using microarray data. Nucleic Acids Res. 2006, 34 (14): 4069-4080. 10.1093/nar/gkl583.PubMed CentralView ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.