Top Page | Upper Page | Contents | About This Site | JAPANESE

Data Mining

Data Mining is the methods to mine useful information from avalanche of data.

I think method called "data mining" has characteristics below.

• To treat large amount of data for which we have to use computers.
• To find particular data in local area rather than whole area.
• The Conclusion is made automatically. ( Machine Learning is used)

Data mining is not only one of multivariable analysis but particular method because "multivariable" is not appropriate to some method.
Upper characteristics 3 is not common in multivariable analysis.

Types of data mining

How to see part of the data

• Decision tree : Find a characteristic area in the data. Classification tree and regression tree , N-try tree , random forest
• Neighboring method : Refer to nearby data. k-NN ,LOF,Minimum distance method

How to look over the data

• Analysis of sample grouping : Cross the table of data into strips to see the similarity of the strips. Visualize by compressing high dimensions into two dimensions , Multi Dimensional Scaling, Self Organizing Map , Cluster Analysis
• Analysis of individual category grouping : See partial similarity of data. Associations Analysis, Rough Sets Analysis, Broadly defined Quantification theory 3, Correspondence analysis
• Many-to-many analysis : See the relationship between items. Network Analysis, Eigenvalue Analysis, Matrix Decomposition , Q-Analysis, Canonical Correlation Analysis

How to quantitatively analyze verbal data

• Quantification theory : Extract quantitative information from qualitative variables.
• Text mining : View sentences as a collection of data.

In the literature, neural networks and Bayesian networks are sometimes introduced as data mining companions. The method that came out after the so-called " multivariate analysis " is collectively called "data mining", so it seems that it is classified in this way.

However, I don't think the point of "mining" the important part from the pile of data is true. This site classifies it as artificial intelligence (AI) .

Difference between Decision Tree and Cluster Analysis

The outputs of Decision Tree and Cluster Analysis are similar. Both of them use the tree-type graph.

The difference between decision Tree and Cluster Analysis is very similar to the difference between Multi-Regression Analysis and Principal Component Analysis . The former use the information of "Y" data in its analysis. But the latter does not use "Y" data. The former analyze the relationship between Y and Xs. The latter analyze the relationship among Xs.

Selection of Methods

Statistical Way of Making Hypothesis

NEXT EDA(Exploratory Data Analysis)