# Hypothesis Testing

Average
is often used to compare categories.
But if unevenness of data is large, the difference of average may be an accident.

Graphs of Disribution
are useful to check the reliability.
But if there are not many samples or the difference is very small,
the visual checking is difficult to judge.

In
statistics
,
hypothesis testing (HT) is used to check the reliability mathematically.

##
Basic Idea (Significance Level and P-value)

There are various HT methods for statistical values (average, deviation and so on).
These methods use different process.
But the final step is similar.
On the final step, the diagram like in this page is used to judge.

If the value we want to judge is in the area of comparing data, the value is a group of comparing data.

The level of judgment is called "significance level".

The index of significance is P-value (provability value).

0.05 (5 percent) is often used as the significance level.
The idea of judgment is that

"P-value is less than 0.05."

= "The phenomenon is difficult to predict."

= "The difference is significant."

###
Effect of the number of samples (feature of P-value)

If there are many samples, P-value likes to be small value.
So the number of samples affects the judgment.

Even if there is a significant difference, it is not useful.
So after the judgment, we need to judge the volume of the difference.

###
Data Mining by Hypothesis Testing

There are functions to calculate P-value in
spreadsheet
software.
If there are many variables, P-value is useful to pick-up suspicious values.
Small P-value means that we should study details of the variables.

This idea it a basic idea of
Decision tree
.

##
Hypothesis Testing of the Difference of Average

In the real work of
SPC(Quality Control)
, hypothesis testing of the difference of average is often used.
It is called "t-test."

T-test is used to compare two average of two groups.

ANOVA
is also for more than two average.
Paired t-test
is used if there is "pair" information.

##
Non-parametric Testing

Parametric testing uses the condition of normal distribution.
In my experience, parametric testing is useful for almost all analysis.

But sometimes, we cannot use normal distribution as a condition.
So we need non-parametric testing.

Non-parametric testing uses the distribution of plus and minus, the order and median.
For the situations,
"Average is not important as the group data",
"We can estimate the order but we cannot estimate concrete (quantity) numbers"
, non-parametric testing may be useful.

Strength and Weakness of Big Data

NEXT Type I error and Type II error